As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath...
So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.
I got stuck at this...
Then a long weekend.
Today is monday. I continue work on this. I should mention that I have read the articles like these:
http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/
http://grepalex.com/2013/02/25/hadoop-libjars/
But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!
Using the libjars option with Hadoop
猜你喜欢
转载自standalone.iteye.com/blog/1872142
今日推荐
周排行