Using the libjars option with Hadoop

As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath...

So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.

I got stuck at this...

Then a long weekend.

Today is monday. I continue work on this. I should mention that I have read the articles like these:

http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

http://grepalex.com/2013/02/25/hadoop-libjars/

But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!

猜你喜欢

转载自standalone.iteye.com/blog/1872142