When we run a pig job which using hive metastore table through hue. We need to locate all related jars to oozie sharelib
Prepare
a. compile hive-0.12.0 and hive-0.13.0 to against hadoop2.3.0
b. compile pig-0.12.0 to against hadoop2.3.0
Update local pig's sharelib
a. backups all jars in share/lib/pig
b. copy pig-0.12.0.jar pig-0.12.0-withouthadoop.jar to share/lib/pig
c. copy oozie-sharelib-pig-4.0.1.jar from backuped jars in share/lib/pig
d. copy all jars in hive-0.13.0-bin/hcatalog/share/hcatalog to share/lib/pig
e. copy all jars in hive-0.13.0-bin/lib to share/lib/pig
f. copy mysql jdbc driver to share/lib/pig
Update hdfs's pig sharelib
a. delete share/lib/pig in hdfs
b. update sharelib in hdfs using
#oozie-setup.sh sharelib upgrade -fs hdfs://192.168.122.1:2014 -locallib share/
Integrate with Hue
a.init hive metastore schema to 0.13.0 using
#schematool -dbType mysql -initSchemaTo 0.13.0
b.set hue configuration to use hive-0.12.0(not hive-0.13.0)
a. backups all jars in share/lib/pig
b. copy pig-0.12.0.jar pig-0.12.0-withouthadoop.jar to share/lib/pig
c. copy oozie-sharelib-pig-4.0.1.jar from backuped jars in share/lib/pig
d. copy all jars in hive-0.13.0-bin/hcatalog/share/hcatalog to share/lib/pig
e. copy all jars in hive-0.13.0-bin/lib to share/lib/pig
f. copy mysql jdbc driver to share/lib/pig