[转]hive产生动态分区太多导致的问题 .

转自:http://blog.csdn.net/chenyi8888/article/details/8236726

做应用开发的也要帮助他们找BUG,真是头疼。遇到如下异常,度娘给出的答案如下:

过多的动态分区会导致job出现以下情况:

org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /tmp/hive-maintain/hive_2012-11-28_22-39-43_810_1689858262130334284/_task_tmp.-ext-10002/part=33436268/_tmp.000004_0 File does not exist. Holder DFSClient_attempt_201211250925_9859_m_000004_0 does not have any open files.

解决方案:

加大动态分区数即可,下面是在当前session下设置。

SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000;

猜你喜欢

转载自dacoolbaby.iteye.com/blog/1904138