1.java.lang.NullPointerException at org.apache.hadoop.io.WritableComparator.compare
如果extends WritableComparator 实现自定义的分组器
在无参构造中一定要添加super:
public NaturalKeyGroupingComparator() {
/**
* 一定要写 super(CompositeKey.class, true); 否则报下面的错 Error:
* java.lang.NullPointerException at
* org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:157)
*
*/
super(CompositeKey.class, true);
}
2.java.lang.RuntimeException: java.io.EOFException at org.apache.hadoop.io.WritableComparator.compare
这是序列化反序列化顺序不一致导致:
原代码:
@Override
public void write(DataOutput out) throws IOException {
out.write(year);
out.write(month);
out.write(day);
out.write(wd);
}
/**
* 反序列化
* @param in
* @throws IOException
*/
@Override
public void readFields(DataInput in) throws IOException {
this.year = in.readInt();
this.month = in.readInt();
this.day = in.readInt();
this.wd = in.readInt();
}
修改后:
/**
* 序列化输出
* @param out
* @throws IOException
*/
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(year);
out.writeInt(month);
out.writeInt(day);
out.writeInt(wd);
}
/**
* 反序列化
* @param in
* @throws IOException
*/
@Override
public void readFields(DataInput in) throws IOException {
this.year = in.readInt();
this.month = in.readInt();
this.day = in.readInt();
this.wd = in.readInt();
}