和ObjectWritable一样,GenericWritable也是一个包装类。一个应用场景就是Reduce段的输入值,如果是key type一样,value type不同的,就可以用GenericWritable来代替。GenericWritable和ObjectWritable区别主要在于:ObjectWritable序列化的时候,会将类类型名称也一起序列化,会造成浪费,特别是大量的网络传输。GenericWritable就是为了解决这样的问题,对于少量类型数量,且事先知道类型,可以用GenericWritable来提供效率
GenericWritable是个抽象类,它有个核心抽象方法getTypes需要用户自己去实现。getTypes方法就是将事先知道的数据类型组成一个数组,序列化和反序列化的时候都需要调用这个方法
/** * Return all classes that may be wrapped. Subclasses should implement this * to return a constant array of classes. */ abstract protected Class<? extends Writable>[] getTypes();GenericWritable有4个属性
private static final byte NOT_SET = -1;//表示没有set实例值 private byte type = NOT_SET;//表示实例的类型(实例类型在getTypes返回数组中的序号) private Writable instance;//实例对象 private Configuration conf = null;//Configuaration对象下面来介绍GenericWritable的主要方法:
set方法 用于赋值实例,并设置type的值
/** * Set the instance that is wrapped. * * @param obj */ public void set(Writable obj) { instance = obj;//对实例赋值 Class<? extends Writable> instanceClazz = instance.getClass();//获得实例的类型 Class<? extends Writable>[] clazzes = getTypes();//获取事先配置的类型数组 //查找实例类型在类型数组中的序号 for (int i = 0; i < clazzes.length; i++) { Class<? extends Writable> clazz = clazzes[i]; if (clazz.equals(instanceClazz)) { type = (byte) i;//设置type的值(序号) return; } } throw new RuntimeException("The type of instance is: " + instance.getClass() + ", which is NOT registered."); }
序列化方法write
@Override public void write(DataOutput out) throws IOException { if (type == NOT_SET || instance == null) throw new IOException("The GenericWritable has NOT been set correctly. type=" + type + ", instance=" + instance); out.writeByte(type);//将类型序号虚拟化 instance.write(out);//序列化实例对象 }
反序列化:
@Override public void readFields(DataInput in) throws IOException { type = in.readByte();//读取类型序号 Class<? extends Writable> clazz = getTypes()[type & 0xff];//通过序号找到对应的类型 try { instance = ReflectionUtils.newInstance(clazz, conf);//通过反射实例化对象 } catch (Exception e) { e.printStackTrace(); throw new IOException("Cannot initialize the class: " + clazz); } instance.readFields(in);//调用实例对象的反序列化方法,初始化属性值 }
从代码中来看,GenericWritable采用类型序号来代替类型名称,从而减少序列化数据的长度。GenericWritable的缺点也显而易见,当数据类型不确定,或者数据类型特别多的时候,GenericWritable不适合使用