在Flume写入HDFS的时候会自动换行问题, 项目要求写入HDFS中数据不需要换行, 查看源码及配置如下: BodyTextEventSerializer.java @Override public void write(Event e) throws IOException { out.write(e.getBody()); if (appendNewline) { out.write('\n'); } } // for legacy reasons, by default, append a newline to each event written out private final String APPEND_NEWLINE = "appendNewline"; private final boolean APPEND_NEWLINE_DFLT = false; private final OutputStream out; private final boolean appendNewline; private BodyTextEventSerializer(OutputStream out, Context ctx) { this.appendNewline = ctx.getBoolean(APPEND_NEWLINE, APPEND_NEWLINE_DFLT); this.out = out; } http://flume.apache.org/FlumeUserGuide.html中查看此配置项为: Body Text Serializer Alias: text. This interceptor writes the body of the event to an output stream without any transformation or modification. The event headers are ignored. Configuration options are as follows: Property Name Default Description appendNewline true Whether a newline will be appended to each event at write time. The default of true assumes that events do not contain newlines, for legacy reasons. Example for agent named a1: a1.sinks = k1 a1.sinks.k1.type = file_roll a1.sinks.k1.channel = c1 a1.sinks.k1.sink.directory = /var/log/flume a1.sinks.k1.sink.serializer = text a1.sinks.k1.sink.serializer.appendNewline = false
Flume写数据换行
猜你喜欢
转载自houshangxiao.iteye.com/blog/2212097
今日推荐
周排行