Hive表如何使用多字符分隔

一般情况下，Hive建表指定分隔符使用”row format delimited fields terminated by”语法，以下是一个最简单的例子，

hive> create external table t2 (a int, b string)
    > row format delimited fields terminated by '|'
    > location '/user/hive/warehouse/t3';
OK
Time taken: 0.045 seconds

使用此方法时，分隔符只能是一个字符，假如想使用多个分隔符如’$%#’，则虽然可以成功创建表，但查询的结果是不正确的，如下

hadoop fs -cat /user/hive/warehouse/t3/t3.txt
1$%#aaa
2$%#bbb
3$%#ccc

hive> create external table t3 (a int, b string)
    > row format delimited fields terminated by '$%#'
    > location '/user/hive/warehouse/t3';
OK
Time taken: 0.051 seconds
hive> select * from t3;
OK
1       %#aaa
2       %#bbb
3       %#ccc
Time taken: 0.105 seconds, Fetched: 3 row(s)

要解决使用多字符分隔的问题，目前从网上查到主要有两种方法，1 MultiDelimitSerDe 2 RegexSerDe

方法1

hive> create external table t4(id INT, name STRING)
    > ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe'
    > WITH SERDEPROPERTIES ("field.delim"="$%#")
    > location '/user/hive/warehouse/t3';
OK
Time taken: 0.045 seconds
hive> select * from t4;
OK
1       aaa
2       bbb
3       ccc
Time taken: 0.092 seconds, Fetched: 3 row(s)

方法2

hive> create external table t5(a string, b string)
    > ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
    > WITH SERDEPROPERTIES ("input.regex" = "^(.*)\\$\\%\\#(.*)$")
    > location '/user/hive/warehouse/t3';
OK
Time taken: 0.046 seconds
hive> select * from t5;
OK
1       aaa
2       bbb
3       ccc
Time taken: 0.09 seconds, Fetched: 3 row(s)

作者：post_yuan
来源：CSDN
原文：https://blog.csdn.net/Post_Yuan/article/details/78845273

但是这样配置会有一个类似以下的问题，
Class org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe not found
解决方法：

将下面的配置语句，加在配置文件： conf/hive-site.xml中，value中hive-contrib-*.jar的路径为你机器上实际的放置。

<property>
  <name>hive.aux.jars.path</name>
  <value>file:///home/develop/hive-0.8.0/lib/hive-contrib-0.8.0.jar</value>
  <description>Added by tiger.zeng on 20120202.These JAR file are available to all users for all jobs</description>
</property>

Hive表如何使用多字符分隔

猜你喜欢