1、编码基本设置
由于hdfs使用的是utf-8的编码格式,所以不管是从mysql导入,还是直接上传文件,编码格式必须是utf-8,否则就会出现乱码,所有有几个编码设置点要注意:
(1)、源数据mysql需要设置my.cnf配置文件 (不同版本可能设置不一样)
[client]
default-character-set=utf8
/*[mysqld]*/
character-set-server=utf8
(2)、源数据库(database编码确认设置)
mysql> show variables like 'char%';
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
如果不是以上格式,使用该指令设置,变换不同参数即可:set character_set_server=utf8;
(3)读取源数据的jdbc连接加上utf-8编码
sqoop import --connect jdbc:mysql://10.0.13.116:3306/test?characterEncoding=UTF-8 --username hive --password for3edc4rfv --table driver --delete-target-dir --hive-overwrite --fields-terminated-by "," --target-dir /apps/hive/warehouse/t_driver
(4)hive的jdbc连接也设置utf-8
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8</value>
</property>
2、执行sqoop指令的权限
由于sqoop需要操作hdfs,所以需要使用hdfs账号执行sqoop指令,否则要报无权限错误;