1. 浏览器下载ipv6地址库。https://lite.ip2location.com/download?id=13
2. 解压
[yeqiang@localhost Downloads]$ unzip IP2LOCATION-LITE-DB3.IPV6.CSV.ZIP
Archive: IP2LOCATION-LITE-DB3.IPV6.CSV.ZIP
inflating: LICENSE_LITE.TXT
inflating: README_LITE.TXT
inflating: IP2LOCATION-LITE-DB3.IPV6.CSV 3. 查看CSV文件
[yeqiang@localhost Downloads]$ less IP2LOCATION-LITE-DB3.IPV6.CSV
4. 用Python脚本，把CSV文件转换成导入到Redis的脚本文件
4.1 脚本内容
#!/usr/bin/python
# -*- coding: UTF-8 -*-
import csv
import json
import logging
import sys
import os
# 导入csv到mysql
# 以下代码不能在main中处理，不生效
reload(sys)
sys.setdefaultencoding('utf8')
if not os.path.exists("./logs"):
os.mkdir("./logs")
my_name = os.path.basename(sys.argv[0])
logging.basicConfig(
level=logging.DEBUG,
format='thread-%(thread)d %(asctime)s %(levelname)s: %(message)s %(filename)s:%(lineno)d',
filename="./logs/%s.log" % my_name,
filemode='a+')
def generate_import_sql():
csv_file = open("/home/yeqiang/Downloads/IP2LOCATION-LITE-DB3.CSV")
csv_reader_lines = csv.reader(csv_file)
count = 0
gen_insert = False
for line in csv_reader_lines:
if count % 2000 == 0:
if count != 0:
print(";")
gen_insert = True
print("insert into ip_country_mapping (`start_ip`, `end_ip`, `country`, `province`, `city`) values ")
print(" ('%s', '%s', '%s', '%s', '%s')" % (line[0], line[1], line[3], line[4], line[5]))
else:
print(",('%s', '%s', '%s', '%s', '%s')" % (line[0], line[1], line[3], line[4], line[5]))
count += 1
print(";")
csv_file.close()
def generate_import_redis_ipv4(country_list):
"""
1. 由于redis特殊型，pipe模式导入的文件要转换成windows格式，即 unix2dos指令转化
2. zadd 命令，对象值类似key，生成的值做成唯一性,因此将score加入到值中
:return:
"""
csv_file = open("/home/yeqiang/Downloads/IP2LOCATION-LITE-DB3.CSV")
csv_reader_lines = csv.reader(csv_file)
for line in csv_reader_lines:
country = line[3]
if len(country_list) > 0 and country not in country_list:
continue
start_ip = int(line[0])
end_ip = int(line[1])
location = {
"start_ip": start_ip,
"end_ip": end_ip,
"country": line[3],
"province": line[4],
"city": line[5]
}
json_str = json.dumps(location)
# print("ZADD z_ip2location %s \"%s\"" % (start_ip, json_str.replace("\"", "\\\"").replace("'", "\\'")))
print("ZADD z_ip2location %s \"%s\"" % (end_ip, json_str.replace("\"", "\\\"").replace("'", "\\'")))
csv_file.close()
pass
def generate_import_redis_ipv6(country_list):
"""
1. 由于redis特殊型，pipe模式导入的文件要转换成windows格式，即 unix2dos指令转化
2. zadd 命令，对象值类似key，生成的值做成唯一性,因此将score加入到值中
:return:
"""
csv_file = open("/home/yeqiang/Downloads/IP2LOCATION-LITE-DB3.IPV6.CSV")
csv_reader_lines = csv.reader(csv_file)
for line in csv_reader_lines:
country = line[3]
if len(country_list) > 0 and country not in country_list:
continue
start_ip = int(line[0])
end_ip = int(line[1])
location = {
"start_ip": start_ip,
"end_ip": end_ip,
"country": line[3],
"province": line[4],
"city": line[5]
}
json_str = json.dumps(location)
# print("ZADD z_ip2location %s \"%s\"" % (start_ip, json_str.replace("\"", "\\\"").replace("'", "\\'")))
print("ZADD z_ip2location_ipv6 %s \"%s\"" % (end_ip, json_str.replace("\"", "\\\"").replace("'", "\\'")))
csv_file.close()
pass
if __name__ == '__main__':
# generate_import_redis(['Malaysia', 'Singapore', 'Hong Kong', 'Taiwan, Province of China'])
# generate_import_redis_ipv4([])
generate_import_redis_ipv6([])
4.2 执行脚本，生成导入redis文件
[yeqiang@localhost cron_task]$ python import_ip2location.csv.py > /tmp/redis_ip2location_ipv6_all.txt
[yeqiang@localhost cron_task]$ 4.3 转为为windows格式文本
[yeqiang@localhost cron_task]$ unix2dos /tmp/redis_ip2location_ipv6_all.txt
unix2dos: converting file /tmp/redis_ip2location_ipv6_all.txt to DOS format...特别说：必须转换为dos格式，否则，导入到redis的时候会报错！！！
5. 导入到Redis中
5.1 查看文本文件大小
[yeqiang@localhost cron_task]$ ll /tmp/redis_ip2location_ipv6_all.txt -h
-rw-rw-r-- 1 yeqiang yeqiang 642M Aug 6 15:11 /tmp/redis_ip2location_ipv6_all.txt5.2 压缩文件，并上传到服务器
[yeqiang@localhost cron_task]$ cd /tmp/
[yeqiang@localhost tmp]$ tar -zcvf redis_ip2location_ipv6_all.tar.gz redis_ip2location_ipv6_all.txt
redis_ip2location_ipv6_all.txt
[yeqiang@localhost tmp]$ ll redis_ip2location_ipv6_all.* -h
-rw-rw-r-- 1 yeqiang yeqiang 60M Aug 6 15:16 redis_ip2location_ipv6_all.tar.gz
-rw-rw-r-- 1 yeqiang yeqiang 642M Aug 6 15:11 redis_ip2location_ipv6_all.txt
5.3 导入到redis
$ cat redis_ip2location_ipv6_all.txt | redis-cli -h localhost -a xxxx -p 6379 --pipe
6. 利用redis zrangebyscore指令查询ip对应的信息
随机从IP2LOCATION-LITE-DB3.IPV6.CSV 取出一条ip记录
"58569024899333240586683499811191128064","58569026484205975881791596750794915839","ZA","South Africa","Western Cape","Cape Town"
其中
58569024899333240586683499811191128064 是ip范围最小值
58569026484205975881791596750794915839 是ip范围最大值
ZA 是国家代码
South Africa 国家名
Western Cape 省
Cape Town 市
redis查询指令
[yeqiang@localhost ~]$ redis-cli
127.0.0.1:6379> auth pwd
OK
127.0.0.1:6379> ZRANGEBYSCORE z_ip2location_ipv6 58569026484205975881791596750794915839 340282366920938463463374607431768211455 limit 0 1
1) "{\"start_ip\": 58569024899333240586683499811191128064, \"country\": \"South Africa\", \"end_ip\": 58569026484205975881791596750794915839, \"province\": \"Western Cape\", \"city\": \"Cape Town\"}"
127.0.0.1:6379> 特别说明：
1. 340282366920938463463374607431768211455 是目前已知的ipv6 最大值（CSV文件最后一行记录）
2. 本方案是利用ip范文最大值最为score，
本方案是已ip范围最大值作为socre， 因此zrangebyscore的到的结果后，必须自己判定340282366920938463463374607431768211455在查询出来的redis记录中的start_ip 和 end_ip之间才有效。否则应当认定未命中数据！（这是额外有上层程序处理的）
7. 关于Java调用方法的一些说明：
1. ipv6 地址占用128bit共16字节存储，因此在java语言内必须使用BigInteger存储
2. ipv6 地址从HttpServletRequest获取出来的格式是采用冒分十六进制表示法
格式为X:X:X:X:X:X:X:X，其中每个X表示地址中的16b，以十六进制表示，例如：
ABCD:EF01:2345:6789:ABCD:EF01:2345:6789
这种表示法中，每个X的前导0是可以省略的，例如：
2001:0DB8:0000:0023:0008:0800:200C:417A→ 2001:DB8:0:23:8:800:200C:417A
需要转换为BigInteger以后，再接祖jedis库查询，注意参数类型long已经不能容纳这么大的数值，采用字符串类型传入。
