问题:运行mapreduce程序出错
报错信息:
Error: java.io.IOException: Cannot obtain block length for LocatedBlock{BP-1241298414-127.0.0.1-1523601065028:blk_1073766720_25914; getBlockSize()=1077; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[172.16.144.58:50010,DS-69e6b461-59df-4e19-90ff-e766c0e3535c,DISK], DatanodeInfoWithStorage[172.16.112.83:50010,DS-3cef3847-e522-4b28-a84e-ff5ec9f2b02a,DISK]]}
at org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:444)
at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:351)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:282)
at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:272)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1132)
at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:329)
at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:326)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:326)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:914)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:85)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:554)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:792)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
解决办法:
1、找出出问题的文件
输入下面命令:
hdfs fsck /logs/product/desktop-ecommerce/2018-05-31 -openforwrite
结果:
Connecting to namenode via http://master:11070/fsck?ugi=root&openforwrite=1&path=%2Flogs%2Fproduct%2Fdesktop-ecommerce%2F2018-05-31
FSCK started by root (auth:SIMPLE) from /172.16.199.39 for path /logs/product/desktop-ecommerce/2018-05-31 at Fri Jun 01 09:33:35 CST 2018
../logs/product/desktop-ecommerce/2018-05-31/FlumeData.1527714183563 1077 bytes, 1 block(s), OPENFORWRITE: .Status: HEALTHY
Total size: 5336068 B
Total dirs: 1
Total files: 4
Total symlinks: 0
Total blocks (validated): 4 (avg. block size 1334017 B)
Minimally replicated blocks: 3 (75.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 1.5
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 2
Number of racks: 1
FSCK ended at Fri Jun 01 09:33:35 CST 2018 in 1 milliseconds
找到了问题文件:/logs/product/desktop-ecommerce/2018-05-31/FlumeData.1527714183563
2、对文件执行修复命令
hdfs debug recoverLease -path /logs/product/desktop-ecommerce/2018-05-31/FlumeData.1527714183563 -retries 3
3、再次执行mapreduce程序 没有报错 完美解决问题!