版本：
Apache Kylin 3.0.0

一、问题

build cube第一步（#1 Step Name: Create Intermediate Flat Hive）报错

Container complete event for unknown container
Hive Runtime Error: Map local work exhausted memory

yarn job日志：

2021-02-08 17:00:18,379 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 3
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e31_1612260368307_6163_01_000008, AllocationRequestId: -1, Version: 0, NodeId: bigdata002:8041, NodeHttpAddress: bigdata002:8042, Resource: <memory:16384, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: 192.168.212.107:8041 }, ExecutionType: GUARANTEED, ] for a map as either  container memory less than required <memory:16384, vCores:1> or no pending map tasks - maps.isEmpty=true
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e31_1612260368307_6163_01_000009, AllocationRequestId: -1, Version: 0, NodeId: bigdata003:8041, NodeHttpAddress: bigdata003:8042, Resource: <memory:16384, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: 192.168.212.108:8041 }, ExecutionType: GUARANTEED, ] for a map as either  container memory less than required <memory:16384, vCores:1> or no pending map tasks - maps.isEmpty=true
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Cannot assign container Container: [ContainerId: container_e31_1612260368307_6163_01_000010, AllocationRequestId: -1, Version: 0, NodeId: bigdata001:8041, NodeHttpAddress: bigdata001:8042, Resource: <memory:16384, vCores:1>, Priority: 20, Token: Token { kind: ContainerToken, service: 192.168.212.106:8041 }, ExecutionType: GUARANTEED, ] for a map as either  container memory less than required <memory:16384, vCores:1> or no pending map tasks - maps.isEmpty=true
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:210432, vCores:129>
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 5
2021-02-08 17:00:18,380 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:16 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:6 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:9 ContRel:3 HostLocal:4 RackLocal:2
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1612260368307_6163: ask=0 release= 3 newContainers=0 finishedContainers=3 resourcelimit=<memory:259584, vCores:132> knownNMs=3
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e31_1612260368307_6163_01_000008
2021-02-08 17:00:19,389 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e31_1612260368307_6163_01_000008
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e31_1612260368307_6163_01_000009
2021-02-08 17:00:19,389 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e31_1612260368307_6163_01_000009
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_e31_1612260368307_6163_01_000010
2021-02-08 17:00:19,389 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container container_e31_1612260368307_6163_01_000010
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=<memory:259584, vCores:132>
2021-02-08 17:00:19,389 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 5

kylin Monitor日志：

java.io.IOException: OS command error exit with return code: 3, error message: WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/hive-common-2.1.1-cdh6.2.0.jar!/hive-log4j2.properties Async: false
OK
Time taken: 3.259 seconds
OK
Time taken: 0.086 seconds
OK
Time taken: 0.418 seconds
OK
Time taken: 0.21 seconds
Query ID = hdfs_20210208165955_31c716ed-007d-44a1-a658-2228c94af16a
Total jobs = 2
Stage-3 is selected by condition resolver.
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 16
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
21/02/08 17:00:04 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm98
Starting Job = job_1612260368307_6163, Tracking URL = http://bigdata004:8088/proxy/application_1612260368307_6163/
Kill Command = /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hadoop/bin/hadoop job  -kill job_1612260368307_6163
Hadoop job information for Stage-3: number of mappers: 6; number of reducers: 16
2021-02-08 17:00:15,008 Stage-3 map = 0%,  reduce = 0%
2021-02-08 17:00:25,373 Stage-3 map = 17%,  reduce = 0%, Cumulative CPU 18.56 sec
2021-02-08 17:00:28,471 Stage-3 map = 33%,  reduce = 0%, Cumulative CPU 42.53 sec
2021-02-08 17:00:33,651 Stage-3 map = 40%,  reduce = 0%, Cumulative CPU 98.55 sec
2021-02-08 17:00:34,686 Stage-3 map = 45%,  reduce = 0%, Cumulative CPU 154.1 sec
2021-02-08 17:00:36,748 Stage-3 map = 56%,  reduce = 0%, Cumulative CPU 158.28 sec
2021-02-08 17:00:39,831 Stage-3 map = 61%,  reduce = 0%, Cumulative CPU 166.56 sec
2021-02-08 17:00:40,853 Stage-3 map = 67%,  reduce = 0%, Cumulative CPU 182.24 sec
2021-02-08 17:00:46,041 Stage-3 map = 74%,  reduce = 0%, Cumulative CPU 191.23 sec
2021-02-08 17:00:47,070 Stage-3 map = 87%,  reduce = 0%, Cumulative CPU 211.97 sec
2021-02-08 17:00:48,098 Stage-3 map = 89%,  reduce = 0%, Cumulative CPU 213.09 sec
2021-02-08 17:00:53,262 Stage-3 map = 96%,  reduce = 0%, Cumulative CPU 220.83 sec
2021-02-08 17:00:55,351 Stage-3 map = 100%,  reduce = 0%, Cumulative CPU 223.2 sec
2021-02-08 17:01:01,587 Stage-3 map = 100%,  reduce = 25%, Cumulative CPU 281.69 sec
2021-02-08 17:01:02,628 Stage-3 map = 100%,  reduce = 50%, Cumulative CPU 345.52 sec
2021-02-08 17:01:03,655 Stage-3 map = 100%,  reduce = 63%, Cumulative CPU 376.68 sec
2021-02-08 17:01:04,692 Stage-3 map = 100%,  reduce = 69%, Cumulative CPU 391.42 sec
2021-02-08 17:01:05,732 Stage-3 map = 100%,  reduce = 88%, Cumulative CPU 441.49 sec
2021-02-08 17:01:06,765 Stage-3 map = 100%,  reduce = 100%, Cumulative CPU 475.2 sec
MapReduce Total cumulative CPU time: 7 minutes 55 seconds 200 msec
Ended Job = job_1612260368307_6163

2021-02-08 17:01:53	Processing rows:	300000	Hashtable size:	299999	Memory usage:	541774872	percentage:	0.284

2021-02-08 17:02:03	Processing rows:	500000	Hashtable size:	499999	Memory usage:	650955648	percentage:	0.3412021-02-08 17:02:14	Processing rows:	700000	Hashtable size:	699999	Memory usage:	947624528	percentage:	0.4962021-02-08 17:02:25	Processing rows:	900000	Hashtable size:	899999	Memory usage:	1054455136	percentage:	0.552
2021-02-08 17:02:43	Processing rows:	1100000	Hashtable size:	1099999	Memory usage:	1398972024	percentage:	0.733
2021-02-08 17:02:53	Processing rows:	1300000	Hashtable size:	1299999	Memory usage:	1436685672	percentage:	0.753
2021-02-08 17:02:59	Processing rows:	1400000	Hashtable size:	1399999	Memory usage:	1461808240	percentage:	0.766
Hive Runtime Error: Map local work exhausted memory
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:271)
	at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:95)
Execution failed with exit status: 3
Obtaining error information

Task failed!
Task ID:
  Stage-12

Logs:

/tmp/hdfs/hive.log
FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
MapReduce Jobs Launched: 
Stage-Stage-3: Map: 6  Reduce: 16   Cumulative CPU: 475.2 sec   HDFS Read: 1022567611 HDFS Write: 1443024061 HDFS EC Read: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 55 seconds 200 msec
The command is: 
hive -e "set mapred.job.name='Create Intermediate Flat Hive Table';
USE default;

DROP TABLE IF EXISTS kylin_intermediate_cube_ho_flight_route_20200810_b8e6474d_03df_68f6_a686_ec9069646ff3;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_cube_ho_flight_route_20200810_b8e6474d_03df_68f6_a686_ec9069646ff3
(
\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_AIR_LINE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_POINTNAME\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_DATE\` date
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_ID\` bigint
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ROUTEINDEX\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DEPARTURE_4_CODE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ARRIVAL_4_CODE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_NUMBER\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_TYPE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_CAPTAIN\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DISPATCH\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ENGINES1\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_TYPE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_PROPERTY\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_STATUS\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLG_STATE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAX_DEPART_WEIGHT\` int
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_SEAT_NUM\` int
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_NUMBER\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACQUIS_DATE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DISTANCE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLAN_DEPARTURE_TIME\` bigint
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACTUAL_DEPARTURE_TIME\` bigint
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_ARRIVAL_TIME\` bigint
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACTUAL_ARRIVAL_TIME\` bigint
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_CALIBRATEDAIRSPEED\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ELAPSEDTIMEMIN\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FUELCONSUMPTION\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FUELFLOW\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_GROUNDSPEED\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_HIGHLAYER\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_LATITUDE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_LONGITUDE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MACH\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAGNETICHEADING\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MINIMUMSAFEALTITUDE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_RESIDUALAIRDISTANCE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_RESIDUALOILCONTENT\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TEMPERATURE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TEMPERATUREDEVIATION\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_THEREMAININGDISTANCE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TIMEREMAINING\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TRUEHEADING\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDDIS\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDFUEL\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDTIME\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_VACUUMSPEED\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDCOMMPONENT\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDDIRECTION\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDSPEED\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_STATUS\` int
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_ALT\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_HEADING\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_GS\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_STATICAIRTEMP\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WINDIR\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WINSPD\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TAS\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WITH_WIND\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TRUELATITUDE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TRUELONGITUDE\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TOTALAIRTEMP\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_FF1\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_MACH\` double
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_COUNTRYCODE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FIRNAME\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_OVERLAP\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ROUTECODE\` string
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ORGNUM\` string
)
STORED AS SEQUENCEFILE
LOCATION 'hdfs://Aero/kylin/kylin_metadata/kylin-30586bda-a726-aac4-b4c7-178f7948266e/kylin_intermediate_cube_ho_flight_route_20200810_b8e6474d_03df_68f6_a686_ec9069646ff3';
ALTER TABLE kylin_intermediate_cube_ho_flight_route_20200810_b8e6474d_03df_68f6_a686_ec9069646ff3 SET TBLPROPERTIES('auto.purge'='true');
INSERT OVERWRITE TABLE \`kylin_intermediate_cube_ho_flight_route_20200810_b8e6474d_03df_68f6_a686_ec9069646ff3\` SELECT
\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`AIR_LINE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_AIR_LINE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`POINTNAME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_POINTNAME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_DATE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_DATE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_ID\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_ID\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ROUTEINDEX\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ROUTEINDEX\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`DEPARTURE_4_CODE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DEPARTURE_4_CODE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ARRIVAL_4_CODE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ARRIVAL_4_CODE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_NUMBER\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_NUMBER\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`PLANE_TYPE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_TYPE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`CAPTAIN\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_CAPTAIN\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`DISPATCH\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DISPATCH\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ENGINES1\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ENGINES1\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_TYPE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_TYPE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_PROPERTY\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_PROPERTY\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_STATUS\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLIGHT_STATUS\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLG_STATE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FLG_STATE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAX_DEPART_WEIGHT\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAX_DEPART_WEIGHT\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`SEAT_NUM\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_SEAT_NUM\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`PLANE_NUMBER\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_NUMBER\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ACQUIS_DATE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACQUIS_DATE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`DISTANCE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_DISTANCE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`PLAN_DEPARTURE_TIME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLAN_DEPARTURE_TIME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ACTUAL_DEPARTURE_TIME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACTUAL_DEPARTURE_TIME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`PLANE_ARRIVAL_TIME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_PLANE_ARRIVAL_TIME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ACTUAL_ARRIVAL_TIME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ACTUAL_ARRIVAL_TIME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`CALIBRATEDAIRSPEED\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_CALIBRATEDAIRSPEED\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ELAPSEDTIMEMIN\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ELAPSEDTIMEMIN\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FUELCONSUMPTION\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FUELCONSUMPTION\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FUELFLOW\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FUELFLOW\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`GROUNDSPEED\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_GROUNDSPEED\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`HIGHLAYER\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_HIGHLAYER\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`LATITUDE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_LATITUDE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`LONGITUDE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_LONGITUDE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MACH\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MACH\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAGNETICHEADING\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAGNETICHEADING\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MINIMUMSAFEALTITUDE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MINIMUMSAFEALTITUDE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`RESIDUALAIRDISTANCE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_RESIDUALAIRDISTANCE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`RESIDUALOILCONTENT\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_RESIDUALOILCONTENT\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`TEMPERATURE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TEMPERATURE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`TEMPERATUREDEVIATION\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TEMPERATUREDEVIATION\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`THEREMAININGDISTANCE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_THEREMAININGDISTANCE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`TIMEREMAINING\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TIMEREMAINING\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`TRUEHEADING\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_TRUEHEADING\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`USEDDIS\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDDIS\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`USEDFUEL\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDFUEL\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`USEDTIME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_USEDTIME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`VACUUMSPEED\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_VACUUMSPEED\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`WINDCOMMPONENT\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDCOMMPONENT\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`WINDDIRECTION\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDDIRECTION\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`WINDSPEED\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_WINDSPEED\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`STATUS\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_STATUS\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_ALT\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_ALT\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_HEADING\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_HEADING\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_GS\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_GS\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_STATICAIRTEMP\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_STATICAIRTEMP\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_WINDIR\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WINDIR\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_WINSPD\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WINSPD\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_TAS\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TAS\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_WITH_WIND\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_WITH_WIND\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_TRUELATITUDE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TRUELATITUDE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_TRUELONGITUDE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TRUELONGITUDE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_TOTALAIRTEMP\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_TOTALAIRTEMP\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_FF1\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_FF1\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`MAP_MACH\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_MAP_MACH\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`COUNTRYCODE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_COUNTRYCODE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FIRNAME\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_FIRNAME\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`OVERLAP\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_OVERLAP\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ROUTECODE\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ROUTECODE\`
,\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`ORGNUM\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810_ORGNUM\`
 FROM \`AERO_HO\`.\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\` as \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`
WHERE 1=1 AND (\`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_DATE\` >= '2021-02-02' AND \`VIEW_CUBE_HO_FLIGHT_ROUTE_20200810\`.\`FLIGHT_DATE\` < '2021-02-08')
;

" --hiveconf hive.merge.mapredfiles=false --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2 --hiveconf hive.exec.compress.output=true --hiveconf hive.auto.convert.join.noconditionaltask=true --hiveconf mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false --hiveconf hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf hive.stats.autogather=true
	at org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:96)
	at org.apache.kylin.source.hive.CreateFlatHiveTableStep.createFlatHiveTable(CreateFlatHiveTableStep.java:59)
	at org.apache.kylin.source.hive.CreateFlatHiveTableStep.doWork(CreateFlatHiveTableStep.java:97)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

二、解决

1 思路

看日志应该是内存不够导致的，加一些内存试试看

2 操作

1 增加hive 进程内存

打开CDH页面 ——> 点击hive ——> 点击配置 ——> 找到客户端java堆大小
java堆大小从4G修改为8G
在这里插入图片描述
2 增加yarn map和reduce内存

打开CDH页面 ——> 点击yarn ——> 点击配置 ——> 点击资源管理 ——> 找到map和reduce内存，从8G修改为16G
在这里插入图片描述
3 增加kylin内存

修改conf/setenv.sh和conf/setenv-tool.sh

vim setenv.sh

注释的内容是修改前的，修改为未注释的参数

#export KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
export KYLIN_JVM_SETTINGS="-Xms32g -Xmx32g -XX:MaxPermSize=1024m -XX:NewSize=6g -XX:MaxNewSize=6g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

vim setenv-tool.sh

注释的内容是修改前的，修改为未注释的参数

#export KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
export KYLIN_JVM_SETTINGS="-Xms32g -Xmx32g -XX:MaxPermSize=1024m -XX:NewSize=6g -XX:MaxNewSize=6g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"

加好内存后，再次构建，已经可以成功构建了。
在这里插入图片描述

反思：

修改的有点粗，未定位到具体组件的具体参数。

5.Apache Kylin 构建第一步报错 Container complete event for unknown container

一、问题

二、解决

1 思路

2 操作

反思：

猜你喜欢

5.Apache Kylin 构建 第一步报错 Container complete event for unknown container

一、问题

二、解决

1 思路

2 操作

反思：

猜你喜欢

5.Apache Kylin 构建第一步报错 Container complete event for unknown container