虚拟内存超出运行
报错信息:
Container [pid=30866,containerID=container_1600927953860_0003_02_000001] is running beyond virtual memory limits. Current usage: 117.3 MB of 1 GB physical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing container.
yarn Container默认虚拟内存设置大小为目前物理机器得2.1倍2.1G,而目前yarn Container占用虚拟内存大小为2.2G超过2.1G。
解决办法配置yarn-site.xml
方法一:增大默认大小:
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>5</value>
</property>
方法二:关闭程序yarn得虚拟内存
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>