Date Posted:
Product: TIBCO Spotfire®
Product: TIBCO Spotfire®
Problem:
MapReduce job fails due to memory setting issues
Solution:
MapReduce job fails with the following error:
================================================================================================
ERROR [pool-610-thread-2] com.alpine.datamining.workflow.AnalyticNodeThread.run(AnalyticNodeThread.java:82) - Unable to recreate exception from backed error: Container [pid=231883,containerID=container_e50_1564730610291_80507_01_000003] is running beyond physical memory limits. Current usage: 1.1 GB of 1 GB physical memory used; 2.9 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_e50_1564730610291_80507_01_000003
===============================================================================================
Check the following parameters on Hadoop Cluster and update the data source configuration with the appropriate parameters and their values.
-
mapreduce.map.memory.mb - mapreduce.reduce.memory.mb
-
mapreduce.map.java.opts