Spotfire Server Cluster instability due to multiple services (Web Players). WARN: Consider increasing the nodemanager.core-pool-size setting

Spotfire Server Cluster instability due to multiple services (Web Players). WARN: Consider increasing the nodemanager.core-pool-size setting

There can be scenarios/cases where there are multiple Spotfire Servers in a cluster and multiple Node Manager Servers (running multiple Web Player) services. Where you may encounter cluster failures and the Spotfire Server nodes may fail.

The Spotfire Server (server.log) shows the below:
--------------------------------------------------------------
WARN 2019-06-17T06:04:13,884-0300 [monitoring@SPOTFIRESYSTEM, #52307, #19748547] nodemanager.executor.ExecutorServiceFactory: The pool size 75, max 100 cannot add more tasks. 100 active tasks. Consider increasing the nodemanager.core-pool-size setting.
--------------------------------------------------------------- - export the active Spotfire server configuration. Using "export-config" command.
- open the exported "configuration.xml" in edit mode.
- find the below tag:
------------------------------------
<nodemanager>
<capacity>xx</capacity>
<worker-threads>xx</worker-threads>
</nodemanager>
------------------------------------

- add the below lines. One may tweak the values that best suits the environment.
<core-pool-size>250</core-pool-size>
<maximum-pool-size>250</maximum-pool-size>

- ensure that the "Node Manager" setting in the Spotfire Server configuration looks something similar as below:
--------------------------
<nodemanager>
<capacity>xx</capacity>
<worker-threads>xx</worker-threads>
<core-pool-size>250</core-pool-size>
<maximum-pool-size>250</maximum-pool-size>
</nodemanager>
--------------------------

- save the file and import the configuration. Using "import-config" command.
- restart the Spotfire server for the changes to take effect.