Details:
The Spotfire Web Player allows administrators to create Scheduled Jobs which will load specified analyses into memory on the Node Manager server. Spotfire Web Player clients then access the loaded version as needed, to reduce the load time observed by end users. When Spotfire's monitoring software senses that resources are very low then it triggers Web Player (WP) instances to 'recycle' which is to say all cached analyses are dropped and need to get rebuilt.
Issue:
When investigating causes of these recycle events it is normally possible to discover the root cause by reviewing log files and performance counter data of Spotfire. Log file entries indicate the type of resource that is low which causes Spotfire to designate a server to be in an 'Exhausted' state. However, in one case recycle events correlated to OutOfMemory entries in the Web Player (debug) log files during a time that Windows server monitoring showed 80 to 83% of RAM used on the Node Manager server. The remaining portion of memory was unused and seemingly there was no reason that WP instances should be recycling.
Cause:
It was determined that recycling of the WP instances was caused by a reaching a .NET Framework limitation called Sync Blocks, at a count of 65 million. This count loosely correlates to the count of DocumentNodes used by Spotfire. Spotfire's performance counters do not track the current count of Sync Blocks, but they do track the current count of DocumentNodes.
Solution/Recommendation:
If your Web Player instance recycles correlate in time to OutOfMemory entries in log files, but monitoring indicates that your Node Manager server still has RAM available, then use Spotfire Server's performance counter data to track is the count of DocumentNodes. If you see spikes that correlate to your recycle events, then you can try increasing the number of Web Player instances. The rationale is that each WP instance has its own .NET process and the Sync Block limit is set per process.
Comments
0 comments
Article is closed for comments.