Memory usage in Spotfire 5.x and 6.x.

Memory usage in Spotfire 5.x and 6.x.

In Spotfire 5.0, the data engine is replaced, compared to previous versions. The data engine is responsible for both storing data and performing computations. When comparing Spotfire 4.x and 5.x, in some scenarios, 5.x may use more memory and behave differently during runtime. Initial RAM usage

============

Spotfire 5.x can use 0% to 50% more RAM after loading analysis files than 4.x does. The exact amount depends on the complexity of the analysis files and the particular features they use. Spotfire 4.x and 5.x are designed to handle similar analysis scenarios, but 5.x should scale to larger data sets more readily. Spotfire 5.x was not designed to use less memory with the same data sets that 4.x can handle. Opening a file saved with an old version (4.x) may include a transformation step, in which a data column is converted to the new 5.x format in memory. Re-saving such an analysis in 5.x allows this conversion step to be skipped when opening the analysis. This can save memory and time, especially for embedded data sets with many rows of data.

RAM usage during runtime

==================

Spotfire 5.x tries to use more RAM to improve performance by caching calculation results. Looking at numbers in Windows Task Manager, the memory used by this cache will be included in the "working set" or "private memory" columns. This cache memory is released when necessary. You can monitor the Web Player Server's cache memory on the Web Player Server Diagnostics page, using the "Web Server" tab's "Memory Used For Caching Calculations" entry. (To see this, you need to be on at least hotfix TS 5.0.1 HF-009 or TS 5.5.0 HF-002.) Although Spotfire releases the cached calculation memory when necessary, such as in low memory situations or if other applications suddenly request lots of memory, it is still recommended to have a dedicated machine for the Spotfire Web Player server. Spotfire 5.x improves performance when paging out unused or rarely used data blocks to disk, by compressing blocks before they are written out. This compression/decompression task uses CPU. Spotfire 4.x did not perform this compression, so CPU usage was lower during paging, but instead, disk I/O took more time.

What features use memory?

=================

The following tasks are known to affect memory consumption:

• Scheduled Updates, which build caches in working memory on the Web Player server.

• Loading of Spotfire analyses (DXP files) that were last saved using the older (4.X) file format.

• Bookmark use.

• User interaction.

Concerning 32-bit vs 64-bit operating systems

===============================

Running Spotfire Professional on a 32-bit operating system will limit the amount of RAM available for use by the Spotfire process, to approximately 1.5 GB (or less). This is a per-process limit that is imposed by the operating system. Spotfire 5.x has been performance optimized for modern 64-bit systems. Upgrading to a 64-bit operating system is recommended, not only to avoid the per-process RAM limit that is inherent in 32-bit operating systems but also to take advantage of other general performance optimizations.

Recommendations for RAM when using Spotfire 5.x

================================

If your 4.x Web Player normally uses 70% or more of the server's available RAM, then you should consider installing more RAM before you upgrade to version 5.x or a later version.

Importance of Applying Hotfixes

=====================

TIBCO continuously strives to optimize memory handling and improve sub-optimal cases. It is important for all customers to apply the latest hotfixes.