Some ways to avoid buildup of large *.sbdf files in the Spotfire Statistics Services server's artifacts folder.

Some ways to avoid buildup of large *.sbdf files in the Spotfire Statistics Services server's artifacts folder.

This article outlines some ways to avoid buildup of large *.sbdf files in the Spotfire Statistics Services server's artifacts folder.
  (1) These temporary Spotfire Binary Data Format (SBDF) files are a necessary part of sending Spotfire data function input and output data to and from TIBCO Spotfire Statistics Services (TSSS).  Large data sets require large SBDF files.  The temporary files are normally deleted as soon as TSSS receives confirmation from the Spotfire client (Analyst desktop, Web Player or Automation Services) that the client has received the output and the TSSS job can be closed.  

Unfortunately, the SBDF files can be left behind as "orphans" in the artifacts folder, if TSSS does not receive this confirmation from the Spotfire client.  This can happen if a Spotfire desktop client session is closed before the data function it launched has finished, for example.

You can remove such "orphan" SBDF files from the artifacts folder manually.  You can also set up a cron job to clean out SBDF files that are older than a certain age.  (An existing enhancement request asks for a built-in TSSS feature that will do this automatically.) 

You can reduce the size of SBDF files that a data function writes into the artifacts folder by reducing the size of the data functions' input and output data sets.  For example, any columns or rows that are not needed for the calculation to be done in the TSSS job can be filtered out of the data function's input parameters, using Spotfire expressions, filters, and/or marking schemes.



(2) A new SBDF file is created each time a web client accesses the dashboard, if the output for the dashboard has not been cached beforehand using the Scheduled Updates feature in Spotfire Web Player.

If the data function output is always the same for all Web Player users, you may find it helpful to use Scheduled Updates to run the analysis that contains these data functions periodically (once a day, for example). 

That way, the temporary SBDF files will only be created when the Scheduled Updates job is run, and the Web Player users will not need to wait for the dashboard's data functions to finish running.


Notes:

  TIBCO Spotfire Statistics Services (TSSS) 7.11 is the supported version for use with Spotfire 7.11, 7.12, 7.13 and 7.14.

  TSSS 7.5 is the supported version for use with Spotfire 7.10, 7.9, 7.8, 7.7, 7.6, and 7.5.
  The following knowledge base articles also provide information on the artifacts folder in Statistics Services:

  https://support.tibco.com/s/article/Artifacts-folder-in-Statistics-Services-directory-taking-up-a-lot-of-space

  https://support.tibco.com/s/article/Performance-issue-when-executing-data-functions-against-TIBCO-Spotfire-Statistics-Services-server-due-to-many-old-SBDF-files-present-inside-data-artifacts-spotfire-folder


The following knowledge base article provides additional information on TSSS 7.5 and TSSS 7.11:

  https://support.tibco.com/s/article/What-is-the-mechanism-for-upgrading-from-one-TSSS-version-to-another