How to permanently delete rows from data tables created by data functions

Date Posted: 2019-05-16
Product: TIBCO Spotfire®

Problem:

Solution:

The delete rows function in TIBCO Spotfire Analyst, available via right clicking in a visualization > Marked Rows > Delete (see Marking in Visualizations reference) is only available for the current session when the data table is linked to source, as it is does not keep record of which rows were deleted (like is shown for deleted columns). The same can be seen in Data Table Properties > Source Information where the "Number of rows Deleted from the data table" would be unknown. See Remove or delete rows in TIBCO Spotfire Wiki article for more details.

Suppose an example where a data table is populated by a data function and you are appending rows to the existing data table, with a data function like:

optable = rbind(inptable,append)

where:

optable = final output data table
inptable = the same optable
append = the data frame object containing the rows that needs to be appended

So the rbind function will insert rows from the "append" data frame to the "inptable" and output the result to "optable".

This forms a cycle as rows are getting appended to the same table and so the "Refresh table automatically" option will be unchecked in Edit Parameters. Trying to check that option, will throw the cyclic dependency error, like:
User-added image

For example, if you have 20 rows in inptable and you append 30 rows, then optable would now have 50 rows.

If you delete 10 rows from optable (by marking 10 rows in the data table > Right click visualization > Visualization Properties > Marked Rows > Delete) so that optable has 40 rows, then save the .dxp, and reopen it you observe that optable again has 50 rows instead of 40 rows. This is because the current data in the data table (calculated through data function) is stored in the analysis and the data table is set to linked to source and not embedded (see Embedded or Linked Data? reference. This can be checked from the Data Table properties). When the data table is embedded, it will never get refreshed from the source so the deleted rows will not return (unless the user explicitly tells it refresh), but when the data table is linked to source it gets refreshed from the data already stored in analysis so the deleted rows will be added again. The delete rows operation will have no impact after the initial session if the data table is linked to source as it will always get refreshed from the source upon next access.

You can use any of the following workarounds in order to delete the rows permanently:

1. Due to the cyclic dependency, the data function will not refresh automatically so it will require a manual trigger to refresh the data table. In this case, you can keep the data table (calculated from the data function like seen above) always embedded. This way you can delete rows as required and then .dxp can be saved and accessed later.

or

2. If you do not intend to keep the data table embedded, then you will need to refresh data function once after you delete the rows from the data table because the data function will take the new data set, which excludes the deleted rows, and append the new rows to that. Then save the .dxp and re-open it and the deleted rows will not reappear. Doc: Marking in Visualizations