TIBCO Spotfire Server upgrade fails with error, "Unable to rewrite data sources using DataDirect and NTLM authentication" in case of external library storage.

TIBCO Spotfire Server upgrade fails with error, "Unable to rewrite data sources using DataDirect and NTLM authentication" in case of external library storage.

This article exaplains a workaround that can be used to upgrade to TIBCO Spotfire Server in case the old Spotfire environment is configured with external library storage. 

The upgrade tool fails with following error logs:
----------------------------------------
Unable to upgrade database
com.spotfire.server.tools.upgrade.DatabaseUpgradeException: Unable to rewrite data sources using DataDirect and NTLM authentication
    at com.spotfire.server.tools.upgrade.DatabaseUpgradeUtil.fixDataDirectNTLMDatasources(DatabaseUpgradeUtil.java:134)
    at com.spotfire.server.tools.upgrade.UpgradeManager.doDatabaseUpgrade(UpgradeManager.java:374)
    at com.spotfire.server.tools.upgrade.UpgradeManager.execute(UpgradeManager.java:244)
    ........
Caused by: org.jdom.input.JDOMParseException: Error on line 1: Premature end of file.
    at org.jdom.input.SAXBuilder.build(SAXBuilder.java:533)
    ... 11 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
    at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
    ... 13 more
------------------------------
Note: The error stack trace should have "Caused by: org.jdom.input.JDOMParseException: Error on line 1: Premature end of file." line like the above.

Cause:

All datasource connection details are stored in files in case of external library storage. The upgrade tool does not check the files on external library storage for the data source information during the upgrade, rather it checks the Spotfire Database which leads to failure of the upgrade process.
  The following workaround can be used to upgrade in case of external library storage.

--------------------------------
1) Take backup of Spotfire DB.
2) Take backup of Spotfire library stored in shared file system.
3) Run following SQL statements on the Spotfire Database.

    --------------------------    

    -- create new 'temporary' library type

    insert into LIB_ITEM_TYPES (TYPE_ID, LABEL, LABEL_PREFIX, DISPLAY_NAME, IS_CONTAINER, FILE_SUFFIX, MIME_TYPE) 
        values ('e3fd56ff-07a1-4a09-b05b-1d148f1f6786', 'datasourcetmp', 'spotfire', 'datasourcetmp', 0, null, null);
 
    -- Change all datasources to the temporary data type.

    update lib_items set item_type = 'e3fd56ff-07a1-4a09-b05b-1d148f1f6786' where item_type = 'e3fd5644-07a1-4a09-b05b-1d148f1f6786';
 
    -- Verify that they have changed.

    select * from lib_items where item_type = 'e3fd56ff-07a1-4a09-b05b-1d148f1f6786';

    ------------------------

4) Run upgrade tool (upgradetool.bat).
5) Run following SQL to rollback the temporary changes performed at step 3.

     -------------------------
     -- Move back to the proper datasource type.

     update lib_items set item_type = 'e3fd5644-07a1-4a09-b05b-1d148f1f6786' where item_type = 'e3fd56ff-07a1-4a09-b05b-1d148f1f6786';
 
     -- Show the datasources.

     select * from lib_items where item_type = 'e3fd5644-07a1-4a09-b05b-1d148f1f6786';
 
     -- Show the temporary library type.

     select *  from lib_item_types where type_id = 'e3fd56ff-07a1-4a09-b05b-1d148f1f6786';
 
     -- Remove the temporary library type.

     delete from lib_item_types where type_id = 'e3fd56ff-07a1-4a09-b05b-1d148f1f6786';
 
     -- Verify that there is only one type starting with e3fd56.

     select * from lib_item_types where type_id like 'e3fd56%'
 
     -- Verify contents of sn_version table depending upon to which Spotfire version you upgraded to.
      select * from sn_version;
     ---------------------------