A complex Regular Expression function returns NULL or causes the Spotfire client to crash.

A complex Regular Expression function returns NULL or causes the Spotfire client to crash.

When using complex regular expressions in TIBCO Spotfire, either the expression will return NULL even if there is a valid match, or the installed client will crash with a "System.Runtime.InteropServices.SEHException: External component has thrown an exception" error. The regular expression might be used in either of the RXReplace() or RXExtract() (version 7.6 and later only) functions, or the ~= operator. The total complexity is a combination of both the string to be searched and the regular expression, so both are factors in reaching the limit.

Symptoms:
In the following versions, the regular expression function will return NULL if it is too complex, even if there is a valid match:

• TS 7.5.0 HF-002 or later
• TS 7.0.1 HF-013 or later

In the versions below, the client will crash with the following errors if the regular expression is too complex:

• TS 7.5.0 HF-001 or earlier
• TS 7.0.1 HF-012 or earlier
• TS 6.5 (all versions)

The following error is seen when the client crashes:

System.InvalidOperationException: SafeExecute failed. ---> System.Runtime.InteropServices.SEHException: External component has thrown an exception.
   at Spotfire.Dxp.Data.Cxx.Internal.NativeMethods64.NativeMethods.QueryExecutorCreateView_(IntPtr session, String expression, Boolean createDot, Boolean importMode, IntPtr callbacks, IntPtr cancelHook, IntPtr& resultingExecutor, IntPtr& resultView)
   at Spotfire.Dxp.Data.Cxx.Internal.NativeMethods64.<>c__DisplayClass73.<QueryExecutorCreateView>b__72()
   at Spotfire.Dxp.Data.Cxx.Internal.Helper.SafeExecute[T](Func`1 nativeDelegate)
   ...

The following events are seen in the Windows Application Event log:

Log Name:      Application
Source:        Application Error
Date:          3/15/2016 10:54:43 AM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myComputer
Description:
Faulting application name: Spotfire.Dxp.exe, version: 18.10.9315.4066, time stamp: 0x561f701a
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17278, time stamp: 0x53eebf2e
Exception code: 0xe0434352
Fault offset: 0x000000000000606c
Faulting process id: 0x16bc
Faulting application start time: 0x01d17ec97f2186be
Faulting application path: C:\Spotfire\Spotfire Professional Client\Spotfire.Dxp.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: d98a5a80-eabd-11e5-8269-083e8ee06634
Faulting package full name:
Faulting package-relative application ID:

 

 

Log Name:      Application

Source:        .NET Runtime
Date:          3/15/2016 10:54:43 AM
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myComputer
Description:
Application: Spotfire.Dxp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.Reflection.TargetInvocationException
Stack:
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at Starter.Program.ProgramLoaderWrapper.Execute(String fullLoaderAssemblyName, Dictionary`2 loaderState, String[] args, Object splash, Version starterVersion, IList`1 moduleFolderList, String upgradePath)
   at Starter.Program+ProgramLoaderWrapper.Execute(System.String, System.Collections.Generic.Dictionary`2<System.String,System.Object>, System.String[], System.Object, System.Version, System.Collections.Generic.IList`1<System.String>, System.String)
   at Starter.Program.Run(Spotfire.Dxp.Starter.StarterSplashWrapper, System.Collections.Generic.Dictionary`2<System.String,System.Object>)
   at Starter.Program.ExecuteRunLoopWithBootstrapFlag(System.String[], Boolean)
   at Starter.Program.Main(System.String[])

Cause:
There is a limit in the complexity of regular expressions allowed in TIBCO Spotfire due to limits in the regular expression libraries. The total complexity is a combination of both the string to be searched and the regular expression, so both are factors in reaching the limit. When the number of internal states in the matching exceed that "max_state_count" limit of 100000000 (which is not customizable) an exception is thrown in the regular expression library. 

This applies to the following functions:

• RXReplace()
• RXExtract() (version 7.6 and later only)
• ~=

To prevent the application from crashing, apply the hotfix corresponding to your TIBCO Spotfire version:

• TS 7.5.0 HF-002 or later
• TS 7.0.1 HF-013 or later

To avoid reaching the regular expression complexity limit, you must simplify the input string, and/or simplify the regular expression being used. This can be done by reducing the length of the input string, or by modifying the regular expression to reduce the number of matches. TIBCO Spotfire Help:

• https://docs.tibco.com/pub/sfire-cloud-p/3.0.0/doc/html/sfire-cloud-p/GUID-44747C22-78D0-48B6-A1E2-3A702DE7FB1E.html