Katalyst:  Experiment save hangs and experiments become locked with 502 Proxy Error

Description

• When saving or exporting certain Katalyst experiments, users experience:
  • Very long export times, followed by a stack trace in the UI.
  • Experiments remaining locked in export_start state.
• The error stack trace includes a 502 Proxy Error similar to:
• com.google.gwt.user.client.rpc.StatusCodeException: 502
• <title>502 Proxy Error</title>

Reason: Error reading from remote server

• Multiple experiments may be affected (e.g., several KAT IDs).

Solution

• Identify the layer producing the 502:
  • Typically, a reverse proxy or load balancer in front of Katalyst.
  • Check its logs around the time exports occur.
• Likely root causes:
  • Long‑running export operations combined with insufficient proxy timeout settings.
  • Backend performance issues causing slow responses from the Katalyst application server.
• Remediation steps:
  1. Proxy configuration
     • Increase request and backend timeouts so that longer exports do not result in 502 errors.
     • Confirm that the proxy correctly routes long‑running requests to the Katalyst server.
  2. Katalyst/export performance
     • Review Katalyst server logs and metrics during export.
     • Identify if particular export types or very large experiments consistently cause timeouts.
     • Optimize database queries, indexing, or export logic where necessary (with Technical Support help).
  3. Unlocking affected experiments
     • Use admin tools or scripts to transition experiments out of the export_start state to a usable state (following internal procedures).
• After adjustments, test:
  • Export of problematic experiments in a controlled environment.
  • Monitor for stability and absence of new 502 errors.