Signals Inventa: Jobs get submitted to Signals Data Factory, but not processed

Problem:

It is possible to see a situation where Signals Inventa jobs get submitted to Signals Data Factory (SDF), but not get processed. The job status doesn't change from "Received" to "Queued" or "Started", for example:

Resolution: The following steps can be performed when this issue happens. This procedure will update the status of current job above in "Received" state, so that job processing can continue.

1. On SDF Admin Console machine, identify the SDF pod that has MongoDB by running "kubectl get pods" command:

2. Run “kubectl exec -it sdf-cs-signals-mongo-5c5474669b-qw798 -- /bin/bash” command. This will log you into the MongoDB container (found in step 1 above).

3. Connect to MongoDB as Root user using command "mongo -u root":

4. Run “use signals--sdf" command. This will switch to the signals--sdf database:

5. Run “db.jobs.find({"status": "Received"});” command. This will retrieve all the jobs that have status “Received”:

6. Run “db.getCollection('jobs').findOneAndUpdate({_id: new ObjectId("6491bf94592ef60010b4ce83")}, {$set: {status: "Failed on Cluster"}})” command. This will update the status of the Job. Note: The required ObjectID of the job is displayed in the Step 5 results above.

7. The jobs will change the status immediately and the subsequent jobs will continue their process.