Image Artist: Index Migration Error Handling in v1.4

During the enforced migration from a file-based index to a PostgreSQL-backed index in Image Artist 1.4, certain datasets may trigger errors due to legacy inconsistencies in Image Artist 1.3. This document outlines two common error types and their root causes.

1. Dataset Indexing Failure

Example Error:

Failed dataset: Got DatastoreError when recreating index for Dataset 'edb3040e-49a3-11ef-8016-02420a000233' under Plate edb3040e-49a3-11ef-8016-02420a000233: (500, 'Internal Server Error', '"Error when saving dataset edb3040e-49a3-11ef-8016-02420a000233 to DB: insert or update on table "datastore_dataset" violates foreign key constraint')

Cause:
The dataset in Image Artist 1.3 contains corrupt or incomplete metadata, most likely due to an interrupted import job. This has resulted in an unusable dataset. Because of this corruption, the migration script correctly blocks the dataset from being migrated to version 1.4. The dataset would not have been usable in version 1.3, or would have triggered an error when accessed.

2. Annotation Metadata Indexing Failure

Example Error:
Failed metadata for Dataset 0bf5320e-6cb0-11f0-9c1f-02420a000230: ["Got DatastoreValidationError when storing WELLRESULT metadata called 10e8d0ee-6cf2-11f0-8050-02420a00020b for Dataset 0bf5320e-6cb0-11f0-9c1f-02420a000230 under Screen 'Screen_001' and Plate 'Plate_001': Refuse to create duplicate metadata. Details: dataset '0bf5320e-6cb0-11f0-9c1f-02420a000230', annotation_location.storage='original', annotation_location.path='r14c21f04p01-t01.xml']

Cause:
The Image Artist 1.3 index contains multiple metadata entries referencing the same file, which likely result from a failed import or batch analysis job. These duplicate entries are unnecessary in version 1.4 and are not migrated.

Resolution & Impact:

• These warnings do not affect any valid or functioning datasets.
• The errors are safe to ignore and indicate that the migration script is correctly filtering out corrupt or redundant data.
• No user action is required unless the affected datasets need to be manually reviewed or re-imported.