| Product | Version / Component |
| Signals Data Factory / Signals Inventa | Migration from SDF 1.0 (Legacy) to SDF 2.0 |
Keywords:
Signals Data Factory, SDF 2.0, SDF migration, Signals Inventa, orphan batches, ambiguous batches, parent measurement type, child measurement type, Batch ID, Compound ID, selectedDataset, dataset ID, LoadAll, publication warnings
Introduction: When preparing a Signals Data Factory (SDF) project for migration from legacy SDF 1.0 to SDF 2.0, administrators and project owners must ensure that all batch and compound relationships are consistent. In projects containing parent and child Measurement Types (MTs), you may encounter scenarios where batches trigger both Orphan Batch and Ambiguous Batch warnings simultaneously. This guide explains the data model requirements in SDF 2.0, root causes of overlapping warnings, and step-by-step instructions on how to locate and resolve them.
What You May Notice:
- During pre-migration validation, the project displays a count of Orphan Batches and/or Ambiguous Batches that does not decrease even after uploading correction files.
- The same Batch ID appears under both orphan batch and ambiguous batch warning lists.
- A circular blocker occurs: following the standard recommendation to "clear orphan batches first before fixing ambiguous batches" stalls because the batch relationship cannot be validated without a compound mapping.
- In projects with parent/child Measurement Types, child datasets contain only Batch IDs without a Compound ID column, while parent datasets contain Compound IDs with missing or duplicate associations.
What Causes It:
- SDF 2.0 Relationship Architecture: Unlike legacy pipelines that may infer compound relationships across joined tables, SDF 2.0 requires every measurement dataset to possess a direct compound association. If a child MT dataset does not define a Compound ID column, its batches are flagged as orphans.
- The "One Parent per Batch" Rule: Across an entire SDF project, a single Batch ID (e.g.,
B001) can have exactly one parent Compound ID (e.g.,C001). IfB001is mapped toC001in one row or file, but toC002in another, it triggers an Ambiguous Batch warning. - Empty Compound ID Mapping: If a dataset has the Compound ID column mapped, but certain rows have empty/blank cells, Data Factory evaluates the parent as
<EMPTY>-B001. If another file or an orphan correction file mapsC001-B001, the system registers two conflicting parents (<EMPTY>andC001), creating an ambiguous batch conflict. - Replicates Are Allowed: Multiple rows having identical compound and batch combinations (e.g., replicates of
C01-B01) are fully supported within or across datasets. The conflict only arises when the parent compound changes for the same batch.
Resolution Workflow:
- Identify Affected Datasets & Rows: Revvity Signals Support and Engineering can run diagnostic scripts to extract detailed CSV reports listing every affected dataset, internal dataset identifier (
selectedDataset), row index, Batch ID, and Compound ID. - Update Child Datasets: Add explicit Compound ID and Batch ID columns directly to child MT source files.
- Standardize Parent Datasets: Eliminate empty, missing, or contradictory Compound ID values in parent source files.
- Perform a Full Reload (
LoadAll): Re-execute a full load so legacy SDF synchronizes and registers the updated source relationships.
Step-by-Step Instructions:
- Request Diagnostic Reports from Support (If Needed)
- If the UI warnings do not indicate which specific files or rows hold the conflict, open a case with Revvity Signals Support.
- Provide your tenant URL and Project Name/ID. Support will extract:
[tenant]-orphan-batches-per-files.csv[tenant]-ambiguous-batches-per-files.csv
- Fix Child Measurement Type (MT) Datasets
- Open the source spreadsheet/file for any child MT dataset identified in the report.
- Verify that both Compound ID and Batch ID columns exist.
- If the Compound ID column is missing, add it and populate each row with the appropriate parent Compound ID value.
- Resolve Parent Ambiguous and Empty Mappings
- Review the datasets listed in the Ambiguous Batches report.
- Ensure each unique Batch ID maps to exactly one Compound ID across all files.
- If a row has a Batch ID but no Compound ID, either supply the valid Compound ID or unmap the column if compound-level association is not applicable. Avoid leaving mapped cells blank.
- Republish and Run
LoadAll- Upload and republish the updated datasets to your project.
- In legacy SDF, initiate a
LoadAlloperation. This forces the system to discard cached associations and re-evaluate all batch-to-compound relationships. - Confirm in the migration pre-check that the orphan and ambiguous batch counts have dropped to zero.
Common Questions:
Q: Can a single batch appear multiple times in a dataset?
Yes. Multiple replicate measurements for the same compound and batch combination (e.g., multiple rows of C001 / B001) are fully valid and supported.
Q: Why did uploading an "Orphan Batch Update" file create ambiguous batches?
If your source dataset already had the Compound ID column mapped but left blank (<EMPTY>), the system considers <EMPTY> to be Parent 1. When your correction file introduced C001 as Parent 2, a conflict was created. You must populate or unmap the blank cell in the source file itself.
Q: Do child datasets really need Compound IDs if the parent already defines them?
Yes. In SDF 2.0 architecture, measurements are verified directly against compound definitions. Child measurement datasets must explicitly provide their parent Compound ID and Batch ID.
Q: What is selectedDataset or Dataset ID in the diagnostic report?
In Signals Inventa / Data Factory, each published table or spreadsheet is assigned an internal UUID/ID known as selectedDataset. Support diagnostic reports list this ID alongside the dataset display name so you can pinpoint the exact file in your project configuration.
Critical Note: Before initiating migration to SDF 2.0, all orphan and ambiguous batch counts must be resolved to zero. Updating source files to include complete Compound ID mappings and executing a LoadAll is mandatory for the legacy pipeline to reflect changes prior to cutover.
Comments
0 comments
Please sign in to leave a comment.