E-Notebook Data Archive: Scheduling Data Extraction Jobs for Large Datasets

Description

When archiving large volumes of E-Notebook data, the following issues may occur during Data Extraction Jobs.

• The ELN Archive job stops during execution and requires a manual restart.
• Using the Retry button may incorrectly mark the job as Completed, even when some records remain unprocessed.

This article explains how to configure scheduled data extraction jobs with the correct recurrence settings to ensure uninterrupted processing.

Solution

The Extraction Management Console allows administrators to:

• Configure and schedule extraction jobs
• Monitor job progress in real time
• Troubleshoot failed or incomplete jobs

Step-by-Step Configuration

1. Access the Console

Log in to the Extraction Management Console with Administrator credentials.

2. Locate or Create a Job

Click the Job Configuration icon, then click "+" to create a new job.

3. Configure Recurrence

Set the run time and recurrence pattern. For large databases, use the following settings:

Note: These settings allow the job to run continuously each day until all collections are fully processed.

4. Start the new job.

Handling an Active or Misconfigured Job

Important: Once a job has started, its configuration cannot be modified.

If the current job is misconfigured or stuck, follow these steps:

1. Stop the current job (if in progress)
2. Mark it as Inactive
3. Create and start a new job with the correct recurrence settings

Performance Estimates

Use the following estimates to plan job duration based on database size:

Recommendation: Configure No End Date with a Daily recurrence for large datasets to avoid premature job termination.