Image Artist: Batch Analysis Job Accounting

Overview

Slurm job accounting is an optional feature in Image Artist that tracks the actual CPU and memory resources consumed by each individual well job during Batch Analysis. It gives administrators visibility into how efficiently the cluster is being utilised, making it easier to tune Slurm configuration and improve Batch Analysis performance.

The feature is disabled by default and must be explicitly enabled by an administrator.

Version Availability

What It Does

When enabled, Slurm accounting records the CPU and memory usage for each well job (a single job corresponding to one well in a Batch Analysis run). After a job completes, this information is:

• Displayed on the Job Status page under the heading Usage: Resource Efficiency of Single Well Job, showing:

  • Memory: utilized X GB of configured Y GB

  • CPU: used XX% of configured Y CPU(s)

• Saved to a log file (sacct_output.log) alongside the job output, allowing administrators to review historical usage data.

This data helps administrators identify over- or under-provisioned jobs and fine-tune Slurm resource allocations accordingly.

Enabling Slurm Accounting

Local Slurm Cluster

1. Open the .env file in the Image Artist installation root directory.

2. Add the following line:

   SLURM_ACCOUNTING_ON=1

3. Restart the Image Artist stack:

   ./undeploy.sh && ./deploy.sh

Once enabled, accounting data for each well job is written to the Job Status page, as well as to the sacct_output.log file within the job output directory. sacct_output.log is included as part of the Batch Analysis job log collection.