Image Artist: "too many open files" messages when collecting server logs

Problem Description

When collecting Image Artist server log files, you may encounter a "too many open files" error that can cause the log collection script to fail. This occurs when the system reaches its maximum limit of open file descriptors.

Example Error Message

*** Collecting logs...

error sending request Get http://127.0.0.1:3100/loki/api/v1/query_range?direction=FORWARD&end=1708528652929831946&limit=100&query=%7Bswarm_stack%3D%22SImA%22%7D&start=1707311613692511039: dial tcp 127.0.0.1:3100: socket: too many open files

Query failed: Run out of attempts while querying the server

Solution

The "too many open files" error indicates that the Linux nofile (open file descriptors) limit is set too low.

For optimal Image Artist log collection, set an appropriate nofile limit on the Image Artist host, as described here: Image Artist: Recommended nofile (open file descriptor) limits

If you cannot modify system limits or need an immediate workaround, reduce the collection period to lower the resource requirements:

get_logs.sh collect --batch 10000 --from $(date -Is -d "2 days ago")

Reduce the batch size (--batch parameter) to process fewer logs at once.