Some users experienced delays and intermittent errors when generating assets with our Flux-based models (LoRAs). The issue has been fully resolved and systems are operating normally.

What Happened:

A component responsible for provisioning new GPU machines (needed to process generation jobs) encountered an unexpected failure during GPU startup while retrieving model files. As a result, new GPUs took longer than normal to become available, which increased queue times and caused occasional timeouts.

Impact:

Some generation jobs experienced slower-than-usual start times.
A subset of requests may have failed intermittently during the affected window.

Resolution:

We identified the failure and applied a fix to restore normal GPU provisioning. We also temporarily expanded capacity to clear the queued workload. The system has remained stable since.