Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

For thousands of catalogue images, use an asynchronous Batch API workflow rather than holding open HTTP requests. Build a JSONL file with one image-generation or image-edit request per line, give every line a deterministic custom_id, upload it, create a batch with its 24-hour completion window, poll the batch status, and download the output and error files when the batch reaches a terminal state.

Track progress at two levels: the API’s job-level status and your own per-image records keyed by custom_id. Download results promptly because OpenAI’s documentation says completed output files are automatically deleted 30 days after batch completion. Retention periods for originals, derivatives, manifests, and logs are business-policy decisions, not values prescribed by the API.

How the workflow works

  1. Create JSONL input. Each line contains a unique custom_id, HTTP method, endpoint URL, and request body.
  2. Upload the file with the official Node.js openai SDK and purpose: "batch".
  3. Create the batch for the image endpoint, using completion_window: "24h".
  4. Persist identifiers. Store the batch ID, input file ID, catalogue job ID, image-version IDs, timestamps, and policy metadata in your database.
  5. Poll status until the batch is completed, failed, expired, or cancelled.
  6. Retrieve output and error files, parse each JSONL line, and update catalogue records by custom_id.

Batch requests are asynchronous and do not support streaming. The OpenAI Batch API FAQ states that results are returned through output files rather than streamed responses, so design a poll-and-retrieve process instead of waiting for an individual HTTP response.

Prepare requests that can be reconciled safely

Use deterministic custom IDs

Make the ID encode the catalogue item and the image revision, for example sku-1842-front-v3. Persist the same value in your job table before submission. Output line order is not guaranteed to match input order, so array position is never a safe way to update a product record.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Keep the JSONL file within service limits

Constraint Documented value Implementation consequence
Requests in one batch 50,000 Split larger catalogues into multiple batches.
Uploaded input file 200 MB maximum Measure the JSONL file itself before upload; this limit is not the combined size of remote images.
Batch creation rate 2,000 batches per hour Queue submissions if an import creates many chunks.
Completion window 24 hours Set alerts and recovery procedures before the window expires.
Pricing 50% lower than synchronous APIs, according to OpenAI’s 2026 Batch API guide Use the discount in cost planning, but do not treat it as a throughput or quality guarantee.

Chunk by both request count and serialized file size, leaving headroom for JSON escaping and metadata. Where an image endpoint permits it, reference an image URL instead of embedding image bytes in every request. The 200 MB limit applies to the uploaded JSONL document; it does not cap the total bytes fetched from remote image URLs by the underlying requests.

Example JSONL line

{"custom_id":"sku-1842-front-v3","method":"POST","url":"/v1/images/generations","body":{"model":"gpt-image-1","prompt":"Clean white-background catalogue image of the supplied product","n":1,"size":"1024x1024"}}

For edits, use the image-edits endpoint and the request shape supported by the model and SDK version you deploy. Validate every line as JSON before upload, reject duplicate IDs, and record the image revision used to construct the prompt or edit.

Submit a batch from Node.js

Install the official SDK and provide an API key through your runtime secret manager:

npm install openai
import OpenAI from "openai");
import fs from "node:fs";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const input = await client.files.create({
  file: fs.createReadStream("catalogue-images.jsonl"),
  purpose: "batch"
});

const batch = await client.batches.create({
  input_file_id: input.id,
  endpoint: "/v1/images/generations",
  completion_window: "24h"
});

console.log({ batchId: batch.id, inputFileId: input.id });

Persist the IDs immediately after each successful call. If your file contains image-edit requests, create the batch for /v1/images/edits and ensure every line uses that endpoint consistently. Derive retry IDs from the same catalogue-item and image-version keys so a retry can be recognized as the same logical work rather than silently creating a second published derivative.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Model progress without pretending there is a percentage

Job-level status

Status Meaning for an operator Action
validating The service is checking the uploaded batch. Keep polling; investigate a transition to failed.
failed The batch could not be accepted or processed as a batch. Read the failure details, correct the input or configuration, and submit a new idempotent attempt.
in_progress Requests are being executed. Poll at a backoff interval and expose elapsed time.
finalizing Results are being assembled into files. Do not assume all output is downloadable until file IDs appear.
completed The batch finished normally. Download output and error files, then reconcile every line.
expired The 24-hour window ended before all work completed. Download both files; retry only the requests represented as expired.
cancelling A cancellation request is being processed. Continue polling until cancelled.
cancelled Processing stopped. Keep completed results, record unfinished requests, and decide which ones to retry.

The API exposes coarse batch states, not a catalogue-specific percentage counter. Calculate an operational view from your own records: queued count, succeeded count, failed count, and the timestamp of the last observed transition. Label that number as your application’s accounting, not as an OpenAI progress measurement.

Per-image records

Create one row per custom_id with fields such as queued_at, state, output_file_line, error_code, attempt, and derivative_uri. Start at queued; change to succeeded when an output line is validated and stored, or failed when an error line is stored. Keep the raw line or a content hash so an audit can reproduce how the state was assigned.

Polling and retrieval example

const terminal = new Set(["completed", "failed", "expired", "cancelled"]);
let current;

do {
  current = await client.batches.retrieve(batchId);
  console.log(current.status, current.request_counts);
  if (!terminal.has(current.status)) {
    await new Promise(resolve => setTimeout(resolve, 30_000));
  }
} while (!terminal.has(current.status));

for (const fileId of [current.output_file_id, current.error_file_id]) {
  if (!fileId) continue;
  const response = await client.files.content(fileId);
  const text = await response.text();
  for (const line of text.split("n")) {
    if (!line.trim()) continue;
    const record = JSON.parse(line);
    // Update by record.custom_id, never by line number.
    await saveBatchRecord(record.custom_id, record);
  }
}

Use exponential backoff and a maximum polling interval appropriate for your operations team. Store the last successful poll and make the downloader restartable; a process crash must not force you to submit the batch again.

Handle expiry, cancellation, and partial results

Expired batches

When a batch expires, unfinished requests are cancelled. Completed responses remain in the output file, while expired requests are written to the error file with a batch_expired message. Reconcile both files before deciding what to retry.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Manual cancellation

Cancellation does not erase work that completed before the cancellation took effect. Those completed requests remain chargeable, and their results should be retained and published or reviewed according to your normal workflow.

Validation and request errors

Distinguish a batch-level failure from a per-request error. A batch can complete while individual image requests fail. Preserve the error line, model and endpoint parameters, attempt number, and the source image-version ID. Retry only errors your application classifies as transient or correctable; do not blindly replay invalid prompts or inaccessible image URLs.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Set a retention policy that survives the 30-day file deadline

Download output and error files to controlled storage as soon as a batch reaches any terminal state. OpenAI’s Batch API guide says the output file is automatically deleted 30 days after the batch is complete. The FAQ also says zero-data-retention settings do not apply to Batch API artifacts: input files, outputs, errors, and intermediate artifacts follow their configured retention policies.

Asset Retain until Decision factors
Original product image No longer needed for reprocessing, takedown handling, contractual review, or audit Recovery value, supplier access, privacy, and storage cost
Approved derivative Through the catalogue publishing period and any replacement or dispute window Whether the derivative can be regenerated exactly and whether downstream URLs must remain stable
Retry copy or staging derivative Until the replacement is verified and rollback is no longer required Recovery value versus duplicate storage and exposure risk
Input manifest Long enough to explain what was requested and which source revision was used Audit, reproducibility, and supplier obligations
Output and error JSONL After download, validation, and the period in which reconciliation or dispute is possible Evidence needed to explain each success, failure, expiry, or cancellation
Operational logs According to your security and incident-response policy Secrets, personal data, access controls, and legal requirements

A practical policy uses four tests for every asset: can the original be recovered, can the derivative be reproduced, is the content privacy- or contract-sensitive, and what will storage and retrieval cost? Document the owner, deletion trigger, legal hold process, and restore procedure. The API documents a 30-day deletion rule for completed output files; it does not prescribe a universal business retention period for your catalogue.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Operational checklist

  • Generate deterministic, unique custom_id values from catalogue and image-version identifiers.
  • Validate JSONL syntax, endpoint consistency, duplicate IDs, request count, and file size before upload.
  • Chunk below 50,000 requests and 200 MB, with headroom.
  • Persist batch and file IDs before beginning polling.
  • Show API status separately from your per-image counts; do not invent a percentage supplied by the service.
  • Poll through finalizing and every terminal state.
  • Download output and error files immediately and verify that every expected custom_id is accounted for.
  • Make retries idempotent and record why each retry was created.
  • Apply access controls and lifecycle rules to originals, derivatives, manifests, errors, and logs.
  • Test recovery from a worker crash, an expired batch, a cancelled batch, and a missing or malformed output line.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.