Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Use BullMQ to move slow AI calls out of your Node.js request handler: a producer adds a job to Redis, and a worker processes it asynchronously. Reliability comes from explicit retry limits, provider-aware pacing, suitable worker concurrency, and Redis and shutdown settings—not from the queue alone.
How the workflow fits together
BullMQ uses Redis to store queue state. Your application acts as the producer, adding a job with the inputs a worker needs; a separate worker retrieves and processes it. BullMQ manages the queue mechanics, but it does not call an AI provider for you. Keep job data minimal: pass identifiers or necessary inputs rather than credentials or unnecessarily large payloads. See the BullMQ introduction.
The request handler can acknowledge that work was queued instead of waiting for model generation to finish. Your application still needs a way to tell the caller where to check the result or how to receive it; that product-level behavior is separate from BullMQ’s queue mechanism.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesStart with a producer and worker
Install BullMQ, run Redis, and start at least one worker process. This minimal example illustrates the queue pattern; it is a starting point, not production-ready configuration.
#1 Best Overall
import { Queue, Worker } from 'bullmq';
const connection = { host: '127.0.0.1', port: 6379 };
const queue = new Queue('ai-tasks', { connection });
// In your request handler or producer process:
await queue.add('generate', {
taskId: 'task-123',
prompt: 'Summarize the supplied report.',
});
// In a worker process:
const worker = new Worker(
'ai-tasks',
async (job) => {
const result = await callYourAIProvider(job.data.prompt);
await saveResult(job.data.taskId, result);
return { taskId: job.data.taskId };
},
{ connection },
);callYourAIProvider and saveResult represent application code, not BullMQ APIs. Split the producer and worker into separate processes or services when that fits your deployment. The official BullMQ quick start covers the basic setup.
How do I retry failed BullMQ jobs?
Set a finite attempts limit and an explicit backoff policy. With no backoff configured, BullMQ retries immediately. Fixed backoff waits the configured delay between retries; exponential backoff increases the delay as attempts progress. Consult BullMQ’s retry guide for the supported options.
Rank #2
await queue.add('generate', data, {
attempts: 4,
backoff: {
type: 'exponential',
delay: 1000,
},
});This example allows up to four attempts with exponential backoff starting from a one-second delay. Choose the ceiling and delay based on the cost of repeating the work and the limits of the downstream service. If many jobs fail together, jitter—a small random variation in retry delays—can reduce synchronized retry bursts; make sure your chosen strategy supports the behavior you intend.
Retries do not provide exactly-once execution. A worker may perform an external action and then fail before BullMQ records completion, so a retry can repeat that action. Make side effects safe to repeat where possible: for example, associate results with a stable task identifier and avoid creating duplicate records for the same completed task.
Rank #3
How do I rate limit AI jobs?
Provider throttling and queue pacing are related but distinct. BullMQ’s rate limiter can pace jobs; rate-limited work remains waiting until it can run. Configure it according to the applicable provider limit and your application’s traffic pattern, rather than treating it as a substitute for handling provider responses. BullMQ’s documentation says QueueScheduler has not been needed for rate limiting since BullMQ 2.0. See BullMQ rate limiting.
The provider SDK may also retry. OpenAI’s documentation says its SDKs automatically retry eligible 429 and 503 responses, subject to retry settings. If both the SDK and the queue retry a failed call, the total work and delay can multiply. Decide which layer should handle each failure, and set queue attempts and SDK retry behavior with that interaction in mind. Refer to OpenAI’s rate-limits guidance for its current SDK and API details; provider behavior can differ.
Rank #4
Choose concurrency and worker topology
For network-bound model calls, asynchronous concurrency lets a worker start another job while an earlier request waits for a response. Multiple worker processes can add capacity and availability, but also increase operational complexity and the number of simultaneous calls. There is no universal concurrency value that guarantees a particular throughput; measure under your workload and respect provider limits.
CPU-heavy synchronous preprocessing or post-processing needs a different strategy. Blocking the Node.js event loop can prevent BullMQ from renewing a job lock, which can cause a job to be marked stalled and processed again. Use sandboxed processors for CPU-intensive work, or otherwise keep the event loop available. See BullMQ’s guides to worker concurrency, stalled jobs, and sandboxed processors.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.When should I split a workflow into stages?
Use one job for a straightforward task. When later work depends on earlier work—for example, preparing inputs, calling a model, then post-processing the result—BullMQ’s FlowProducer can express parent-child dependencies. A parent waits until its children complete successfully. These stages are an application design choice, not a required AI architecture. Details are in the BullMQ flows guide.
Prepare Redis and workers for production
Queue reliability depends on the Redis deployment and worker lifecycle as well as the job code. BullMQ’s production guidance recommends configuring Redis persistence and setting maxmemory-policy to noeviction; eviction of queue data can undermine reliable processing. Plan how workers handle Redis reconnects, and review the deployment guidance at Going to production.
- Persistence: choose Redis persistence settings appropriate to the recovery requirements of your queue.
- Memory policy: configure
maxmemory-policy noevictionrather than allowing Redis to evict queue keys under memory pressure. - Reconnects: define and test how the application behaves during transient Redis disconnects and recovery.
- Retention: choose how long completed and failed jobs remain available so debugging needs are balanced against storage use.
- Shutdown: on
SIGINTandSIGTERM, stop accepting new work as appropriate and close workers cleanly so active work can finish within the available grace period.
Graceful shutdown reduces disruption; it does not guarantee that jobs will never stall. An active task that outlasts the deployment’s shutdown grace period may be interrupted and become eligible for processing again. Test Redis outages, transient disconnects, and shutdowns in the environment you deploy to; a successful local run does not establish production reliability.
Recommended Free Tools
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
