Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To create a delivery once when a Node worker retries, make the delivery operation idempotent. Give each logical delivery a stable identity, and enforce that identity at the point where the side effect is committed, so replaying the same job cannot create a second delivery. Queue retries and job deduplication help control repeated work, but neither one makes an external side effect exactly once on its own.
Why a retry can send the same delivery twice
A retry is a second attempt at work that may already have partly succeeded. BullMQ runs a job again after a processor failure when retries are configured, and its retry guide describes when a failed job is attempted again. It does not decide whether the work is safe to repeat. That decision belongs to your code. (BullMQ: Retrying failing jobs)
The failure that matters is the gap between the side effect and the acknowledgement. Your worker may send an email, insert a delivery row, or call a third-party API, and then crash, lose its connection, or time out before the job is marked complete. The queue sees an unfinished job and runs it again. Amazon SQS documents the same class of problem for standard queues: a message can be received more than once in rare cases, and AWS advises designing consumers to be idempotent. (Amazon SQS: At-least-once delivery)
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →So the design question is not how to stop the queue from ever running a job twice. It is how to make the second run leave the system in the same state as the first.
#1 Best Overall
Define idempotence as a final state
BullMQ defines an idempotent job by its outcome: the final system state should be the same whether the job succeeds on the first attempt or only after one or more retries. That is the target to design for. A job that inserts a row, sends a message, and then increments a counter is not idempotent by default, because a retry after a partial run can insert a second row or send a second message. (BullMQ: Idempotent jobs)
BullMQ’s guidance is to keep jobs simple and atomic where possible. Large jobs that do many things make partial progress harder to track and rollback harder to reason about. Splitting a workflow into smaller jobs, each with one clear side effect, makes it easier to see which step needs protection.
Give each logical delivery a stable key
The key identifies the business event, not the attempt. A retry of the same delivery should reuse the same key. A genuinely new delivery, such as the same customer receiving a second notice for a new order, should get a different key.
Rank #2
Good keys usually come from data that already identifies the event: an order ID plus a notification type, a source message ID from your upstream system, or a request ID your API issued. Avoid keys built from the job ID that the queue generates for each attempt, the current timestamp, or a random value. Any of those changes on retry, so the key stops protecting you.
This is an implementation recommendation drawn from how BullMQ and AWS use identity keys for deduplication, not a rule either vendor prescribes for your domain. (BullMQ: Deduplication, Amazon SQS: Exactly-once processing)
Enforce the key where the side effect is committed
A key only helps if the write that records the delivery refuses a second logical delivery with the same key. For a relational database, a primary key or unique constraint does this. The insert succeeds once, and any later attempt with the same key finds the existing row instead of creating another.
Rank #3
CREATE TABLE deliveries (
delivery_key text PRIMARY KEY,
recipient text NOT NULL,
status text NOT NULL, -- 'pending', 'sent', 'failed'
created_at timestamptz NOT NULL DEFAULT now()
);
-- Run before the external send
INSERT INTO deliveries (delivery_key, recipient, status)
VALUES ($1, $2, 'pending')
ON CONFLICT (delivery_key) DO NOTHING;After the insert, check the row’s status. If it already reads sent, the worker can acknowledge the job and stop. If it reads pending, the worker needs a rule for whether to try the send again, which depends on the provider (see below).
This pattern is practical design guidance. The queue documentation does not promise that your database will behave this way; the uniqueness check is what enforces the rule.
The gap between your database and the external send
A database row and an external call cannot normally be committed in one transaction. If you insert pending, call the provider, and crash before updating to sent, a retry sees pending and must decide what to do. If you send first and record afterward, a crash between the two steps can produce a delivery with no record.
Rank #4
Two approaches reduce the risk:
- Use the provider’s idempotency mechanism if it has one. Pass your logical key to the provider so it can reject or return the earlier result for a repeated request. Check the current documentation for that specific API before relying on it, because support varies.
- Reconcile pending rows. Run a periodic check that finds rows stuck in
pendingbeyond a threshold and looks up whether the provider accepted the message, using a provider-side lookup or a status webhook. Where no lookup exists, decide explicitly whether a resend or a manual review is acceptable for that message type.
What queue deduplication does and does not do
BullMQ and Amazon SQS both offer deduplication features. They operate at the queue layer, which is useful for controlling admission of repeated jobs or messages. They do not reach into your email provider, payment API, or database to undo a side effect that already happened.
| Mechanism | What the documentation says | What it protects against | What it does not protect against |
|---|---|---|---|
| Application-level idempotence (your code) | BullMQ describes an idempotent job as one whose final state is the same after a first-attempt success or a retry success. (BullMQ: Idempotent jobs) | Repeated execution of the operation, provided the logical key is enforced at the write. | Nothing outside your system unless the external API also enforces a key. |
| BullMQ job ID and deduplication | Repeated additions can be ignored while a matching job exists, or according to a configured deduplication mode or TTL. (BullMQ: Deduplication) | Duplicate jobs added to the queue within the matching window. | Side effects of a job that already ran; the guarantee ends when the matching job is removed or the TTL expires. |
| BullMQ reused job ID after removal | BullMQ warns that a removed completed or failed job no longer counts as an existing duplicate for a reused job ID. (BullMQ: Throttle jobs) | Nothing after removal; the queue no longer recognises the earlier job. | Any duplicate that arrives after the earlier job was removed. |
| Amazon SQS standard queue | A message may be delivered more than once in rare cases, and AWS advises idempotent consumers. (Amazon SQS: At-least-once delivery) | Nothing on its own; the consumer must tolerate repeats. | Duplicate processing by the consumer. |
| Amazon SQS FIFO queue | Duplicate sends within the documented five-minute deduplication interval are suppressed when a message deduplication ID is used (content-based or explicit). (Amazon SQS: Exactly-once processing) | Duplicate sends to the queue inside that five-minute interval. | Duplicates sent after the interval, and any side effect your consumer performs after receiving the message. |
The practical reading is that queue deduplication narrows the window in which duplicates enter the system. Your consumer still has to be safe when a duplicate does reach it.
Free tools Windows power users keep installed
One-click scans. No signup required.
Keep retries bounded and visible
Retries should handle transient failures, such as a brief network error or a rate-limit response. They should not hide a permanent failure or make correctness depend on repetition. BullMQ documents a number of attempts and a backoff strategy, either fixed or exponential, for failing jobs. (BullMQ: Retrying failing jobs)
await deliveryQueue.add('send-notice', payload, {
jobId: 'notice:' + payload.orderId + ':shipped',
attempts: 5,
backoff: { type: 'exponential', delay: 2000 },
});In this example the jobId is derived from the business event rather than generated per attempt. It is a queue-level guard only. The database key in the previous section is still what protects the delivery itself.
Configure your worker so exhausted failures are visible. A job that has used all its attempts should move to the failed set where an alert or dashboard can pick it up, rather than disappearing silently.
Test the failure window that matters
Testing a happy path proves little. The case that creates duplicates is a crash after the side effect and before the job is recorded as complete. Reproduce that case deliberately:
- Start a worker and enqueue one job for a single logical delivery.
- Let the send complete and the
sentstatus commit to the database. - Stop the worker process before the queue records completion, for example with a forced kill.
- Restart the worker so the job runs again with the same logical key.
- Query the table for that key and confirm exactly one row exists, with one provider send recorded.
Repeat the sequence with the crash placed after the provider call but before the status update, and confirm the reconciliation path resolves the pending row without a second send. The results depend on your provider and your code, so run the steps against your own stack rather than assuming them from this outline.
Check versions and current documentation
The options and behaviours above reflect BullMQ and Amazon SQS documentation as checked on 7 October 2026. Option names, deduplication modes, and retention behaviour can change between releases. Confirm the settings against the BullMQ version and SQS configuration you actually run before deploying, and check your email, payment, or messaging provider’s current documentation for any idempotency feature.
Quick Recap
“
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
