Webhook retries: how to use idempotency, backoff, and queues

Webhook retries make messaging integrations faster because they allow a system to receive events as soon as something happens: a message arrives, is processed, delivered, or read, or an operation fails.

The problem is that an HTTP request alone does not guarantee that the event will be processed correctly. The connection may drop, the endpoint may respond too slowly, the server may return an error, the workflow may fail after receiving the request, or the same event may be sent again.

That is why a reliable integration needs to account for four situations:

1. The event may not arrive;
2. The event may arrive more than once;
3. The event may arrive out of order;
4. Processing may fail after the event has already been received.

The architecture needs to combine retries, backoff, idempotency, queues, dead-letter queues (DLQs), and monitoring.

It is also important to separate two flows that are often confused in Z-API integrations:

• Webhook delivery: Z-API sends events to the company’s endpoint;
• Message sending queue: the application sends a message to Z-API, which places it in a queue for processing toward messaging app.

The two flows may be related, but they are not the same thing. Webhook delivery retries need to be handled at the receiving layer. The sending queue belongs to the path between the application, Z-API, and messaging app.

In this guide, you will learn how to design an architecture that reduces event loss, prevents duplicates, and allows failures to be recovered without turning every error into another message sent to the customer.

What is a webhook?

A webhook is an HTTP call triggered by a system when an event occurs. Instead of your application asking every minute whether something new has happened, the provider sends a request to a configured URL.

A traditional polling flow looks like this:

Your application → “Is there a new message?”
Your application → “Is there a new message?”
Your application → “Is there a new message?”

With a webhook, the flow is event-driven:

Event occurs → Provider calls your URL → Your application processes it

n8n, for example, uses a Webhook node to receive external data and start a workflow. In an architecture using Z-API, webhooks can report interactions and changes related to message processing.

In the sending flow, the application sends the message to the API, Z-API adds the request to a queue, and returns an identifier. Later events can then report the progress of processing, delivery, and read status.

The key point is that receiving the webhook and completing its processing are two different stages.

Why do webhooks fail?

A webhook failure does not always mean the provider lost the event. The event may have arrived correctly, but processing may have failed immediately afterward.

Timeout

The server takes too long to respond. This can happen when the endpoint calls a CRM, queries a database, processes an AI request, or waits for another service before returning an HTTP status.

The webhook should respond quickly. Heavy processing should continue in an internal queue or a separate worker.

HTTP error

Responses such as 500, 502, 503, and 504 generally indicate a server or dependency failure. A 400 response may indicate an invalid payload. A 401 or 403 points to an authentication or authorization problem.

Not every error should trigger a retry. Repeating a request with invalid JSON does not fix the payload.

Network failure

The provider may be unable to reach your domain. DNS, firewall rules, TLS certificates, routing, proxies, or downtime can prevent the connection.

Deployment or maintenance

An update may restart the service at the exact moment the webhook is delivered. Without a queue on either the provider side or the receiving side, the event may be lost.

Partial workflow failure

The endpoint returns 200, but the workflow fails afterward.

From the sender’s perspective, the event was accepted. From the company’s perspective, the action was not completed.

This is one of the reasons for separating receipt from processing. The endpoint should durably record the event before responding.

Duplicate event

The same event may arrive again after a timeout or retry.

If the application creates a ticket, issues a charge, or sends a reply without checking whether the event has already been processed, duplicate actions will occur.

The difference between webhook retries and the Z-API sending queue

This distinction is essential.

Webhook delivery retry

In this flow, Z-API acts as the sender and your application acts as the receiver.

Z-API → POST to your endpoint → Your application responds

If the endpoint is unavailable or returns a temporary error, the sender may retry according to the policy applicable to the service.

The receiver should be prepared to receive the same event more than once.

The exact retry policy, interval, and duration should be confirmed in the current documentation and service agreement. Your application should not depend on the assumption that a specific number of retries will always happen.

Z-API message sending queue

In this flow, your application acts as the sender and Z-API receives the request.

Your application → API → API queue → messaging app

The application sends a message to Z-API. The API receives the request, places the send operation in a queue, and returns an identifier.

From there, later events can report processing, delivery, and read status.

This queue handles messages sent by the company. It does not replace the queue your own application may need for receiving and processing webhooks.

Aspect	Webhook retry	Z-API sending queue
Direction	Z-API → your application	your application → Z-API → messaging app
Goal	Deliver an event to your endpoint	Process a message being sent
Main control	Webhook sender and receiver	Z-API and your sending instance
Common risk	Lost or duplicated event	Duplicate message or misinterpreted status
Identifier	Event ID or message ID	ID returned by the API
Handling	Idempotency on the consumer side	Sending control and status tracking

A mature architecture uses both layers: its own queue for consuming events and message-ID tracking for outbound messages.

Retry and backoff: how to retry without overloading the system

A retry is another attempt to execute an operation that failed.

Retries are useful for temporary errors, but they can make an outage worse when many instances repeat the same request at the same time.

A retry strategy should combine:

• a maximum number of attempts;
• progressive intervals;
• random jitter;
• error classification;
• a circuit breaker;
• a reprocessing queue.

Exponential backoff

With exponential backoff, the interval increases after each attempt.

A simple formula is:

interval = min(max_delay, base_delay × 2^attempt)

Example with a base delay of 1 second:

Attempt	Approximate interval
1	1 second
2	2 seconds
3	4 seconds
4	8 seconds
5	16 seconds

Do not use identical fixed values for every worker.

The goal is to prevent an unavailable system from receiving another wave of requests immediately after every failure.

Jitter

Jitter adds random variation to the interval, distributing attempts over time.

wait = random(0, min(max_delay, base_delay × 2^attempt))

Jitter is recommended in distributed systems because it reduces unintended synchronization among clients that failed at the same time.

When should you retry?

In general, retries make sense for temporary failures such as:

• timeout;
• terminated connection;
• 502 Bad Gateway;
• 503 Service Unavailable;
• 504 Gateway Timeout;
• temporary DNS failure;
• rate limiting, while respecting the indicated retry interval.

Avoid automatic retries for errors that require correction:

• 400 Bad Request caused by an invalid payload;
• 401 Unauthorized;
• 403 Forbidden;
• invalid recipient;
• missing required field;
• event incompatible with the workflow.

In these cases, the event should be routed for analysis or to a specific correction routine.

Maximum number of attempts

Never depend on infinite retries.

Set a maximum number of attempts that makes sense for the flow. Once that limit is reached, move the event to a failure queue.

It should not disappear silently.

You may also want to explore about: Automated digital communication and how to build an elite customer service operation in 2026.

What is idempotency?

An operation is idempotent when executing it once or multiple times produces the same final effect.

Imagine that the “order shipped” webhook is received three times. The customer should receive only one notification.

Likewise, if a “new lead” event is duplicated, the CRM should keep one opportunity instead of creating three.

Idempotency does not simply mean ignoring repeated requests.

It means recognizing that they represent the same operation and reusing the result that has already been recorded.

Idempotency for inbound events

The event needs a stable key. Use, in this order, an identifier provided by the provider or a reliable composite key.

Examples:

messageId
messageId + eventType
instanceId + messageId + eventType
orderId + eventType + version

Avoid using the receipt timestamp as the key. The same event may arrive at different times.

Idempotency for outbound messages

Before calling the API to send a message, create your own operation key.

For example:

order-9482:shipped

If the workflow fails after sending, the retry should check whether that key already has a messageId.

If it does, do not send a second message.

Deduplication pseudocode

The logic below shows a consumer that records the event before executing the main side effect.

function processWebhook(event):

    eventKey = buildKey(
        provider = "z-api",
        instance = event.instanceId,
        message = event.messageId,
        type = event.eventType
    )

    existing = idempotencyStore.find(eventKey)

    if existing.status == "COMPLETED"
        return HTTP 200

    if existing.status == "PROCESSING":
        return HTTP 202

    idempotencyStore.insertIfAbsent(
        key = eventKey,
        status = "PROCESSING",
        receivedAt = now()
    )

    queue.publish({
        key: eventKey,
        payload: event
    })

    return HTTP 202

The worker processes the queue:

function consume(message):

    record = idempotencyStore.find(message.key)

    if record.status == "COMPLETED":
        acknowledge(message)
        return

    try:
        result = executeBusinessAction(message.payload)

        idempotencyStore.update(
            key = message.key,
            status = "COMPLETED",
            result = result,
            completedAt = now()
        )

        acknowledge(message)

    catch error:
        if isTransient(error) and message.attempts < MAX_ATTEMPTS:
            retryWithBackoff(message)

        else:
            deadLetterQueue.publish(message, error)

            idempotencyStore.update(
                key = message.key,
                status = "FAILED",
                error = error.message
            )

            acknowledge(message)

The insertIfAbsent method must be atomic. Two workers must not be able to insert the same key at the same time.

How should you choose an idempotency key?

The key should identify the operation, not just the contact. A contact can receive multiple different messages.

Using only the phone number or LID as the key would make a second legitimate message look like a duplicate.

A good key combines:

• provider;
• instance or channel;
• event identifier;
• event type;
• operation version.

If the event does not have a reliable ID, create a fingerprint of the normalized payload.

However, be careful: two legitimate messages with identical content can produce the same hash.

Include contextual fields such as event type, source, and object ID.

Queues: why not process everything inside the webhook?

The endpoint should do only what is necessary:

1. validate authentication;
2. validate the basic structure;
3. generate or extract the idempotency key;
4. persist the event;
5. place the event in a queue;
6. respond quickly.

Heavy processing should be handled by a worker. This includes:

• querying the CRM;
• calling an AI model;
• updating an ERP;
• sending an email;
• creating a ticket;
• calculating a price;
• calling other services.

When the webhook waits for all these steps to finish, any slowdown can be interpreted as a delivery failure.

The sender may retry while the first process is still running, creating duplicates.

In-memory queue vs. durable queue

An in-memory list is fast, but it can lose events if the process restarts. For important events, use durable storage.

Common options include:

• Redis with appropriate persistence configuration;
• RabbitMQ;
• Amazon SQS;
• Kafka;
• a database with an events table;
• a queue managed by the automation platform itself.

The choice depends on volume, ordering, retention, cost, and the team’s capabilities.

Ordering

Not every event can be processed in arbitrary order.

A READ status arriving before RECEIVED may require different handling, just as an order update may.

If order matters, use an ordering key based on the conversation, contact, or order.

Avoid blocking the entire queue because of a single conversation.

Dead-letter queue: what should you do with events that fail?

A dead-letter queue, or DLQ, is a queue for messages that could not be processed after the maximum number of attempts.

The DLQ prevents two extremes:

• discarding the event without a record;
• retrying an operation forever when it will never succeed.

Queue services such as Amazon SQS use a redrive policy to define how many times a message can be received before being moved to a DLQ.

The failure queue can be inspected, analyzed, and reprocessed after the underlying issue is fixed.

What should be stored in the DLQ?

Store:

• original payload;
• idempotency key;
• attempt counter;
• latest error;
• stack trace when appropriate;
• timestamps;
• consumer version;
• workflow name;
• webhook source;
• reason for the decision.

Do not store sensitive data unnecessarily. Protect access to the queue and define an appropriate retention policy.

Safe reprocessing

After the issue is fixed, the event can be sent back to the main queue. Reprocessing should preserve the same idempotency key.

If the action has already completed and only the internal record is inconsistent, the worker should recognize the state and avoid repeating the external side effect.

Need to know how to diagnose REST API instability? Read the definitive 2026 guide and learn what to investigate.

Failure monitoring

An integration without observability can appear healthy while still losing events. Monitor the entire path.

Inbound metrics

• number of webhooks received;
• 2xx response rate;
• 4xx and 5xx responses;
• endpoint latency;
• timeouts;
• events by type;
• inbound queue size.

Processing metrics

• events processed;
• queue wait time;
• worker execution time;
• success rate;
• retries per event;
• permanent failures;
• messages in the DLQ;
• duplicates detected.

Sending metrics

• API calls accepted;
• message IDs generated;
• sending failures;
• delivery status;
• read status;
• time between send and delivery;
• discrepancies between request and confirmation.

Z-API documents a sequence in which the API returns an ID, the queue processes the request, and webhooks report later processing, receipt, and read events.

Use that ID to correlate the initial send with later statuses.

Useful alerts

Create alerts for:

• a sudden increase in 5xx responses;
• latency above the defined threshold;
• a queue that keeps growing;
• increasing retry counts;
• the first message entering the DLQ;
• a drop in delivery rate;
• too many events without an idempotency key;
• duplicate rates above normal;
• no webhook events during an expected period.

An alert should not simply say that an error occurred. It should identify the service, instance, event type, and next procedure.

Webhook security

Retries and queues do not compensate for an unprotected endpoint.

Authentication

Use header authentication, tokens, signatures, or another mechanism supported by the provider.

n8n, for example, offers Basic Auth, Header Auth, and JWT authentication for its webhooks.

Payload validation

Validate types, sizes, required fields, and origin. Do not execute critical actions simply because the request body contains an event field.

HTTPS

Use HTTPS with a valid certificate. Do not expose customer data in URLs or query strings unnecessarily.

Rate limiting

Limit requests by source and protect the endpoint against abuse. The limit should be compatible with the provider’s legitimate traffic volume.

Secure logging

Record IDs and metadata for troubleshooting. Masking phone numbers, content, and tokens reduces exposure in logs.

How to implement this with n8n, Make, or Zapier

n8n

Use the Webhook node with the test URL during development and the production URL after publishing the workflow. Configure authentication, validate the payload, record the idempotency key, and respond quickly.

A simple architecture is:

n8n Webhook ↓ Validation ↓ Data Store / database: has this key already been processed? ↓ no Queue or asynchronous execution ↓ Business action ↓ Record COMPLETED or send to DLQ

Make

Use a Custom Webhook as the entry point.

The scenario can store the key in a Data Store, database, or external application before updating the CRM or sending a response.

Keep in mind that instant webhooks may be processed in parallel.

If order matters, configure sequential processing or implement sequence control in the destination system.

Zapier

Use Catch Hook to receive the event and filter duplicates using an identifier stored in a database. For critical processes, evaluate whether the available history, retention, and plan limits are appropriate.

In any visual automation platform, do not confuse “workflow executed” with “effect completed”. Record the result of the external action and handle failures at each step.

Complete example: incoming message and automatic response

Suppose a customer sends: “I want to track my order.”

1. Z-API sends the event to the webhook.
2. The endpoint validates the token.
3. The application extracts messageId, contact, and content.
4. The idempotency key is created.
5. The event is stored as PROCESSING.
6. The endpoint responds with 202.
7. The worker queries the CRM and logistics system.
8. The worker builds the response.
9. The application calls the Z-API API.
10. Z-API returns a new message ID.
11. The application stores that ID.
12. Status webhooks update the record.

If the original event arrives again, the existing key prevents a second lookup or a second response.

If the CRM is unavailable, the worker retries with backoff.

If the problem continues, the event moves to the DLQ. The customer does not receive five identical messages, and the team can investigate the case.

Implementation checklist

• Use HTTPS on the endpoint.
• Configure webhook authentication.
• Validate the payload and request size.
• Respond quickly after persisting the event.
• Separate event receipt from heavy processing.
• Create a stable idempotency key.
• Implement atomic deduplication.
• Record PROCESSING, COMPLETED, and FAILED states.
• Use a durable queue for important events.
• Retry only temporary errors.
• Use exponential backoff with jitter.
• Define a maximum number of attempts.
• Create a DLQ.
• Monitor the queue and DLQ.
• Store message IDs returned by Z-API.
• Distinguish acceptance, processing, delivery, and read status.
• Test duplicate events.
• Test out-of-order events.
• Test timeouts and 500 responses.
• Test reprocessing after a fix.
• Review log retention and protection.

Webhook retries are necessary in real-world integrations

Retries are necessary, but retries without idempotency can turn a temporary failure into duplicated messages, repeated tickets, and inconsistent updates.

• The safest design separates webhook receipt from business processing.
• The endpoint validates, records, and places the event in a queue.
• The worker processes the event using an idempotency key.
• Temporary errors use backoff and jitter.
• Permanent failures go to a dead-letter queue.
• Monitoring covers intake, queueing, processing, sending, and status updates.
• It is also essential to distinguish the two communication directions.

Webhook delivery retries refer to the event that Z-API sends to the company’s endpoint.

The Z-API sending queue handles the message that the application sends to the API and that will be processed toward messaging app.

A robust integration does not assume that events will arrive exactly once, in perfect order, or without failures.

It records, deduplicates, retries, monitors, and provides a controlled way to recover anything that was not successfully processed.

See the Z-API webhook documentation and implement your consumer with idempotency, queues, and monitoring from the very first workflow.

Frequently asked questions

What is a webhook retry?

It is another attempt to deliver or process an event that failed. The strategy should limit the number of attempts and distinguish temporary errors from permanent ones.

Can the same webhook arrive twice?

Yes. The consumer should assume that events may be duplicated and use an idempotency key before performing actions with external side effects.

Should I return 200 only after the entire process is complete?

In many cases, no. It is safer to persist the event, place it in a queue, and respond quickly. A worker can then complete the processing asynchronously.

What is the difference between retry and backoff?

A retry is another attempt. Backoff is the progressively increasing interval between attempts. Jitter adds random variation so multiple workers do not retry at exactly the same time.

What is a dead-letter queue?

It is a queue that receives events that failed after the maximum number of attempts. It preserves the case for investigation and controlled reprocessing.

Is the Z-API sending queue the same as my webhook queue?

No. The Z-API queue processes messages sent by your application toward messaging app. Your webhook queue processes events received from Z-API.

Is the message ID enough to deduplicate everything?

It can be an important part of the key, but idempotency should also consider the event type, instance, and business action. The correct key depends on the payload and the workflow.

How can I test an integration with retries?

Simulate timeouts, 500 responses, duplicate events, out-of-order events, CRM failures, invalid payloads, and reprocessing of a message from the DLQ.