Production-ready Webhooks 2026: Standard Webhooks, JWKS, and the replay problem

Customer reports: "The order is paid, but your system doesn't show it." You check the logs — the webhook was fired, returned a 503 because the other side was deploying, and... that's it. No retry, no queue, no way to resend. That event is lost forever, and you are the one manually reconciling the database.

That is the gap between a webhook that "works" and a webhook that is "reliable." In 2026, this gap is no longer something to DIY — there are standards, patterns, and ready-to-use infrastructure. This article is a checklist for both sides: Vietnamese SaaS providers exposing webhooks to customers, and backend teams receiving webhooks from Stripe, GitHub, or Shopify.

The era of every vendor having a different signing method is over.

Previously, receiving webhooks from 5 vendors meant writing 5 different verify functions: some sign HMAC-SHA256 on the raw body, some sign on the body + timestamp, some place the signature in the header. X-Hub-Signature-256, elsewhere Stripe-Signature with a custom format. Copying a single line wrong will silently skip verification.

Spec Standard Webhooks was created to end that. Three unified headers:

webhook-id:        msg_2ZxTpM...      # định danh duy nhất của message
webhook-timestamp: 1752739200         # Unix timestamp lúc gửi
webhook-signature: v1,K5oZfzN95Z9U... # chữ ký trên (id + timestamp + body)

Signature is calculated on both id and timestamp, not just the body — meaning even if an attacker captures a valid request, they cannot modify the timestamp for reuse. Official SDKs are available for most popular languages, so the receiver doesn't have to implement crypto manually.

Regarding the payload, CloudEvents is the default choice for the envelope (type, source, id, time, data), and AsyncAPI to describe the contract — equivalent to OpenAPI for REST. If you are designing a new webhook for your SaaS in 2026, follow this trio instead of inventing a custom format: your customers will integrate in one afternoon instead of a week.

A 3-year HMAC secret is a time bomb.

Common pattern: issue a shared secret to the customer, the customer pastes it into an env var, and that secret lives... forever. If the secret is leaked (wrong log, accidental commit, employee departure), an attacker can forge valid webhooks indefinitely — and you have no way of knowing.

2026 Trend: replace static HMAC secrets with short-lived signing keys, asymmetric signing, automatic rotation, and publishing via a JWKS endpoint — exactly the OIDC model used for decades. The sender signs with a private key and rotates keys on a schedule (a few days to a few weeks); the receiver fetches the public key from /.well-known/jwks.json, cache by kid, encounter kid if unusual, refresh. No secrets are exposed on the receiver side, and rotation requires no client action.

The counterpart to signature is prevent replay using timestamp tolerance: request has webhook-timestamp Reject immediately if the drift is more than 5 minutes compared to the server clock, even if the signature is valid. Without this step, a valid request captured today could be replayed next month and still pass verification.

Sender side: retry is a feature, not luck.

Receiver downtime is a certainty. The question is what your system does when that happens. 2026 production standards, looking at how Stripe, GitHub, and Shopify operate:

The most overlooked point is Replay after DLQWebhook failure is not scary; losing data after failure is. Stripe or Shopify dashboards in 2026 will all show customers the history of every delivery and resend for each event — that is the standard your customers will use to compare you.

Receiver side: duplicate is the default, not an exception

At-least-once means you will receive the same event twice — the sender times out before receiving your 200 and retries, even if you have already processed it. Two principles:

Idempotency key is mandatory. Use webhook-id Implement dedup lock: if already processed, return 200 and skip. A unique constraint in the database is enough; don't let "double charging" become a production ticket.

Ack quickly, process asynchronously. Verify signature, write event to internal queue, and return 200 within a few hundred ms. Offload heavy business logic to workers. A handler taking 20 seconds to call three other services will consistently timeout on the client side, trigger continuous retries, and turn you into an "unreliable endpoint" in the vendor's dashboard.

@app.post("/webhooks/payments")
def handle(request):
    wh = Webhook.from_jwks(JWKS_URL)          # SDK Standard Webhooks
    event = wh.verify(request.body, request.headers)  # sig + timestamp tolerance

    if not store.insert_once(event_id=request.headers["webhook-id"]):
        return 200                             # duplicate — đã xử lý rồi

    queue.publish(event)                       # xử lý async ở worker
    return 200

Build from scratch or use existing infrastructure?

For receiving webhooks, just do it yourself — SDK verify + queue + idempotency is enough. However, at scale, a serious webhook system is a complete queue-retry-DLQ-dashboard system, and that is when you consider specialized infrastructure: Hookdeck if you want managed, Outpost if you want open-source self-hosted. A reasonable threshold: under a few thousand events/day with one or two customers, build it yourself using existing queues; if you offer webhooks as a product feature for hundreds of customers, do not rewrite the retry engine.

Finally, look at the true nature: a webhook is just event-driven architecture extending beyond the company's boundaries. Internally, you have Kafka with consumer groups, retry, DLQ; webhooks are exactly those same problems, but the consumer is someone else's system, running on infrastructure you don't control, written by teams you don't know. Throw away all optimistic assumptions about network and uptime — the rest is just familiar engineering, just do it properly.