Recovering from failures
What to do when a request fails or its answer never arrives, case by case, with where your money is in each.
A failure you did not see the end of is unknown, not failed. Find out where the money is before you act: each case below says how, and none of them is ever fixed by creating a second order.
Unknown outcomes
- A timeout means unknown. It never means failed. The order may exist, with its amount held.
- Retry with the same idempotency key. Send the same request, with the same body and the same
Idempotency-Key. You get the order it made, never a second one. - Never treat a broadcast as delivered. A request handed to a provider, or a transaction seen on a network, is not a delivery. Only
succeededmeans delivered, and onlyfailedmeans your money is back. - Keep the order under review.
under_reviewmeans the outcome could not be confirmed, so your money stays held until it is. Show it as pending.
Failure cases
| What happened | Where your money is | What to do |
|---|---|---|
A timeout or a dropped connection on POST /orders | Unknown: the order may exist, with its amount held. | Send the same request with the same Idempotency-Key, or find the order with GET /orders?reference=. |
internal_error | Unknown. | Send the same request with the same key. A retry never repeats an operation that went through. |
service_unavailable or rate_limited | The request was not processed. | Wait at least the seconds in Retry-After, then send the same request with the same key. |
idempotency_in_progress | The first attempt is still running. | Wait a moment, then send the same request with the same key. |
insufficient_funds | Nothing was reserved, and no order was created. | Fund the balance, then send the same order again; see Pricing and costs. |
quote_expired | Nothing was reserved. | Quote again, and order from the new quote. |
An order is failed | Released back to available. | Read status_reason. Order again, with a new key, only if it still makes sense. |
An order is under_review | Held. | Wait for succeeded or failed. Do not order again meanwhile. |
| Your webhook endpoint was down | Unchanged: an event moves no money. | Failed attempts are retried. Catch up from GET /events, and replay a delivery with POST /. |
| A key may have leaked | At risk while the key works. | Revoke it in your dashboard and create another. |
After a crash or a restart
- A worker that stored each
Idempotency-Keybefore sending can send every unfinished request again, safely. - Compare what you believe with what the API holds:
GET /ordersby yourreferenceor bystatus, and the statement,GET /balance/transactions, where each order's reserve, capture or release names itsorder_id. - Catch up on events from
GET /events, which keeps the last 30 days, and ignore any event id you have already handled.
Uncredited GRAM deposits
Read its status and hold_reason on GET /funding/deposits. A deposit that cannot be credited automatically is recorded and reviewed, never lost from the record; see Funding with GRAM. Do not send the amount again to find out.
Common mistakes
- Treating a timeout as a failure. The order may exist. Retry with the same key, or look it up by
reference. - Ordering again to find out. A second order is a second payment. Ask about the first one instead.
- Retrying in a loop. Back off for the seconds in
Retry-After. - Acting on a test event. Acknowledge it, and change nothing.
Related guides
- Idempotency and safe retries: how to send a request again safely.
- Errors: every code, and whether to retry it.
- Webhooks: the event log and replay.
- Transaction statuses: where the money is in each status.
