Skip to content

Recovering from failures

What to do when a request fails or its answer never arrives, case by case, with where your money is in each.

A failure you did not see the end of is unknown, not failed. Find out where the money is before you act: each case below says how, and none of them is ever fixed by creating a second order.

Unknown outcomes

  • A timeout means unknown. It never means failed. The order may exist, with its amount held.
  • Retry with the same idempotency key. Send the same request, with the same body and the same Idempotency-Key. You get the order it made, never a second one.
  • Never treat a broadcast as delivered. A request handed to a provider, or a transaction seen on a network, is not a delivery. Only succeeded means delivered, and only failed means your money is back.
  • Keep the order under review. under_review means the outcome could not be confirmed, so your money stays held until it is. Show it as pending.

Failure cases

What happenedWhere your money isWhat to do
A timeout or a dropped connection on POST /ordersUnknown: the order may exist, with its amount held.Send the same request with the same Idempotency-Key, or find the order with GET /orders?reference=.
internal_errorUnknown.Send the same request with the same key. A retry never repeats an operation that went through.
service_unavailable or rate_limitedThe request was not processed.Wait at least the seconds in Retry-After, then send the same request with the same key.
idempotency_in_progressThe first attempt is still running.Wait a moment, then send the same request with the same key.
insufficient_fundsNothing was reserved, and no order was created.Fund the balance, then send the same order again; see Pricing and costs.
quote_expiredNothing was reserved.Quote again, and order from the new quote.
An order is failedReleased back to available.Read status_reason. Order again, with a new key, only if it still makes sense.
An order is under_reviewHeld.Wait for succeeded or failed. Do not order again meanwhile.
Your webhook endpoint was downUnchanged: an event moves no money.Failed attempts are retried. Catch up from GET /events, and replay a delivery with POST /webhook-deliveries/{id}/replay.
A key may have leakedAt risk while the key works.Revoke it in your dashboard and create another.

After a crash or a restart

  1. A worker that stored each Idempotency-Key before sending can send every unfinished request again, safely.
  2. Compare what you believe with what the API holds: GET /orders by your reference or by status, and the statement, GET /balance/transactions, where each order's reserve, capture or release names its order_id.
  3. Catch up on events from GET /events, which keeps the last 30 days, and ignore any event id you have already handled.

Uncredited GRAM deposits

Read its status and hold_reason on GET /funding/deposits. A deposit that cannot be credited automatically is recorded and reviewed, never lost from the record; see Funding with GRAM. Do not send the amount again to find out.

Common mistakes

  • Treating a timeout as a failure. The order may exist. Retry with the same key, or look it up by reference.
  • Ordering again to find out. A second order is a second payment. Ask about the first one instead.
  • Retrying in a loop. Back off for the seconds in Retry-After.
  • Acting on a test event. Acknowledge it, and change nothing.