---
title: "Retry Web API Calls Without Paying Twice | Perceptlayer"
description: "Retry only 429, 502 and 503. Failed responses return X-Credits-Used: 0, but a timeout is the gap. The credit arithmetic, a Python loop and a balance check."
url: https://perceptlayer.com/blog/retry-web-api-calls-without-paying-twice
---

[Blog](https://perceptlayer.com/blog) / [Data engineering](https://perceptlayer.com/blog/category/data-engineering)

# Retry Web API Calls Without Paying Twice

31 Jul 2026 Perceptlayer 10 min read

Retry only the statuses that can succeed on a second try: `429`, `502` and `503`. Wait a random time below a doubling cap, give up after a handful of attempts, and fail fast on everything else. On Perceptlayer a failed response costs nothing and returns `X-Credits-Used: 0`, so a retry after an error you received adds nothing to your bill. A timeout is different: you never saw the response, and Perceptlayer requests have no idempotency key, so a retry after a timeout can be a second paid call.

## Key takeaways

-   Retry `rate_limited` (429), `source_error` (502) and `temporarily_unavailable` (503). Do not retry `invalid_api_key` (401), `restricted_endpoint` (403), `not_found` (404) or `invalid_parameters` (422): the same request fails the same way.
-   Treat `insufficient_credits` (402) as a stop signal for the whole job, not a request to retry. The balance only changes when you top up.
-   Add jitter to the wait, a random delay below a doubling cap, so rejected clients don't all retry at the same moment.
-   Every failed response you receive returns `X-Credits-Used: 0`. Ten failed responses followed by one success cost 1 credit, the same as a first-try success. A timeout is the exception to plan for: check `X-Credits-Remaining` between successes.
-   Cap attempts and total wait. A retry loop with no ceiling turns an outage into a hung job.

## Which errors are worth retrying?

Retry an error only if the next attempt could plausibly return a different answer. A rate limit is about request pace, and the `502` and `503` codes signal trouble on the service side, so waiting and trying again is a reasonable bet. That is our recommendation, not a guarantee, which is why the loop caps attempts. A malformed parameter or a revoked key will fail identically every time, so retrying it only wastes time.

Perceptlayer errors come back as an HTTP status plus a JSON body, `{"error": {"code", "message", "request_id"}}`. Branch on the status or the `code`, and keep the `request_id` for any support conversation. The full list is in the [errors docs](https://docs.perceptlayer.com/errors).

| Status | Code | Retry? | Why |
| --- | --- | --- | --- |
| 401 | `invalid_api_key` | No | The key is wrong or revoked. Fix the key. |
| 402 | `insufficient_credits` | No, stop the job | The prepaid balance is empty and nothing more is debited. |
| 403 | `restricted_endpoint` | No | Accept the restricted-endpoint terms in the dashboard first. |
| 404 | `not_found` | No | The path or resource doesn't exist. |
| 422 | `invalid_parameters` | No | Correct the parameters, then send a new request. |
| 429 | `rate_limited` | Yes, with backoff | Slow down and try again. See [rate limits](https://docs.perceptlayer.com/rate-limits). |
| 502 | `source_error` | Yes, with backoff | Our recommendation: a later attempt may get a different answer. Cap the attempts. |
| 503 | `temporarily_unavailable` | Yes, with backoff | The name says the condition is temporary, so a later attempt may succeed. |

Network errors and client timeouts belong in the retry set too, with one caveat covered below: when you never saw a response, the zero-cost rule doesn't tell you what happened.

## Why does a retry not cost you twice here?

A call is debited only when it succeeds. Failed calls cost nothing and return `X-Credits-Used: 0`, and a successful response carries `X-Credits-Used` (what the call cost) and `X-Credits-Remaining` (the balance after it). So when you receive an error response and retry, the cost of the request is the cost of its one success: 1 credit for nearly every endpoint, however many error responses came first. The rule covers responses you actually received. It says nothing about an attempt that timed out.

That changes how you design the loop. On an API that bills per attempt, you'd weigh each retry against its price and keep the retry count low. Here the price of an extra attempt that returns an error is zero credits, so the limits you set are about time, load and your own deadline, not money. The [credits docs](https://docs.perceptlayer.com/credits) describe the rule in full.

### The arithmetic

Say a job fetches 10,000 items at 1 credit each.

-   Credits spent: 10,000 calls × 1 credit = 10,000 credits, the size of one Starter pack. The [pricing page](https://perceptlayer.com/pricing) shows what credits cost.
-   If 800 of those calls hit a `503` first and succeed on the second attempt, you made 10,800 requests and still spent 10,000 credits.

What retries do cost is time and traffic, and time is the budget you tune. With the loop below, the worst case for one call is five attempts. Without a `Retry-After` header that is at most 7.5 seconds of sleeping; when the server sends one, the loop honours it up to the 30-second cap per wait, so four sleeps can reach 120 seconds. Add up to 30 seconds per attempt if each one runs into the client timeout. In the example above, the 800 retried calls add 800 round trips and their waits, not a single credit. Credits stay flat while time grows with the retry count, so you set `max_attempts` and `cap` against your deadline, not your balance.

## How do you retry with exponential backoff and jitter?

Wait longer after each failure and randomise the wait. The ceiling doubles per attempt (0.5 s, 1 s, 2 s, 4 s) and the actual sleep is a random value below it, so clients rejected together don't return together. The pattern is described in AWS's [Exponential Backoff and Jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) (2015). The loop below implements it; if you run several workers against one key it matters, and a single sequential script benefits less.

Pacing beats retrying. Request limits are per account, shared by all its keys, and counted per minute: free accounts get 1 request a second (60 a minute). Every response carries `RateLimit-Remaining` (requests left this minute) and `RateLimit-Reset` (seconds until the minute resets), so a worker can slow down or sleep until the reset before it draws a `429`. See [rate limits](https://docs.perceptlayer.com/rate-limits) for paid rates.

### The request

This article doesn't pick an endpoint for you, because the loop works the same on any of them. Open an API from the [API list](https://perceptlayer.com/apis), choose an endpoint in its playground, fill in its parameters and run a real call with your key in the playground, then copy the endpoint URL for `ENDPOINT_URL`. Send your key in the `X-API-Key` header (see [authentication](https://docs.perceptlayer.com/authentication)):

```bash
curl -i "$ENDPOINT_URL" -H "X-API-Key: $API_KEY"
```

The `-i` flag prints the response headers, so you can see `X-Credits-Used` and `X-Credits-Remaining`. On a failed call you'll see `X-Credits-Used: 0`.

If a one-off shell script is all you need, curl can retry for you:

```bash
curl --retry 5 --retry-max-time 60 -i "$ENDPOINT_URL" -H "X-API-Key: $API_KEY"
```

The [curl manual](https://curl.se/docs/manpage.html#--retry) documents `--retry` and `--retry-max-time`; check it for which failures curl retries. It has no notion of your credit balance, so it won't stop a whole job on a `402`. For anything that runs unattended, write the loop yourself.

### The loop

This Python function retries only the retryable statuses, applies full jitter, and raises immediately on everything else. It also waits for the `Retry-After` header, which Perceptlayer sends with `429` and `503` responses (see the [errors docs](https://docs.perceptlayer.com/errors)). It needs `pip install requests`.

```python
import os
import random
import time

import requests

RETRYABLE = {429, 502, 503}

class ApiError(Exception):
    def __init__(self, status, code, request_id):
        super().__init__(f"{status} {code} (request_id={request_id})")
        self.status = status
        self.code = code
        self.request_id = request_id

def call_with_retry(url, params=None, method="GET",
                    max_attempts=5, base=0.5, cap=30.0, timeout=30):
    headers = {"X-API-Key": os.environ["API_KEY"]}
    last = None

    for attempt in range(max_attempts):
        resp = None
        try:
            resp = requests.request(method, url, params=params,
                                    headers=headers, timeout=timeout)
        except (requests.ConnectionError, requests.Timeout):
            last = ApiError(0, "network_error", None)

        if resp is not None:
            if resp.ok:
                return resp
            try:
                err = resp.json().get("error", {})
            except ValueError:
                err = {}
            last = ApiError(resp.status_code, err.get("code"), err.get("request_id"))
            if resp.status_code not in RETRYABLE:
                raise last  # 401, 402, 403, 404, 422: retrying cannot help

        if attempt == max_attempts - 1:
            break

        wait = random.uniform(0, min(cap, base * 2 ** attempt))
        retry_after = resp.headers.get("Retry-After") if resp is not None else None
        if retry_after and retry_after.isdigit():
            wait = max(wait, min(cap, int(retry_after)))
        time.sleep(wait)

    raise last
```

Three details are worth a second look. The loop raises on a non-retryable status before it sleeps, so a bad key fails in one round trip. The `Retry-After` branch uses the wait Perceptlayer sends with a `429` or `503`, and falls back to jittered backoff when a response has none. This code reads only a plain number of seconds and ignores any other form of the header. And the final `raise last` surfaces the last real error, with its `request_id`, rather than a generic "retries exhausted".

### The parameters

| Parameter | Default | What it controls |
| --- | --- | --- |
| `max_attempts` | `5` | Total tries, including the first. The loop sleeps at most 4 times. |
| `base` | `0.5` | Seconds for the first delay ceiling. Doubles per attempt. |
| `cap` | `30.0` | Upper bound on any single wait, in seconds. |
| `timeout` | `30` | Seconds before a request counts as a network failure. |
| `method` | `"GET"` | Pass the method your endpoint expects. |

With these defaults the delay ceilings are 0.5, 1, 2 and 4 seconds, so a call that never recovers and gets no `Retry-After` header gives up after at most 7.5 seconds of waiting plus the time of the five attempts themselves. If the server sends `Retry-After`, each wait can rise to the 30-second `cap`, which makes the worst case 120 seconds of sleeping.

## What should you do when a call times out?

A timeout means you never saw the response, so you don't know how that attempt ended. The zero-cost rule applies to failed responses you receive. If the request succeeded on the server and the reply was lost, it may have been debited, and Perceptlayer requests have no idempotency key that would stop the retry from being debited again. Retry anyway, with the same cap on attempts, but check the balance rather than assume.

The check uses `X-Credits-Remaining`. Between two consecutive successes from a single sequential worker, the balance should drop by exactly the second call's `X-Credits-Used`. A bigger drop means a debit you didn't see, most likely a timed-out attempt that went through. To cover the first call too, read a starting balance before the loop: error responses carry `X-Credits-Remaining` as well, so a request with no parameters, which the endpoint should reject, gives you the baseline for free.

```python
import os

url = os.environ["ENDPOINT_URL"]
baseline = requests.get(url, headers={"X-API-Key": os.environ["API_KEY"]}, timeout=30)
previous = int(baseline.headers["X-Credits-Remaining"])

spent = 0
for item in items:
    resp = call_with_retry(url, params={"q": item})
    used = int(resp.headers["X-Credits-Used"])
    remaining = int(resp.headers["X-Credits-Remaining"])
    spent += used
    if previous - remaining != used:
        print(f"unexpected debit: balance fell {previous - remaining}, call cost {used}")
    previous = remaining

print(f"spent {spent} credits on confirmed calls, {previous} remaining")
```

This snippet continues from the loop above and reuses its `requests` import. If the endpoint accepts an empty request, the baseline call succeeds and is billed like any other, so pick an endpoint with a required parameter.

Two limits on this check. It only works when nothing else spends from the same key at the same time, so don't run it alongside a second worker or the playground. And it detects extra debits after the fact; it doesn't prevent them. The `params` here are placeholders: use the parameter names the playground shows for your endpoint. Read `X-Credits-Used` from the response, not from a constant, so the check stays right on an endpoint that costs more than 1 credit.

## How do you stop a retry loop from running away?

Bound it three ways: attempts per call, wait per attempt, and failures across the whole job. The first two are the `max_attempts` and `cap` parameters. The third is a circuit breaker. If every call in a window fails with `503`, more retries won't help, and stopping early and resuming later is cheaper than hammering a service that is already down.

A simple version counts consecutive exhausted calls. After a few in a row, pause the job for a minute or two, or exit and let your scheduler start it again. For `402`, don't pause at all: the balance is empty, there is no automatic top-up, and every further call will fail until you add credits. Raise an alert and stop. If the same calls repeat across a job, [caching responses](https://perceptlayer.com/blog/cache-web-data-api-responses) removes some of them before they can fail, and [safe page loops](https://perceptlayer.com/blog/paginate-web-data-api-python) cover the other common long-running pattern. More like this is in the [data engineering category](https://perceptlayer.com/blog/category/data-engineering).

## Can you retry through an AI agent or MCP tool?

Yes, and the cost rule is the same. The `call_endpoint` tool on the MCP server spends the endpoint's credits on success only, and its result reports the credits used and remaining. `search_endpoints` and `get_endpoint` look up the catalogue and cost nothing. An agent that retries a failed `call_endpoint` therefore pays once, on the attempt that works.

Put the retry policy in the tool wrapper or the agent's instructions, not in the model's judgment. Tell it which errors to retry, how many times, and to stop on `insufficient_credits`. A model left to improvise will sometimes retry a `422` with the same parameters. The [MCP docs](https://docs.perceptlayer.com/ai-agents) show how to connect the server.

## Next step

Get a [free API key](https://perceptlayer.com/signup): 1,000 free credits every month after you verify an email address, no credit card. Pick an endpoint in the playground on any [API page](https://perceptlayer.com/apis), paste its URL into the loop above and run it. Check the headers on the first response, then send a request with a deliberately bad parameter and confirm it fails once with `X-Credits-Used: 0` and no retries. What credits cost is on the [pricing page](https://perceptlayer.com/pricing).

## Frequently asked questions

### Does a retried request cost extra credits?

Not after an error response you received. A failed call costs nothing and returns `X-Credits-Used: 0`, and only a successful response debits credits. If a request returns three errors and then succeeds, you pay the endpoint's price once. The exception is a timeout: you never saw the response, and Perceptlayer requests have no idempotency key, so a retry can be a second paid call. Compare `X-Credits-Remaining` between successes to catch it.

### Should I retry a 402 insufficient\_credits response?

No. The service is prepaid, and when the balance runs out calls return `insufficient_credits` (402) with nothing further debited. There is no automatic top-up, so a retry will return the same error until you add credits. Stop the job, raise an alert, and resume after the balance is topped up.

### How many retries are reasonable?

Five attempts with a 30-second cap on any single wait is a sound default for most jobs. Retries that return errors cost no credits, so the limit is about your own deadline and load, though timeouts are worth keeping few. If the same call fails after five tries, the problem is probably not transient, so surface the error with its `request_id` rather than looping longer.

### What should I log when a call fails?

Log the HTTP status, the error `code` and the `request_id` from the JSON error body, plus the attempt number. The `request_id` is what you quote in a support conversation. Leave the response body and your key out of the logs.

### How can I test the loop without spending credits?

Send a request with a deliberately invalid parameter. It returns `invalid_parameters` (422) with `X-Credits-Used: 0`, and the loop should raise on the first attempt without retrying. That confirms the fail-fast path and that a failed call is not debited.

retries backoff rate limits credits error handling

## More from the blog

1.  [Data engineering](https://perceptlayer.com/blog/category/data-engineering) 22 Sept 2026 Perceptlayer 7 min read
    
    ## [Handle Web Data API Errors in Python: 4xx, 5xx and Timeouts](https://perceptlayer.com/blog/handle-web-data-api-errors-python)
    
    Map the API's error codes to retry, fix or stop in Python: 429 and 503 honour Retry-After, 4xx stops, and failed calls cost 0 credits.
    
2.  [Data engineering](https://perceptlayer.com/blog/category/data-engineering) 19 Oct 2026 Perceptlayer 7 min read
    
    ## [Concurrent API Requests in Python: Asyncio Under Rate Limits](https://perceptlayer.com/blog/concurrent-api-requests-python-rate-limit)
    
    A Python asyncio fetcher that paces calls to your per-second limit, honours Retry-After and prices a 20,000-call batch at 20,000 credits.
    
3.  [Data engineering](https://perceptlayer.com/blog/category/data-engineering) 7 Oct 2026 Perceptlayer 10 min read
    
    ## [Scheduling Web Data Pulls: Cron, Queues or Workflows](https://perceptlayer.com/blog/schedule-web-data-pulls-cron-queue)
    
    Cron, a job queue or a workflow engine for recurring API pulls? Compare failure handling and credit cost, with a verdict per team size and a cron script.