Blog / Data engineering

Concurrent API Requests in Python: Asyncio Under Rate Limits

19 Oct 2026Perceptlayer7 min read

To run async API requests in Python under a rate limit, bound two things separately: how many requests are in flight (an asyncio.Semaphore) and how often new ones start (a pacer). The semaphore alone doesn't enforce a rate. This tutorial builds both with httpx, reads the account's limit from response headers, and backs off on 429 using Retry-After.

Why doesn't a semaphore alone respect a rate limit?

A semaphore limits how many coroutines run a block at once, not how many start per second. If each call takes 200 milliseconds and you allow 10 in flight, you start about 50 requests a second, whatever the account allows. Concurrency and rate are different dials, and a rate limit only cares about the second one (asyncio.Semaphore docs).

The relationship is simple: requests per second ≈ in-flight requests ÷ latency. Your latency isn't fixed, so a semaphore sized for today's latency overshoots the limit when the API gets quicker. The pacer makes the rate explicit. The semaphore stays as a safety valve, so a slow stretch can't pile up hundreds of open connections.

This matters most on one endpoint with a batch of queries, such as keyword lists against the Google Search API. Every call is identical in cost, so the only thing that varies is how fast you're allowed to send them.

What are the account's rate limits?

Limits are per account, shared by all its keys and counted per minute. Free accounts get 1 request a second (60 a minute). Paying accounts run at the rate of the credits they're using, and at least 5 a second (300 a minute). Starting a second process with a second key doesn't add capacity: both draw from the same minute.

A faster rate can be chosen at checkout for a higher price, and credits bought at a faster rate are used first. The pricing page has the details, and the rate limits docs describe how the limit is counted.

Every response also carries RateLimit-Limit (requests a minute), RateLimit-Remaining, RateLimit-Reset (seconds until the minute resets) and RateLimit-Policy. The code below reads RateLimit-Limit after each call, so it adapts if the account's rate changes mid-run instead of trusting a constant.

How do you build the fetcher?

The fetcher has three parts: a Pacer that schedules request starts, a fetch coroutine that retries rate-limit responses, and a run_batch function that wires them to one shared httpx.AsyncClient. It targets GET /v1/google-search/search, which takes query as its only required parameter and costs 1 credit per successful call.

Step 1: Install the client and set your key

Install httpx and export your key. Keys start with sk_live_ and go in the X-API-Key header (authentication docs). If you don't have one yet, get a free API key: 1,000 free credits every month after you verify an email address, no credit card.

pip install httpx
export API_KEY=sk_live_your_key_here

Step 2: Write the pacer

The pacer hands out start times. Each caller reserves the next slot under a lock, then sleeps until that slot arrives. sync converts RateLimit-Limit into an interval, with 10% headroom (HEADROOM = 0.9) so jitter at a minute boundary doesn't push you over. pause pushes every future slot back, which is how a 429 slows the whole batch rather than one task.

import asyncio
import os
import time

import httpx

BASE_URL = "https://api.perceptlayer.com"
PATH = "/v1/google-search/search"
MAX_IN_FLIGHT = 10
HEADROOM = 0.9
MAX_ATTEMPTS = 4


class Pacer:
    """Spaces request starts so the account stays under its per-minute limit."""

    def __init__(self, per_second=1.0):
        self.interval = 1 / per_second
        self._next = 0.0
        self._lock = asyncio.Lock()

    def sync(self, limit_per_minute):
        # RateLimit-Limit is requests a minute; keep some headroom under it.
        self.interval = 60 / (limit_per_minute * HEADROOM)

    def pause(self, seconds):
        self._next = max(self._next, time.monotonic() + seconds)

    async def wait(self):
        async with self._lock:
            now = time.monotonic()
            start = max(now, self._next)
            self._next = start + self.interval
        if start > now:
            await asyncio.sleep(start - now)

The pacer starts at 1 request a second, the free rate, and speeds up after the first response reports the real limit. A paying account therefore begins slowly for a handful of requests, then moves to its own rate. Starting low is the safe direction.

Step 3: Fetch with retries

fetch takes a semaphore slot, waits for the pacer, sends the request and decides what the status means. The semaphore is acquired first, so no more than MAX_IN_FLIGHT tasks are ever waiting on the pacer or the network.

async def fetch(client, sem, pacer, halt, query):
    async with sem:
        for attempt in range(1, MAX_ATTEMPTS + 1):
            if halt.is_set():
                return {"query": query, "status": "skipped"}
            await pacer.wait()
            try:
                r = await client.get(PATH, params={"query": query, "hl": "en"})
            except httpx.TimeoutException:
                # The call may have succeeded and been billed. Don't retry blindly.
                return {"query": query, "status": "unknown"}
            if "RateLimit-Limit" in r.headers:
                pacer.sync(int(r.headers["RateLimit-Limit"]))
            if r.status_code == 200:
                return {
                    "query": query,
                    "status": "ok",
                    "credits": int(r.headers.get("X-Credits-Used", 0)),
                    "data": r.json(),
                }
            if r.status_code in (429, 503):
                wait = float(r.headers.get("Retry-After", 1))
                pacer.pause(wait)
                continue
            if r.status_code in (401, 402):
                halt.set()
            return {"query": query, "status": r.status_code}
        return {"query": query, "status": "gave_up"}

Retry-After is a number of seconds on these responses (the header can also carry an HTTP date, per RFC 9110 and the MDN reference, but the API sends seconds). The halt event stops the batch on 401 (invalid_api_key) or 402 (insufficient_credits). Neither fixes itself, and sending the remaining thousands of requests would only produce more errors.

Step 4: Run the batch

run_batch opens one client, so connections are reused, and gathers one fetch per query. A single Pacer and Semaphore are shared by every task.

async def run_batch(queries):
    headers = {"X-API-Key": os.environ["API_KEY"]}
    async with httpx.AsyncClient(
        base_url=BASE_URL, headers=headers, timeout=30
    ) as client:
        sem = asyncio.Semaphore(MAX_IN_FLIGHT)
        pacer = Pacer(per_second=1.0)
        halt = asyncio.Event()
        return await asyncio.gather(
            *(fetch(client, sem, pacer, halt, q) for q in queries)
        )


if __name__ == "__main__":
    queries = [f"best restaurants in {city}" for city in ["Berlin", "Munich", "Hamburg"]]
    started = time.monotonic()
    results = asyncio.run(run_batch(queries))
    for r in results:
        if r["status"] == "ok":
            top = (r["data"].get("organic_results") or [{}])[0].get("title")
            print(f'{r["query"]!r}: {top}')
        else:
            print(f'{r["query"]!r}: {r["status"]}')
    spent = sum(r.get("credits", 0) for r in results)
    print(f"{len(results)} queries, {spent} credits, {time.monotonic() - started:.1f}s")

Save it as fetch_batch.py and run python fetch_batch.py. For lists in the tens of thousands, pass the queries in chunks of a thousand or so: gather creates one task per query up front, and chunking keeps memory flat and gives you a natural place to save progress.

What does the result look like?

Each ok result holds the endpoint's JSON, and the script prints the first organic result's title plus a one-line summary of queries, credits and elapsed time. The credits figure is the sum of X-Credits-Used across successful calls, so it matches what the balance dropped by. For best restaurants in Berlin, the recorded response starts like this:

{
  "search_information": {
    "query_displayed": "best restaurants in Berlin",
    "organic_results_count": 6
  },
  "organic_results": [
    {
      "position": 1,
      "title": "The Best Restaurants in Berlin - Food Guide & Map",
      "link": "https://andershusa.com/best-restaurants-berlin-germany-guide/",
      …
    }
  ],
  …
}

So the first line the script prints for that query is the title of position 1. Failed entries print their status instead: unknown for a timeout, gave_up after four rate-limited attempts, skipped once the batch halted, or the HTTP status for other errors. Rerun only the non-ok queries, and check unknown ones against your balance first (see retrying without paying twice).

The Google Search search endpoint takes many more optional parameters, such as gl, location and device. Add them to the params dict as needed.

How long does a batch take and what does it cost?

Cost is the number of successful calls times 1 credit, and the minimum time is the number of calls divided by your rate. A 429 costs nothing, but each one adds a pause, so real runs finish a little after the minimum. The headroom setting adds roughly 11% to the times below (1 ÷ 0.9).

Credits for those batches: 600 calls × 1 credit = 600 credits, inside the 1,000 free credits a month. 5,000 calls × 1 credit = 5,000 credits, inside the Starter pack's 10,000. 20,000 calls × 1 credit = 20,000 credits, which needs the Build pack's 50,000 or larger. The credits docs explain billing, and pricing shows what each pack costs.

If the same queries repeat across runs, caching saves both credits and time. See caching API responses in Python.

What should the fetcher do on errors?

Treat each status differently: wait and retry on 429 and 503, stop on 401 and 402, and don't retry blindly on a timeout. Failed calls cost nothing and return X-Credits-Used: 0, so retrying a 429 can't double-bill you. A timeout is the exception: there's no idempotency key, so a request whose reply was lost may have succeeded and been billed.

Errors come back as JSON in the form {"error": {"code", "message", "request_id"}}; log the request_id with each failure. The errors docs list every code, and handling web data API errors in Python goes deeper on the non-rate-limit cases.

If you page through results instead of sending many queries, the same pacer works unchanged: see paginating a web data API in Python. For agents that call tools in parallel, MCP server rate limiting covers the same limits from the tool side.

Next step: get a free API key, run the three-city script above, and watch RateLimit-Remaining fall as the batch runs.

pythonasynciorate limitsconcurrencygoogle search