Blog / Web data by industry

Job Posting Data for Labour-Market Research

21 Oct 2026Perceptlayer8 min read

Job postings show what employers want as soon as they advertise a role: titles, locations and dates are public at once, with no survey cycle to wait for. Three fields carry most of the trend analysis: title, location and date_published. Postings measure demand, not hiring, so treat every series as an index of advertised openings. One call to the Indeed jobs endpoint returns these fields as JSON for 1 credit, and a repeatable weekly collection fits in a few thousand credits a month.

Which posting fields matter for trend analysis?

Five fields do most of the work: title, location, date_published, id and company.name. Title tells you the role, location tells you where, the date tells you when it was advertised, the id lets you count each posting once, and the company lets you see whether a few employers drive a trend. Everything else is useful context but rarely the unit you count.

The table below maps fields from the recorded response to the job they do.

Be careful with attributes. In the recorded sample it mixes employment type ("Full-time") with skill-like tags ("UI design", "Go", "Collaboration with Sales"), so it's a signal to test, not a clean taxonomy. For employment type, filter on the job_type request parameter instead of parsing tags.

What about salary?

Pay is the field researchers ask for most, and the recorded response doesn't contain a salary field. The attributes list in the sample holds no pay figures either. Run your own queries, inspect the JSON, and count how many postings carry any pay information before you build a series on it. If coverage is thin or uneven by country, say so in your write-up rather than imputing values.

How do you pull the data?

You send GET /v1/indeed/jobs with a required query and optional filters for place, type and ordering. For a time series, set sort to date and spot-check the ordering on your own queries, then follow cursor until has_more is false or you hit your page budget. Each successful call costs 1 credit, and failed calls cost nothing.

curl "https://api.perceptlayer.com/v1/indeed/jobs?query=software+engineer&location=Berlin&country=DE&job_type=full_time&limit=20" \
  -H "X-API-Key: $API_KEY"

The recorded response for that request returns a jobs array and a pagination object. An excerpt:

{
  "success": true,
  "data": {
    "jobs": [
      {
        "id": "805a70464f904dce",
        "title": "Senior Product Manager (Enterprise)",
        "company": { "name": "n8n", "profile_url": "/cmp/N8n" },
        "location": { "city": "Berlin", "country": "DE", "is_remote": true, … },
        "attributes": ["Collaboration with Sales", "Go"],
        "date_published": "2026-08-12 05:00:00",
        …
      }
    ],
    "pagination": { "next_cursor": "ABQAAYEAFAAAAAAAAAAAAAAAAmYRNdcAMgEBAQcpfSGrk7wi9QBuLZ6BEihQwmDA0xA2nDAGrqtuJIdLnPZKyLE0IcnT2+IETfyRAAA=", "has_more": true },
    "metadata": { "total_results": 20, "timestamp": "2026-10-21T13:25:37.233649Z" }
  }
}

For the endpoint basics, see Indeed Jobs API: Get Job Postings as JSON. The Indeed API: Jobs page lists the endpoint, and the playground on the Indeed API page runs a real call with your key.

Do job postings measure hiring?

No. A posting records that an employer advertised a role, not that anyone was hired. The US Bureau of Labor Statistics counts both separately in its Job Openings and Labor Turnover Survey: in August 2026 it reported 7.1 million job openings and 5.2 million hires (BLS, JOLTS news release, 2026). The gap between the two is the first thing to keep in mind when reading a postings chart.

Several biases follow from that, and each changes how you should read a trend.

• Duplicates and reposts. The same role can appear more than once. Count by id, and consider also collapsing on company, normalised title and city.
• Ghost and evergreen postings. Some listings stay up with no intent to fill them soon. A rising count can mean more advertising, not more demand.
• Channel coverage. One site's postings over-represent the employers and occupations that use it. Small firms and informal hiring are under-counted.
• Multiple hires per posting. One posting can fill several seats, and many hires never involve a posting.
• Query shape. Keyword search returns what matches the words, so a title that changes name drops out of your series.

None of this makes postings useless. It means the right claim is "advertised demand for X rose" and the right check is a comparison with an official series. JOLTS draws on a survey of about 21,000 establishments (BLS, JOLTS Handbook of Methods, 2026), so it's a sound benchmark for direction, though not for individual roles or cities. JOLTS covers the US only, so it can't check a Berlin panel. For Germany and other EU countries, Eurostat publishes quarterly job vacancy statistics collected from national statistical authorities, and reported that 2.0% of jobs in the EU were vacant in Q2 2026 (Eurostat, Job vacancy statistics, 2026). Compare the direction of your index with that series, not the level.

How do you build a repeatable collection plan?

Fix a panel of queries and run it on a schedule that never changes. A panel is a list of title and location pairs, such as 25 roles across 10 cities. Each run pages through a set number of calls per pair, saves the raw JSON with a timestamp, and deduplicates by id at analysis time. Consistency matters more than breadth, because trend breaks come from changing the panel.

Here's a minimal Python collector. It saves each page as a JSON line and stops after a fixed page budget.

import json, os, time, datetime, requests

API = "https://api.perceptlayer.com/v1/indeed/jobs"
HEADERS = {"X-API-Key": os.environ["API_KEY"]}
PAIRS = [("software engineer", "Berlin", "DE"), ("data analyst", "Berlin", "DE")]
MAX_PAGES = 3

def collect(query, location, country, out):
    cursor = None
    for _ in range(MAX_PAGES):
        params = {"query": query, "location": location, "country": country,
                  "sort": "date", "limit": 20}
        if cursor:
            params["cursor"] = cursor
        r = requests.get(API, headers=HEADERS, params=params, timeout=30)
        if r.status_code in (429, 503):
            time.sleep(int(r.headers.get("Retry-After", "5")))
            continue
        r.raise_for_status()
        data = r.json()["data"]
        out.write(json.dumps({
            "collected_at": datetime.datetime.utcnow().isoformat(),
            "query": query, "location": location, "jobs": data["jobs"],
        }) + "\n")
        if not data["pagination"]["has_more"]:
            break
        cursor = data["pagination"]["next_cursor"]

with open(f"postings-{datetime.date.today()}.jsonl", "a") as out:
    for q, loc, c in PAIRS:
        collect(q, loc, c, out)

A client timeout may hide a call that succeeded and was billed, as Retry Web API Calls Without Paying Twice explains. For the loop itself, Paginate a Web Data API in Python covers safe patterns. Rate limits and Retry-After are in the rate limits docs.

How do you turn the saved pages into an index?

Read every saved line, keep one row per id, count postings per week of date_published, then divide each week by a base week and multiply by 100. This sketch reads the files the collector wrote and assumes one fixed panel. It uses the earliest week as the base, so choose the base week before you look at the results.

import glob, json, datetime
from collections import Counter

seen = {}
for path in sorted(glob.glob("postings-*.jsonl")):
    for line in open(path):
        for job in json.loads(line)["jobs"]:
            seen.setdefault(job["id"], job)  # one row per posting id

weekly = Counter()
for job in seen.values():
    day = datetime.datetime.strptime(job["date_published"][:10], "%Y-%m-%d").date()
    year, week, _ = day.isocalendar()
    weekly[(year, week)] += 1

weeks = sorted(weekly)
base = weekly[weeks[0]]
for w in weeks:
    print(w, weekly[w], round(100 * weekly[w] / base, 1))

Each printed row is the ISO week, the unique postings and the index, where the base week is 100. Weeks near your first collection date are incomplete, because older postings that were already taken down never appear. Start the index a few weeks after the first run, and report the index with the panel and the base week stated.

What does the plan cost?

Cost is calls times 1 credit. Take 25 roles across 10 cities, which is 250 pairs. At up to 3 pages each, that's 250 × 3 = 750 calls a week, or 750 credits. Over four weeks that's 750 × 4 × 1 credit = 3,000 credits a month.

The last row fits inside the 1,000 free credits you get each month after verifying an email, with no card. The Starter pack holds 10,000 credits, so the one-run-a-week panel (3,000 credits a month) uses it up in just over three months, and purchased credits last 12 months. The Build pack holds 50,000 credits: at 6,000 credits a month, the two-run panel lasts 8 months. See pricing for what credits cost and Credits for how balances work.

When is another source a better fit?

If you need hires, wages or national benchmarks, use official statistics: they're built for that, and postings can't replace them. Postings fit when you need speed, fine geography, or specific titles and skills that official categories don't carry. Use both, and report the postings series as an index with a clear base period.

For other industries' fields, see Real Estate Listing Data: Fields That Matter for Valuation Search Trend Data for Demand Forecasting, and the Web data category. For pay data from another source, see Glassdoor API: Company Reviews and Salaries as JSON. To start collecting, get a free API key and run your first panel query in the playground.

jobslabour marketindeedjob postingstrend analysis