Job Listings Scraping Proxies: Postings as Market Data

Quick answer

For job listings you need residential proxies with country and city targeting, because job boards rank and filter postings by the visitor's location and throttle datacenter ranges quickly. Geonode's residential pool (2.5M IPs online, 195+ countries) returns the local result set, and the Unlimited Scraper has ready targets for Indeed, Glassdoor and LinkedIn so you receive postings as JSON instead of HTML.

Why job listings scraping breaks without the right proxies

Job postings are a strong public signal: who is hiring, for what, where and at what salary band. Collecting them across boards, aggregators and career pages at analytical scale runs into protections designed for exactly this traffic pattern.

Large boards apply aggressive rate limiting to repeated searches from one IP, answering with HTTP 429, then CAPTCHAs, then shadow-limited result pages that return a handful of listings instead of hundreds. Results are also region-locked: a search for "data engineer" from a US server returns US-weighted listings even when you asked for Manchester, and salary fields are shown or hidden per country. Professional networks serve a login wall to unknown visitors instead of the public posting.

Pagination depends on session state. If page 3 arrives from a different IP than page 1, boards often reset the search and hand you page 1 again, inflating duplicates and hiding the long tail of postings.

Which Geonode proxy type fits

For job listings we recommend residential proxies with the Unlimited Scraper for the major boards. Residential IPs from 195+ countries with country, state and city targeting let you run a Manchester search from Manchester and a Toronto search from Toronto, so the ranking, salary display and local-language postings match what a candidate sees. Sticky sessions up to 24 hours hold paginated searches together.

The Unlimited Scraper has ready targets for Indeed, Glassdoor and LinkedIn public pages, renders JavaScript and returns JSON, which removes parser maintenance for the sources that change layout most often. ISP proxies suit a fixed set of company career pages you poll daily from one stable identity. Datacenter proxies from $0.14/GB handle open career sites and applicant-tracking-system feeds that do not challenge visitors.

Residential — Best for: Job boards, aggregators, regional and city-level searches; IP source: Real consumer devices, 195+ countries; Session: Rotating or sticky up to 24 h; Price from: $0.27/GB at volume
ISP (static residential) — Best for: Daily polls of a fixed set of career pages; IP source: Static IPs from real ISPs; Session: Unlimited sessions, unlimited bandwidth per IP; Price from: $1.25/IP
Datacenter — Best for: Open career sites, ATS feeds, sitemaps; IP source: Rotating datacenter ranges; Session: Rotating; Price from: $0.14/GB

How to set it up

Define the dataset: titles or keyword families, locations at the level you analyse (country, metro, city), and the fields you keep (title, company, location, posted date, salary band if public, description text).
Route the major boards through the Unlimited Scraper ready targets for Indeed, Glassdoor and LinkedIn. Use residential proxies with your own parser for aggregators and company career pages.
Create a residential proxy user per market with -country- and -city- targeting, and a sticky session per search query so pagination stays consistent to the last page.
Run searches on the cadence postings change, usually daily, with low concurrency per board. Log source, query, region, timestamp and HTTP status for every page.
Deduplicate on company plus title plus location plus posted date, normalise salary and location fields, and store the first-seen and last-seen dates to measure time-to-fill.

# verify endpoint against docs.geonode.com before publishing
import requests

USER = "USERNAME-country-gb-city-manchester-session-jobs-mcr-01"
proxy = f"http://{USER}:PASSWORD@premium-residential.geonode.com:9000"
proxies = {"http": proxy, "https": proxy}

for page in range(1, 11):
    url = f"https://www.example-jobboard.co.uk/jobs?q=data+engineer&l=Manchester&page={page}"
    r = requests.get(url, proxies=proxies, timeout=30)
    print(page, r.status_code, len(r.text))
    if r.status_code != 200:
        break

Get the exact snippet for your language from the API Code Generator in the dashboard.

Job-listings-specific settings that matter

City targeting for local labour markets

Boards rank postings by the visitor's location and show salary differently per country. Use -country- plus -city- so each metro's search runs from that metro. State and city cannot be combined in one targeting string, so pick the level that matches your analysis unit.

Sticky sessions through the last page

Hold one session id for a full search from page 1 to the end, then rotate for the next query. This keeps the board's search state intact and prevents the page-1-again problem that inflates duplicates and truncates the long tail.

Pacing per board

Keep per-board concurrency low and add random delays between pages. Geonode has no fixed thread limit, so scale by running many queries across boards in parallel rather than many pages of one query at once. On 429 or CAPTCHA, back off exponentially and resume on a new session.

First-seen and last-seen tracking

Time-to-fill and hiring velocity come from when a posting appeared and disappeared, not from its content. Re-run each query daily, record first-seen and last-seen dates per deduplicated posting, and treat a missing posting as closed only after two consecutive absences.

What to log for compliance

Store the source URL, query, proxy region, timestamp and the fields extracted for every posting. This proves the data was public at collection time and lets you remove records on request. Never collect applicant data, recruiter contact details behind a login, or anything that requires an account.

What Geonode gives you for job listings data

2.5M residential IPs online, 30–50M unique IPs monthly, 195+ countries
Country, state and city targeting for metro-level labour-market searches
Sticky sessions up to 24 hours for consistent pagination
Unlimited Scraper ready targets for Indeed, Glassdoor and LinkedIn with JSON output
Scraper API free tier of 1.5K requests per month to validate parsing

Pricing for job listings data

Posting pages are text-heavy, so residential bandwidth stretches far: pay-as-you-go from $0.27/GB at volume, a 10 GB entry plan at $7.92 per month and a 3-day trial for $5, no contract. The Scraper API is free up to 1.5K requests per month, then from $3.50 per 10K requests, with monthly unlimited plans from $47 for continuous board coverage. ISP proxies start at $1.25/IP and datacenter at $0.14/GB. See the pricing page.

FAQ

Why do I need job listings scraping proxies instead of a board's API?

Most boards offer no analytics-grade API, and the ones that exist cover their own postings only. Proxies let you collect public postings across boards, aggregators and career pages from the right region, which is what labour-market and competitor-hiring analysis needs.

Can job listings scraping proxies return the same results a local candidate sees?

Yes. Residential proxies with country and city targeting run the search from inside the metro, so ranking, local-language postings and salary visibility match the candidate view. Sticky sessions keep pagination consistent to the final page.

Does Geonode support Indeed, Glassdoor and LinkedIn?

The Unlimited Scraper has ready targets for Indeed, Glassdoor and LinkedIn public pages and returns postings as JSON. Collect only public postings, stay within each platform's terms, and do not attempt to access logged-in or applicant-facing content.

What are the limits and compliance rules for job listings data?

Collect only publicly available postings, honour GDPR, CCPA and platform terms, and never collect credentials, applicant data or private content. Use proxies only on accounts you own or manage with authorisation and respect rate limits. Geonode's AUP prohibits fraud, spam and credential abuse.

How do I measure competitor hiring signals from postings?

Track first-seen and last-seen dates per deduplicated posting for each competitor, then compute open roles, new roles per week and time-to-fill by function and location. Sudden growth in one function or city is the signal.

How much bandwidth does a daily job listings crawl use?

Posting pages are small text documents, so tens of thousands of pages per day usually fit in a few gigabytes. Block images and trackers, measure consumption during the $5 trial, and move to a larger residential plan when volumes settle.

Start

Turn public job postings into labour-market and hiring-signal datasets with region-accurate collection. Get started with a 3-day trial for $5, or contact sales for multi-country coverage.