Classifieds scraping proxies for multi-category listing aggregation

Quick answer

Classifieds scraping proxies need to be residential with city or state targeting, because classifieds sites rank, filter and sometimes hide listings by the visitor's location and apply strict per-IP limits on search and detail pages. Geonode residential proxies cover 195+ countries with sticky sessions up to 24 hours, so an aggregator can crawl vehicles, jobs, housing and goods categories per region with one consistent method.

Why classifieds aggregation breaks without the right proxies

Classifieds platforms are local by design. Search results default to the visitor's region, distance filters are computed from the IP, and categories such as vehicles and housing are listed within a radius. A crawler from a datacenter range in another country gets a default region, a country landing page or an empty result set.

Scale is the second problem. A multi-category aggregator touches search pages for every category and region, then detail pages for every new listing, which adds up to large daily volumes. Classifieds sites respond to repeated hits from one IP with HTTP 429, CAPTCHA challenges, degraded search pages with fewer results or temporary blocks on the whole range.

Freshness makes it harder. Listings expire, get edited and are reposted within hours, so an aggregator must revisit often. Without a large rotating pool and stable regional sessions, the revisit rate needed for accuracy is exactly the pattern that trips rate limits.

Which Geonode proxy type fits

The recommended product is Residential proxies. The pool has 2.5M IPs online and 30–50M unique IPs rotating monthly across 195+ countries, with country, state, city and ASN targeting. That lets one pipeline crawl the vehicle category in one state, the housing category in a city and the jobs category nationally, each from an IP that matches the region the platform expects.

Rotating sessions suit search-result discovery, where every request can come from a fresh IP. Sticky sessions, which hold an IP for up to 24 hours, suit detail-page crawls where the platform ties listing visibility or contact details to a consistent session and region.

ISP proxies at $1.25 per IP fit a fixed set of regions you re-check continuously, such as a watchlist of high-value vehicle or housing listings, with unlimited bandwidth per IP.

Datacenter proxies at $0.14/GB suit sitemaps, taxonomies and image downloads that are not region-dependent. The Scraper API complements the proxies when listing pages render client-side, returning HTML, Markdown or JSON.

Residential — Best for: Regional search and detail pages across all categories; IP source: Real household devices, 195+ countries; Session: Rotating or sticky up to 24 h; Price from: $0.27/GB at volume
ISP (static residential) — Best for: Continuous re-checks of a fixed regional watchlist; IP source: Static IPs from real ISPs; Session: Unlimited, same IP; Price from: $1.25/IP
Datacenter — Best for: Sitemaps, taxonomies, image assets; IP source: Rotating datacenter ranges; Session: Rotating; Price from: $0.14/GB

How to set it up

Map each platform's category tree to your own schema: vehicles, jobs, housing, goods and services, with the region granularity each category uses.
Create residential credentials and generate usernames per region with country and state or city targeting. Use rotating sessions for search pages and sticky sessions for detail pages.
Run discovery crawls on search pages per category and region, collecting listing ids, titles, prices and posted dates, and queue only new or changed ids for detail fetches.
Fetch detail pages through sticky sessions, extract attributes per category, such as mileage, salary range, bedrooms or condition, and store them with region and timestamp.
Re-check listings on a per-category schedule, since goods churn fastest and jobs slowest, mark expired ids, retry 429 and degraded pages with back-off, and log the reported region.

# verify endpoint against docs.geonode.com before publishing
import requests

USER_SEARCH = "USERNAME-country-us-state-texas"           # rotating
USER_DETAIL = "USERNAME-country-us-state-texas-session-tx01"  # sticky
PASS = "PASSWORD"

def proxies(user):
    url = f"http://{user}:{PASS}@premium-residential.geonode.com:9000"
    return {"http": url, "https": url}

search = requests.get(
    "https://example-classifieds.com/tx/cars?sort=newest",
    proxies=proxies(USER_SEARCH), timeout=30,
)
detail = requests.get(
    "https://example-classifieds.com/listing/987654",
    proxies=proxies(USER_DETAIL), timeout=30,
)
print(search.status_code, detail.status_code)

Get the exact snippet for your language from the API Code Generator in the dashboard.

Classifieds-specific settings that matter

Region granularity per category

Vehicles and housing are usually searched within a radius, jobs by metro and goods by city or state. Match the targeting level to the category and remember that state and city targeting cannot be combined. Verify the region the platform reports in its own header or filter bar before trusting a result set.

Rotating for discovery, sticky for details

Search pages benefit from rotation, so each page of results comes from a fresh IP and no single address repeats a query pattern. Detail pages, contact buttons and image galleries often depend on a consistent session, so fetch them through a sticky session tied to the same region.

Concurrency and back-off

Geonode sets no fixed thread limit, but classifieds sites throttle per IP quickly. Scale by adding regional sessions, keep per-session rates modest, and back off exponentially on HTTP 429, CAPTCHA pages and search pages that return fewer listings than the stated count.

What to log

Log platform, category, region targeted, region reported, session id, status code, listing count per search page and detail-page completeness. Region mismatches and count shortfalls are the earliest signals of a degraded response.

What Geonode gives you for classifieds data

Country, state, city and ASN targeting across 195+ countries for region-accurate listings.
2.5M residential IPs online with 30–50M unique IPs rotating monthly for high revisit rates.
Sticky sessions up to 24 hours for detail pages tied to a consistent region.
Scraper API with JavaScript rendering returning HTML, Markdown or JSON.
Pay-as-you-go residential traffic from $0.27/GB at volume, no contract.

Pricing for classifieds data

Residential traffic is pay-as-you-go with no contract. Entry is 10 GB for $7.92 per month ($0.792/GB), 50 GB costs $34.65, and the rate falls to $0.27/GB at volume, with a 3-day trial for $5. Accounts above 500 GB per month qualify for the "1TB on us" program. ISP proxies start at $1.25 per IP, and rotating datacenter traffic starts at $0.14/GB for non-regional assets. Full tiers are on the pricing page.

FAQ

How is this different from real estate or automotive data pages?

Those pages cover one vertical in depth, with portal-specific fields and valuation use cases. Classifieds aggregation spans every category on general-purpose listing platforms, so the pipeline is built around a shared listing schema, per-category region rules and churn-based revisit schedules.

Why do classifieds scraping proxies need city or state targeting?

Classifieds platforms default search results to the visitor's region and compute distance filters from the IP. Without matching targeting you get another region's listings or an empty set. City or state residential IPs reproduce what a local visitor sees.

How do classifieds scraping proxies handle the volume of detail pages?

Rotation gives access to the full pool, so you scale by running more regional sessions at a modest rate each. Queue only new or changed listing ids for detail fetches, and request JSON endpoints where available to keep bandwidth and cost down.

Is aggregating classifieds listings compliant?

Collect only publicly visible listings, respect platform terms, robots policies and rate limits, and do not collect seller contact details beyond what the platform publishes or use accounts you do not own or manage with authorisation. Geonode's acceptable use policy prohibits fraud, spam and credential abuse.

Should I use the Scraper API for classifieds pages?

Use it when listing pages render client-side or when you want Markdown or JSON output instead of HTML. It includes JavaScript rendering and 1.5K free requests per month. Many aggregators use proxies for search pages and the Scraper API for rendered detail pages.

How do I keep listing status accurate?

Set revisit intervals per category based on observed churn, re-check listings near their typical expiry age first and mark ids that return a removed page. Log the platform's own posted and updated dates so your freshness metrics are measured against the source.

Start

Aggregate one category in one region on the 3-day trial, verify the reported region, then expand. Get started with pay-as-you-go traffic or Contact sales for aggregator volumes above 500 GB per month.