Grocery price scraping proxies for regional price indexes and category data

Quick answer

Grocery price scraping proxies have to resolve to the region whose store you are pricing, because supermarket sites assign a store by location and serve that store's shelf price. Geonode residential proxies give you country, state and city targeting with sticky sessions that hold the store selection, and the Scraper API renders JavaScript-heavy product grids when you prefer structured output to raw HTML.

Why grocery price tracking breaks without the right proxies

Supermarket sites are store-scoped. The first visit sets a store from the postcode or the IP's location, and every price, promotion and stock flag after that belongs to it. A crawler from a datacenter range in another region gets a default flagship store or a store picker, and your index prices the wrong shelf.

Volume is the second constraint. A single chain can have tens of thousands of SKUs, and category pages load through APIs with per-IP rate limits. Burst through them and you get HTTP 429, throttled responses that silently drop items, or a CAPTCHA challenge that your parser reads as an empty category.

Session consistency matters more here than on most targets. The store cookie, delivery slot and sometimes a loyalty-pricing flag live in the session. If the IP changes mid-crawl, the site may reset the store and your dataset mixes two regions without any marker. A price index is only useful if the region is stable over time.

Which Geonode proxy type fits

The recommended product is Residential proxies, paired with the Scraper API for rendered pages. The residential pool has 2.5M IPs online and 30–50M unique IPs rotating monthly across 195+ countries, with country, state and city targeting. For a national grocery index you run one sticky session per region, set the store once and crawl the categories under the same IP for up to 24 hours.

The Scraper API adds JavaScript rendering and returns HTML, Markdown or JSON. Its free tier of 1.5K requests per month is enough to check that a chain's product grid parses cleanly before committing to volume.

ISP proxies fit a fixed panel of stores you reprice daily. A static IP from a real ISP at $1.25 per IP with unlimited bandwidth behaves like one shopper returning every morning, which suits a time series. Datacenter proxies at $0.14/GB work for national, non-store-scoped pricing and product metadata such as images and nutrition labels.

Residential — Best for: Regional store prices, promos, stock across many areas; IP source: Real household devices, 195+ countries; Session: Rotating or sticky up to 24 h; Price from: $0.27/GB at volume
ISP (static residential) — Best for: Fixed store panel repriced daily; IP source: Static IPs from real ISPs; Session: Unlimited, same IP; Price from: $1.25/IP
Datacenter — Best for: National pricing, product metadata, images; IP source: Rotating datacenter ranges; Session: Rotating; Price from: $0.14/GB

How to set it up

Build the SKU panel per chain and region. Store the product URL or internal id, the region, a representative postcode and the store id the site assigns to it.
Create residential credentials, then generate one username per region with country and state or city targeting and a session id, so each region keeps a stable IP.
Start each run by setting the store through the site's own store selector, then crawl category and product pages under the same session so the store cookie persists.
Extract shelf price, unit price per kilogram or litre, promotion type, loyalty price, pack size and stock status into a normalised row with chain, store id, region and timestamp.
Schedule daily runs, retry 429 and truncated category pages with back-off, and validate each run by checking that the store id in responses matches the one you set.

# verify endpoint against docs.geonode.com before publishing
import requests

USER = "USERNAME-country-gb-session-mcr-day01"
PASS = "PASSWORD"
proxy = f"http://{USER}:{PASS}@premium-residential.geonode.com:9000"
proxies = {"http": proxy, "https": proxy}

with requests.Session() as s:
    s.proxies.update(proxies)
    s.get("https://example-grocer.com/store-locator?postcode=M11AA", timeout=30)
    r = s.get("https://example-grocer.com/category/dairy?page=1", timeout=30)
    print(r.status_code, r.headers.get("content-type"))

Get the exact snippet for your language from the API Code Generator in the dashboard.

Grocery price tracking-specific settings that matter

One sticky session per region

Store selection lives in the session, so keep one session id for the whole regional crawl and never share it across regions. Rotate to a new session id on the next scheduled run. If the IP changes early because the device went offline, re-run the store selector before continuing.

Region targeting that matches the store

Choose the targeting level that maps to how the chain assigns stores. Some chains use the postcode you enter and only need country targeting; others geolocate the IP and need city targeting. Remember that state and city cannot be combined, so pick the one the site actually uses.

Concurrency across sessions, not within

There is no fixed thread limit on Geonode, but grocery sites rate-limit per IP. Run many regions in parallel, each on its own session, and keep per-session request rates low. Add exponential back-off on HTTP 429 and treat a category page with far fewer items than yesterday as a throttled response.

What to log

Log session id, requested region, the store id returned by the site, status code, item count per category and whether a CAPTCHA appeared. A store id mismatch is the earliest sign of a session reset.

What Geonode gives you for grocery price tracking

Country, state and city targeting across 195+ countries to match store assignment.
Sticky sessions up to 24 hours so the selected store stays fixed per run.
Scraper API with JavaScript rendering returning HTML, Markdown or JSON.
Pay-as-you-go residential traffic from $0.27/GB at volume, no contract.
24/7 support and payment by card or crypto.

Pricing for grocery price tracking

Residential traffic is pay-as-you-go. Entry is 10 GB for $7.92 per month ($0.792/GB), 50 GB costs $34.65, and the rate reaches $0.27/GB at volume. A 3-day trial is $5, with no contract. ISP proxies start at $1.25 per IP, and rotating datacenter traffic starts at $0.14/GB. The Scraper API is free for 1.5K requests per month, then from $3.50 per 10K requests. See the pricing page for volume tiers.

FAQ

Why do grocery price scraping proxies need to be residential?

Supermarket sites assign a store from the IP location or postcode and serve that store's prices. Residential IPs resolve to real regions and hold the store across a session, while datacenter ranges are often assigned a default store or asked to pick one manually.

How do I keep a price index consistent over months?

Fix the region, store id and SKU panel, and use the same session structure for every run. Log the store id the site returns on each visit and reject runs where it differs, so a session reset never leaks another region's prices into the series.

Can grocery price scraping proxies handle thousands of SKUs per day?

Yes. There is no fixed thread limit, so you scale by running many regional sessions in parallel at a modest per-session rate. Bandwidth is the cost driver, so request JSON endpoints where available and avoid downloading images.

When is the Scraper API a better fit than proxies?

When product grids render client-side or you want structured JSON instead of HTML. The Scraper API renders JavaScript, offers 1.5K free requests per month, and its unlimited plans run from $47 with 2 threads to $1,799 with 100 threads.

Is grocery price tracking compliant?

Collect only publicly displayed prices and respect each retailer's terms, robots policies and rate limits. Use accounts only where you own them or have authorisation. Geonode's acceptable use policy prohibits fraud, spam and credential abuse, and compliance remains your responsibility.

How do I detect that a site changed my store mid-crawl?

Read the store id or store name from each response and compare it with the one you set. Also watch for a sudden change in delivery slot text or currency. On mismatch, discard the batch, start a new session id and re-run the selector.

Start

Price one chain in one region on the 3-day trial, confirm store ids match, then add regions. Get started with pay-as-you-go traffic or Contact sales for national index volumes.