Web Archiving Proxies: Record Pages as a Region Saw Them

Quick answer

For web archiving that has to stand up to a regulator or a court, you need residential proxies with country targeting so each capture reflects what a visitor in that region actually saw, plus a capture method that stores the page faithfully. Geonode residential IPs cover 195+ countries, and the Scraper API returns rendered HTML or Markdown snapshots with JavaScript rendering built in.

Why web archiving breaks without the right proxies

A page is not one document. Pricing, disclaimers, cookie banners, promotional terms and product availability change by the visitor's country, and often by state within it. An archive taken from a single datacenter location records one version and silently misses the one a customer in another market was shown, which is exactly the version a complaint will be about.

Public archive services do not solve this. They crawl from fixed locations, skip pages that block crawlers, and have no notion of a regional variant. For regulatory record-keeping you also need control over timing, frequency and retention, which a third-party archive cannot promise.

Access is the next issue. Sites serve datacenter ranges a stripped page, a consent interstitial or an HTTP 403, and repeated captures from one IP trigger HTTP 429 or CAPTCHA responses. A capture that records a block page instead of the content is worse than no capture, because it looks complete in the index.

Lastly, modern pages render client-side. Saving the raw HTML response misses content that appears only after JavaScript runs, so your archive shows an empty shell where the disputed text used to be.

Which Geonode proxy type fits

Residential proxies fit web archiving proxies because they give the archive a real visitor's viewpoint from the chosen country or city. Geonode has 2.5M IPs online across 195+ countries with country, state, city, ASN and OS targeting, and sticky sessions up to 24 hours so a multi-page capture of a site stays on one IP.

Pair them with the Scraper API when pages depend on JavaScript. It renders the page and returns HTML, Markdown or JSON, which gives you both a faithful render and a readable text version for the record. The crawl endpoint can snapshot a whole section of a site in one job.

ISP proxies suit scheduled captures of your own properties from a fixed identity, where a stable IP makes it easy to whitelist the archiver in your own analytics. Datacenter proxies are fine for archiving sources that do not vary by region and accept datacenter traffic, such as public filings or documentation sites.

Residential — Best for: Region-accurate captures of third-party and own pages; IP source: Real consumer devices, 195+ countries; Session: Rotating or sticky up to 24 h; Price from: $0.27/GB at volume, 10 GB for $7.92
ISP (static residential) — Best for: Scheduled captures of your own properties from a stable IP; IP source: Static IPs from real ISPs; Session: Unlimited sessions, unlimited bandwidth per IP; Price from: $1.25/IP
Datacenter — Best for: Non-regional sources such as filings and documentation; IP source: Datacenter ranges, rotating; Session: Rotating; Price from: $0.14/GB

How to set it up

Define the capture register: URL, reason for archiving, regions required, frequency, retention period and the regulation or policy that drives it. This is what an auditor will ask for first.
Decide the capture format per page. Store raw HTML and headers for every capture, add a rendered HTML or Markdown snapshot via the Scraper API for JavaScript-heavy pages, and a screenshot where layout matters.
Run each capture through a residential IP in the required region using a sticky session per site visit, so the cookie banner, consent state and any follow-on pages share one identity.
Hash every file, record the UTC timestamp, IP country, session id and response headers, and write everything to immutable storage with the register entry it belongs to.
Verify captures automatically: flag any file that contains a consent interstitial, a block page or far fewer bytes than the previous capture, and re-run it from a fresh session.

# verify endpoint against docs.geonode.com before publishing
# Regional capture from Germany with headers, then a rendered snapshot via the Scraper API
SESSION=reg-de-$(date -u +%Y%m%d)
PROXY="http://USERNAME-country-de-session-${SESSION}:PASSWORD@premium-residential.geonode.com:9000"
URL="https://www.example-retailer.com/terms"
STAMP=$(date -u +%Y%m%dT%H%M%SZ)

curl -sS --proxy "$PROXY" -L \
  -H "Accept-Language: de-DE,de;q=0.9" \
  -D "terms_de_${STAMP}.headers" \
  -o "terms_de_${STAMP}.raw.html" "$URL"

# Rendered snapshot (HTML or Markdown) via Scraper API extract endpoint
curl -sS -X POST "https://api.geonode.com/scraper/extract" \
  -H "X-Api-Key: YOUR_API_KEY" -H "Content-Type: application/json" \
  -d "{\"url\":\"$URL\",\"format\":\"markdown\",\"country\":\"de\"}" \
  -o "terms_de_${STAMP}.md"

sha256sum terms_de_${STAMP}.* > "terms_de_${STAMP}.sha256"

Get the exact snippet for your language from the API Code Generator in the dashboard.

Archiving-specific settings that matter

One sticky session per site visit

A capture often spans a landing page, a terms page and a checkout step. Keep them on one sticky session so consent state and cookies carry over and the record reads as one visit. Start a new session for each scheduled run so captures are independent of each other.

Region and language together

Set the IP country and the Accept-Language header to the same market. Many sites choose the language from the header and the legal variant from the IP, and a mismatch produces a page no real visitor would see. Add state targeting where consumer law differs by state.

Capture the response, not just the body

Archive status code, headers, redirects and final URL with every body. A 302 to a regional edition or a Set-Cookie for a consent banner is part of what the visitor experienced and may matter in a dispute.

Low frequency, high reliability

Archiving is about regularity, not throughput. Run captures at fixed intervals with low concurrency per site, retry with exponential back-off on HTTP 429 or CAPTCHA, and alert when a capture fails twice so gaps in the record are visible immediately.

Log for the auditor, not for the developer

Keep a per-capture manifest: register entry, URL, UTC time, IP country and city, session id, file hashes, capture method and operator. Store manifests with the files and never edit them afterwards.

What Geonode gives you for web archiving

Residential IPs in 195+ countries with country, state and city targeting for region-accurate captures.
Sticky sessions up to 24 hours so multi-page captures share one visitor identity.
Scraper API with JavaScript rendering, returning HTML, Markdown or JSON snapshots.
Crawl and map endpoints to snapshot whole site sections in one job.
Pay-as-you-go, no contract, 24/7 support, payment by card or crypto.

Pricing for web archiving

Text pages are small, so a daily multi-region capture programme fits in the 10 GB residential entry plan at $7.92 per month, with volume pricing from $0.27/GB for large registers. The 3-day trial for $5 lets you validate regional variants before you commit. The Scraper API includes 1.5K free requests per month and a 48-hour trial, then from $3.50 per 10K requests, with monthly unlimited plans from $47. ISP proxies start at $1.25/IP and datacenter at $0.14/GB. Full details on the pricing page.

FAQ

Why use web archiving proxies instead of a public archive service?

Public archives capture from fixed locations, skip pages that block crawlers and cannot show regional variants. With your own proxies you control the region, timing, frequency, retention and the format of each capture, which is what regulators expect of records you are required to keep.

Which format should I archive, HTML, Markdown or screenshot?

All three serve different purposes. Raw HTML and headers are the primary record, a rendered HTML or Markdown snapshot from the Scraper API makes JavaScript-built content readable and searchable, and a screenshot shows layout and placement of disclosures where that matters.

Can web archiving proxies capture my competitors' pages for legal purposes?

You can capture publicly accessible pages as any visitor would see them, within the site's terms and without logging in or circumventing access controls. How the captures are used in a dispute is a question for your legal team, so keep provenance complete.

How do I prove a capture has not been altered?

Hash every file at capture time, store hashes in a manifest alongside the UTC timestamp, IP country and session id, and keep files in write-once storage. A consistent, automated process repeated at fixed intervals is more convincing than any single capture.

Is web archiving allowed under Geonode's terms?

Yes, when you capture publicly available pages, respect the source's terms and rate limits, and keep only what your record-keeping obligation requires. Geonode's AUP prohibits fraud, spam and credential abuse, so archiving must never involve accounts you do not own or manage with authorisation.

How many regions should I capture?

Capture every market where the page is served to customers and where the obligation applies. For consumer-facing terms and pricing that usually means each country you sell in, and sometimes individual states. Start with the markets where disputes are most likely and expand from there.

Start

Keep a defensible record of what each market was shown and when. Get started with a 3-day trial for $5, or Contact sales if you need a large register across many regions with a combined proxy and Scraper API setup.