Geonode logo
Available

Connect your stack to the Geonode Scraper API

Extract web content over HTTP. Add Markdown, HTML and asynchronous data jobs to the language and backend you already use.

HTTPS client
Get API keyRead documentation

What You'll Need

  • Geonode API key

    Copy a Scraper API key from your dashboard.

  • An HTTP client

    Use cURL or the HTTP library in your backend.

Send your first request

Set GEONODE_SCRAPER_API_KEY, then request a page as Markdown.

curl https://scraper.geonode.io/v1/extract \
  -H "X-Api-Key: $GEONODE_SCRAPER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "processing_mode": "sync"
  }'

One API, several workflows

Choose how content is returned and how the job runs.

  • API-key authentication

    Send your key with every request from your backend.

    X-Api-Key: YOUR_API_KEY

  • Output and rendering

    Request Markdown or HTML; enable JS when needed.

    "formats": ["markdown"],
"render_js": true

  • Async jobs

    Submit work, store the job ID and retrieve results.

    "processing_mode": "async"

GET /v1/extract/{job_id}

API overview

Connection details and the operations available over HTTP. Open an operation for its full schema and examples.

Base URL
https://scraper.geonode.io
Authentication
X-Api-Key: <your-api-key>
Get API key
Request format
application/json

  • Extract

    POST /v1/extract
    Input
    A URL, output formats and optional browser rendering.
    Output
    Markdown or HTML with metadata; a job ID in async mode.

  • Batch

    POST /v1/batch
    Input
    A list of URLs and extraction options.
    Output
    A job ID and accepted URL count. Retrieve results when complete.

  • Crawl

    POST /v1/crawl
    Input
    A seed URL, maximum depth and page limit.
    Output
    A job ID. Retrieve extracted pages as the crawl completes.

  • Map

    POST /v1/map
    Input
    A base URL and optional discovery filters.
    Output
    Discovered URLs and their sources, without page content.

  • Search

    POST /v1/search
    Input
    A query, with optional locale, page and search filters.
    Output
    Search results, suggestions and a search job ID.

Request behavior

  • Synchronous requests and background jobs

    Extraction supports sync and async modes. Batch and crawl return a job ID; poll for completion and paginate results. Webhooks can notify your service when a job finishes.

    Job lifecycle

  • Rendering and usage

    Enable render_js when the page requires a browser. Rendering is slower and uses more credits. Keep API keys on your server; usage is charged to your Geonode account.

    Extraction options

  • Errors and retries

    Handle authentication, validation and balance errors explicitly. Respect Retry-After on rate limits. A timed-out submission may still have created a job; avoid blindly submitting it again.

    Error handling

Developer documentation

Get the full setup guide, authentication details and technical reference.

Read the developer docs

Capabilities

  • Extract webpage content as Markdown or HTML.

  • Process multiple URLs together with batch extraction.

  • Crawl a website and retrieve the resulting pages.

  • Discover website URLs before choosing what to extract.

  • Search the web for pages relevant to your query.

  • Track background jobs and receive completion notifications through webhooks.

Common workflows

# Submit an asynchronous extraction
curl https://scraper.geonode.io/v1/extract \
  -H "X-Api-Key: $GEONODE_SCRAPER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "processing_mode": "async"
  }'

# Set this to the job_id returned above
JOB_ID="YOUR_JOB_ID"

# Check status; read data once completed
curl "https://scraper.geonode.io/v1/extract/$JOB_ID" \
  -H "X-Api-Key: $GEONODE_SCRAPER_API_KEY"

Runtime and access

Runtime

HTTPS client

Account and access

Keep API keys on the server or in your client’s credential store. Usage is charged to your Geonode plan.

Troubleshooting

  • Authentication failed

    Verify your X-Api-Key header.

  • Invalid request

    Check the URL, field names and accepted values.

  • Content missing

    Check the chosen format and whether the page needs JS.

  • Job not complete

    Inspect status before consuming async results.

Build a dependable workflow

  • Keep keys on the backend

    Use environment variables for credentials.

  • Choose the right workflow

    Extract, batch, crawl and map solve different jobs.

  • Handle job states

    Keep IDs and account for unsuccessful jobs.

  • Monitor usage

    Review concurrency and statistics.

FAQ

Any language or tool that can send HTTP requests. Send JSON to https://scraper.geonode.io and read the JSON response; for Python, you can also use the typed Python SDK.

Send your API key in the X-Api-Key header with every request. Keep the key on your backend and never expose it in frontend code, public repositories or logs.

Yes. Set "render_js": true to load the page in a headless browser before extraction. Use wait_config to wait for network idle or a specific CSS selector.

Send "processing_mode": "async" to get a job_id, then call GET /v1/extract/{job_id} until the job is completed. You can also register a webhook to be notified when extract, batch or crawl jobs finish.

Related integrations

Your stack. Fresh web data.

Make a first request with your Geonode API key.

Get API keyRead documentation