Tutorials, guides, and close looks from the team
that owns the infrastructure.
XPath is a query language for navigating XML and HTML documents. It has a small core and a large number of ways to misuse it. This is a reference organised the way you actually need it: syntax first, then the functions worth knowing, then the patterns that come up repeatedly, then the traps. Everything here is XPath 1.0, which is what browsers, Selenium and most parsing libraries implement.
`curl -A "MyBot/1.0" https://example.com` sets the User-Agent header. That is the syntax, and the interesting question is what to put in it. The instinctive answer — copy a Chrome string — is frequently the wrong one, because it creates a contradiction between what you claim to be and what your connection demonstrably is. This guide covers the syntax, the removal case, and the more useful strategy of identifying yourself honestly.
`curl -u username:password https://example.com` sends basic authentication. That works, and it also puts your password somewhere you probably did not intend. curl's own manual is unusually blunt about it: sensitive data "should be retrieved from a file instead or similar and never used in clear text in a command line". This guide covers the syntax, the three safer alternatives, how basic auth differs from the other schemes, and what changes when a proxy is in the path.
`curl -d "key=value" https://example.com` sends a POST. That is the whole basic answer, and it is also the source of the most common error people hit next. `-d` sets a form-encoded content type. Send JSON with it and forget the header, and many APIs will reject you with a 415. This guide covers form data, JSON including the shortcut most people do not know about, file uploads, and how to tell which of the several data options you actually want.
"Clean proxies" is one of the most-used phrases in this industry and one of the least defined. Nobody agrees what it means, which makes it excellent marketing and poor information. Here is the version that survives contact with reality: cleanliness is not a property of an address. It is a relationship between an address and a specific target, and it changes over time. That reframing has consequences for what you should test and what you should refuse to believe.
curl fetches a URL and prints the result. Everything else is options, and there are over two hundred of them. You need about eight to be productive. This guide covers those eight, the mental model that makes the rest readable, and the handful of mistakes that catch everyone at the start. By the end you will be able to send requests, read responses, debug what actually went over the wire, and know which manual page to open next.
`await page.screenshot({ path: 'shot.png' })` takes a screenshot. That is the easy part, and it is also where most guides stop. The harder part is making screenshots that are the same twice — which matters enormously if you are comparing them, and matters not at all if you are just looking at one. This guide covers both, plus the options that control what gets captured and the case where screenshots are genuinely the right verification tool.
wget has no `--proxy` option. That surprises people coming from curl, and it is the first thing to know. Proxies are configured through environment variables, a configuration file, or exclusion flags — and the exclusion variable, `no_proxy`, matches in a way that is narrower than most people assume. This guide covers all three methods, the SOCKS limitation nobody mentions, and how to read wget's exit codes when something goes wrong.
A proxy is a machine you voluntarily route your traffic through. Whether that is safe depends almost entirely on who runs it and what your traffic looks like when it passes. The short version: a reputable paid proxy carrying HTTPS traffic is a modest, manageable risk. A free public proxy is a genuinely bad idea, and there is peer-reviewed research explaining why. Here is the full picture, including what proxies do not protect you from at all.
A proxy pool is a collection of IP addresses you can route requests through, usually behind a single endpoint that picks one for you. The number everyone advertises — how many addresses are in it — is close to the least useful thing about it. This guide covers what actually determines whether a pool works for your job, how to measure it before committing, and how many addresses you genuinely need, which is almost always fewer than you think.
A subnet ID is the part of an IP address that identifies the network rather than the individual machine on it. Split the address in two, and the left-hand part is the subnet. That is a networking fundamental with an unexpectedly practical consequence: sites that block addresses frequently block whole subnets, which is why a thousand proxies from one block can be worth less than fifty from fifty. This guide covers what a subnet ID is, how to work one out, and what it means when you are buying addresses.
`contains()` is the XPath function people reach for most and understand least. It looks like a substring test and behaves like one only when you give it a string. Given a node-set — which is what most expressions produce — it silently uses only the first node and ignores the rest. This guide covers that trap, the class-matching idiom nobody gets right first time, and the whitespace and case handling that XPath 1.0 makes harder than it should be.
"Timeout of 30000ms exceeded" is the most common Playwright error and the most commonly misdiagnosed. It rarely means your test is slow. It usually means an element never became actionable, and the timeout is simply where Playwright gave up waiting for a condition that was never going to be satisfied. This guide covers the six separate timeouts, which one you are actually hitting, and why raising the number is usually the wrong fix.
Proxy chaining routes your traffic through two or more proxies in sequence rather than one. The intuition is that more hops means more privacy. The arithmetic disagrees. Latency adds up, reliability multiplies downward, and the trust model improves less than you would think — because each operator in the chain still sees a hop. There is a design that does chaining properly, and it is not a stack of commercial proxies.
`curl -I` sends a HEAD request. `curl -X HEAD` looks like it should do the same and does something subtly broken, which is a fine way to lose twenty minutes to a hanging terminal. The curl manual says so directly, and the reason is worth understanding because it applies to every method you might set by hand. This guide covers the correct syntax, the specification's rules about what servers may omit, and the cases where a HEAD tells you something a GET would not.
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
Geonode Team
Published: September 2, 2026
