Foundations

Client, Server, DNS, IP, the Architecture

The four moving parts behind every web request, what each does, and why scrapers need to understand all of them.

Lab: /

HTTP Protocol: Methods, Status Codes, Headers

The actual on-the-wire format your scraper speaks. Methods you'll use, status codes you'll interpret, headers that change behaviour.

Lab: /

HTTPS, TLS, and Why It Matters for Scraping

What TLS actually does, the certificate verification you'll be tempted to disable but shouldn't, and the fingerprint that gets your scraper blocked before you ever send a request.

Lab: /

Cookies, Sessions, and Authentication Basics

How servers remember who you are between requests, and how scrapers persist that state correctly without re-logging-in every time.

Lab: /challenges/static/cookies/set-on-visit

HTML Structure and the DOM

What HTML actually is, what a parser turns it into, and the tree-shaped mental model your scraper needs to navigate.

Lab: /products

CSS Selectors, Complete Reference

Every CSS selector you need to know, organised by what you'll actually use them for in scrapers.

Lab: /challenges/static/lists/cards

XPath, Complete Reference

The more powerful, less familiar query language for DOM nodes. When CSS runs out, XPath keeps going.

Lab: /challenges/static/tables/nested

Choosing Between CSS Selectors and XPath

When to use which. A short decision framework with concrete examples, not 'XPath is more powerful so always use XPath.'

Lab: /products/1-yellow-ceramic-mug

Elements Panel, Inspect, Edit, Copy Selectors

Your eyes on a live page. How to use the Elements panel to plan a scrape before writing a single line of code.

Lab: /products

Network Panel, The Scraper's Most Important Tool

Where the data actually lives. Use Network to find the JSON endpoint the page is hitting and skip HTML scraping entirely.

Lab: /products/1-yellow-ceramic-mug

Console, Sources, Application Tabs

The three other DevTools panels every scraper should know, for interactive prototyping, reading minified JS, and inspecting client-side storage.

Lab: /account/login

Copy as cURL and Why It's a Superpower

Right-click → Copy → Copy as cURL. The single most time-saving habit in web scraping. Here's how to use it, what to strip, and how to translate to Python or PHP.

Lab: /products

Python, pip, venv, uv, Modern Toolchain

Install Python correctly, isolate every project, and meet the tooling that actually makes Python pleasant in 2026.

Python Crash-Course for Scrapers

The 5% of Python you'll use 95% of the time when writing scrapers. Strings, lists, dicts, comprehensions, file I/O, error handling, and the f-string.

Lab: /

JSON, CSV, and Regex Essentials in Python

The three data-handling skills you'll use in every scraper: parsing JSON responses, writing structured output, and reaching for regex without abusing it.

Lab: /api/products

PHP, Composer, and a Modern Dev Environment

PHP 8.x is a serious language. Here's how to set it up cleanly and use Composer the way modern PHP projects expect.

PHP Crash-Course for Scrapers

The PHP you'll actually use to write scrapers. Arrays, strings, file I/O, error handling, JSON, and the modern PHP 8 features that make it pleasant.

Lab: /

Why PHP Is a Legitimate Scraping Language (and When to Pick It)

PHP is not just for WordPress. It has a serious scraping ecosystem and three concrete advantages over Python in specific situations. Here's when to reach for which.

Git and GitHub for Scraper Projects

The minimum Git you need to keep scraper projects sane, collaborate, and ship to production. Plus the patterns specific to scraper repos.

Legal & Ethical Scraping, Your Compass

robots.txt, Terms of Service, GDPR, the CFAA, the hiQ vs LinkedIn ruling, the legal and ethical scaffolding every scraping project should consider before code is written.

Lab: /robots.txt