TLS Fingerprinting (JA3/JA4), `curl-cffi` and `tls-client`

You've trimmed your headers. You've set a realistic User-Agent. Your requests look identical to Chrome's, to a human eye. The server still 403s.

The server is looking at the TLS handshake.

What is TLS fingerprinting

When a client opens a TLS connection, the first thing it sends is a ClientHello with:

• The TLS version it supports.
• The cipher suites it offers, in its preferred order.
• The extensions it requests (SNI, ALPN, supported groups, signature algorithms, etc.).
• The order of those extensions.

Different TLS libraries fingerprint differently:

• Chrome, boringssl with Chrome's cipher list, extension order, GREASE values.
• Firefox, NSS with different cipher preferences and extension order.
• Python requests, urllib3 → cpython OpenSSL with default settings. Distinct from any browser.
• curl, uses the system OpenSSL/SChannel/SecureTransport, also distinct from browsers.
• Go net/http, Go's own TLS stack, different again.

The server can fingerprint your TLS stack from the ClientHello alone. JA3 (Salesforce) and JA4 (FoxIO) are standard ways to express that fingerprint as a string.

JA3 and JA4

JA3 = MD5 hash of: TLSVersion,Ciphers,Extensions,EllipticCurves,EllipticCurvePointFormats

Example JA3:

769,49195-49199-49196-49200-...,0-23-65281-10-11-...,29-23-24,0
→ MD5 → cd08e31494f9531f560d64c695473da9

JA4 is the newer standard, more granular (includes ALPN, TLS version negotiation result). Format is t13d1516h2_8daaf6152771_b186095e22b6.

Anti-bot vendors map known fingerprints to known clients. A requests fingerprint is immediately identifiable.

Who fingerprints

Heavy users:

• Cloudflare, particularly its bot management.
• Akamai, strong fingerprinting in Bot Manager.
• DataDome, TLS + JS + behavioral.
• PerimeterX (HUMAN), same.
• Imperva, F5, enterprise WAFs with TLS components.
• Google, limited TLS fingerprinting on some endpoints.

Light or none: most non-WAF sites, internal APIs, smaller targets.

If your script works on Catalog108 but fails on a Cloudflare-protected site with otherwise-identical headers and cookies, TLS fingerprinting is the prime suspect.

The fix: impersonate

Two main Python libraries:

curl-cffi

pip install curl-cffi

from curl_cffi import requests

r = requests.get(
  "https://target.example.com/api/data",
  impersonate="chrome120",  # or "chrome116", "edge99", "firefox", "safari17_0"...
)
print(r.status_code, r.json())

Uses libcurl-impersonate under the hood, a fork of curl with patched cipher lists matching real browsers. Drop-in replacement for requests for most use cases.

tls-client

pip install tls-client

import tls_client

s = tls_client.Session(
  client_identifier="chrome120",
  random_tls_extension_order=True,
)
r = s.get("https://target.example.com/api/data")
print(r.status_code, r.json())

Built on Bogdanfinn/tls-client (a Go library) with a Python wrapper. Different impersonation profile coverage than curl-cffi; sometimes one works where the other doesn't.

Sample impersonation profiles

# curl-cffi
"chrome99", "chrome100", "chrome101", "chrome104", "chrome107", "chrome110",
"chrome116", "chrome119", "chrome120", "chrome123", "chrome124",
"edge99", "edge101", "safari15_3", "safari15_5", "safari17_0",
"firefox", "ios_15_5", "ios_15_6_1"

# tls-client
"chrome_103" through "chrome_120", "safari_15_3" through "safari_16_0",
"safari_ios_15_5", "safari_ios_16_0", "firefox_102" through "firefox_115",
"okhttp_4"

Match the impersonation to a real browser version that's common on the target's userbase.

A diagnostic workflow

When your scraper fails on a site:

1. Capture a working browser request via DevTools → Copy as cURL. Run it, confirm it works.
2. Translate to Python requests. Run, observe failure (403, 5xx, captcha).
3. Headers and body are identical. The difference must be lower-level.
4. Switch to curl-cffi with impersonate="chrome120". Try again.
5. If it works → fingerprinting was the cause.
6. If it still fails → look at JS challenges, IP reputation, header order.

Why two libraries

Each library has different impersonation coverage and different bugs. curl-cffi has wider browser support; tls-client sometimes handles specific Cloudflare edge cases better. Standard practice is to try both:

def fetch(url, **kw):
  try:
  from curl_cffi import requests
  return requests.get(url, impersonate="chrome120", **kw)
  except Exception:
  import tls_client
  s = tls_client.Session(client_identifier="chrome120")
  return s.get(url, **kw)

Performance

Both libraries are slower than requests:

• requests: ~5ms per call (local).
• curl-cffi: ~10–20ms.
• tls-client: ~10–30ms.

For most scraping, the overhead is irrelevant. For high-throughput async work, it adds up.

Catalog108 lab

The /challenges/antibot/tls-fingerprint endpoint inspects the JA3 / JA4 of the incoming request and rejects fingerprints that look like Python's requests. Hit it with vanilla requests to see the rejection; hit with curl-cffi impersonating Chrome to see success.

# Will fail
import requests
r = requests.get("https://practice.scrapingcentral.com/challenges/antibot/tls-fingerprint")
print(r.status_code, r.text[:200])

# Will succeed
from curl_cffi import requests as cffi
r = cffi.get("https://practice.scrapingcentral.com/challenges/antibot/tls-fingerprint",
  impersonate="chrome120")
print(r.status_code, r.text[:200])

PHP support

PHP's standard tooling has weaker TLS impersonation:

• curl-impersonate for PHP, wraps the same libcurl-impersonate. Requires a custom-built curl binary.
• Symfony's HttpClient with the cURL handler can use it if linked correctly.
• Otherwise, PHP scrapers facing TLS fingerprinting often shell out to Python.

This is one of the few places PHP genuinely lags Python for scraping.

When fingerprinting isn't enough to bypass

TLS fingerprint impersonation defeats JA3/JA4 checks. It doesn't help against:

• JS-based challenges (Cloudflare Turnstile, hCaptcha). Need a real browser.
• Behavioral analysis (mouse movements, scroll patterns). Need full automation.
• IP reputation (residential vs datacenter). Need quality proxies.

Anti-bot is layered. TLS fingerprinting is one layer; you may bypass it and still face others.

Hands-on lab

Hit /challenges/antibot/tls-fingerprint with plain requests and observe the 403. Switch to curl-cffi with impersonate="chrome120" and see the 200. Compare the JA3 fingerprints (the response often includes the detected fingerprint). You've now physically observed TLS fingerprinting and bypassed it, the most important "magic" anti-bot technique demystified.