Using curl_cffi for TLS Impersonation in Python Scraping

curl_cffi is a Python binding for curl-impersonate that lets you make HTTP requests with TLS fingerprints matching real browsers. It is the best solution when you need HTTP-level scraping but sites block based on TLS fingerprinting.

Installation

pip install curl_cffi

Basic Usage

from curl_cffi import requests

# Impersonate Chrome 136
response = requests.get(
    "https://example.com",
    impersonate="chrome136"
)
print(response.status_code)
print(response.text[:500])

Available Browser Profiles

curl_cffi supports multiple browser impersonation targets:

from curl_cffi import requests

# Chrome versions
response = requests.get(url, impersonate="chrome136")
response = requests.get(url, impersonate="chrome124")
response = requests.get(url, impersonate="chrome120")

# Firefox versions
response = requests.get(url, impersonate="firefox133")

# Safari versions
response = requests.get(url, impersonate="safari18_0")

# Edge
response = requests.get(url, impersonate="edge131")

Using Sessions for Cookie Persistence

from curl_cffi import requests

session = requests.Session(impersonate="chrome136")

# Cookies persist across requests
session.get("https://example.com/login")
response = session.get("https://example.com/dashboard")
print(response.text)

Async Support

curl_cffi supports async operations for concurrent scraping.

import asyncio
from curl_cffi.requests import AsyncSession

async def scrape_urls(urls):
    async with AsyncSession(impersonate="chrome136") as session:
        tasks = [session.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        return responses

urls = [
    "https://example.com/page/1",
    "https://example.com/page/2",
    "https://example.com/page/3"
]

results = asyncio.run(scrape_urls(urls))
for r in results:
    print(r.status_code)

Combining with Proxies

from curl_cffi import requests

proxies = {
    "http": "http://user:pass@proxy.example.com:8080",
    "https": "http://user:pass@proxy.example.com:8080"
}

response = requests.get(
    "https://protected-site.com",
    impersonate="chrome136",
    proxies=proxies,
    timeout=30
)

When to Use curl_cffi vs ScraperAPI

Use curl_cffi when:

• You need HTTP-level speed (no browser overhead)
• TLS fingerprinting is the primary detection method
• You manage your own proxy infrastructure

Use ScraperAPI when:

• The site also requires JavaScript rendering
• You need managed proxy rotation
• You want to avoid infrastructure maintenance

Verifying Your Fingerprint

from curl_cffi import requests

response = requests.get("https://tls.peet.ws/api/all", impersonate="chrome136")
fp = response.json()
print(f"JA3 hash: {fp['tls']['ja3_hash']}")
print(f"HTTP/2 fingerprint: {fp['http2']['akamai_fingerprint']}")

curl_cffi is the go-to library when you need fast, HTTP-level scraping that passes TLS fingerprint checks without the overhead of a full browser.