How to Scrape Real Estate Listings (Zillow, Realtor)

Real estate data scraping is popular for investment analysis, market research, and building property comparison tools. Sites like Zillow, Realtor.com, and Redfin contain rich property data. Here is how to extract it.

Challenges

Real estate sites present unique scraping challenges:

• Heavy JavaScript rendering required
• Aggressive bot detection (especially Zillow)
• Dynamic loading with infinite scroll
• Geographic restrictions on some data

Scraping Zillow with ScraperAPI

Zillow is notoriously difficult to scrape. ScraperAPI's proxy rotation and rendering are essential:

import requests
from bs4 import BeautifulSoup
import json

API_KEY = "YOUR_SCRAPERAPI_KEY"

def scrape_zillow_search(location):
    url = f"https://www.zillow.com/homes/{location}_rb/"

    response = requests.get("https://api.scraperapi.com", params={
        "api_key": API_KEY,
        "url": url,
        "render": "true",
        "country_code": "us"
    })

    soup = BeautifulSoup(response.text, "html.parser")

    # Zillow stores listing data in a script tag
    script_tag = soup.find("script", {"id": "__NEXT_DATA__"})
    if script_tag:
        data = json.loads(script_tag.string)
        results = data.get("props", {}).get("pageProps", {}).get("searchPageState", {})
        listings = results.get("cat1", {}).get("searchResults", {}).get("listResults", [])

        properties = []
        for listing in listings:
            properties.append({
                "address": listing.get("address", "N/A"),
                "price": listing.get("price", "N/A"),
                "beds": listing.get("beds", "N/A"),
                "baths": listing.get("baths", "N/A"),
                "sqft": listing.get("area", "N/A"),
                "url": listing.get("detailUrl", "")
            })
        return properties

    return []

properties = scrape_zillow_search("San-Francisco-CA")
for p in properties:
    print(f"{p['address']} - {p['price']} ({p['beds']}bd/{p['baths']}ba)")

Scraping Realtor.com

Realtor.com is somewhat easier to scrape:

import requests
from bs4 import BeautifulSoup

response = requests.get("https://api.scrapingant.com/v2/general", params={
    "x-api-key": "YOUR_SCRAPINGANT_KEY",
    "url": "https://www.realtor.com/realestateandhomes-search/San-Francisco_CA",
    "browser": "true"
})

html = response.json()["content"]
soup = BeautifulSoup(html, "html.parser")

listings = soup.select('[data-testid="card-content"]')
for listing in listings:
    price = listing.select_one('[data-testid="card-price"]')
    address = listing.select_one('[data-testid="card-address"]')

    if price and address:
        print(f"{address.text.strip()} - {price.text.strip()}")

Data Points You Can Extract

• Property address and coordinates
• List price and price history
• Bedrooms, bathrooms, square footage
• Year built, lot size
• Property type (single family, condo, etc.)
• Days on market
• Tax history

Legal Considerations

Real estate listing data is generally considered factual information, but sites' Terms of Service may restrict scraping. Zillow, in particular, has taken legal action against scrapers. Always review each site's ToS and robots.txt.

Verdict

Real estate scraping requires robust proxy rotation and JavaScript rendering. ScraperAPI is our top recommendation for Zillow thanks to its high success rates on protected sites. ScrapingAnt works well for Realtor.com and similar platforms. Always scrape responsibly and respect site terms.