How to Scrape Shopify Store Products

Shopify stores are among the easiest e-commerce sites to scrape because they expose product data through built-in JSON endpoints. Over 4 million stores use Shopify, making this technique widely applicable.

The Shopify JSON Trick

Every Shopify store exposes product data at predictable URLs:

• All products: https://store.com/products.json
• Single product: https://store.com/products/product-handle.json
• Collections: https://store.com/collections/collection-name/products.json

import requests

url = "https://example-store.myshopify.com/products.json?limit=250"
resp = requests.get(url)
products = resp.json()["products"]

for product in products:
    print(f"Title: {product['title']}")
    print(f"Price: {product['variants'][0]['price']}")
    print(f"Available: {product['variants'][0]['available']}")
    print(f"Images: {len(product['images'])}")
    print("---")

Paginating Through All Products

Shopify paginates with a page parameter (or cursor-based pagination for newer stores).

import requests

page = 1
all_products = []

while True:
    url = f"https://store.com/products.json?limit=250&page={page}"
    resp = requests.get(url)
    products = resp.json().get("products", [])
    
    if not products:
        break
    
    all_products.extend(products)
    page += 1

print(f"Total products: {len(all_products)}")

Available Data Fields

Field	Description
title	Product name
body_html	Product description (HTML)
vendor	Brand or manufacturer
product_type	Category
variants	Sizes, colors with individual prices
images	Product image URLs
tags	Product tags
created_at	When the product was added

When You Need ScraperAPI

Some Shopify stores disable JSON endpoints or add Cloudflare protection. In those cases, use ScraperAPI to scrape the HTML pages.

API_KEY = "YOUR_SCRAPERAPI_KEY"
url = "https://protected-store.com/collections/all"
resp = requests.get(f"http://api.scraperapi.com?api_key={API_KEY}&url={url}")

Discovering Shopify Stores

Use ScrapingAnt to identify Shopify stores by checking for the Shopify meta tag or the /products.json endpoint.

Best Practices

1. Always try /products.json first, It is the easiest approach
2. Use limit=250, The maximum per page
3. Check the sitemap, https://store.com/sitemap.xml lists all products
4. Monitor inventory changes, Track available status on variants
5. Respect rate limits, Shopify throttles at about 2 requests per second