Handling Dropdowns, Forms, and Clicks

Many websites require interaction before showing the data you need. You might need to select a category from a dropdown, submit a search form, or click through filters. Both Playwright and Selenium provide methods to automate these interactions.

Filling and Submitting Forms with Playwright

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://quotes.toscrape.com/login")

    # Fill in form fields
    page.fill("#username", "admin")
    page.fill("#password", "admin")

    # Click the submit button
    page.click("input[type='submit']")

    # Wait for navigation after form submission
    page.wait_for_load_state("networkidle")

    print(f"Logged in. Current URL: {page.url}")
    browser.close()

Handling Select Dropdowns with Playwright

# Select by value
page.select_option("select#country", value="US")

# Select by visible text
page.select_option("select#country", label="United States")

# Select multiple options
page.select_option("select#tags", value=["python", "scraping"])

Handling Dropdowns with Selenium

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome()
driver.get("https://example.com/form")

# Native <select> dropdown
dropdown = Select(driver.find_element(By.CSS_SELECTOR, "select#country"))
dropdown.select_by_visible_text("United States")
dropdown.select_by_value("US")
dropdown.select_by_index(2)

driver.quit()

Checkboxes and Radio Buttons

# Playwright
# Check a checkbox
page.check("#agree-terms")

# Uncheck
page.uncheck("#newsletter")

# Check if already checked before toggling
if not page.is_checked("#remember-me"):
    page.check("#remember-me")

Clicking Elements

# Playwright, various click methods
page.click("button#load-more")          # Standard click
page.dblclick("div.item")               # Double-click
page.click("button#menu", button="right")  # Right-click

# Click with position offset (useful for canvas or maps)
page.click("canvas#map", position={"x": 100, "y": 200})

Handling Custom Dropdowns (Non-Native)

Many modern sites use custom dropdown components instead of native <select> elements. These require click-based interaction:

# Playwright, custom dropdown
page.click(".dropdown-trigger")                # Open the dropdown
page.wait_for_selector(".dropdown-menu")       # Wait for menu to appear
page.click(".dropdown-menu >> text=Python")    # Select an option

File Upload

# Playwright
page.set_input_files("input[type='file']", "document.pdf")

# Upload multiple files
page.set_input_files("input[type='file']", ["file1.pdf", "file2.pdf"])

Practical Example: Search and Scrape

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://books.toscrape.com")

    # Interact with page elements to navigate
    page.click("a:has-text('Travel')")
    page.wait_for_selector(".product_pod")

    books = page.query_selector_all(".product_pod h3 a")
    for book in books:
        print(book.get_attribute("title"))

    browser.close()

Next Steps

• Learn about browser fingerprinting and stealth mode
• Use proxies with Playwright and Selenium
• Scrape single-page applications