Python Selenium Module
Selenium is a powerful tool for controlling web browsers through programs and performing browser automation. It is widely used for testing web applications and scraping dynamic websites.
Installation
pip install seleniumNote: Modern Selenium (4.6+) automatically manages browser drivers, so you typically don't need to manually download chromedriver or geckodriver anymore.
Basic Usage
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
# Initialize the browser (Chrome)
driver = webdriver.Chrome()
# Open a website
driver.get("https://www.google.com")
# Get page title
print(driver.title)
# Close the browser
# driver.quit()Locating Elements
You can locate elements using the By strategy.
# Find element by Name attribute
search_box = driver.find_element(By.NAME, "q")
# Find element by ID
# el = driver.find_element(By.ID, "some-id")
# Find element by CSS Selector
# el = driver.find_element(By.CSS_SELECTOR, "div.classname")
# Find element by XPath
# el = driver.find_element(By.XPATH, "//button[@type='submit']")Interacting with Elements
# Typing
search_box.send_keys("Python Selenium")
# Pressing Enter
search_box.send_keys(Keys.RETURN)
# Clicking
# button.click()Waits
Websites load dynamically. Using time.sleep() is unreliable. Use Explicit Waits instead.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
try:
# Wait up to 10 seconds for the element to be present
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "result-stats"))
)
print("Page loaded!")
finally:
driver.quit()Headless Mode
Run the browser in the background without a UI.
options = webdriver.ChromeOptions()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)