A Complete Guide to Using find_element and find_elements in Selenium Python

Selenium is one of the most powerful tools for browser automation and web scraping. Whether you’re building a bot to automate tasks or scraping structured data from websites, the first step is locating elements on a page.

In this guide, we’ll explore how Selenium’s find_element and find_elements methods work, why they’re essential, and how to use them effectively for different scenarios.

What is find_element in Selenium?

find_element is a Selenium method used to locate a single HTML element on a webpage. It’s often the starting point when interacting with elements, such as clicking a button, filling a form, or extracting data.

Basic Syntax

driver.find_element(by=By.<method>, value='<locator>')

driver.find_element(by=By.<method>, value='<locator>')

Here, By is a class that tells Selenium what strategy to use for locating elements (e.g., by ID, class name, tag name, XPath, etc.).

To use it, you must import the necessary classes:

from selenium import webdriver
from selenium.webdriver.common.by import By

from selenium import webdriver
from selenium.webdriver.common.by import By

Common Locator Strategies

Selenium offers multiple strategies for locating web elements. Here are the commonly used ones:

By ID

So the code will be

element = driver.find_element(By.ID, "productTitle")

element = driver.find_element(By.ID, "productTitle")

Fast and reliable, assuming the ID is unique.

By Name

So the command will be:

element = driver.find_element(By.NAME, "next-head-count")

element = driver.find_element(By.NAME, "next-head-count")

Useful for form fields.

By Class Name

So the command will be:

element = driver.find_element(By.CLASS_NAME, "a-section")

element = driver.find_element(By.CLASS_NAME, "a-section")

Can return unexpected results if multiple elements share the same class.

By Tag

We can scrap the whole tags like h1,h2, or h3, etc.

element = driver.find_element(By.TAG_NAME, "h1")

element = driver.find_element(By.TAG_NAME, "h1")

Best used when you’re certain there’s only one tag of that kind.

By Link Text/ Partial Link Text

driver.find_element(By.LINK_TEXT, "Forgot Password?")
driver.find_element(By.PARTIAL_LINK_TEXT, "Forgot")

driver.find_element(By.LINK_TEXT, "Forgot Password?")
driver.find_element(By.PARTIAL_LINK_TEXT, "Forgot")

Great for interacting with anchor tags.

By XPath

element = driver.find_element(By.XPATH, "//div[@class='price']")

element = driver.find_element(By.XPATH, "//div[@class='price']")

Very flexible but harder to read. Prone to breaking with minor layout changes.

Example: Finding and Clicking a Button

Here’s a simple real-world example using Selenium and Chrome WebDriver:

from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://example.com")
# Locate and click a button
button = driver.find_element(By.ID, "subscribe")
button.click()
driver.quit()

from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://example.com")
# Locate and click a button
button = driver.find_element(By.ID, "subscribe")
button.click()
driver.quit()

find_element vs find_elements

The key difference is:

• find_element returns a single WebElement.
• find_elements returns a list of WebElements.

If the element is not found:

• find_element throws a NoSuchElementException.
• find_elements returns an empty list.

Example

elements = driver.find_elements(By.CLASS_NAME, "product")
print(len(elements))  # Useful when scraping product listings

elements = driver.find_elements(By.CLASS_NAME, "product")
print(len(elements))  # Useful when scraping product listings

Best Practices When Using find_element

Always Use Explicit Waits

Web pages are often dynamic, meaning elements don’t appear immediately. Use WebDriverWait to wait for elements.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
element = wait.until(EC.presence_of_element_located((By.ID, "login")))

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
element = wait.until(EC.presence_of_element_located((By.ID, "login")))

Validate Element Presence Before Interaction

Always verify the element exists before calling click () or text.

try:
    element = driver.find_element(By.ID, "submit")
    element.click()
except NoSuchElementException:
    print("Element not found!")

try:
    element = driver.find_element(By.ID, "submit")
    element.click()
except NoSuchElementException:
    print("Element not found!")

Use Relative Locator or Contextual Locators

Instead of hardcoded IDs, use relationships:

element = driver.find_element(By.XPATH, "//label[text()='Email']/following-sibling::input")

element = driver.find_element(By.XPATH, "//label[text()='Email']/following-sibling::input")

Useful when IDs are dynamic or missing.

Common Errors and How to Fix Them

NoSuchElementException: Element doesn’t exist or hasn’t loaded yet.

• Use waits, or double-check the locator.

ElementNotInteractableException: The element is found but can’t be clicked or typed into.

• Scroll into view or check visibility with JavaScript.

Dynamic elements: Use stable attributes like data-* fields, or use custom XPath functions like contains().

Conclusion

Mastering find_element in Selenium is essential for automating browser tasks or scraping structured data. The key is choosing the right locator strategy, handling exceptions properly, and always using waits for dynamic content.

When working with sites that implement anti-scraping measures or geo-restrictions, integrating Selenium with a dependable proxy solution becomes essential.

That’s where tools like Proxying can make a significant difference, offering stable and scalable proxy infrastructure to help ensure your Selenium scripts remain efficient, undetected, and consistent across sessions.