Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Use driver.findElements(By.xpath("your XPath")) to get every match in the page currently loaded in Selenium. It returns a List<WebElement>; when there are no matches, Selenium returns an empty list. findElement returns only the first match. To collect matches from several pages, run the lookup on each page, wait for that page’s content to be ready, copy the text or attributes you need, then advance using the site’s actual pagination mechanism.
A lookup never combines elements from pages that are not loaded in the current browsing context. The following pattern is the essential distinction:
What “duplicate XPath matches” means
There are two different cases:
- Several matches on one page: the same XPath identifies multiple elements in the current DOM. Use
findElementsand iterate the returned list. - Matches on separate pages: each page has its own DOM. You must navigate to page 1, collect its matches, navigate to page 2, collect again, and continue until the site’s final-page condition is met.
The official Selenium finding-elements documentation states that an empty list is returned when there are no matches. That makes findElements safe for optional result areas and for deciding whether a page contains any records.
Find every match on the current page
Minimal Java example
Import the locator, element, and collection types, then call findElements:
#1 Best Overall
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import java.util.List;
List<WebElement> matches = driver.findElements(
By.xpath("//div[@class='result']")
);
for (WebElement match : matches) {
System.out.println(match.getText());
}matches.size() is the number of elements in the active page’s DOM. If the count is zero, Selenium does not throw the “no such element” exception that findElement would throw; it returns an empty list instead.
When you need only one element
Use findElement when the first matching element is the intended target:
WebElement first = driver.findElement(
By.xpath("//div[@class='result']")
);Do not use it to count duplicates. It selects one match and gives no indication that additional matches exist.
Collect matches across paginated pages
The loop has four responsibilities: identify matches, copy their values, advance through the site’s pagination, and stop reliably. Copy values before navigation because elements belong to the page in which they were found; after a navigation or DOM replacement, old references may become unusable.
Rank #2
Complete example with a “Next” button
This example assumes result cards use //div[contains(@class,'result')], a next button has data-testid='next-page', and the button becomes disabled on the final page. Replace every selector and readiness condition with those of the target application.
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
public class CollectAcrossPages {
public static void main(String[] args) {
WebDriver driver = new ChromeDriver();
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(20));
List<String> collected = new ArrayList<>();
try {
driver.get("https://example.com/results");
while (true) {
// Wait for the page-specific result state, not an arbitrary sleep.
wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(
By.xpath("//div[contains(@class,'result')]")
));
List<WebElement> matches = driver.findElements(
By.xpath("//div[contains(@class,'result')]")
);
for (WebElement match : matches) {
// Copy data while this page is still loaded.
collected.add(match.getText());
}
WebElement next = driver.findElement(
By.cssSelector("button[data-testid='next-page']")
);
boolean disabled = !next.isEnabled()
|| "true".equals(next.getAttribute("aria-disabled"));
if (disabled) {
break;
}
// Save an old-page marker so the wait observes a real change.
WebElement firstResult = matches.get(0);
next.click();
wait.until(ExpectedConditions.stalenessOf(firstResult));
wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(
By.xpath("//div[contains(@class,'result')]")
));
}
System.out.println("Collected " + collected.size() + " values");
} finally {
driver.quit();
}
}
}The sample is a pattern rather than a site-specific test. A real application may replace the result nodes instead of making them stale, may expose a numbered link, or may change the URL without replacing the same elements. Adjust the wait and stop test to the observable state that actually changes.
Numbered links or URL-based pagination
If each page has a predictable URL, navigating directly can be simpler and less fragile:
for (int page = 1; page <= 20; page++) {
driver.get("https://example.com/results?page=" + page);
wait.until(ExpectedConditions.presenceOfElementLocated(
By.cssSelector("main.results")
));
for (WebElement match : driver.findElements(
By.xpath("//div[contains(@class,'result')]") )) {
collected.add(match.getAttribute("data-id"));
}
}Use a known final page count only when the application supplies one. Otherwise, stop when a next link is absent or disabled, when a page-specific “no results” state appears, or when a URL repeats. A repeated URL guard prevents an accidental loop caused by a broken pagination control.
Preserve structured data, not just visible text
Read each required value before leaving the page:
for (WebElement result : matches) {
String title = result.findElement(By.cssSelector("h2")).getText();
String href = result.findElement(By.cssSelector("a")).getAttribute("href");
String id = result.getAttribute("data-id");
// Store title, href and id in your own record object.
}Keeping a stable key such as a record ID lets you remove an item that appears on two pages because of overlapping pagination or changing results. Do not retain the WebElement itself as your cross-page record.
XPath scope: // versus .//
When the lookup starts from driver, an XPath such as //article searches the document. When it starts from a WebElement, the difference matters:
WebElement card = driver.findElement(By.cssSelector("section.cards"));
// Searches from the document root and can match articles elsewhere.
List<WebElement> broad = card.findElements(By.xpath("//article"));
// Restricts the search to descendants of card.
List<WebElement> local = card.findElements(By.xpath(".//article"));The Selenium WebElement API documents this context behavior. Use .// when you have already located a container and want only its descendants; it prevents unrelated matches from other parts of the page.
Wait for the page state that proves it is ready
findElements is affected by the configured implicit wait, as described in the WebDriver API. An implicit wait alone does not tell Selenium that an AJAX request has finished or that the new page’s results differ from the old page’s. Prefer an explicit, page-specific signal:
- Result presence: wait for the result container or at least one result element.
- Old content gone: after clicking Next, wait for the previous first result to become stale.
- Loading indicator hidden: wait for the application’s spinner to disappear when that is its documented readiness signal.
- URL changed: for URL-driven pagination, wait for the expected query parameter or path.
- Empty state: treat a visible “no results” panel as a valid terminal state rather than an exception.
A fixed Thread.sleep can be too short on a slow run and unnecessarily long on a fast one. The correct condition depends on the application; Selenium’s locator and wait APIs do not know which business state means “complete.”
Make the XPath maintainable across page templates
Identical templates can share one XPath only while their relevant structure and attributes remain stable. Selenium’s locator guidance recommends compact, readable locators and favors unique, predictable IDs where available. CSS is often a good alternative; XPath is useful when you need text relationships, ancestor traversal, or other expressions CSS cannot represent.
- Prefer a unique
idor stable data attribute over a long chain of positional indexes. - Scope a broad expression to a known container, for example
//main[@id='results']//article. - Use class-token checks such as
contains(concat(' ', normalize-space(@class), ' '), ' result ')when class attributes contain several tokens. - Avoid selectors tied to generated CSS-module names or changing visual text.
- Log the current URL and match count on every iteration so a template change is visible immediately.
Validate the XPath against every relevant page type. A locator that works on page 1 may match a sponsored block, a different mobile template, or nothing on a final empty page.
Common failures and precise fixes
| Symptom | Likely cause | Fix |
|---|---|---|
findElement throws, but the page legitimately has no records |
The singular API requires one match. | Use findElements and handle an empty list. |
| Only one duplicate is collected | The code uses findElement or reads index zero. |
Iterate the complete List<WebElement>. |
| Results from the previous page appear again | The click returned before the new DOM was ready, or the site reused nodes. | Wait for URL change, staleness, a loading indicator, or a page-specific marker; then call findElements again. |
StaleElementReferenceException |
A saved element belongs to a replaced DOM. | Copy text and attributes before navigation and locate fresh elements after navigation. |
| Nested lookup returns unrelated elements | The XPath starts with // from a WebElement. |
Use .// to limit the search to that element’s descendants. |
| Loop never ends | Next remains clickable, URL does not change, or the stop condition is wrong. | Track visited URLs or page keys, inspect disabled/ARIA state, and set a defensible maximum-page guard. |
| Intermittent zero matches | Results are rendered asynchronously or an iframe is involved. | Wait for the application’s readiness condition; if content is inside an iframe, switch to the correct frame before locating it. |
| XPath works in one layout but not another | Responsive or authenticated templates differ. | Inspect each template, use stable attributes, and keep separate locators when the DOM genuinely differs. |
Performance, reliability and data quality
Collect only the fields required by the job. Calling getText() on every large subtree can be slower than reading a specific heading or attribute. Keep one driver session for the pagination run, but do not mix implicit and explicit waits without understanding their compounded delays. Set a reasonable explicit timeout and record failures with the URL and page identifier.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPagination can expose changing data: records may move between pages while the run is in progress, causing duplicates or omissions. If the application offers a stable sort order or an API, use it; otherwise, store a unique key and deduplicate after collection. Respect authentication, rate limits, robots policies, and the site’s terms. Selenium itself adds no purchase requirement for this method; the cost is the browser session and the runtime resources of your environment.
Or skip the browser setup
If your actual goal is to obtain page images or PDFs rather than interact with every DOM element, ScreenshotNeo provides a single HTTP request. It accepts cookie and consent banners before capture and removes more than 60 known consent platforms, newsletter popups and chat widgets; each step can be disabled. Bot checks, CAPTCHAs, blank pages, timeouts, failed loads and cache hits are not billed, and the response identifies the result with X-Page-Verdict and X-Billed headers. Its MCP server includes take_screenshot, get_page_info and capture_pdf for Claude, Cursor and other MCP clients.
cURL
curl -G "https://api.screenshotneo.com/v1/shot"
-d access_key=YOUR_API_KEY
--data-urlencode url=https://stripe.com
-o shot.webpPython
import requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)Node.js
const q = new URLSearchParams({
access_key: 'YOUR_API_KEY',
url: 'https://stripe.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const data = Buffer.from(await res.arrayBuffer());
require('fs').writeFileSync('shot.webp', data);See the ScreenshotNeo documentation for the full option set, including full-page and element capture, device presets, custom CSS and JavaScript, waits, request blocking, cookies and headers, PDF controls, caching, signed links, asynchronous jobs and bulk capture. The Free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000 shots. Create a free ScreenshotNeo account to try it.
Frequently Asked Questions
Does findElements search every tab or browser window?
No. It searches the current browsing context. Switch to the required window or tab first, then run the lookup.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How can I prove that two pages were actually processed?
Log the URL (or another page identifier), the match count, and the unique keys collected before each navigation. A repeated URL or repeated key exposes a pagination loop.
Should I deduplicate by visible text?
Prefer a stable record ID or URL. Visible text can legitimately repeat for different records and can change with localization or formatting.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
