Use Playwright when a page’s content appears only after browser rendering or interaction. Navigate to the page, wait for a meaningful content signal, then extract and validate the fields you need. If the same content is already present in the ordinary HTTP response, a request and HTML parser may be simpler.
When Playwright is the right tool
Playwright automates a real browser, which is useful when the page depends on JavaScript, user interaction, or browser behavior before the information is available. For a static page whose response already contains the needed content, a plain HTTP client and HTML parser avoid unnecessary browser setup. The tradeoff is implementation complexity and browser resource use; the available documentation does not establish comparative performance or cost benchmarks.
- Choose Playwright when content is rendered after navigation, revealed by a control, or otherwise depends on browser behavior.
- Choose a request-and-parser approach when the response already contains the fields you need.
- Check the target site’s terms, access controls, and applicable requirements before collecting or reusing its data. Playwright’s technical capabilities do not grant permission.
Set up a minimal Node.js scraper
Install Playwright in a Node.js project, then install the browser it will automate:
- Run
npm init -yif you do not already have a project. - Run
npm install playwright. - Run
npx playwright install chromiumto install Chromium. - Save the script below as
scrape.mjsand run it withnode scrape.mjs.
import { chromium } from 'playwright';
const url = 'https://example.com/catalog';
const browser = await chromium.launch();
try {
const page = await browser.newPage();
const response = await page.goto(url);
if (!response || !response.ok()) {
throw new Error(`Navigation failed: ${response?.status() ?? 'no response'}`);
}
const heading = await page
.getByRole('heading', { name: 'Catalog' })
.textContent();
const names = await page.locator('[data-product-name]').allTextContents();
if (!heading || names.length === 0) {
throw new Error('Expected catalog content was not found');
}
const records = names.map(name => ({ name: name.trim(), source: url }));
console.log({ heading: heading.trim(), records });
} finally {
await browser.close();
}Replace the example URL and locators with ones that match a site you are allowed to access. The role and accessible name must exist on the actual page; the data attribute is only an example of a potentially stable site-specific contract. Do not assume either selector matches without inspecting the page.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
Find content with locators
Locators identify content or controls and are Playwright’s preferred way to interact with the page. Its documentation describes locators as the central piece of auto-waiting and retry behavior. Prefer a role, label, or visible text when it clearly identifies the target; use a site-specific attribute when it is deliberately provided and stable. Deeply nested CSS or XPath selectors that depend on incidental markup tend to be more brittle.
page.getByRole('heading', { name: 'Catalog' })targets an accessible heading.page.getByLabel('Search')can target a form control by its label.page.getByText('Load more')can locate visible text, though a role-based locator is preferable when it better describes the control.page.locator('[data-product-name]')targets elements matching a CSS selector.
For repeated content, inspect the page’s structure and choose a locator that matches the intended records rather than every element of a broad type. If a locator returns no results or too many, refine it and validate the output before saving.
Wait for the page state you need
Do not use an arbitrary sleep as the default synchronization strategy. It can be too short on a slow response and waste time on a fast one. Playwright locator actions auto-wait, and its Page API discourages waitForSelector in favor of locator waits or web assertions that express the required state.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
await page.goto('https://example.com/catalog');
await page.getByRole('list', { name: 'Products' }).waitFor({ state: 'visible' });
const rows = await page.getByRole('listitem').allTextContents();The example assumes that the page exposes a list with the accessible name “Products.” Inspect and adapt those roles and names for the actual page. A useful wait signal is the result container becoming visible, or a loading indicator disappearing, rather than an estimate of how many milliseconds the site usually takes.
Extract and validate records
Decide which fields each record needs before collecting data—for example, a title, publication date, and canonical page URL. Extract only those fields, then check required values before writing results to a file or database. Playwright does not automatically establish that scraped data is complete or correct.
- Flag records missing required fields or containing implausible values.
- Check for unexpected duplicates and for visible error or access-denied states.
- Keep the source URL and retrieval time with each result so its origin can be traced.
- Handle navigation failures and incomplete results explicitly rather than treating an empty list as successful collection.
Use network activity to diagnose a page
Playwright can observe and route HTTP and HTTPS traffic, including XHR and fetch requests. Network monitoring can help you understand how a page obtains data or test an application you control. An endpoint visible in browser traffic is not, by itself, permission to collect or reuse its data. Review the target site’s terms, access controls, and applicable requirements first.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
For work involving multiple pages or shared browser settings, a BrowserContext can contain several pages. Pages in a context share context-level settings such as viewport emulation and network routes.
Troubleshoot common scraping problems
The locator finds nothing
Confirm that navigation reached the intended page, inspect the visible page state, and verify that the role, accessible name, text, or selector actually exists. The page may render the content later or present an error or access-denied state instead.
The result is empty or partial
Wait for the relevant content state, check that the locator matches the intended records, and validate required fields before saving. Do not assume a successful navigation means the page has finished producing the data you need.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
The scraper races dynamic content
Wait for a meaningful locator or assert the expected page state instead of guessing a delay. Prefer locator-based synchronization over a manual selector wait where it expresses the same condition.
A CSS or XPath selector breaks
Prefer a role, label, or text locator when it identifies the intended target. If a structural selector is necessary, keep it as narrow and stable as possible, and revisit it when the site changes its markup.
Browser network traffic reveals a data endpoint
Treat the endpoint as a technical observation, not authorization. Check the site’s terms, access controls, and applicable requirements before collecting or reusing data from it.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesBest Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Or skip the browser setup
If your goal is to capture a page as an image or PDF rather than extract structured records, ScreenshotNeo offers a screenshot API and MCP server. A single request can return a PNG, JPEG, WebP, or PDF; it is not a substitute for a scraper that needs structured fields.
For a screenshot, see the ScreenshotNeo API documentation and send one GET request:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/catalog -o shot.webpScreenshotNeo accepts cookie or consent banners and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and whether the request was billed. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents using Claude, Cursor, or another MCP client. The Free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000 shots.
Sign up for 1,000 free screenshots a month, with no card required.
Recommended Free Tools
Frequently Asked Questions
Can Playwright scrape a page without launching a browser?
Playwright is browser automation. If the needed content is already in the ordinary HTTP response, use an HTTP client and HTML parser instead.
Does Playwright make scraping a website permitted?
No. Its documentation explains technical capabilities, not permission for a particular site. Check that site’s terms, access controls, and applicable requirements.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
