To automate a browser with an API, your program launches or attaches to a browser, opens a page, performs actions, and observes results through an automation library or protocol. For most new Chrome-based projects, start with a pinned Chrome for Testing build and a maintained library such as Puppeteer or Playwright. Choose Selenium with WebDriver BiDi when standards-based control, multiple language bindings, or Grid orchestration matters.
“Web API” here means browser-control APIs—not only JavaScript APIs exposed by a website. The sections below show a reproducible workflow, explain CDP and WebDriver BiDi, and help you choose the right tool for local development and CI.
What browser automation APIs actually do
A browser automation stack has three layers:
- Browser: Chrome, Chromium, Firefox, or WebKit renders pages and executes JavaScript.
- Protocol: CDP or WebDriver (including WebDriver BiDi) carries commands and events between your code and the browser.
- Library or framework: Puppeteer, Playwright, Selenium, WebdriverIO, or Nightwatch provides locators, waits, assertions, fixtures, and language bindings.
A typical test or job launches a browser, creates a context or session, navigates with a URL, finds controls, clicks or types, waits for a state change, checks the result, captures diagnostics, and closes the session. Use automation only on sites and accounts where you have authorization, and follow applicable terms and operational limits.
Choose the protocol: CDP or WebDriver BiDi?
Chrome DevTools Protocol (CDP)
CDP exposes commands and events for Chromium, Chrome, and other Blink-based browsers. It is useful for debugging, performance instrumentation, network interception, console logs, screenshots, and emulation. Its tip-of-tree definitions change frequently and have no guaranteed backward compatibility, so use a library’s supported API where possible and pin compatible browser and library versions.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problems#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
WebDriver BiDi
WebDriver BiDi is the W3C bidirectional protocol. A WebSocket connection lets automation code receive events such as network requests, console messages, and JavaScript errors while also sending commands. Selenium documents BiDi as the long-term standards path; its CDP integration is described as temporary while BiDi implementations mature.
Classic WebDriver is primarily request/response oriented. In Selenium, enable the webSocketUrl capability in your browser options before using BiDi logging, network, or script APIs.
Practical distinction
| Need | Better fit | Reason |
|---|---|---|
| Chromium-specific inspection or debugging | CDP through a supported library | Direct access to Chromium commands and events. |
| Standards-oriented events across browser vendors | WebDriver BiDi | Bidirectional WebSocket model and W3C direction. |
| Stable test abstractions | Playwright, Puppeteer, or Selenium APIs | Locators, waits, fixtures, and version handling reduce protocol coupling. |
Pin a reproducible Chrome environment
Chrome for Testing is a Chrome distribution intended for web-app testing and automation. Its versioned downloads let a team pin the browser, and releases are paired with matching ChromeDriver binaries. Pinning prevents a CI run from silently changing browser behavior.
- Choose a Chrome for Testing version supported by your framework.
- Install that exact browser in development and CI images.
- Use the matching ChromeDriver when your framework requires a driver.
- Record browser, driver, library, operating-system, and headless-mode versions in build logs.
Modern headless Chrome uses the same browser implementation as headful Chrome. Run headless on servers and CI; use a visible window locally when diagnosing selectors or layout.
Automate Chrome with Puppeteer (JavaScript)
Puppeteer is a JavaScript library maintained by Chrome’s Browser Automation team. It supports Chrome and Firefox. Its documented defaults use CDP for Chrome and BiDi for Firefox, and it also has production-ready BiDi support for both browsers. Puppeteer can download a compatible Chrome for Testing binary for its typical workflow.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
Install and run a minimal flow
mkdir browser-automation
cd browser-automation
npm init -y
npm install puppeteerconst puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({ headless: true });
try {
const page = await browser.newPage();
await page.goto('https://example.com', { waitUntil: 'networkidle2', timeout: 30000 });
const title = await page.title();
if (title !== 'Example Domain') {
throw new Error(`Unexpected title: ${title}`);
}
console.log({ title, url: page.url() });
} finally {
await browser.close();
}
})();Replace the example assertion with a user-visible outcome. Prefer locators or stable roles and labels over brittle XPath or generated CSS classes. Wait for a meaningful state—such as a visible result or URL change—rather than sleeping for an arbitrary number of milliseconds.
Useful Puppeteer patterns
- Navigation: set an explicit timeout and a suitable
waitUntilcondition. - Interaction: locate an element, ensure it is visible and enabled, then click or type.
- Events: subscribe to console, request, response, or page-error events before the action that triggers them.
- Isolation: create a fresh browser context per test or account to avoid cookies and local-storage leakage.
- Diagnostics: save a screenshot, HTML, console log, and trace when a test fails.
Playwright: a multi-engine test framework
Playwright provides launch APIs for Chromium, Firefox, and WebKit and its own protocol connection. A minimal JavaScript example is:
const { chromium } = require('playwright');
(async () => {
const browser = await chromium.launch({ headless: true });
try {
const page = await browser.newPage();
await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });
await page.getByRole('heading', { name: 'Example Domain' }).waitFor();
console.log(await page.title());
} finally {
await browser.close();
}
})();Playwright’s connectOverCDP path supports Chromium-based browsers only and is significantly lower fidelity than Playwright’s own protocol connection. Launching an external browser with incompatible arguments can also break features. Use the native Playwright connection unless you specifically need to attach to an existing Chromium process.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteSelenium and WebDriver BiDi
Selenium has bindings for more languages than Puppeteer or Playwright and supports Selenium Grid for distributed execution. Use it when your organization already has WebDriver infrastructure or needs broad language and browser coverage.
Python example with classic WebDriver
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
options.add_argument("--window-size=1280,900")
driver = webdriver.Chrome(options=options)
try:
driver.get("https://example.com")
heading = driver.find_element(By.TAG_NAME, "h1").text
assert heading == "Example Domain"
finally:
driver.quit()Enable BiDi events
Set the webSocketUrl capability in your browser options, then use Selenium’s higher-level BiDi logging, network, and script APIs. Exact method names vary by Selenium language binding and release, so match the examples to the version you pin. The important distinction is that events arrive over the WebSocket instead of requiring repeated polling.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
How to select Selenium, Playwright, or Puppeteer
| Question | Puppeteer | Playwright | Selenium |
|---|---|---|---|
| Browser engines | Chrome and Firefox; CDP default for Chrome, BiDi default for Firefox | Chromium, Firefox, WebKit | Browser support through WebDriver implementations |
| Languages | JavaScript/TypeScript | JavaScript/TypeScript plus official language clients | Broad language bindings |
| Event protocol | CDP or BiDi | Native Playwright protocol; limited CDP attachment | WebDriver and increasingly BiDi |
| Distributed runs | Usually supplied by your test runner or platform | Runner and platform integrations | Selenium Grid is a mature orchestration option |
| Best starting point | Chrome-focused JavaScript automation | Cross-engine end-to-end testing | Standards, existing Grid, or many languages |
Make the decision from seven constraints: required browser engines, programming language, need for a standards-based event stream, framework-specific features, distributed orchestration, version alignment, and whether runs are visible or headless.
Run automation reliably in CI
- Build a container or runner image with a pinned Chrome for Testing version and matching dependencies.
- Install the exact library and driver versions from a lockfile.
- Run headless with a fixed viewport, timezone, locale, and (when needed) geolocation.
- Use explicit waits for selectors, navigation, and network conditions; avoid global sleeps.
- Give each test isolated context, credentials, and test data.
- On failure, retain a screenshot, page source, browser logs, console errors, and network evidence.
- Retry only known transient failures. A retry must not hide deterministic selector or assertion defects.
Keep credentials in the CI secret store, not source control. Redact tokens from logs and disable video or HAR capture where it could expose sensitive data.
Common failures and fixes
Browser or driver version mismatch
Symptom: session creation fails or commands behave inconsistently. Fix: pin Chrome for Testing and use its matching ChromeDriver; align the Puppeteer or Playwright release with its supported browser.
“Element not found” or intercepted clicks
Cause: the page has not reached the required state, an overlay covers the control, or a locator is unstable. Fix: wait for a role, label, or stable test attribute; handle consent dialogs; scroll into view; and capture a failure screenshot.
Timeouts in headless CI
Cause: slow dependencies, constrained CPU, blocked resources, or an incorrect readiness condition. Fix: wait for the application’s ready selector, set realistic per-step timeouts, inspect console and network errors, and avoid treating “network idle” as universal proof of readiness.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
CDP command rejected
Cause: a tip-of-tree command changed or is unsupported by the browser version. Fix: replace raw CDP calls with the library API, or pin a known-compatible browser and library pair.
BiDi events never arrive
Cause: the WebSocket capability was not enabled, the browser/driver lacks the feature, or the binding version is too old. Fix: verify webSocketUrl, update within your supported matrix, and log the negotiated capabilities.
Automation behaves differently from a person
Web pages can distinguish trusted and untrusted events through the isTrusted flag and related event patterns. Puppeteer-generated input events are trusted, but that does not defeat bot detection or grant permission to access a site. Treat authorization and site rules as separate requirements.
Or skip the browser setup
If your task is simply producing a clean image or PDF of a URL, ScreenshotNeo provides a website screenshot API and MCP server. One GET request returns PNG, JPEG, WebP, or PDF. It accepts cookie and consent banners before capture and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each cleanup step can be disabled. Bot checks, CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers report the page verdict and billing status.
See the ScreenshotNeo API documentation for all options, including full-page lazy-image loading, CSS-element capture, dark mode, 12 device presets or custom viewports, retina scale, PDF paper and page ranges, custom CSS and JavaScript, clicks, selector waits, network-idle waits, request blocking, headers, cookies, user agents, authorization, timezone, geolocation, transparent backgrounds, resizing, TTL caching, signed image links, asynchronous webhooks, 100-URL bulk capture, usage data, and an OpenAPI specification. An MCP server exposes take_screenshot, get_page_info, and capture_pdf to Claude, Cursor, and other MCP clients.
Free tools Windows power users keep installed
One-click scans. No signup required.
cURL
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpPython
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)Node.js
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);The Free plan includes 1,000 shots per month with no card. Paid plans start at $5 for 3,000 shots; every feature is included on every plan. Create a free ScreenshotNeo account to try it.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Operational, cost, and security considerations
- Performance: reuse a browser process when safe, create isolated contexts, block unnecessary resources, and use a selector-based readiness condition.
- Reliability: pin versions, collect artifacts, and distinguish application failures from browser-start failures.
- Cost: parallel browsers consume CPU and memory; Grid or hosted runners add infrastructure costs. Limit concurrency to what the runner can sustain.
- Security: never expose remote-debugging ports publicly, restrict proxy and navigation targets, protect cookies and authorization headers, and sanitize downloaded files.
- Compliance: obtain permission before automating third-party sites, accounts, or scraping workflows.
Frequently Asked Questions
How do I automate a browser with an API?
Install a browser automation library, launch or attach to a browser, navigate to a URL, interact through stable locators, wait for observable state, assert the result, collect diagnostics, and close the session.
What is the difference between CDP and WebDriver BiDi?
CDP is a Chromium-oriented command/event protocol with fast-changing definitions. WebDriver BiDi is a W3C bidirectional WebSocket protocol intended for interoperable browser events and commands.
Can Puppeteer automate Firefox?
Yes. Puppeteer supports Firefox; its documented default is BiDi for Firefox, while Chrome uses CDP by default.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →How do I run browser automation in CI?
Use a pinned Chrome for Testing image, matching driver and library versions, headless mode, explicit waits, isolated test data, and failure artifacts such as screenshots and logs.
Should I use Selenium, Playwright, or Puppeteer?
Choose Puppeteer for Chrome-focused JavaScript work, Playwright for its multi-engine workflow, and Selenium when broad language support, WebDriver standards, or Grid orchestration is central.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
