The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To capture an infinite-scroll search page with Puppeteer, first identify whether the document or a results panel scrolls. Then scroll in bounded increments, wait for evidence that more results appeared, and stop at a confirmed end marker or after a defined number of attempts without progress. Only then take the screenshot. fullPage: true captures the page as it exists at that moment; it does not load the rest of the feed for you.
What you need to adapt for the search page
There is no universal results selector or end-of-results signal: search pages differ, and some scroll a nested panel instead of the document. Inspect the target page and choose selectors and a completion rule that match its visible behavior. The example below uses result-count growth and an optional end marker; replace both selectors with ones verified on your page.
- Result selector: a selector that matches each rendered search result.
- Scroll container selector: the element that actually scrolls, or
nullwhen the document scrolls. - End marker selector: an element that appears when the feed is complete, or
nullif the page has no reliable marker.
Load results with a bounded Puppeteer loop
This ES module example navigates to a search URL, scrolls the selected owner in increments, and waits briefly for the result count to grow or an end marker to appear. It has a maximum round count and stops after three rounds without progress. The wait timeout is a fallback, not evidence that the feed is complete.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
import puppeteer from 'puppeteer';
const searchUrl = 'https://example.com/search?q=puppeteer';
const resultSelector = '.search-result'; // Replace with the page's result selector.
const scrollContainerSelector = null; // Use a selector for a nested scrolling panel, if needed.
const endMarkerSelector = '.end-of-results'; // Use null if there is no reliable marker.
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
await page.goto(searchUrl, { waitUntil: 'domcontentloaded' });
let previousCount = 0;
let stalledRounds = 0;
const maxRounds = 40;
for (let round = 0; round < maxRounds && stalledRounds < 3; round++) {
const currentCount = await page.evaluate(selector =>
document.querySelectorAll(selector).length,
resultSelector,
);
if (currentCount > previousCount) {
previousCount = currentCount;
stalledRounds = 0;
} else {
stalledRounds++;
}
const done = endMarkerSelector
? await page.evaluate(selector => Boolean(document.querySelector(selector)), endMarkerSelector)
: false;
if (done) break;
const scrollResult = await page.evaluate(selector => {
const container = selector
? document.querySelector(selector)
: document.scrollingElement;
if (!container) throw new Error('Scroll container not found');
container.scrollBy(0, Math.max(300, container.clientHeight * 0.8));
return true;
}, scrollContainerSelector);
if (!scrollResult) break;
const countBeforeWait = previousCount;
try {
await page.waitForFunction(
(resultSel, endSel, count) =>
document.querySelectorAll(resultSel).length > count ||
Boolean(endSel && document.querySelector(endSel)),
{ timeout: 5000 },
resultSelector,
endMarkerSelector,
countBeforeWait,
);
} catch {
// Recheck on the next round; a timeout alone does not prove completion.
}
}
await page.screenshot({ path: 'search-results.png', fullPage: true });
} finally {
await browser.close();
}Save this as an .mjs file and install Puppeteer in the project with npm install puppeteer. Replace the example URL and selectors before running it. The code is an adaptable pattern, not a verified script for any particular search site.
#1 Best Overall
- [INTEL POWERED CONTENT] - Built with a 8th Generation Hexa-Core Intel i5 and 32GB of DDR4 RAM; Modern, Windows 11 ready, with 4K support, Executive multitasking, media streaming and smooth, multi-tab web browsing; Perfect as an all-purpose multimedia computer; built for content creators; Plenty of RAM and Mass storage for photo and video editing powered by Intel HD 630
- [LATEST WIRELESS TECH] - This Dell Desktop Computer easily connects to the internet through the Built In WiFi / Bluetooth
- [SOLID STATE STORAGE] - This Dell Computer setup comes with an ultra-fast 1TB Solid State Drive (SSD); Setup as the primary boot device; Boot and load programs with lightning speed ; Additional expansion available
- [BUY & OWN WITH CONFIDENCE] - From the world's largest Microsoft Authorized Refurbisher; Quality Guarantee and Free Tech Support; Award-winning Customer Service; | Support Sustainable Business
- [MODERN HI-SPEED PORTS] - USB 3.0 (x4) | USB 2.0 (x4) | DisplayPort (x1) | HDMI Port (x1) | Audio Combo Jack (x1) | Audio Out (x1) | RJ-45 Ethernet (x1) | Internal SATA (x3)
Important implementation details
- The loop checks the result count at the start of each round. If a scroll produces results asynchronously, the next round detects the increase and resets the stalled counter.
- Pass both selectors into
waitForFunction()as arguments. Avoid depending on globals injected into the page context. - If there is no end marker, the no-progress limit and
maxRoundsprovide a bounded exit, but cannot prove that all results loaded. Increase the limits only when the page’s behavior justifies it. - If a scroll panel cannot be found, the example throws an error rather than silently scrolling the wrong element.
Choose the right scroll and progress signals
Document or nested results panel
Check which element’s scroll position changes when you scroll the results. For document scrolling, document.scrollingElement is the example’s default. For a nested panel, set scrollContainerSelector to that panel’s selector. Scrolling the wrong owner may leave the feed stationary even though the script continues.
Some sites respond specifically to user-like wheel events. Puppeteer’s Locator interaction guide describes scrolling an element with mouse-wheel events; use that approach if direct scrollBy() does not trigger the site’s loading behavior. See Puppeteer page interactions.
Rank #2
- Model: Dell OptiPlex 7050 Small Form Factor (SFF)
- Processor: Intel Core i7-7700 3.60 GHz
- Memory: 32GB DDR4 Ram
- Storage: 1TB Solid State Drive (SSD) Fast Boot + Storage
- Operating System: Windows 11 Pro (64-bit)
Result growth, end markers, and network quiet
A changed result count or a site-provided end marker is more closely tied to feed completion than a quiet network. Puppeteer’s waitForNetworkIdle() waits for network activity to become idle, with at least the configured idle interval; it does not establish that a search feed has no more results. A feed may pause between batches and request more only after another scroll. Treat network idleness as a settling signal, not a completion test. See Page.waitForNetworkIdle().
Recommended Free Tools
Page.waitForFunction() waits until a page-context function returns a truthy value. Use it to observe a condition such as a larger result count or an end marker. If it times out, recheck progress or stop safely; do not interpret the timeout as proof that every result loaded. See Page.waitForFunction().
Rank #3
- IMMERSIVE 24 INCH DISPLAY: Experience stunning clarity on a Full HD IPS screen with ultra-thin bezels, offering a 90% screen-to-body ratio that makes everything from spreadsheets to streaming come alive with vibrant colors and crisp details.
- POWERFUL INTEL PROCESSING: Tackle demanding tasks with ease thanks to the Intel processor and 16GB of high-speed memory, delivering smooth performance whether you're multitasking between applications or running productivity software.
- GENEROUS STORAGE: Store all your important files, photos, and programs with blazing-fast solid state drive technology that ensures quick boot times, rapid file access, and plenty of space for your digital life.
- ENHANCED PRIVACY AND COLLABORATION: Work confidently with the pop-up privacy camera that tucks away when not in use, plus dual microphones with noise reduction for crystal-clear video calls that keep you connected professionally.
- ECO-CONSCIOUS DESIGN: Feel good about your purchase with an EPEAT Gold registered and ENERGY STAR certified computer that combines premium performance with responsible environmental manufacturing practices.
Capture one image, a section, or several segments
One full-page screenshot
After the feed reaches the intended endpoint, capture the page with await page.screenshot({ path: 'search-results.png', fullPage: true }). fullPage requests a full-page image; its documented default is false. It captures the page at capture time and does not itself scroll an infinite feed to load missing results. See Puppeteer ScreenshotOptions.
A result element or viewport-sized segments
For a particularly long feed, a single image may be unwieldy. Puppeteer supports screenshots of a specific element through ElementHandle.screenshot(); use that when the results occupy a distinct element. Alternatively, capture deliberate viewport-sized segments if separate, readable files are more useful than one very tall image. Puppeteer documents page and element screenshots in its screenshots guide.
Rank #4
- This Certified Refurbished product is tested and certified to look and work like new. The refurbishing process includes functionality testing, basic cleaning, inspection, and repackaging. The product ships with all relevant accessories, a minimum 90-day warranty, and may arrive in a generic box. Only select sellers who maintain a high-performance bar may offer Certified Refurbished products on Amazon.com.
- Dell Optiplex 3050 SFF Desktop computer PC, Intel Quad Core i5-6500 up to 3.6GHz, 16GB DDR4, 256GB SSD
- Includes: USB Keyboard & Mouse, USB WiFi adapter, Microsoft office 30 days free trail.
- Port: Front: USB 3.0(2), USB 2.0(2); Rear: DP, HDMI, USB 3.0(2), USB 2.0(2), RJ-45.
- Support 4K (3840x2160) Dual display, makes it easy to connect two monitors at the same time, and you can expand working Windows, mirror content, or expand a single window across multiple monitors.
Troubleshoot missing or incomplete results
- The screenshot shows only the first batch:
fullPagedoes not trigger feed loading. Run the scroll loop first and verify that the result count grows. - Scrolling has no effect: confirm whether the document or a nested panel owns scrolling. Set the panel selector if needed; if the site requires wheel events, try Puppeteer’s Locator scroll interaction.
- The loop stops too early: make sure the result selector matches individual results, not a wrapper whose count never changes. Check whether an apparent end marker is present before the real end, and revise the completion rule.
- The loop runs to its limit: the page may have no end marker, the selector may be wrong, or the feed may be returning no additional results. Inspect the rendered DOM and increase bounds only if the page’s behavior warrants it.
- The wait times out between batches: a timeout is only a failed wait for the chosen predicate. Recheck the count and marker on the next round; do not equate a quiet interval with completion.
- The image is excessively tall: capture the results element or use multiple viewport-sized captures instead of one full-page image.
Or skip the browser setup
ScreenshotNeo offers a screenshot API and MCP server. Its one-call API can return an image or PDF; the example below saves a WebP screenshot. See the ScreenshotNeo API documentation for the request options.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpScreenshotNeo accepts cookie and consent banners before capture and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each of those steps can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and responses identify the page verdict and billing status in headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for AI agents. The Free plan includes 1,000 screenshots a month without a card; paid plans start at $5 for 3,000 shots. Sign up for free and get 1,000 screenshots a month with no card.
Frequently Asked Questions
Does Puppeteer support infinite-scroll screenshots directly?
Puppeteer provides the browser controls and screenshot methods; your script must load the feed and decide when it has reached the intended endpoint.
Best Value
- Connectivity: Includes WiFi, Bluetooth, and LAN for wireless and wired connections
- Memory: Features 16GB DDR4 RAM for smooth multitasking and performance
- Storage: Combines 500GB SSD and 1TB HDD for ample storage space
- Graphics: Integrated Intel UHD Graphics 630 for crisp visuals and video playback
- Design: Sleek desktop tower with black color and slim profile for modern look
Can I know every result loaded if the site has no end marker?
Not from a no-progress timeout alone. It shows only that your chosen progress signal did not change during the wait; validate the page-specific behavior before treating the capture as complete.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
