Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A website monitoring script should check the failure that matters to you: whether an endpoint responds, whether important content is present, or whether a browser can load and render the page correctly. Start with a timed HTTP check for simple sites; use Playwright when JavaScript, lazy loading, or user interaction affects the result. Then schedule the check, keep useful failure evidence, and alert only after a defined threshold.

Choose what the monitor should prove

“The website is up” can mean several different things. Decide which condition counts as healthy before writing code; otherwise a script can report success while the page is unusable, or report failure because of an irrelevant detail.

Check type What it establishes Typical failure signal
Uptime The server can be reached and returned a status your policy accepts. Connection or timeout exception, or an unacceptable HTTP status.
Content health The response or rendered page includes a required phrase, element, or data value. A required marker is absent, even though the page returned a response.
Browser behavior A browser navigated and the relevant user-facing state appeared. Navigation failure, missing target element, or a relevant browser/request error.

For a simple status endpoint, an HTTP GET is usually enough. For a JavaScript application, an HTTP response may contain only a shell that fills in later. A browser check is appropriate when the monitored condition depends on that rendered state.

Build a lightweight HTTP check first

Use a bounded timeout so a stalled server cannot hang the monitor indefinitely. Record when the check ran, the status, elapsed time, and exception details. The example below uses Python’s urllib from the standard library, accepts only HTTP 200 as healthy, and makes at most three attempts. Adjust the accepted status policy for your endpoint—for example, a deliberate redirect may be normal for one URL and a defect for another.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Runnable Python example

Save as http_monitor.py and run with python http_monitor.py. Replace the example URL and expected phrase with the endpoint and content that matter to your service.

import json
import time
from datetime import datetime, timezone
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

URL = "https://example.com/health"
EXPECTED_TEXT = None  # For example: "service ready"
TIMEOUT_SECONDS = 10
ATTEMPTS = 3
RETRY_DELAY_SECONDS = 2


def check_once():
    started = time.monotonic()
    record = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "url": URL,
        "status": None,
        "elapsed_ms": None,
        "healthy": False,
        "error": None,
    }
    request = Request(URL, headers={"User-Agent": "website-monitor/1.0"})

    try:
        with urlopen(request, timeout=TIMEOUT_SECONDS) as response:
            body = response.read().decode("utf-8", errors="replace")
            record["status"] = response.status
            record["healthy"] = response.status == 200
            if EXPECTED_TEXT is not None and EXPECTED_TEXT not in body:
                record["healthy"] = False
                record["error"] = "Expected text was not present in the response body"
    except HTTPError as exc:
        # HTTPError is a received HTTP response, not a transport failure.
        record["status"] = exc.code
        record["error"] = f"HTTP status {exc.code}"
    except (URLError, TimeoutError, OSError) as exc:
        record["error"] = f"{type(exc).__name__}: {exc}"
    finally:
        record["elapsed_ms"] = round((time.monotonic() - started) * 1000, 1)

    return record


def main():
    last_record = None
    for attempt in range(1, ATTEMPTS + 1):
        last_record = check_once()
        last_record["attempt"] = attempt
        print(json.dumps(last_record))
        if last_record["healthy"]:
            return 0
        if attempt < ATTEMPTS:
            time.sleep(RETRY_DELAY_SECONDS)

    return 1


if __name__ == "__main__":
    raise SystemExit(main())

Each attempt emits one JSON line, suitable for a scheduler or log collector. A returned 404 or 503 is still an HTTP response; this example classifies it as unhealthy because its policy accepts only 200. Keep transport failures distinct from received statuses in your own records so that you can tell whether the server answered at all.

Use Playwright for JavaScript-rendered pages

Use a real browser when the check depends on client-side rendering, lazy-loaded content, or an interaction. Install Playwright’s Python package and its browser binaries with pip install playwright followed by playwright install. The synchronous API below navigates, waits for a meaningful page element, captures a screenshot on failure, and logs browser request and response events.

page.goto() waits for a navigation load event, but modern pages may continue fetching data or changing the interface afterward. Treat a specific visible element or phrase as the health assertion, rather than assuming that navigation alone proves the page works.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
AT-A-GLANCE Undated Website Address Book and Password Keeper, Black, 3.63 x 6.13 x .21 Inches (80-500-05)
  • Bookbound planner helps you keep track of passwords and favorite websites
  • Room for over 200 entries; 3.5 x 6 inch page sizes
  • User name and security questions field
  • Tips for what makes a strong password; web resources; notes pages
  • Printed on quality paper containing 30% post-consumer waste; black simulated leather cover; 3.63 x 6.13 x .21 inches

Runnable browser-monitor example

Save as browser_monitor.py. Set URL, READY_SELECTOR, and optionally EXPECTED_TEXT to match a stable, user-visible part of the page.

import json
import time
from datetime import datetime, timezone
from pathlib import Path
from playwright.sync_api import sync_playwright

URL = "https://example.com/app"
READY_SELECTOR = "main"
EXPECTED_TEXT = None  # For example: "Account overview"
TIMEOUT_MS = 15000


def main():
    events = []
    started = time.monotonic()
    record = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "url": URL,
        "healthy": False,
        "status": None,
        "elapsed_ms": None,
        "error": None,
        "diagnostic": None,
        "screenshot": None,
        "events": events,
    }

    with sync_playwright() as playwright:
        browser = playwright.chromium.launch(headless=True)
        page = browser.new_page()
        page.on("request", lambda request: events.append({
            "type": "request", "url": request.url, "method": request.method
        }))
        page.on("response", lambda response: events.append({
            "type": "response", "url": response.url, "status": response.status
        }))
        page.on("requestfinished", lambda request: events.append({
            "type": "requestfinished", "url": request.url
        }))
        page.on("requestfailed", lambda request: events.append({
            "type": "requestfailed", "url": request.url,
            "failure": request.failure
        }))

        try:
            response = page.goto(URL, wait_until="load", timeout=TIMEOUT_MS)
            record["status"] = response.status if response else None
            if response is None or response.status >= 400:
                raise RuntimeError(f"Main document returned {record['status']}")

            page.locator(READY_SELECTOR).wait_for(state="visible", timeout=TIMEOUT_MS)
            if EXPECTED_TEXT is not None:
                page.get_by_text(EXPECTED_TEXT, exact=False).wait_for(
                    state="visible", timeout=TIMEOUT_MS
                )
            record["healthy"] = True
        except Exception as exc:
            record["error"] = f"{type(exc).__name__}: {exc}"
            record["diagnostic"] = "Navigation or required page assertion failed"
            Path("monitor-evidence").mkdir(exist_ok=True)
            screenshot_path = Path("monitor-evidence") / (
                datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") + ".png"
            )
            try:
                page.screenshot(path=str(screenshot_path), full_page=True)
                record["screenshot"] = str(screenshot_path)
            except Exception as screenshot_exc:
                record["diagnostic"] += (
                    f"; screenshot failed: {type(screenshot_exc).__name__}: "
                    f"{screenshot_exc}"
                )
        finally:
            record["elapsed_ms"] = round((time.monotonic() - started) * 1000, 1)
            print(json.dumps(record))
            browser.close()

    return 0 if record["healthy"] else 1


if __name__ == "__main__":
    raise SystemExit(main())

The recorded event list can grow quickly on pages that make many requests. In a long-running monitor, retain only the most useful events or write them to a separate log. Also decide which browser errors matter: a third-party analytics request failing may not mean your own page is unhealthy.

Interpret responses, failures, and page state correctly

A non-2xx response and a failed network request are different conditions. A 404 or 503 means an HTTP response arrived; Playwright documents that such HTTP error responses still complete as requests, rather than triggering requestfailed. Reserve that event for transport-level failures such as network errors or timeouts. Your health policy must classify received statuses explicitly.

  • For the main document, decide which status codes represent success for your site and record the actual status.
  • For required content, assert an element or text that indicates the expected state. Prefer stable selectors over styling classes likely to change during a redesign.
  • For browser-dependent behavior, capture the page and diagnostic record when an assertion fails.
  • For subresources, distinguish a failed critical API request from an optional image, tracker, or unrelated third-party request.

A useful failure record includes the URL, check name, timestamp, status if available, elapsed time, exception, assertion or diagnostic message, and the screenshot path when one was captured. Keep sensitive headers, cookies, and page data out of logs unless you have a secure reason to retain them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Schedule checks and keep alerts actionable

Run the script from cron, a task scheduler, or a worker process at an interval that suits the service and its access limits. Scheduling is separate from the check itself: the scheduler starts the process, while the script’s exit code and structured output indicate the result.

  1. Run the script manually and confirm it exits with code 0 for healthy and 1 after exhausted failures.
  2. Set the schedule and capture standard output and errors in a log or monitoring system.
  3. Alert after a defined number of consecutive failures or a retry policy, rather than notifying on every transient blip.
  4. Include the latest status, elapsed time, error, and evidence location in the notification so someone can act on it.
  5. Test alert delivery with a controlled failure before relying on it.

Retries help smooth over brief network interruptions, but they also increase the time before a definitive failure and multiply requests. Keep attempts and delays bounded, and account for them when choosing schedule frequency. Alert thresholds should reflect how quickly you need to know about an outage without producing noise from a single transient error.

Respect access rules and avoid unnecessary load

Before monitoring a site, review its terms, authentication boundaries, and request-rate limits. Inspect /robots.txt as one signal about crawler access; Google describes it as telling search engine crawlers which URLs they can access. AWS also treats it as an indication of sections a crawler is allowed to visit. A robots.txt rule does not, by itself, grant permission for every monitoring purpose.

  • Monitor only systems and pages you are authorized to access.
  • Use credentials through a secure mechanism when an authenticated check is necessary; do not put secrets in source code or logs.
  • Choose a reasonable interval and avoid retries that create excessive traffic.
  • Do not attempt to bypass access controls, bot checks, or other restrictions.

Decide whether to maintain a script or use a service

A script offers control over assertions, evidence, and integrations. In exchange, you own scheduling, alert delivery, log retention, dependency updates, browser installation, and recovery when the monitor itself fails. A hosted service can reduce that operational work, but the right fit depends on its supported checks, access model, alerting, diagnostic evidence, and cost. No single approach is best for every site.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Need Script is a better fit when… Hosted monitoring may fit when…
Check behavior You need custom logic or a specific in-house assertion. A supported built-in check covers the requirement.
Maintenance You can own runtime, scheduler, dependencies, and browser updates. You want less infrastructure to operate.
Evidence and alerting You want records and notifications integrated with your own systems. The service’s evidence and alert channels meet your needs.
Access and compliance You need the monitor to run within a controlled environment. The service’s access model is acceptable for the target site.

Or skip the browser setup

If your monitoring workflow needs screenshots as evidence, ScreenshotNeo provides a website screenshot API and MCP server. A GET request can return an image or PDF. For a one-off capture, request a screenshot of the page you want to inspect:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

See the ScreenshotNeo documentation for request options. Cookie banners, newsletter popups, and chat widgets are removed before the shot; each cleanup step can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers indicate the page verdict and billing result. Its MCP server gives AI agents tools to take screenshots, inspect page information, and capture PDFs. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000.

Sign up free for ScreenshotNeo to start with 1,000 screenshots a month and no card.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Troubleshooting common failures

The HTTP check reports failure, but the site opens in a browser

Inspect the recorded status and exception. The endpoint may redirect, require authentication, reject the script’s request, or be reachable only from a particular network. Confirm the exact URL and define which status codes are acceptable; do not treat every response as a transport failure.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The browser loads but the content assertion times out

Check whether the selector exists in the current page and whether it becomes visible only after a user action. Use a stable, meaningful marker and wait for that marker rather than extending a generic sleep. If the page’s content is behind a consent prompt or login, the monitor may need an authorized test account and an explicit interaction.

Best Value
Password Book with Alphabetical Tabs, Password Keeper for Seniors 5.3"x7.7"
  • 【Featured A-Z Tabs & Untitle for Security】Our password books have recognizable alphabetical tabs with the colorful design allow you to locate quickly and save time. The anonymous cover of our password keeper is unobtrusive and stays secure.
  • 【Premium Quality & Perfect Size】This password journal features a eco-leather hardcover and 100gsm no-bleed paper, equipped with an elastic band, inner pocket, pen loop and bookmark. It comes in medium format (5.3 x 7.7 inches) which is the perfect size you need.
  • 【Clean Layout & Plenty of Space】 Each tab has 6 pages with 4 entries per page and contains more than 552 passwords in our password organizer. This password notebook also provides more password space in case you need to change your password.
  • 【Perfect Organization & Safe Placement】We ensure this password log book provides you with a secure space to keep passwords and web addresses. You won't have to worry about passwords being leaked or hacked.
  • 【Thoughtful Gift & Warm Heart】 Considering for practical gifts for family or friends? Our specially designed internet password book is sturdy and easy to use. Ideal for any occasion, it's a gift that truly shows care.

There is a 404 or 503 but no request failure event

This is expected for an HTTP response. Read the response status and apply your own health policy. A request-failure event is for transport problems, not every unsuccessful HTTP status.

The monitor reports intermittent timeouts

Compare elapsed time, attempt number, and timestamp across runs. Check whether the timeout is too short for the endpoint, whether the network or target is unstable, and whether retry volume is appropriate. Keep retries bounded and report the final outcome rather than silently treating a slow response as healthy.

The screenshot is missing after an error

The page or browser may have closed before capture, or the output directory may not be writable. The example creates its evidence directory and records a screenshot error separately; verify the process has permission to write there and inspect the diagnostic output.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Scheduled runs behave differently from manual runs

Schedulers often use a different working directory, environment, PATH, or credentials. Use an explicit Python environment and absolute paths for scripts and evidence, and ensure browser binaries installed for Playwright are available to the scheduled user.

Frequently Asked Questions

Can a website return HTTP 200 and still be unhealthy?

Yes. A 200 response proves that a response arrived, not that the expected content rendered or that critical browser behavior worked. Add a content or browser assertion when those conditions matter.

Should I use a fixed sleep before checking a JavaScript page?

A fixed delay can waste time or still be too short. Waiting for a meaningful element or text is a more direct test of the page state you care about.

Can robots.txt alone tell me whether monitoring is permitted?

No. It communicates crawler access guidance, but you should also consider the site’s terms, authorization, and rate limits.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Quick Recap

SaleBestseller No. 1
Bestseller No. 2
AT-A-GLANCE Undated Website Address Book and Password Keeper, Black, 3.63 x 6.13 x .21 Inches (80-500-05)
AT-A-GLANCE Undated Website Address Book and Password Keeper, Black, 3.63 x 6.13 x .21 Inches (80-500-05)
Bookbound planner helps you keep track of passwords and favorite websites; Room for over 200 entries; 3.5 x 6 inch page sizes
$9.22

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.