Heritrix
Heritrix: A configurable crawler for teams operating their own web-archiving infrastructure. Ranked #6 of 15 in Internet Archiving Software by our editors (8.0/10); pricing: Free plan; best for technical teams building configurable crawl workflows.
At a glance
- Editor score8.0 / 10
- PricingFree plan
- Best forTechnical teams building configurable crawl workflows
- Free planYes
- Paid fromFree
- Capture scopeWebsite
- Facts checked24 Sep 2026
Where it wins
- Set crawl scope, inclusion and exclusion rules, and resource limits
- Manage jobs through a web interface or REST API
- Write crawled content with WARC writer components
Where it doesn't
- Requires teams to operate their own infrastructure
- Job configuration and crawl policies require technical oversight
- Recurring schedules and retention depend on deployment and configuration
Our verdict on Heritrix
Heritrix is an open-source web crawler developed by the Internet Archive for archival-quality crawling. It is aimed at technical teams and organizations that operate their own web-archiving infrastructure, rather than buyers seeking a hosted capture service. Operators define jobs with seed URLs and configuration files, then set the scope and rules that determine which content the crawler includes or excludes. Heritrix is primarily used on Linux, requires Java 17 or later, and is also listed for Windows and API use. Its Apache License 2.0 makes it available as open-source software.
The crawl controls are the central strength. Teams can set limits based on downloaded bytes, document count, or elapsed time, and configure politeness policies. Heritrix can respect robots.txt and META nofollow directives, follow links within configured scope, and write collected material using WARC writer components. Job management is available through a web-based user interface as well as a REST API, giving technical operators both a graphical option and a way to control jobs programmatically. These capabilities suit workflows where teams need to shape crawling behavior and produce WARC output.
Heritrix is a crawler engine, not a complete archive-management experience. It does not provide a hosted capture service, and recurring schedules and retention depend on how an organization deploys and configures it. That distinction matters for teams planning the surrounding infrastructure and ongoing operations: the crawler handles configurable collection, while the supplied product description does not position it as a turnkey system for managing an archive end to end. Choose Heritrix if your team can operate its own crawling environment and values scope, limits, API control, and WARC output. A team looking for hosted capture or a complete archive-management service should consider a different type of product.
Your product? Our editors can test Heritrix in depth and publish a review, labelled as paid · from $149/mo →
Heritrix pricing
Heritrix fact sheet
| Free plan | Yes |
|---|---|
| Paid from | Free |
| Capture scope | Website |
| Scheduled crawls | Not verified |
| Capture frequency | Not verified |
| Archive export | Warc |
| Retention period | Not verified |
| API access | Yes |
| Deployment | Self-hosted |
| Platforms | Linux, Windows |
| Built for | Mid-market, Enterprise (editorial estimate) |
| Pricing | Free plan |
| Website | crawler.archive.org |
| Facts checked | 24 Sep 2026 |
Alternatives to Heritrix
- Archive-ItA managed web-archiving service with scheduled capture, replicated WARC storage, and collection tools.9.2
- InnoARCHIVEA broad web-archiving platform with flexible hosting and governance tools.8.8
- MirrorWeb Web ArchivingPreserves dynamic websites and communications with immutable records, search and replay.8.6
See all Heritrix alternatives →
Heritrix vs the competition
- Heritrix vs Archive-It
- Heritrix vs InnoARCHIVE
- Heritrix vs MirrorWeb Web Archiving
- Heritrix vs Pagefreezer
- Heritrix vs ArchiveBox
- Heritrix vs ArchiveWeb.page
Compare Heritrix with any tool side by side →
Used Heritrix? Be the first to review it
The editor score above is our own research. What this page doesn't have yet is a reader's view — what you used Heritrix for, what worked and what didn't. No stars are seeded and no review is paid for; an editor reads every one before it appears.
Write a reviewTwo minutes · verified accounts only · read by an editor before it appears
Featured on iTechGuides
Heritrix is listed in our Internet Archiving Software directory. Add the badge to your site — it links back to this page.
<a href="https://www.itechguides.com/products/heritrix/"><img src="https://www.itechguides.com/best/badge/heritrix.svg" alt="Featured on iTechGuides" width="230" height="46"></a>
Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026
Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.
Last updated · How we research and update