Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
If your backups grow much faster than the files you edit, fixed-size chunking may be part of the problem. A small insertion near the start of a large file can shift every later block boundary, making mostly unchanged data look different and causing it to be stored again. Content-defined chunking (CDC) places boundaries according to patterns in the data, which can help a backup reuse unchanged content after those shifts. It is not a guaranteed storage reduction: results depend on your files, settings, compression, and retained history.
First, identify which backup size is growing
“Backup size” can describe several different measurements. A snapshot may represent the full logical contents of your files while storing only newly encountered chunks in the repository. Before changing software or settings, find out which number is actually too large.
- Source size: how much data exists on the computer or server.
- Bytes scanned or read: how much data the backup process examines. This can be high even when little new data is stored.
- Bytes transferred: how much data is sent to the destination, which may differ from both source size and repository growth.
- Newly stored data: new payload added during a run after reuse of existing chunks.
- Total repository size: data retained across all snapshots, including unique historical content still needed for restore points.
Borg describes repository-wide deduplication across backups and machines that share a repository; its scope is a Borg behavior, not a universal property of backup programs. Borg’s project overview describes content-defined chunking as deduplicating everything in its repository.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsHow a small edit can make a backup much larger
Fixed-size chunks follow offsets
A backup program can divide a file into blocks at regular byte offsets. If you insert or remove bytes near the beginning of a large file, the later boundaries shift. The later blocks then contain different combinations of bytes, even where the underlying content is mostly unchanged. A system comparing blocks may consequently need to store many of them again. Restic’s design documentation explains this boundary-shift problem.
#1 Best Overall
- Easily store and access 2TB to content on the go with the Seagate Portable Drive, a USB external hard drive
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop
- To get set up, connect the portable hard drive to a computer for automatic recognition no software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
Content-defined chunks follow patterns in the data
CDC uses a rolling fingerprint to find suitable cut points from the content rather than relying only on fixed offsets. After an insertion, a familiar pattern farther along can allow the chunker to establish a matching boundary again. The chunks beyond that point may then match content already in the repository and be reused. Restic explains this approach in its overview of content-defined chunking.
For example, imagine a large virtual-machine image with a small change near its beginning. Fixed-offset blocks after the change may all start at new positions. With CDC, later matching content can produce familiar boundaries, limiting how much unchanged data appears new. This is a mechanism that can help; it does not ensure a particular reduction for every file or backup.
Chunking, deduplication, compression, and retention do different jobs
- Chunking divides data into pieces and determines where boundaries fall.
- Deduplication checks whether a chunk is already present and reuses it instead of storing another copy.
- Compression encodes stored data in fewer bytes, trading storage against processing time and sometimes transfer time.
- Retention determines how many restore points remain. Keeping more snapshots can retain unique historical data even when repeated chunks are deduplicated.
CDC improves the chance that unchanged regions will produce reusable chunks after edits shift offsets. It is not itself compression, and neither CDC nor deduplication removes the storage cost of unique changes that remain in retained history.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWhen CDC helps—and when it may not
Large files with small internal changes
Virtual-machine images, disk images, databases, and archive files can change internally even when the user-facing change seems small. When a change shifts later offsets, CDC may let the backup identify reusable regions that fixed-size chunking would miss.
Rank #2
- Easily store and access 1TB to content on the go with the Seagate Portable Drive, a USB external hard drive.Specific uses: Personal
- Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop. Reformatting may be required for Mac
- To get set up, connect the portable hard drive to a computer for automatic recognition no software required
- This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
- The available storage capacity may vary.
Data that is already compressed
Many media formats and other compressed files may have little room for another compression pass. Compression settings are separate from chunking, and the available sources do not establish a universal compression result for each file type.
Workloads suited to fixed-size chunks
CDC is not automatically the best choice for every workload. Borg notes that fixed chunking can be efficient for block devices and raw disk images. Consider how your data changes, how it is restored, and how the repository is maintained before choosing a chunking method. Borg’s internals documentation describes its available chunker options.
What the documented implementations actually specify
Algorithm details below are specific to the named tools, not general requirements for CDC. Defaults and options can differ by release, so check the documentation for the version you have installed.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →| Tool and documentation context | Chunking details | Deduplication and related behavior |
|---|---|---|
| Restic design documentation | Uses a 64-byte sliding window and Rabin fingerprints. Its documented blob sizes range from 512 KiB to 8 MiB, with a 1 MiB average target. | Matching content can remain reusable after bytes are inserted or removed. Figures are implementation details, not a promised savings rate. |
| Borg 2.x internals documentation | Lists FastCDC as the default in the documented 2.x development internals, alongside Buzhash variants, keyed AES-based chunkers, and fixed-size chunking. | Supports repository-wide deduplication. Verify the documentation for your installed release before treating a setting as its default. |
Neither implementation detail predicts how much a particular backup will shrink. A benchmark winner for your workload cannot be inferred from these descriptions alone.
Rank #3
- High capacity in a small enclosure – The small, lightweight design offers up to 6TB* capacity, making WD Elements portable hard drives the ideal companion for consumers on the go.
- Plug-and-play expandability
- Vast capacities up to 6TB[1] to store your photos, videos, music, important documents and more
- SuperSpeed USB 3.2 Gen 1 (5Gbps)
Diagnose backup growth before changing settings
- Compare the right measurements. Check whether the increase is in source data, bytes scanned, transferred data, newly stored payload, or total repository size. Use the backup tool’s own reporting and destination accounting where available.
- Look for frequently changing large files. Identify whether VM images, disk images, databases, or archives are changing internally. Their apparent small edits may affect many blocks under offset-based chunking.
- Check the tool’s documented chunking and deduplication behavior. Confirm whether it supports CDC, what its deduplication scope is, and whether the repository can be shared across snapshots or machines. Restic and Borg are documented examples, but a migration also depends on your operating systems, repository format, encryption, destination, restore process, and maintenance needs.
- Inspect compression as a separate setting. Borg’s quickstart describes lz4 as its default compression and offers zstd and other choices; stronger compression can use more CPU. Consult the guide for your installed release and weigh time and resource use against storage needs. Borg’s quickstart covers its compression choices.
- Review retention and pruning. Deduplication can avoid storing repeated payloads, but unique data kept for older restore points still occupies space. A longer retention policy can therefore increase repository size.
- Do not casually change chunker parameters in a populated repository. Different boundaries can make touched files store again. Borg warns that the effect can accumulate as files are touched and old archives are pruned. Test on a copy or use a fresh repository when appropriate, and verify the migration and restore plan before making a change. See Borg’s notes for parameter-related caveats.
Choose a backup approach around your recovery needs
Compare tools and configurations on the factors that affect both storage and a successful restore:
- Chunking: algorithm, adjustable chunk sizes, and compatibility with the repository you already use.
- Deduplication scope: whether reuse applies within a file, across snapshots, or across machines sharing a repository.
- Compression: storage reduction versus CPU use and transfer speed for your actual data.
- Security: encryption behavior and how keys are created, stored, and recovered.
- Fit: supported source platforms and destination types.
- Recovery and upkeep: restore workflow, repository verification, pruning behavior, and migration complexity.
Borg documents local disks and USB drives as possible repository destinations. A local backup drive can provide capacity or an offline copy, but it does not perform CDC or reduce the amount of data stored by itself. Borg’s official site describes its repository destinations.
There is no generally applicable CDC savings percentage established by these implementation descriptions. To estimate the effect on your own data, use a representative test and compare newly stored repository data under the same retention and compression settings; do not infer the result from a tool’s chunk size or algorithm name alone.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
