Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Yes—but only if the pipeline checks business rules, not just whether a file can be loaded. In Jigon Yoo’s reported 2026 example, a CSV order batch loaded without errors even though one numeric amount, $4.5 million, overwhelmed the revenue total. Explicit dbt tests caught the planted defects and prevented the downstream revenue mart from rebuilding. The example also shows what such a gate cannot guarantee: tests only catch encoded assumptions, and a blocked build can leave old output looking current.
How could a successful load contain a $4.5 million error?
A loader answers a mechanical question: can it read the file and move its rows into the warehouse? It does not necessarily answer whether an amount makes sense for an order. In Yoo’s example, a small order warehouse used CSV, DuckDB, dbt staging, and a daily revenue mart. The bad amount was valid numeric data, so type checking alone would accept it.
Yoo reports that the clean batch contained 900 orders and $395,751.28 in revenue. The sabotaged batch contained 901 orders and $4,905,051.18. One row, order 401, carried an amount of $4,500,000; removing it brought the sabotaged total to $405,051.18. The author says that remainder was about 2% above the clean batch, a difference he considered plausible given separate draws and smaller defects. These are results of his example, not an independently audited measurement or an industry statistic.
Recommended Free Tools
As Yoo puts it, “Moving rows is the load’s whole job, and it did that job.” The gap is between successful ingestion and trustworthy data.
#1 Best Overall
What did the data contract catch?
The example defines checks in dbt using generic tests and custom tests. The listed checks cover different failure classes; a single “data quality” pass is not one universal guarantee.
- Structure and required values: non-null checks catch missing values in required fields.
- Uniqueness and relationships: tests can catch duplicate keys and references that do not match expected records.
- Allowed values: accepted-value checks limit fields to declared categories.
- Business plausibility: custom checks cover non-negative amounts, amount magnitude, future signup dates, and a reporting window.
Yoo says the example had twelve planted defects and fifteen tests. For the sabotaged batch, he reports twelve failures and one skipped mart; the clean batch built without test errors. His reported command output was PASS=22 … ERROR=0 for the clean run and PASS=9 … ERROR=12 SKIP=1 for the sabotaged run. Those results show the behavior of this particular setup, not a guarantee that every data contract or dbt project will respond the same way.
Rank #2
How does a failed test stop the mart?
In this workflow, dbt build runs the staging checks before building the revenue mart on top of staging. When staging tests fail, the dependent mart is skipped rather than rebuilt from data that violated the contract. That is a useful gate: downstream reporting does not silently consume a batch the project has declared unacceptable.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsA skipped build is not the same as removing prior output. The last successful mart may remain available, which means its figures can be stale even though a dashboard still displays them. A pipeline should make the failed run visible and communicate output freshness or status, rather than letting an unchanged mart appear current.
Why can fixed thresholds still miss a bad batch?
A fixed maximum is good at catching values far outside an expected range, but it can miss an error that remains plausible under the chosen ceiling. Yoo notes that multiplying a sub-$900 order by 100 would still produce less than $90,000—below this example’s $100,000 threshold. A unit or decimal-place mistake can therefore pass a broad upper-bound test.
As Yoo writes, “A fixed ceiling is a check for impossible values, not for wrong ones; a unit error needs something relative, like the value against its own history.” Relative checks—such as comparing an order with its prior values or a relevant historical distribution—can help detect shifts that an absolute ceiling misses. They still need carefully chosen assumptions: legitimate business changes can also create unusual values.
Rank #4
Which layer should a check inspect?
Validation after normalization can lose evidence of defects in the original input. For example, trimming whitespace or converting case may make a raw-value problem disappear before a staging test sees it. The checks should match the failure being prevented and run at the layer where that evidence still exists.
| Check area | Useful layer | Possible failure response |
|---|---|---|
| File shape, column types, and raw formatting | Raw arrival, before cleanup | Reject or quarantine malformed input |
| Required fields, keys, relationships, and allowed values | Raw arrival and/or normalized staging, depending on whether normalization can hide the issue | Fail validation and block dependent builds |
| Business bounds and historical anomalies | Staging or a layer with the necessary history | Block, quarantine, or alert for review |
| Volume and freshness | Pipeline/job monitoring alongside data tests | Alert, mark output stale, or prevent consumers from treating it as current |
This is a practical way to reason about the contract, not a comparison or benchmark of tools. Choose the response deliberately: rejecting input is safest when it is unusable; quarantine preserves it for investigation; blocking a downstream build prevents propagation; and alerting while marking output stale makes the operational state clear.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What does this example not guarantee?
- Tests only cover written assumptions. Twelve catches in this demonstration correspond to twelve planted defects, not every possible failure mode. An unanticipated but harmful value can still pass.
- The demonstrated contract does not check volume or freshness. Yoo notes that an empty batch with declared column types could pass the listed checks and produce an empty mart. A job that never ran is not itself a failed data test.
- A passing normalized model does not prove the raw input was clean. Cleanup can erase signals that would have mattered at arrival.
- A blocked mart may be old, not absent. Consumers need an explicit failure and freshness signal to distinguish a current result from a retained prior build.
The example answers “can a pipeline stop a bad batch?” with a qualified yes: it can stop batches that violate checks the team has implemented, and only if failures actually gate downstream work and are visible to consumers. It does not establish whether an agent running a load would independently spot a defect; an automated agent needs the same explicit rules and operational safeguards.
How to reproduce the reported example
Yoo identifies the reproduction project as jigonyoo/warehouse-quality-gate. The article describes generating both batches and running the evidence commands from that repository. The reported environment was dbt-core 1.12.5 and dbt-duckdb 1.11.0 when the author last checked it on 2026-09-30; these are dated environment details, not a current compatibility promise. Yoo says he reran both cases from a fresh clone on that date. The reported experiment and figures are described in the case study.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
