iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
When an app turns test scores into written feedback, each sentence is a claim about what the numbers mean. In a case study published on DEV Community on October 1, 2026, Daniel Pertu describes auditing CogniPrep’s practice-test reports and finding problems that ranged from awkward number phrasing to judgments the scores did not support. His central lesson: validate the meaning of generated feedback, not just its grammar.
What the audit uncovered
Pertu describes three kinds of defects in reports that translate test results into prose. One example also exposed a deeper scoring issue: ranking raw scores can misidentify a student’s weakest area when sections differ in difficulty.
Number wording can undermine clarity
Some generated phrases were grammatically awkward, including “0 of 6 right,” “You answered 0 of 16 correctly,” and “more than 1 seconds.” The article describes using small helper functions to improve pluralization, choose the right article before numerals, and spell out some numbers when they begin a sentence. These are copy fixes; they do not establish whether the underlying feedback is accurate.
Recommended Free Tools
Quantifiers and praise need evidence
The audit found words such as “mostly,” “majority,” “clearly,” and “reliably” appearing without evidence strong enough to support them. Calling the largest category “mostly” does not show that it represents a majority. Calling an ability reliable based on a result only slightly above a focus threshold overstates what that result says. A generic positive sentence also appeared for a low count of a desirable answer.
#1 Best Overall
The distinction matters: a sentence can be grammatically polished and still misrepresent a score. The threshold for an adjective or quantifier should match the meaning readers will reasonably take from it.
The page can differ from the generated string
One percentile heading rendered as “1 st percentile” because separately styled parts of the ordinal were laid out with a gap. Pertu says reducing the CSS gap fixed the visual defect. The example shows why checking the text string alone is not enough: layout can change how a claim appears to a reader.
Rank #2
Raw-score rankings can give the wrong comparison
A summary identified the lowest raw score as the weakest area even though the sections did not share the same difficulty. Pertu says the correction was to compare each section’s performance with its own expected line rather than rank raw scores alone. For systems that report multiple scores, the comparison is only meaningful when the relevant baselines are considered.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →How to audit generated feedback
Pertu says the release audit generated reports for each test using synthetic weak, middling, and strong sittings. The continuing test pattern asserted conditions over generated report text and cross-checked different sentences about the same metric. The practical value of this approach is that it tests the claims a reader actually sees, across more than one type of result.
Make claims conditional on explicit checks
- For each adjective, ranking, and quantifier, define the data condition that makes it true. Do not treat a grammar correction as a truth check.
- Check words such as “mostly” against the threshold their ordinary meaning implies, rather than using them as synonyms for “largest.”
- Make confidence language such as “reliably” depend on evidence that supports reliability, not merely a result just over a focus threshold.
- Compare performance against the relevant expected line when sections have different difficulty or baselines.
Allow the system to say nothing
A metric may be present without justifying praise or criticism. Pertu recommends preserving a valid no-claim state: if the evidence does not warrant a judgment, the system should not force one. He describes this as building “the vocabulary of silence first.” That is more honest than turning every metric into a binary good-or-bad verdict.
Check consistency across the report
Feedback often refers to the same metric in more than one sentence. Cross-check those sentences so an unsupported claim cannot disappear from the main insight and then reappear in related feedback. The audit’s text assertions are useful not only for catching a bad phrase but also for testing whether claims about the same result agree with one another.
Read the rendered page at realistic widths
Test representative weak, middling, and strong inputs, then inspect both the generated wording and the page as it appears at realistic screen widths. Pertu’s advice is concise: “assert on the text. Then read the rendered page.” That catches semantic defects in the output and visual defects introduced by styling.
What this case study does—and does not—show
The article is a first-person account of one product audit, not a comparative study of feedback systems or an independently reproduced experiment. Its examples illustrate how generated reports can fail; they do not establish how often the same problems occur elsewhere. CogniPrep describes itself as a psychometric-test practice platform with tests and feedback features on its official homepage. The case study’s broader engineering lesson is applicable beyond that product: generated prose should be treated as data-dependent claims, and each claim should be defensible from the data that produced it.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
