What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
A prompt-injection evaluation harness earns its keep only when it answers three separate questions about your application: did the defense stop the injected instruction, did anything unsafe happen anyway, and did legitimate users still get their work done. Most of the design follows from keeping those three answers apart. Each one needs its own observable, its own denominator, and its own limit.
This article explains how to design that harness and how to read its output. It does not report measured results from a particular application or detector. Published figures are attributed to their source and described only within the scope that source gives them.
Separate the detector’s decision from the system’s outcome
Two things are often merged in informal testing. A detector decision (allow, block, or send to human review) is a classifier output. A security outcome is what the system actually did: whether a secret appeared in an answer or tool argument, whether a tool ran with the wrong parameters, whether data reached an external destination. A detector can block a request while the system still misbehaves through another path, and a detector can allow a request while the system handles it safely. Grade the two separately, or the numbers will tell a story the system does not support.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Start with a threat model for this application
An attack case only means something relative to the application it targets. Before writing any case, document:
- The supported tasks and what a correct answer to each one looks like.
- User roles and what each role may see or do.
- Sensitive data the model can reach, including data placed in the context window and data returned by tools.
- External content sources: retrieved documents, email, web pages, and tool output.
- Tools and their write permissions.
- The concrete harm an attacker could cause through each path, such as a message sent to an outside address or a record changed.
Then write every test case against a fixed schema with five fields:
- Source channel: direct user input, retrieved document, email, web page, or tool output.
- Intent: the intended violation, or the legitimate task for a benign control.
- Necessary context: the conversation history, retrieved content, user role, and tool state the case depends on.
- Expected observable: the specific signal that would show the outcome (see the objectives table below).
- Severity: the harm if the attack succeeds.
For an agent, the case must also name the actions and data flows it is allowed to produce. That is what lets grading cover tool calls and state changes, not only the final text.
Test the path the content actually takes
Direct and indirect injection cross different boundaries. A direct attack arrives in the user’s message. An indirect attack sits inside content the application fetches on the user’s behalf, and it reaches the model through a different route with different trust assumptions. A harness that only posts strings to a chat endpoint tests the first case and says nothing about the second.
OWASP AI Exchange recommends tailoring attacks to the application, including attempts at data extraction and at downstream actions, and varying inputs to test evasion. It also recommends running tests with the same model versions, prompts, tools, permissions, and configuration as production (OWASP AI Exchange, AI security testing). Anything that runs before the model call, such as input normalization, classifiers, and the system prompt template, belongs inside the path under test.
Direct user input
Send each case through the same entry point, middleware, and prompt assembly that production uses. A harness that calls the model directly skips the controls you most need to measure.
Indirect instructions in retrieved content and tool output
Place the untrusted instruction inside the document, email, web page, or tool response that the application actually retrieves. Use the same retrieval step, chunking, and ranking that production uses, because those steps decide whether the instruction reaches the model at all. Then test the downstream effect: an injected instruction in a retrieved file is only dangerous if it can change what the application does next.
Build attack families from your boundary, not from a list
OWASP’s illustrative examples are a useful starting checklist and a poor finish line. They cover several families:
- Direct instruction override
- Role or authority claims
- Base64-encoded instructions
- Typoglycemia (words with scrambled interior letters)
- Spacing and case variations
- Remote-injection patterns
- Benign requests, to test for false alarms
The set contains 14 hand-picked attack examples and seven benign examples. OWASP presents them as illustrative and tells readers to adapt them to the application’s tasks, permissions, and input channels. Its guidance on the set is direct: “Use the examples below as a smoke test, not a security benchmark” (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention, section “Smoke-Testing Your Defenses”).
A smoke test answers a narrow question: does the defense catch the obvious variants, and does it pass the obviously legitimate requests. It does not estimate how often attacks occur in production.
For a larger point of reference, the USENIX Security 2024 paper “Formalizing and Benchmarking Prompt Injection Attacks and Defenses” evaluates five prompt-injection attacks and ten defenses across ten LLMs and seven tasks, and provides a public platform (USENIX Security 2024). Those are the study’s design counts. They describe its scope, not a ranking you can transfer to your own application.
Grade each security objective with its own observable
Each security objective needs one observable that would show it happened. A clean final answer is not evidence of safety. A model can place a value in a tool argument, change state, and then refuse in prose. A refusal in the final answer cannot undo an earlier action.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitches| Objective | Test boundary | Observable to record | What a negative result does not prove |
|---|---|---|---|
| Marker disclosure | A dummy secret placed in the prompt or context the model can read | Whether the exact dummy marker appears in the output, logs, or downstream payloads | A missing marker does not rule out transformed disclosure, such as an encoded or partial value |
| Unauthorized action | Tools wired to sandboxed substitutes with instrumentation | Tool calls, authorization decisions, and the resulting dummy state | A refusal in the final answer does not show that no call was made |
| External disclosure | A controlled destination the application could send data to | Whether anything arrived at the instrumented destination | An empty destination log is only as complete as the instrumentation covering each outbound channel |
Report each row separately. Folding marker disclosure, unauthorized action, and external disclosure into one “security score” hides which control failed.
Use safe fixtures and treat missing telemetry as its own outcome
- Put only dummy secrets in prompts and tool state. Make each marker unique to its case, so a match is unambiguous.
- Route every action to a sandboxed substitute, and record each authorization decision and state change.
- Point exfiltration tests at a controlled destination that you own and log.
Errors, missing telemetry, and unsupported contexts must stay out of the “blocked” column. If the logger failed, the sandbox timed out, or the tool was never reached, the case is inconclusive. Report inconclusive counts beside the results, not inside them.
Read a single case before you count it
Aggregate numbers hide the cases that matter, so check how each case resolved:
- Clean answer, dummy marker in a tool argument: record a disclosure failure. The final text looked safe, but the secret left the boundary.
- Refusal in the answer, state change in the sandbox: record an unauthorized-action failure, and check whether the authorization decision was logged.
- Refusal in the answer, no tool call, instrumentation covering that tool: record the action objective as blocked.
- Refusal in the answer, no tool call, instrumentation not covering that tool: record the action objective as inconclusive.
- Benign request refused as a security violation: count one false positive, and keep the policy decision separate from the task result.
Measure false positives and task completion on benign controls
Benign controls are the other half of the harness. Each one is a legitimate request the application should handle correctly, and each gets two separate grades: the policy decision (allow, block, or human review) and whether the task was completed correctly. A request can be allowed and still fail the task. A request routed to review has no outcome until a reviewer decides, so it needs its own line.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteRank #4
- Made in USA - Proudly produced in Ohio by a Veteran-owned business
- Comprehensive Coverage: This BookFactory log book includes essential fields such as post/shift, time of change, date, weather conditions, and a designated space for detailed notes. This ensures that all relevant information is captured and easily accessible.
- Sturdy Cover: The trans-lux cover protects the log book from wear and tear, ensuring its longevity and maintaining the integrity of your recorded data.
- Essential Security Tool: This log book is an indispensable tool for any organization that values security and accountability. It helps to prevent misunderstandings, improve communication, and ensure a smooth transition between shifts.
- Wire-O with Trans-lux cover, 100 Pages, Dimensions 8.5" x 11" - (Security-Pass-Down) Reorder SKU: LOG-100-7CW-PP(Security-Pass-Down)
| Metric | Numerator | Denominator | Notes |
|---|---|---|---|
| False-positive rate | Benign requests incorrectly refused as security violations | Applicable benign requests | Includes refusals generated by the model itself, not only by the detector. Do not classify refusals by searching for phrases. |
| Pending reviews | Benign requests routed to human review and not yet decided | Applicable benign requests | Reported on its own line, not added to false positives or completions |
| Task-completion rate | Benign requests completed correctly | Applicable benign requests | An empty response does not count as success. State whether pending reviews are excluded from the denominator. |
A system that refuses every benign request has a 100% false-positive rate under this measure, however strong its blocking looks in the attack results. That is why the two sets of numbers have to be read together.
What zero false positives can and cannot establish
Small benign sets produce wide uncertainty. OWASP illustrates this with zero false positives in seven independent trials sampled from a defined benign workload. The 95% Wilson confidence interval for that outcome runs from approximately 0% to 35.4% (OWASP Cheat Sheet Series page, accessed October 4, 2026). This is an illustration of uncertainty, not a measured result, and it does not show that the false-positive rate is near zero.
An interval also cannot correct for how cases were chosen. If the benign set leaves out the kinds of legitimate requests that look most like attacks, such as long pasted documents or instructions that quote other instructions, the measured rate will look low for reasons unrelated to the detector. The same applies to attack classes: a benign set and an attack set are only as representative as their selection.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Repeat runs and keep every setting versioned
Every reported rate needs its numerator, denominator, corpus source, model and defense versions, settings, and number of repeated runs. Keep per-case outcomes next to the summary, so a reader can see which cases drove the number. Break findings out by security objective rather than merging them into a single score.
Recommended Free Tools
Repeated runs and related variants are not independent cases by default. Ten paraphrases of one injection are one attack idea tested ten ways. Five runs of the same case against a stochastic model are five samples of one case. Count results at the level where cases are genuinely independent, and state which level you used.
When comparing two defenses, run both on the same cases and keep the results paired, so each case produces an outcome under both. Pairing removes differences that come from case selection rather than from the defense itself.
Rerun the suite before deployment and whenever the threat picture changes. The AI Exchange guidance frames testing as something repeated over time, not a one-off certification (OWASP AI Exchange, AI security testing).
Where a guardrail model fits
A model-based guardrail, an LLM that classifies inputs before they reach the application model, is one possible layer. It is not a complete boundary. OWASP is direct about this: “A guardrail LLM is itself an LLM and is itself susceptible to prompt injection” (OWASP Cheat Sheet Series, LLM Prompt Injection Prevention, section “Model-Based Guardrails”).
Free tools Windows power users keep installed
One-click scans. No signup required.
OWASP advises using such a guardrail alongside:
- Input validation
- Structured prompts
- Least-privilege tool scopes
- Human approval for destructive actions
Guardrail calls add latency and cost, and their decisions should be logged and watched for drift. Those logs are also an input to your harness: a guardrail decision that changes between runs of the same case is a finding in its own right.
OWASP also describes a capability-oriented pattern. A privileged planner never inspects risky documents, a quarantined parser has no tools, and a custom interpreter tracks data flow and enforces policy. The page describes its code as a research artifact with limitations and not a supported security component. Treat the pattern as a design to test against, not a fix to drop in.
Practice on a deliberately vulnerable target first
OWASP Basileak is an intentionally vulnerable fine-tune of Falcon 7B, built as a capture-the-flag sparring target for prompt-injection training, red-team education, and research. OWASP states that it should not be deployed in production or used with real users, real data, or real credentials (OWASP Basileak project). Use it to learn how the attack families behave, and keep it entirely separate from the harness you run against your own application.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
