iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
When an AI agent cannot retrieve a how-to it needs, treat the miss as a potential documentation bug: record the failed query, add the missing working example, and rerun the same search to verify that the guide is now discoverable. Jon Udell’s 2026 XMLUI case study shows how this turns an agent’s dead end into a testable documentation fix—not just a prompt to try harder.
What happened when an agent could not find a radio-group guide?
While working on Bram, a desktop app used with XMLUI, an agent asked: “How do I convert a select widget to a radio group?” The XMLUI MCP server was expected to surface verified How To examples, but it did not return a guide for that task. The agent fell back to the RadioGroup component documentation and worked out the conversion after some fumbling, Udell reports in his InfoWorld case study published Aug. 26, 2026.
Udell reviewed the MCP search logs and found two unsuccessful queries: “radio group mutually exclusive choice change value form input” and “choose one of two options horizontal selector.” The first returned form-related results without radio content; the second expressed the need in user-experience terms and also failed to find the how-to. Those misses pointed to a gap in the documentation catalog, rather than proving that the agent’s task was impossible.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWhy treat a retrieval miss as a documentation bug?
An agent can only use the guidance its retrieval system surfaces. If a product promises verified how-to examples but a needed, implemented pattern is absent or consistently hard to retrieve, the defect may be in the documentation—or in how it is indexed—not in the agent’s ability to improvise. Calling it a bug gives the team a concrete object to investigate: the query, the returned results, and the expected guide.
The distinction matters. A user can sometimes coax a useful answer from component reference material, as happened in this example, but that does not establish that the intended how-to is easy to find or reliably supported. A successful fallback may still conceal a documentation and retrieval problem.
How to turn the miss into a testable fix
- Capture the search evidence. Keep the original query and the results the MCP server returned. Include alternative phrasings when they expose different ways of expressing the same task.
- Check whether the working pattern is documented. Compare the task with the available guides and reference pages. In Udell’s example, the agent eventually used the RadioGroup component documentation, but the expected how-to was missing.
- File the gap as an issue. Describe the user task, the query or queries that failed, the relevant results, and the expected documentation. This makes the issue actionable without assuming that every retrieval failure has the same cause.
- Add the missing how-to. Document the working pattern so the guide answers the task the agent was trying to complete, not merely the name of a component.
- Rerun the search. Test the original query and useful variants against the updated catalog. Confirm that the new guide appears prominently enough to be useful, rather than treating publication alone as proof that the problem is fixed.
Udell summarizes the key idea: “That new doc is a testable fix.” The guide is the change; retrieval checks provide evidence about whether the change addressed the original failure.
What did the before-and-after search show?
Udell reports that the earlier searches had a top relevance score of around 0.7, while the new guide ranked first at 4.2—“six times higher than anything else.” He says version pinning in the MCP server enabled a direct before-and-after comparison. After adding the guide, several synthetic queries also found it, including “radio buttons for a small set of options.”
These values describe one case study, not general thresholds for a good or bad search result. Relevance scores depend on the search system and its scoring scale; the reported figures do not establish that another MCP server, documentation set, or retrieval system should produce comparable numbers. The useful practice is to compare results in a controlled setup and see whether the intended guide becomes easier to retrieve.
Where can teams look for more documentation gaps?
A single failed task is a useful starting point. Teams can also look across searches and code to find patterns that have not yet been turned into discoverable guidance. Udell describes three signal sources, each answering a different question:
| Signal source | What it reveals | Limitation |
|---|---|---|
| MCP search logs | Queries agents made and the results they received; repeated misses or weak matches can expose retrieval or content gaps. | They show search behavior and output, but a miss alone does not establish why retrieval failed. |
| Application code | Patterns developers actually implemented across a portfolio of applications, which can suggest how-to topics worth checking. | Code provides evidence of implemented work; teams still need to infer likely questions and test them against the documentation catalog. |
| Session logs | Potential dead ends encountered by agents or developers, especially when correlated with MCP searches by timestamp. | Udell says reliably surfacing episodes of struggle from session logs remains difficult. |
Use application code to suggest candidate guides
A team can ask an agent to identify recurring or notable patterns across a set of applications, then formulate the how-to searches that would lead a developer to those patterns. Udell’s example approach asks an agent to find top patterns across five apps and run corresponding MCP searches. The code can reveal that a technique exists in practice; search tests show whether documentation helps someone discover it.
Use session logs cautiously
Session logs may help connect a developer’s struggle to a search made at the same time, but identifying frustration or a dead end reliably is harder than observing a query and its returned results. Treat this as a potentially useful signal, not an automated verdict about documentation quality.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What the example does—and does not—establish
Udell’s account demonstrates a practical workflow: notice an agent’s retrieval failure, inspect the search evidence, add a missing how-to, and check retrieval again. It also shows how version pinning can support a direct comparison in the described MCP setup. Because this is a first-person case study rather than an independent controlled evaluation, it does not establish a universal improvement rate or a score threshold for documentation quality.
The broader lesson is modest but useful: an agent’s workaround does not make a missing guide irrelevant. When a needed pattern is supposed to be documented, a failed search can become a reproducible issue—and a new guide can be evaluated by whether searches for that task now find it.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
