JudgeStack, a Magic: The Gathering rules-question agent, did better than a one-shot keyword search on a small blind test—but that does not prove it can reliably answer Magic rules questions. In Joshua R. Gutierrez’s 2026 evaluation, structured evidence gathering got 9 of 10 held-out verdicts right, compared with 1 of 10 for keyword retrieval. The useful result is narrower: choosing and connecting the right sources can help an agent ground its answers, while gaps in the evidence model and flaws in evaluation can still make confident answers wrong.
Why a Magic rules answer needs more than a search result
A rules agent must establish not only an answer, but which source supports each part of it. The right evidence depends on the question:
- For what a card says now, use its current Oracle text.
- For how that wording interacts with game mechanics, use the Oracle text alongside the current Comprehensive Rules.
- For a historical question, use rules and card wording appropriate to the date being asked about.
- For whether a card is legal in a format now, use current format-legality data.
- For when a ban or restriction took effect, use the dated announcement—not just a current status record.
- To explain why a physical card differs from its current text, compare its printed wording with current Oracle text.
That distinction between present status and historical effective date matters. A current legality record can establish a card’s status now; it does not, by itself, establish when that status began. Combining those questions can lead a system to supply a plausible-sounding date without evidence for it.
Wizards of the Coast describes the Comprehensive Rules as a reference for rules and corner cases, meant to be consulted for specific questions rather than read from beginning to end. The Magic Judges rules resource lists the current version as effective September 25, 2026; rules change, so a claim about current gameplay should be checked against the applicable version.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
How JudgeStack organized its evidence
Gutierrez reports that the project corpus contained 496 documents across ten types: card, printing, ruleParagraph, glossaryTerm, formatEvent, claim, decision, textDifference, adjudicationCase, and authoritySource. The reported counts included 30 cards, 77 printings, 16 rule paragraphs, 208 legality claims, and 74 detected differences between printed wording and current Oracle text. These are project-specific figures from 2026, not counts for a general Magic database.
The implementation used two Sanity Context MCP endpoints: one exposed structured documents for filtered GROQ queries, and the other exposed the Comprehensive Rules as a knowledge-base file. They had to remain separate because, according to Gutierrez, an endpoint configured with a dataset source ignores its knowledge-base sources. Combining the sources would have removed access to the rules file.
The public structured dataset contained only the 16 rule paragraphs cited by reviewed cases; the full rules document was a retrieval source rather than hundreds of separately republished dataset records. Gutierrez describes this as a way to keep duplicated public rules text narrower, not as a resolution of the licensing question.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
What the blind comparison found—and what it did not
The evaluation covered 30 questions across printed wording versus current Oracle text, current format legality, and historical rules changes. Ten questions were held out and not run during development. Both conditions used the same answer model and prompt, DeepSeek Flash, but differed in how they gathered evidence:
| Condition | Evidence gathering | Blind verdicts correct | Reasoning rested on something unretrieved |
|---|---|---|---|
| One-shot keyword retrieval | One BM25 search over a flattened corpus; the top 12 chunks supplied in one pass | 1/10 | 7/10 |
| Structured evidence gathering | Could query the Sanity dataset with GROQ, read the rules knowledge base, follow references, and take up to ten model steps | 9/10 | 1/10 |
Gutierrez reports that the 20 holdout answers—ten from each condition—were shuffled and stripped of labels before judging. The judge saw the questions, expected verdicts, and rubric, but not the condition labels or counts. The answering and judging models came from different vendors, although the exact judge build was not pinned because grading took place in the ChatGPT interface. The evaluation pack, rubric, and raw judgments were made public so others can repeat the grading with another judge.
The comparison is encouraging but not a controlled BM25-versus-GROQ experiment. The structured system had more opportunities to act: it could choose what to query next and use as many as ten model steps, while the lexical condition performed one retrieval pass. Temperature and output-token limits were not set, leaving provider defaults in effect. The result therefore compares two evidence-gathering architectures with different interaction budgets, not retrieval algorithms in isolation.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Across all 30 questions, including those used during development, the structured condition retrieved required cards in 30/30 cases versus 22/30 for keyword retrieval, and required rules in 30/30 versus 19/30. Cited rules were actually retrieved in 30/30 versus 24/30, while unsupported citations numbered zero versus six. These full-suite diagnostics are useful for understanding the system, but the questions used during development make them less suitable as an unbiased estimate of performance on new questions.
Why retrieving the relevant card still was not enough
The clearest retained failure concerned Sol Ring. JudgeStack retrieved claims that included its restricted status in Vintage, yet concluded that the card could only be registered in Commander. The answer was wrong because the corpus supplied the status without explaining what “restricted” means; the model effectively treated “restricted” as “banned.”
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
This is a schema problem, not simply a search problem. The system needs a legality-term concept that defines legal, banned, and restricted and explains how restrictions apply. Without that connective definition, retrieving the card’s status and other relevant records still does not give the model enough information to reason correctly.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
Why the date-discipline score was withdrawn
Gutierrez initially reported a 10/10 automated “date discipline” score for structured retrieval, then withdrew it. The check looked for retrieval of any format event, not whether the event concerned the card in the question. The two stored events involved an unrelated card, so the check could pass even when an answer invented an effective date.
It also failed to distinguish a card being banned as of a date from a claim that its ban became effective on that date. Because the saved evaluation rows did not retain the retrieved IDs, the score could not be recomputed correctly. The 10/10 figure is withdrawn, not evidence of date accuracy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Integration bugs can look like reasoning failures
The author reports three implementation problems that initially obscured JudgeStack’s behavior:
Recommended Free Tools
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
- Incompatible AI SDK dependency versions caused tool-call validation failures.
- Identical tool names from the two endpoints collided when their tool sets were merged, dropping the dataset schema overview.
- Stringifying the MCP response instead of extracting
content[].textleft document IDs escaped. Card-ID parsing then failed even though rule-number retrieval appeared to work; flattening the response fixed card retrieval.
These are reported implementation findings, not independently reproduced tests. They illustrate why an agent’s output should not be diagnosed as a model-reasoning problem until the tool calls, schemas, and parsed results have been checked.
Gutierrez also reports that a local Qwen3 configuration made no successful dataset-endpoint calls across three runs and produced invalid arguments for parameterless tools. A separate workflow—having the model produce a JSON retrieval plan and executing that plan externally—could use the corpus. The three-run result is specific to that configuration; it is not a claim about all local models.
What this experiment supports
The strongest conclusion is that, in this project’s small blind holdout, structured, multi-step evidence gathering was associated with more correct verdicts and fewer answers whose reasoning depended on unretrieved material than one-shot keyword retrieval. It also surfaced problems that a single accuracy score would hide: a missing definition, an invalid date metric, and bugs in tool integration.
Ten held-out questions are not enough to establish broad reliability, and the two conditions differed in both architecture and available model turns. JudgeStack is best understood as a rules laboratory, not a replacement for a judge. Its results show how to make an evaluation more informative; they do not demonstrate that an AI system can safely answer Magic rules questions in general.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
