Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsiTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
The short answer: the larger cloud model produced the clearest positive movement, but it still failed the promotion gate. The local 4B model was stable and inexpensive yet produced no net accuracy gain, while the role-separated runner produced zero candidate proposals at all. The experiment shows that a better model can improve the quality of a signal without producing a prompt change you should ship—and that changing the executor can alter the evidence an analyzer has available.
What the experiment compared
The AgentSelfEdit v0.3.0 field test compared three versions of the same self-edit loop. Each loop evaluated a prompt, examined failures, proposed an edit, and tested whether that edit should replace the current prompt. The comparison was not simply “small model versus large model.” It also tested whether assigning execution, analysis, and judging to different models changed the search behavior.
The three setups were:
- Local 4B:
Qwen3-4B-Instruct-2507-4bitused as the single model. - Single cloud model:
mistralai/mistral-small-3.2-24b-instructused for the loop. - Role-separated runner:
qwen/qwen3-30b-a3b-instruct-2507as executor and the Mistral 24B model as analyzer, with judging separated as configured by the runner.
The evaluation used a 40-task promotion corpus and a 25-task held-out set. A candidate could pass only if its result met both parts of the promotion rule: a statistical result of p < 0.05 and a minimum effect size of at least 5 percent. Passing one condition was not enough. This matters because a visible increase in a held-out score is not the same as an accepted prompt revision.
Results at a glance
| Setup | Baseline accuracy | Final or reported artifact | Iterations | Promotions | Proposal behavior |
|---|---|---|---|---|---|
| Local Qwen3 4B | 60.0% | 60.0% | 5 | 0 | Mostly null or cancelling edits |
| Single Mistral 24B | 64.0% | 68.0% held-out artifact | 8 | 0 | First clean positive movement |
| Role-separated Qwen 30B + Mistral 24B | 60.0% | No candidate artifact | 3 | 0 | Zero proposals |
The table contains three different kinds of outcome. The local run completed with no net change. The cloud run generated a higher reported artifact but did not promote its candidate. The separated run stopped before candidate generation, so it cannot be judged as a failed edit or a successful one.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
What happened with the local 4B model
Stable execution, unchanged accuracy
The local Qwen3-4B-Instruct-2507-4bit run started at 60.0 percent accuracy and ended at 60.0 percent after five iterations. It produced zero promotions. That is not evidence that the model was unusable: the run was operationally stable and completed its mechanics. It is evidence that the loop did not discover a prompt edit that improved performance enough to pass the gate.
The iteration pattern explains the flat result
Iterations 1, 2, and 4 recorded no improvements and no regressions. Iterations 3 and 5 each recorded one improvement and one regression. In other words, the occasional positive case was exactly offset by a negative case, leaving the net score unchanged.
The model was described as useful for executor or baseline work because it was cheap and predictable. Its weakness appeared in the analyzer role. Instead of repeatedly finding a coherent edit that fixed a recurring failure, it often returned a null edit or a change that cancelled out another change. A model can therefore be valuable for running tasks and still be a poor search engine for prompt revisions.
What the larger cloud model changed
A positive signal appeared
The single-model run used mistralai/mistral-small-3.2-24b-instruct. Its baseline was 64.0 percent, and the reported final held-out artifact was 68.0 percent after eight iterations. This was the first setup to generate a clean positive movement rather than a flat or cancelling sequence.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
Why the result was not a promotion
The best A/B iteration improved two cases and regressed none. Its reported mean delta was 0.05, its effect size was 0.0625, and its p-value was 0.79. The effect-size figure exceeded the minimum 5 percent requirement, but the p-value did not come close to the required p < 0.05. Because both conditions were mandatory, the candidate was not promoted.
The 64.0-to-68.0 percent change must therefore be described as a reported artifact, not as a deployed improvement. There was no accepted prompt revision behind it. If the candidate was never promoted, later runs did not inherit it as the new production prompt.
This is the central distinction in the experiment: a stronger model can generate a more promising edit and still fail to produce evidence strong enough to trust. “Better is not the same thing as good enough to trust,” author Debashish Ghosal wrote in 2026.
Why role separation produced zero proposals
The separated configuration
The role-separated runner used qwen/qwen3-30b-a3b-instruct-2507 as executor and the Mistral 24B model as analyzer. It ran for three iterations, with a held-out sample of five tasks and a promotion sample of ten tasks. The baseline was sane at 60.0 percent.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Zero proposals is a specific failure mode
The run directories and analysis.json files existed, showing that the runner and its baseline evaluation worked. However, prompt-b.md and ab-comparison.json never appeared. The loop did not reach candidate generation and comparison, so there was no proposal to score, reject, or promote.
That outcome should not be collapsed into “bad proposals.” Bad proposals mean the analyzer generated candidates that failed evaluation. Zero proposals means the system never produced a candidate in the first place. It points to a breakdown in the path from executor output to analyzer action, not to a measured defect in a particular prompt edit.
Why a stronger model did not automatically improve the prompt
The analyzer sees traces, not task names
The most useful explanation from the field test is that the analyzer does not operate on abstract task labels. It operates on failure traces. Those traces contain the concrete outputs, errors, omissions, and near-misses produced by the executor. Change the executor and you change the evidence the analyzer receives.
This makes role separation behavior-shaping rather than merely organizational. A more capable executor may solve a task cleanly, leaving fewer failures for the analyzer to study. A different executor may fail in a more consistent way, making a pattern easier to turn into a prompt edit. Conversely, a mixture of irregular failures can make the analyzer unable to identify a stable intervention even when the analyzer itself is strong.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
The search stayed narrow
Across the runs, proposals clustered around urgency-boundary rewrites, local wording clarifications, and narrow rule tightening. Structural prompt changes, broad task-decomposition changes, and genuinely diverse proposal families were uncommon. The larger model improved the quality of the initial signal, but it did not demonstrate a broad search over fundamentally different prompt strategies.
This helps explain why model size alone was insufficient. A stronger analyzer can phrase a sharper local edit, yet still be constrained by the failures it is shown and by the loop’s willingness to generate, test, and retain different classes of candidates.
How to interpret a no-promotion benchmark
Separate the five possible outcomes
When reviewing a self-edit benchmark, identify which of these outcomes occurred:
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11- Execution failure: the system could not run the baseline or complete its evaluation.
- Zero proposals: the analyzer never produced a candidate for comparison.
- Rejected proposals: candidates were generated but did not improve the measured result.
- Promising but statistically weak movement: the measured score rose, but the gate was not met.
- Promotion: the candidate met both the statistical threshold and the minimum effect size and became the accepted prompt.
The three setups occupy different categories. The local run completed and produced offsetting changes. The cloud run reached a promising but statistically weak result. The separated run stopped at zero proposals. Treating all three as “the model failed to improve” hides the engineering information each outcome provides.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Check the gate before quoting an accuracy number
Always report the baseline, the candidate or artifact score, the number of evaluated tasks, the effect size, and the p-value together. A headline score without promotion status can imply that a change shipped when it did not. In this experiment, the 68.0 percent cloud artifact is informative about the search signal, but it is not a production result because no prompt was promoted.
What each setup is useful for
Use the local 4B model where cost and repeatability matter
The local model’s 60.0 percent to 60.0 percent result does not erase its operational strengths. It can serve as a baseline executor when you need a cheap, stable loop and clear mechanics. It should not be assumed to provide strong analyzer search merely because it can complete the surrounding workflow.
Use the larger cloud model for stronger candidate signals, not automatic trust
The Mistral 24B run demonstrated that a larger model can find a clean positive movement that the local run did not. Its failed p-value gate shows why every candidate still needs the same statistical check. A stronger model raises the chance of a useful proposal; it does not remove the need for controlled evaluation.
Free tools Windows power users keep installed
One-click scans. No signup required.
Treat role separation as an experiment in evidence quality
Separating executor and analyzer roles changes the failure traces and therefore changes what can be learned from them. The zero-proposal result shows that a capable analyzer paired with a different executor is not guaranteed to produce more edits. Before judging the quality of separated roles, verify that the pipeline reaches candidate generation and comparison.
A practical reading of the experiment
The local run establishes a useful baseline for stability but no net improvement. The single cloud run establishes that better signal generation is possible, while its statistics reject premature deployment. The role-separated run establishes that pipeline behavior depends on the executor’s outputs and that a healthy baseline can coexist with an empty proposal stage.
Together, the results argue for reporting self-edit systems as pipelines rather than as single model scores. Record which model executed the tasks, which model analyzed failures, whether proposals were generated, how many improvements and regressions occurred, and whether the formal gate accepted a candidate. Without those fields, a reader cannot tell whether a no-change result reflects model weakness, evidence scarcity, a broken handoff, or a correctly conservative promotion policy.
The experiment’s most defensible conclusion is therefore narrow but important: a local 4B model can be useful without improving its prompt; a larger cloud model can produce a positive but untrusted signal; and role separation can reshape—or halt—the search because executor output determines the evidence available to the analyzer.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
