Closed-Loop Iteration Beats Blind Review (23,760 Benchmarks)

Developers iterate. They generate code, run tests, fix failures, run again. The question isn’t whether to iterate — it’s whether the loop has signal. Blind “review and improve” prompts destroy quality. But a closed loop with real test feedback lifts the hardest model-language combinations by up to +19 points.

I ran 23,760 benchmarks testing eleven iteration patterns across four languages. The results: blind iteration makes things worse, but closed-loop iteration — tests → code → run tests → fix → pass — works. The catch: front-loading comprehensive instructions in a single turn still wins on efficiency. Closed-loop shines when you’re working with a model-language pair that struggles (Haiku+C#, Haiku+Go).

This is the ninth experiment in our benchmark series, following the context pollution, GSD methodology, and capstone best-practices studies. All data is open source at claude-benchmark.

Two Types of Iteration

Not all multi-turn patterns are equal. The critical distinction:

• Open-loop (blind): “Review and improve your code.” No test results, no compiler errors, no signal. The model guesses what to change.
• Closed-loop (evidence-based): Run tests between turns. Feed failures back. The model fixes what’s actually broken.

Open-loop iteration is how most people prompt AI today. Closed-loop iteration is how Claude Code’s tool-use actually works. The gap between them is enormous.

Experiment Design

I tested six variants in v1 (blind iteration) and five in v2 (test-feedback iteration):

V1 — Blind iteration (12,960 runs):

V2 — Closed-loop with test feedback (10,800 runs):

Each variant was run across:

• 3 models: Claude 3.5 Sonnet, Claude 3.5 Haiku, Claude 3 Opus
• 48 cross-language tasks: Python, Go, JavaScript, C#
• 15 repetitions per model-task-variant combination

The Scoreboard: Blind Iteration Fails

V1 results — open-loop variants:

Why Blind Review Fails

The three-turn pattern is intuitive: generate code, review it, improve it, review again, polish. It’s how humans work. But LLMs don’t have the metacognitive ability to recognize when code is already correct. They can’t say “this is fine, don’t change it.” Every turn becomes an opportunity to break something that worked.

The C# data is especially stark: three-turn-bare drops to 61.13, down from 63.17 in single-turn-bare. That’s a 2.04-point regression on the language that’s already hardest for Claude. More turns amplify model weaknesses.

Even “specific feedback” doesn’t save blind review. Two-turn-specific-feedback (71.49) still regresses below single-turn-bare (71.86). Telling the model “check edge cases” doesn’t help when it can’t actually run the code to see if the edge cases pass.

Closed-Loop Changes Everything

V2 results — test feedback between turns (10,800 runs):

But the bigger story is where closed-loop shines:

Low-Baseline Combos: Where Closed-Loop Dominates

The overall averages hide the most important finding. When we break results by model-language pair, closed-loop iteration shows massive gains for combinations where Claude struggles:

This maps directly to the low-baseline rule we’ve seen across experiments: techniques that help when Claude is struggling (weak model + hard language) often don’t help — or actively hurt — when it’s already performing well (Sonnet + Python).

The Efficiency Tradeoff

Front-loading still wins on pure efficiency:

Single-turn-kitchen-sink: 79.79 at 2,391 tokens. Three-turn-bare: 68.22 at 5,433 tokens. That’s 2.3x the cost for 11.57 fewer quality points. If you’re paying per token, front-loading is the obvious choice for high-baseline combinations.

But for low-baseline combos (Haiku+C#, Haiku+Go), the closed-loop approach is worth the extra tokens. A +19 point lift justifies the cost.

The Decision Framework

Practical Advice

Limitations

• Simplified test feedback. Our v2 experiment injected pass/fail results. Real Claude Code tool-use workflows have richer feedback (compiler errors, linter output, stack traces). The real-world gains from closed-loop may be larger than +19pts for hard cases.
• Two turns only. We didn’t test 3+ turn closed loops. Real agentic workflows may iterate more times with diminishing returns per turn.
• Claude models only. Other model families may handle iteration differently, though the pattern of “signal helps, blind iteration hurts” likely generalizes.
• Single-file tasks. Multi-turn may add more value for architectural work, planning, or multi-file coordination where test feedback spans integration boundaries.

Try It Yourself

The multi-turn conversation experiment configuration and all data are open source:

pip install claude-benchmark

# Run the multi-turn experiment
claude-benchmark experiment run experiments/multi-turn-conversation.toml

# Generate the report
claude-benchmark report

# See the results
open results/*/report.html

You can also define your own turn patterns in the experiment TOML and test them against your specific task types. If you find a closed-loop strategy that beats front-loading for high-baseline combinations, I’d like to see the data.

Full experiment configuration: experiments/multi-turn-conversation.toml

For the complete picture of what works and what doesn’t in AI prompt engineering, see the capstone best-practices experiment where we combine the winning strategies from all experiments into a single optimized configuration.