For the last couple of weeks I’ve been heads-down building something I didn’t have a name for yet: a loop that lets an AI agent develop and debug verification tests on its own. I started with Cursor and Copilot as the harness, then layered in shell and Python scripts that tail simulation logs, digest pass/fail status, timestamps, and wall-clock time, all so the agent gets exactly what it needs with the fewest possible tokens.
Three problems kept surfacing, and none of them were “the AI can’t write SystemVerilog.”
The long-running job problem. A SoC Verification simulation takes 1.5 hours. A Palladium run takes 20 minutes. Out of the box, no agent framework I tried just waits. They either block synchronously and burn compute doing nothing, or they lose their train of thought entirely by the time the job finishes. Copilot on Autopilot told me it will keep polling the watcher shell script then just passed me the turn. WHY?!?!
The caching problem. You have zero control over what gets cached unless you’re disciplined about sending the exact same conversation history with every new prompt. One extra token reordered at the start of your context and you silently pay full price again.
The guardrail problem. Where is the agent allowed to look? What is it allowed to change? Without hard boundaries, an agent debugging a testbench will happily “fix” things it was never supposed to touch.
I started prototyping a LangGraph-based orchestrator just to test the waters on these three fronts.
Then yesterday, NVIDIA published HORIZON — and it answered almost exactly what I’d been fighting with, from a completely different angle.
What HORIZON does
HORIZON treats hardware design as repository-level code evolution. Instead of a Python framework holding state in memory, the agent works inside an isolated Git worktree. When it proposes a fix, the system stages it, runs the evaluator (the compile/simulate/check loop), and either commits the change or logs the failure as a Git note attached to that commit. The agent’s entire memory is just git log.
What struck me, and I think should strike anyone who’s been fighting Redis- or any DB-backed agent memory, is that there’s no external state store at all. Version control is the memory layer.
The numbers that matter
HORIZON hit a 100% pass rate across every major RTL benchmark suite it was tested on, including Verilog-Eval, RTLLM-2.0, ChipBench, and multiple CVDP verification categories. Some of these converged almost instantly: RTLLM-2.0 passed at 78% on the very first iteration and needed only 2 more rounds to close.
But here’s the part that should reframe how everyone thinks about “AI writes correct Verilog now”: a handful of stubborn tasks ate almost all the budget. The CVDP checker-generation category (CID 002) started at just 3.2% first-pass and needed 82 iterations to reach 100%. Across the whole benchmark suite, HORIZON burned roughly 210 million tokens total — and the hard categories like CID 002, CID 003, and CID 012 alone accounted for over 60% of that spend.
That’s a cost-and-iteration-count problem rather than pure prompt engineering problem, and it’s exactly the toolchain question I’ve been circling in my own loop.
And the WOW number that really shined for me: 91% of HORIZON’s total token consumption was cached. That’s the caching discipline I’ve been fighting to enforce by hand, achieved almost as a side effect of using Git as the substrate – because the stable context (the harness, the spec, the accumulated commit history) never has to be re-sent or reshuffled <insert mind blowing wow omg moment>. Only the newest diff and the newest log get appended. Brilliant if you ask me. Thank you Git :).
The warning We Must Take Into Consideration
The paper is upfront about two limitations that map directly onto what I’ve been worried about:
First, reward hacking. Passing the visible testbench is not the same as proving the design correct. An agent chasing a green checkmark will find the shortest path to green, even if that path is a hardcoded stub that satisfies the test but not the spec. This is exactly why UVM’s reference models, scoreboards, and coverage aren’t going anywhere – they’re the “strong gate” that keeps an agent honest.
Second, long-latency reward. HORIZON’s benchmarks ran fast enough for tight edit-evaluate-repair loops. But real chip design (full synthesis, place-and-route, timing closure) can push feedback turnaround to days or weeks. The authors flag this explicitly as an open research problem, because a naive loop simply can’t survive waiting that long between actions.
That second point is where I think a human in the loop, or a hardened harness config, has to step in. If your last iteration ate 30% of your token budget and still failed, that’s not a “let the agent keep trying” moment, that’s a “stop, look at what it’s doing, and tighten the guardrails” moment. And the agent needs to inform you when it happens. How do you formulate this?
Where this leaves my own loop
HORIZON didn’t make my problem go away. It confirmed the shape of it. The path forward for my own project isn’t a bigger model or a cleverer prompt, maybe it’s a Git-native memory layer, stricter acceptance gates than a single pass/fail check, and an explicit budget ceiling that forces a human checkpoint before the agent burns through 80+ iterations chasing a fix that might just be reward hacking.
