Testing Multi Agent UVM Loops on Complex RTL Corner Cases

MosChip recently detailed an agentic approach to digital design, deploying a dual-model framework where specialized code models handle RTL generation while orchestrating agents manage testbench construction and error triage. They are not alone. Across the verification sector, startups and research groups are shifting away from monolithic prompt-and-code autocomplete toward multi-agent feedback loops. These architectures simulate the traditional dynamic between a design engineer and a verification engineer, passing SystemVerilog source, UVM components, and simulator logs back and forth until the regression clears.

For design leads and verification managers running tight chip programs, the promise sounds straightforward. Building Universal Verification Methodology (UVM) testbenches eats up roughly 60 to 70 percent of front-end engineering hours. Writing class-based drivers, scoreboards, coverage collectors, and sequences by hand is notoriously tedious. Yet early attempts to automate this with generic code-generation models failed miserably. Autoregressive language models write plausible-looking SystemVerilog that fails compilation inside Questa, VCS, or Xcelium due to subtle phase-order mistakes, broken factory registrations, or nonsensical sequence items.

Multi-agent loops attempt to fix this by introducing closed-loop tool feedback. The verification agent writes a testbench, a local simulator executes it, and the compiler errors or assertion failures are piped back into the agent context for self-correction. But moving from single-turn code generation to multi-agent UVM orchestration introduces a new set of risks. Teams now face massive token budgets, infinite correction loops, and the dangerous phenomenon of self-confirming scoreboards, where an agent unconsciously mirrors an RTL bug inside its verification reference model.

The Multi-Agent Separation of Concerns

Single-prompt language models fail at verification because RTL design and functional verification require fundamentally opposing mindsets. A designer focuses on hardware efficiency: area, timing paths, state encoding, and power gating. A verification engineer focuses on breaking the specification: identifying unhandled protocol states, corner cases in backpressure, FIFO overflows, and asynchronous reset recovery.

When a single prompt asks a model to generate both the design under test (DUT) and its test environment, the model inevitably builds a symmetrical mental model. If the model misinterprets the specification regarding how an AXI ready/valid handshake behaves under backpressure, it will write RTL with that bug and immediately construct a testbench that expects that exact faulty behavior. The test passes, code coverage reaches 100 percent, and the bug sails directly into the tape-out package.

Multi-agent architectures address this failure mode by enforcing role separation and strict context boundaries:

1. The RTL Architect Agent receives the natural language and microarchitectural specification, producing synthesizable SystemVerilog or Verilog with strict lint checks.
2. The DV Lead Agent parses the specification independently to produce a verification plan, defining functional coverage points, assertion properties (SVA), and stimulus cross-coverage matrices.
3. The UVM Scaffolding Agent implements the testbench topology: uvm_sequence_item, uvm_sequencer, uvm_driver, uvm_monitor, uvm_scoreboard, and the top-level uvm_env.
4. The Tool Interop Agent invokes the simulator (such as Siemens Questa, Synopsys VCS, or open-source engines like Verilator and Icarus with Cocotb), captures STDERR, parse errors, and assertion reports, and routes structured failure packets back to the responsible agent.

MosChip's implementation relies on a split-tier model strategy, pairing high-throughput models like DeepSeek-Coder for low-level Verilog emission with higher-order reasoning models for architectural planning and error triage. The goal is to drive syntax and UVM phase errors down to zero before a human verification engineer ever opens the file.

Anatomy of the UVM Generation Bottleneck

Generating synthesizable RTL is relatively simple compared to generating a robust UVM testbench. RTL is bounded by hardware primitives, clock domains, and combinational paths. UVM is a massive, complex object-oriented software framework built on top of SystemVerilog classes, macros, dynamic memory, and strict phasing semantics.

There are four distinct layers where multi-agent generation either succeeds or derails:

1. Factory Registration and Phasing Boilerplate

LLMs excel at standard boilerplate. Injecting uvm_component_utils, setting up constructors with super.new(name, parent), and connecting ports in the connect_phase is repetitive work. Multi-agent workflows achieve near 100 percent reliability on this mechanical layer because any compilation error (such as a missing build_phase super call or a mismatched analysis port type) produces explicit simulator error messages that the agent parses and patches in one iteration.

2. Transaction Sequence and Randomization Constraints

This is where single-agent models collapse and multi-agent loops begin to strain. A proper UVM test requires constrained-random verification. The agent must write uvm_sequence_item classes with valid SystemVerilog constraint blocks that respect the protocol rules without causing constraint solver contradictions.

When an agent generates over-constrained sequences, the simulator's constraint solver errors out immediately. In a multi-agent loop, the Tool Interop Agent feeds the solver failure log back to the Scaffolding Agent, which relaxes the conflicting boundary. However, if the agent under-constrains the transaction, the simulation runs without error, but the sequence drives illegal bus combinations that real hardware will never see, triggering false-positive assertions in the monitor.

3. Golden Reference Models and Scoreboarding

Writing an independent scoreboard requires the agent to model the cycle-approximate or transaction-level behavior of the DUT using dynamic arrays, queues, or DPI-C imports. If the DUT is an encryption core, an AXI crossbar, or an SPI controller, the scoreboard must compute the expected output without duplicating the RTL's internal state machine bugs. If the prompt context shares the RTL implementation code with the scoreboard generator, the agent frequently copies the RTL logic directly into the scoreboard. When that happens, the test validates nothing.

4. Functional Coverage and Cross-Bins

Line, branch, and toggle coverage are mechanical metrics handled by simulator flags. Functional coverage requires writing explicit covergroup and coverpoint structures with cross-bins that capture complex protocol collisions (for example, receiving an interrupt while a FIFO is at maximum depth during a backpressure cycle). Multi-agent systems can draft these covergroups, but converging on the final 10 to 15 percent of coverage holes requires intelligent stimulus steering that current LLM agents struggle to direct autonomously.

Receipts Block: Dual-Agent vs Single-Pass Benchmark

To evaluate how multi-agent closed-loop architectures compare against baseline single-pass LLM generation, we analyze performance across four standard IP blocks of increasing complexity. The data below represents an illustrative composite drawn from published agentic RTL workflows, academic benchmarks on automated UVM generation (such as arXiv:2504.19959), and empirical multi-agent EDA evaluations.

Setup parameters for the composite benchmark:

• Environment: Python orchestration engine driving QuestaSim 2023.4 and Verilator 5.020.
• Models: DeepSeek-Coder-33B (specialized RTL/testbench coding) managed by Claude 3.5 Sonnet / GPT-4o tier reasoning controllers.
• Targets: AXI4-Stream Asynchronous FIFO, Wishbone SPI Master, Round-Robin Arbiter with Priority Weighting, and a Pipelined AXI4-Lite Register Slice.
• Loop limit: Maximum 6 self-correction iterations per block.

Table 1: Generation Success, Verification Quality, and Compute Cost

Source: Illustrative composite benchmark aggregating data from MosChip architectural briefs, ChipAgents frameworks, and recent LLM-aided UVM academic literature (arXiv:2504.19959v1).

The metrics show a clear trade-off. Single-pass generation is cheap in terms of tokens, but it yields non-functional verification environments that fail compilation or miss most cross-coverage points. Dual-agent loops push compilable UVM testbench rates above 80 to 90 percent, but token consumption scales non-linearly as designs grow in state complexity. An asynchronous FIFO with multi-clock domain crossings requires over 90,000 tokens of feedback and five tool interactions to resolve testbench race conditions, pointer wraps, and scoreboard synchronization.

The Token and Compute Tax in Agentic EDA

Multi-agent loops introduce hidden computational overhead that development teams rarely budget for. When an engineer writes a UVM environment, they read the compiler warning, jump directly to line 142 of spi_driver.sv, change a blocking assignment to a non-blocking assignment, and re-run.

An LLM agent operates under different constraints. In most multi-agent frameworks, when the Tool Interop Agent reports a compiler failure:

1. The simulator log is truncated or summarized.
2. The entire SystemVerilog class file (often 300 to 800 lines of code) is re-injected into the prompt context along with the error log.
3. The model re-generates the entire class file or outputs an edit patch.
4. If the patch introduces an unintended side effect in an analysis port, the next simulation run fails with an elaboration error, triggering another context re-injection.

By iteration four, the context window contains thousands of tokens of historical error back-and-forth. This context bloat causes two specific engineering problems: attention degradation and high API costs.

Attention degradation occurs when the code model loses track of subtle constraints defined in early prompts because the context is flooded with repetitive simulator stack traces. The model might fix a compilation error in the driver while accidentally reverting a clock-period parameter in the top module.

On the economic side, running 90,000 tokens through frontier reasoning models for a single peripheral block costs several dollars per run. If an engineering team runs these loops across hundreds of internal IP modules on every Git commit, API costs quickly rival commercial EDA license fees without providing the deterministic guarantees of traditional static analysis tools.

The Verification Illusion and Symmetrical Hallucination

Syntax closure does not equal correct verification. A testbench that compiles cleanly, runs 10,000 random cycles without assertion errors, and reports 95 percent code coverage can still be completely blind to critical silicon bugs.

Consider a concrete failure mode observed in automated AXI4-Stream testbenches. The specification requires that the TVALID signal, once asserted, must remain high until TREADY is asserted by the receiver (the handshake rule).

// Faulty Agent-Generated RTL
always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    tvalid <= 1'b0;
  end else if (internal_drop_condition) begin
    // Bug: dropping TVALID before TREADY handshake completes
    tvalid <= 1'b0;
  end else if (data_ready) begin
    tvalid <= 1'b1;
  end
end

If the RTL agent generates this invalid de-assertion logic, what happens in the verification agent?

If the verification agent extracts protocol rules from the same high-level prompt without an independent formal property suite, it often writes an AXI monitor that checks packet payload data only when TVALID && TREADY are both active, completely omitting the temporal check that TVALID must stay asserted until TREADY.

The monitor captures the valid transactions, passes them to the scoreboard, the scoreboard matches the transformed data, and the simulator reports a clean pass. The code coverage tool shows the branch was taken. The functional coverage group shows transactions occurred. Yet the block will lock up or drop packets the moment it connects to an off-the-shelf third-party AXI interconnect on silicon.

This is the Verification Illusion. Agentic loops are exceptionally good at finding harmony between two generated artifacts. They are not naturally equipped to identify what neither agent thought to check.

Making Multi-Agent Verification Usable: A 4-Step Guardrail

Silicon teams looking to deploy agentic workflows like MosChip's dual-model architecture need defensive engineering constraints around the toolchain. AI agents should accelerate testbench drafting, not act as the final sign-off authority.

+-------------------------------------------------------------------------+
|                        Specification (Micro-Arch)                       |
+------------------------------------+------------------------------------+
                                     | 
         +---------------------------+---------------------------+
         v                                                       v
+-----------------------+                               +-----------------------+
|   RTL Coder Agent     |                               |   UVM Generator Agent |
|  (Synthesizable Core) |                               | (Scaffolding/Drivers) |
+-----------+-----------+                               +-----------+-----------+
            |                                                       |
            |                                                       |
            v                                                       v
+-----------------------+                               +-----------------------+
| Synthesizable Verilog |                               | Complete UVM Testbench|
+-----------+-----------+                               +-----------+-----------+
            |                                                       |
            +---------------------------+---------------------------+
                                        | 
                                        v
                   +-----------------------------------------+
                   |     Deterministic Gatekeeper Suite      |
                   |   - Vendor Formal SVA VIP (AXI/Wishbone)|
                   |   - Verilator / SpyGlass Lint Suite     |
                   |   - Commercial Simulator (VCS/Questa)   |
                   +--------------------+--------------------+
                                        |
                    +-------------------+-------------------+
                    v                                       v
            [Regression Passed]                     [Violation Caught]
                    |                                       |
                    v                                       v
           Human Sign-Off Review                 Log Packet Routed to
                                                Responsible Agent Loop

1. Decouple Context Completely

Never feed the generated RTL file into the prompt context of the UVM testbench generator. The verification agent must only receive the register map, interface definitions, and functional requirements. If the verification agent needs to know signal names, provide a strictly defined SystemVerilog interface wrapper rather than the internal RTL implementation.

2. Bind Pre-Verified Formal VIP

Do not rely on the agent to invent protocol temporal assertions. For standard bus protocols (AXI, AHB, APB, PCIe, SPI), bind commercial or battle-tested open-source Verification IP (VIP) and SystemVerilog Assertions directly into the simulation wrapper. If the agent-generated RTL violates protocol rules, let hardcoded formal properties catch it, triggering the error loop before the testbench scoreboard is even evaluated.

3. Cap Agent Repair Iterations

Set a strict limit on self-correction loops, typically three to four iterations. If an agent fails to clear a compilation error or constraint contradiction within four turns, kill the process and alert a human engineer. Beyond four turns, language models frequently enter recursive thrashing, modifying unrelated working code to bypass an error message they do not understand.

4. Enforce Independent Reference Scoreboards

Whenever possible, implement the scoreboard reference model using a golden software implementation (such as a C++ or Python model linked via DPI-C) created outside the agentic loop. Forcing the UVM environment to compare RTL transactions against an external compiled executable eliminates the risk of symmetrical hallucination.

Tools designed for rigorous hardware development, such as Silicode (silicode.ai), focus specifically on this requirement: ensuring that generated logic is bound to verified formal assertions and deterministic test harnesses rather than relying on unconstrained language model autocomplete.

What to Track Next

As multi-agent EDA tools move into production environments, watch how vendors handle verification plan traceability. The differentiator will not be how fast an agent can generate standard UVM boilerplate. Any modern coding model can emit a uvm_driver class.

The real test of these multi-agent architectures is how systematically they close coverage holes on non-standard, custom microarchitectures without human hand-holding. Track whether tools provide deterministic proof of coverage closure, whether they support direct integration with commercial formal property checkers, and how cleanly they isolate verification reference models from design artifacts.

Until agentic frameworks demonstrate that they can catch subtle asynchronous race conditions and protocol violations without human-written assertions in the loop, treat auto-generated UVM environments as accelerated rough drafts. They save typing time; they do not remove the need for deeply skeptical verification engineering.

Sources

• Design & Reuse: Accelerating RTL Design with Agentic AI
• MosChip AI Engineering: Accelerating RTL Design with Agentic AI
• Semiconductor Digest: Accelerating RTL Design Through Multi-Agent AI Collaboration
• arXiv (2504.19959): An Automated LLM-aided UVM Machine for RTL Verification
• ChipAgents: AI Agents for UVM: the Next Frontier
• MosChip Technologies Corporate Overview