IDO
Insight

silicode · 2026-09-09 · 13 min

Nemotron 3 Ultra and the 97% RTL Pass Rate

Nvidia claims a 97.1% RTL pass rate with Nemotron 3 Ultra running agentic loops. We examine what that 6,629-token budget actually fixes, and what still fails in lint and CDC.

Abstract diagram of semiconductor logic gates, clock domain crossings, and state verification trees.

Nvidia recently published benchmark figures for Nemotron 3 Ultra running agentic RTL generation tasks, reporting a 97.1% pass rate across nine categories of digital design work. The model uses a hybrid Mamba-Attention Mixture-of-Experts (MoE) architecture and consumes an average of 6,629 tokens per iteration to reach closure against the benchmark suite.

On paper, 97.1% sounds like autonomous front-end design is solved. In a production ASIC or FPGA workflow, that number requires aggressive qualification. Anyone who has spent nights trying to close timing on an FPGA or debugging a silicon failure traced back to an unconstrained clock domain knows that passing functional simulation on a set of unit-level testbenches is only the initial hurdle.

To understand what Nemotron 3 Ultra actually delivers, we have to look past the top-line accuracy metric and analyze two separate mechanisms: how the hybrid Mamba-Attention architecture handles long EDA error traces, and why high pass rates on benchmarks like CVDP (Comprehensive Verilog Design Problems) do not translate directly to lint-clean, synthesis-ready, CDC-safe RTL.

Inside the 6,629-Token Iteration Loop

The 97.1% figure is not a zero-shot completion score. It is the result of an agentic loop where the model generates Verilog, passes it to an external simulator or parser, reads the compilation and runtime failure logs, and rewrites the code until the testbench outputs match the golden reference.

The 6,629 tokens consumed per iteration tell a specific mechanical story. Standard dense Transformer models choke when fed thousands of lines of verbose EDA tool logs. A single ModelSim, VCS, or Verilator error report on a multi-module design easily runs several hundred lines. When a syntax error causes cascaded type-checking failures or uninstantiated module warnings, the log output explodes.

In standard self-correction setups, an LLM burns most of its context budget absorbing compiler stderr streams. The prompt contains:

  1. The original natural language specification and interface definition.
  2. The previously generated SystemVerilog module.
  3. The full simulation log, containing syntax errors, assertion violations, or mismatched output waveforms.
  4. Instructions to diagnose the failure and output a patched implementation.

A single round of feedback often takes 4,000 to 8,000 tokens once you include the module text and compiler stack traces. Averaging 6,629 tokens across iterations indicates that Nemotron 3 Ultra is actively ingesting multi-page error dumps, modifying its internal state representation, and regenerating substantial code blocks rather than applying localized diffs.

This brute-force convergence works remarkably well for resolving syntax quirks, fixing off-by-one bit slicing, aligning bus widths, and reconciling state machine enumeration mismatches. If an output bus is defined as [7:0] but driven by a 16-bit intermediate accumulator, Verilator or VCS will throw a width mismatch warning or error. The agent reads the exact line number, adjusts the truncation or expands the target register, and re-runs the simulation. The iteration counter ticks, tokens are spent, and the test passes.

However, syntax-fixing loops create a false sense of security. An agent optimized solely to satisfy a testbench will write whichever construct silences the simulator, regardless of whether that construct translates into sensible hardware.

The Mamba-Attention MoE Architecture in Hardware Workflows

To make iterative loops practical without prohibitive latency and hosting costs, Nvidia built Nemotron 3 Ultra as a hybrid Mamba-Attention model using Mixture-of-Experts routing. Understanding this architecture explains why it outperforms standard dense models on agentic throughput.

Pure Attention mechanisms scale quadratically with sequence length. In an agentic chip design environment where the context window accumulates previous RTL drafts, multiple simulator logs, and intermediate test vectors, standard KV caches grow rapidly. Inference slows down, memory consumption spikes, and cost per iteration climbs to impractical levels.

State Space Models (SSMs) like Mamba process sequence data with linear computational complexity relative to context length. They do this by maintaining a selective, compressed hidden state rather than caching the full key-value history of every preceding token. This makes Mamba efficient at ingesting voluminous, sequential text, such as raw cycle-by-cycle simulation traces, execution logs, and repetitive register transfer dumps.

Mamba alone has historically struggled with precise associative recall across long distances. It can lose track of exact identifier definitions, pinout mappings, and structural port lists established thousands of tokens earlier. That is a fatal flaw for Verilog, where a single flipped bit in an instantiation parameter or an inverted active-low reset signal breaks the build.

By interleaving Attention layers into the Mamba backbone and routing active weights through an MoE framework, the model splits the workload:

  • The Mamba layers stream through large compilation logs, waveform summaries, and multi-turn conversational history at high throughput.
  • The Attention layers preserve exact syntactic relationships, module hierarchies, and variable binding across the codebase.
  • The MoE routing activates only a subset of parameters per token, keeping the active parameter footprint low during generation.

Nvidia reports up to 5x higher throughput and 30% lower generation costs compared to equivalent open dense models. For teams deploying local inference nodes for EDA scripting, that throughput jump matters. It transforms an agentic loop from a multi-minute coffee break into an interactive background task that returns in seconds.

What the CVDP Benchmark Measures

Nemotron 3 Ultra was evaluated on the Comprehensive Verilog Design Problems (CVDP) benchmark, an evolution beyond early suites like VerilogEval.

Early RTL benchmarks were notoriously primitive. VerilogEval and its immediate successors tested language trivia: simple combinational multiplexers, basic Gray-code counters, shift registers, and trivial 4-state Mealy FSMs. Models trained on general software coding could often solve those problems simply by treating Verilog as C with different syntax, relying on blocking assignments (=) inside unclocked blocks.

CVDP introduces more practical design patterns across nine categories, including:

  • Multi-stage arithmetic pipelines.
  • Packet framing and streaming protocol decoders (AXI-Stream, UART, SPI).
  • Synchronous FIFO buffers with programmable watermarks.
  • Multi-state control paths with nested condition handling.
  • Algorithmic accelerators (such as CRC calculators and sorting networks).

Reaching a 97.1% pass rate on CVDP is a genuine achievement in automated code synthesis. It demonstrates that the model understands SystemVerilog grammar, handles parameterized module declarations, correctly instantiates sub-modules, and constructs functional digital logic that meets input-output assertions under nominal testbench conditions.

Yet, passing CVDP proves that a block works under functional simulation. It does not prove that the block is tape-out clean.

The Real Filter: What Happens in Lint and Synthesis

When you take agentic LLM outputs that passed functional simulation and run them through industrial static analysis tools like Synopsys SpyGlass, Real Intent Ascent Lint, or strict Verilator lint checks (-Wall), clean pass rates drop sharply. Functional simulators are forgiving. Silicon is not.

Here are the specific structural failure modes that frequently survive agentic testbench loops.

+-------------------------------------------------------------------------+
|                    Agentic Generation Loop                              |
|                                                                         |
|   +------------+      +---------------+      +---------------------+    |
|   | Spec / RTL | ---> | Functional    | ---> | Passes CVDP Tests?  |    |
|   | Generator  |      | Simulation    |      | (97.1% Pass Rate)   |    |
|   +------------+      +---------------+      +---------------------+    |
|         ^                     |                         |               |
|         | (6,629 tokens/iter) |                         | Yes           |
|         +-- Feedback on Fail -+                         v               |
+-------------------------------------------------------------------------+
                                                          |
                                                          v
+-------------------------------------------------------------------------+
|                    Industrial Static Verification                       |
|                                                                         |
|   +-----------------------------------------------------------------+   |
|   | Static Lint Checks (SpyGlass, Verilator -Wall)                  |   |
|   | - Inferred latches from incomplete unique case statements       |   |
|   | - Blocking assignments mixed into sequential always_ff blocks   |   |
|   | - Implicit wire declarations causing silent 1-bit truncations   |   |
|   +-----------------------------------------------------------------+   |
|                                 |                                       |
|                                 v                                       |
|   +-----------------------------------------------------------------+   |
|   | CDC / RDC Analysis (Meridian CDC, VC SpyGlass)                  |   |
|   | - Unsynchronized control signals across asynchronous clocks     |   |
|   | - Glitch-prone combinational logic feeding synchronizers        |   |
|   | - Missing 2-FF or handshake synchronizers on multi-bit buses    |   |
|   +-----------------------------------------------------------------+   |
|                                 |                                       |
|                                 v                                       |
|   +-----------------------------------------------------------------+   |
|   | Logic Synthesis & Timing Closure (Design Compiler, Vivado)      |   |
|   | - Multi-cycle paths generated without timing constraints (SDC)  |   |
|   | - Excessive logic depth on critical paths violating setup time  |   |
|   | - Area bloat from unshared arithmetic resources                 |   |
|   +-----------------------------------------------------------------+   |
+-------------------------------------------------------------------------+

Inferred Latches and Incomplete Priority Decoding

LLMs love always @(*) or always_comb blocks with nested if-else and case statements. If any branch fails to assign every single output variable, the synthesis tool infers a transparent latch.

Functional simulators running directed testbenches often do not catch inferred latches because the test vectors only evaluate valid, expected operational states. In simulation, an unassigned variable simply retains its previous value in the event loop, exactly matching the behavioral description. But during synthesis, latches create asynchronous feedback loops, complicate static timing analysis (STA), and make scan-chain insertion during Design-for-Test (DFT) painful.

Models often patch this incorrectly when prompted about simulation mismatches. Rather than refactoring the state decoding into clean, parallel combinational paths, they slap default assignments at the top of the block. While that avoids the latch, it can introduce unintentional priority logic, inflating cell count and adding unnecessary gate delays to the critical path.

Mixed Assignment Types in Sequential Logic

A classic bug in LLM-generated SystemVerilog is the improper mixing of blocking (=) and non-blocking (<=) assignments inside clocked always_ff @(posedge clk) blocks.

// Typical agent-generated pattern that passes simulation but violates lint
always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
        count     <= '0;
        limit_hit = 1'b0; // Error: blocking assignment inside sequential block
    end else begin
        count <= count + 1'b1;
        if (count == max_val) begin
            limit_hit = 1'b1;
        end else begin
            limit_hit = 1'b0;
        end
    end
end

In event-driven simulation, the blocking assignment updates immediately in the active region, allowing downstream logic within the same cycle to read limit_hit. In physical hardware, this represents a register whose output must drive downstream logic within the same clock edge, creating simulation-synthesis mismatches and race conditions during post-synthesis gate-level simulation.

Implicit Net Declarations

Unless the RTL explicitly sets `default_nettype none at the head of every file, Verilog-2001 and SystemVerilog compilers treat undeclared signals as 1-bit implicit nets (wire).

When an LLM misspells a multi-bit bus identifier inside an instance port map, such as typing data_in_bus instead of data_in_bus_reg[31:0], the compiler does not fail. It silently creates a 1-bit wire named data_in_bus, connects the lowest bit of the driver, and leaves the remaining 31 bits unconstrained or floating. The module might pass a weak functional test if the testbench only exercises lower-order bits, but fails completely on physical hardware.

The Structural Blindspot: Clock Domain Crossing and Resets

Where the 97.1% benchmark metric completely decouples from production engineering is in asynchronous boundaries and physical constraints.

Benchmark testbenches run on ideal, single-clock, zero-delay environments. Production chips operate across multiple asynchronous clock domains, varying voltage islands, and non-ideal reset distributions.

Asynchronous Reset Deassertion and Glitches

LLMs generate reset logic that looks standard on the surface:

always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
        q <= 1'b0;
    end else begin
        q <= d;
    end
end

What the model misses is the reset tree implementation. Asynchronous assertion is safe, but asynchronous deassertion within a few picoseconds of a clock edge causes flip-flop metastability, known as reset recovery and removal timing violations. Agentic models rarely generate the necessary reset synchronizer modules (two-stage flip-flop synchronizers with asynchronous assert and synchronous deassert) unless explicitly forced by a rigid structural prompt.

Clock Domain Crossing (CDC)

When a design requires transferring data between two asynchronous clocks (clk_a and clk_b), an agentic model will frequently generate a direct register read, or attempt a naive dual-flip-flop synchronizer across a multi-bit data bus.

Running dual-rank synchronizers on a multi-bit bus is a fundamental hardware bug. Due to routing delays and process-voltage-temperature (PVT) variations, individual bits arrive at the destination flip-flops at slightly different times. The destination clock domain captures an intermediate, corrupted data word.

A human designer knows you must use an asynchronous FIFO with Gray-coded read/write pointers, a 4-phase req/ack handshake, or a toggle-based capture mechanism. Because zero-delay functional simulators do not model physical setup/hold windows or multi-bit skew, the agentic model's naive multi-bit bus synchronizer passes 100% of functional test vectors while being guaranteed to fail in silicon.

Formal Property Verification vs Agentic Iteration

One promising avenue to bridge this gap is replacing simulation testbenches with formal property verification (FPV) inside the agentic loop.

In a standard simulation loop, the agent optimizes against specific input vectors. If the testbench does not assert a specific corner case (like back-to-back FIFO push/pop operations when full, or asserting valid without ready for 10,000 cycles), the model will write code that ignores that state.

When an agent is paired with a formal tool (such as JasperGold, VC Formal, or open-source SymbiYosys), the feedback loop changes completely. Instead of consuming cycle logs, the model consumes counterexamples: formal traces showing the exact cycle sequence that breaks a SystemVerilog Assertion (SVA).

However, this exposes a symmetrical failure mode: if the LLM is tasked with writing both the RTL and the SVA properties, it duplicates its own cognitive blind spots. If the model does not understand that an AXI-Stream ready signal can be deasserted indefinitely, it will omit that assumption from both the RTL state machine and the SVA assume/assert blocks. The formal engine reports a clean mathematical proof on an incomplete specification.

Deterministic static analysis must remain outside the model's direct control. Platforms like Silicode (silicode.ai) focus on wrapping model generation within strict verification pipelines, forcing the output through external lint, formal safety properties, and CDC checks before declaring a task closed.

PPA Realities: Area, Power, and Timing

Even when generated RTL is functionally correct and lint-clean, it often scores poorly on Power, Performance, and Area (PPA) metrics compared to human-optimized code.

Agentic models write structurally flat code. To satisfy timing in high-performance designs, engineers balance pipeline depths, perform manual register retiming, and deliberately structure arithmetic operations to map efficiently to dedicated FPGA DSP blocks or ASIC standard-cell library adders.

An LLM generating a 64-bit multiply-accumulate unit will write out <= out + (a * b); inside a single clock cycle. If the target frequency is 1 GHz on a 5nm process, or 400 MHz on an AMD Versal FPGA, this creates a massive combinational path that breaks setup time.

When the agent is prompted with timing slack reports from an EDA synthesis engine, it often struggles to distribute pipeline stages cleanly. Adding a pipeline register requires adjusting the control logic, stalling mechanisms, and backpressure signals throughout the entire pipeline. That structural refactoring spans multiple modules and state machines, where local agentic token edits often create state desynchronization bugs.

How to Use Nemotron 3 Ultra Today

Nemotron 3 Ultra's 97.1% score is a milestone for automated syntax generation, state-machine scaffolding, and unit-level module drafting. It proves that hybrid Mamba-Attention MoE architectures can process massive verification logs efficiently, lowering the operational cost of running agentic coding loops.

Engineering teams looking to deploy this model should treat it as an accelerated drafting tool rather than an autonomous silicon designer. To extract real value from it without introducing severe bugs into your repositories:

  1. Do not rely on simulation-only testbenches. Never accept agent-generated RTL based solely on Icarus, Verilator, or VCS functional pass rates. Run every generated block through strict lint rules (verilator --lint-only -Wall at minimum; commercial tools like SpyGlass or Ascent Lint if available).
  2. Isolate clock domains manually. Keep multi-clock infrastructure, asynchronous reset trees, and CDC bridges out of the LLM prompt scope. Use verified, parameterized internal IP libraries for FIFOs, synchronizers, and PLL wrappers. Task the model only with single-clock synchronous logic.
  3. Provide rigorous assertion suites upfront. Write the SVA properties and interface assertions before prompting the model. Do not let the model generate its own acceptance criteria.
  4. Inspect critical paths post-synthesis. Run the generated code through logic synthesis early. Check the worst negative slack (WNS) and total negative slack (TNS). If the model generated an unpipelined combinational tree, refactor the architectural pipeline manually rather than burning tokens asking the model to guess register balancing.

Nemotron 3 Ultra gives digital design teams a fast, high-throughput engine for initial RTL generation. The leverage it provides is real, provided you maintain an uncompromising static analysis and formal gatekeeping pipeline between the model output and your tape-out build.

Sources

RTL DesignASICEDA AutomationVerilog