NVIDIA Nemotron 3 Ultra and the Reality of 97 Percent RTL Pass Rates

NVIDIA reports a 97.1% pass rate on CVDP using Nemotron 3 Ultra and ACE-RTL. We examine the token costs, CDC pitfalls, and synthesis realities behind the number.

NVIDIA published benchmark results showing its Nemotron 3 Ultra model achieving a 97.1% average pass rate across nine hardware design task categories on the CVDP benchmark. Paired with the ACE-RTL agent framework, the model completed these tasks with an average consumption of 6,629 tokens per iteration. The underlying architecture is a 550-billion-parameter Mixture-of-Experts (MoE) system with 55 billion active parameters per token, built as a hybrid combining Mamba state-space layers with standard self-attention.

For digital design engineers who have spent the last two years testing code-generation models against real ASIC and FPGA requirements, a 97.1% pass rate triggers immediate skepticism. We have seen high pass rates before on benchmarks like VerilogEval and RTLLM, only to run the generated code through Synopsys Design Compiler, Cadence Genus, or a strict SpyGlass lint run and find it riddled with inferred latches, broken clock domain crossings, and nonsensical reset trees.

Nemotron 3 Ultra represents a real shift in inference efficiency and architectural capability. But understanding what that 97.1% figure actually means, and where it falls apart, requires looking past the headline metrics into the mechanics of agentic RTL generation.

Anatomy of the Benchmark: What CVDP Actually Tests

The Chip Verification and Design Productivity (CVDP) benchmark evaluates language models on discrete hardware blocks across nine categories. These tasks typically cover standard register-transfer level (RTL) building blocks: arithmetic logic units, finite state machines, basic pipelined execution units, priority encoders, synchronous FIFO buffers, and common bus interface shims (such as simple APB or AXI-Lite peripherals).

In the ACE-RTL framework, the model does not just spit out a single Verilog file blindly. It operates in an agentic loop. The agent drafts a SystemVerilog module, invokes a simulator (typically Verilator or Icarus Verilog) along with an automated testbench, captures standard output and compiler error logs, and feeds those diagnostics back into the prompt for iterative correction.

Reaching a 97.1% pass rate in this context means that within a bounded number of iterations, the model produced code that compiled cleanly under the target simulator and passed the functional assertions defined in the testbench.

This is a major improvement over older dense models like early Llama variants or generic coding models, which often drifted into C++ syntax, confused blocking (=) and non-blocking (<=) assignments inside procedural blocks, or failed to resolve basic port list mismatches. Nemotron 3 Ultra gets the basic syntactic and immediate functional wiring right.

Passing a unit-level testbench in Verilator is not equivalent to producing tapeout-ready, synthesisable SystemVerilog. Real design constraints live in the physical and structural details that unit testbenches do not see.

The Token Economics of the 6,629-Token Iteration

NVIDIA's reported figure of 6,629 tokens per iteration gives us concrete visibility into the cost profile of agentic RTL generation.

In a standard software agent (such as one writing Python or TypeScript), an iteration loop usually consumes 1,500 to 3,000 tokens. Why does RTL require more than double that volume?

First, hardware descriptions carry high syntactic overhead. A fully parameterized SystemVerilog module with explicit typedefs, package imports, and detailed interface declarations takes substantial prompt real estate.

Second, the feedback loop in hardware design is verbose. When an EDA tool like Verilator or a linter rejects a design, it produces multi-line warnings covering unused bits, implicit net declarations, width mismatches, and potential race conditions. Feeding that compiler log back into the context window, alongside the original specification and the previous iteration's code, drives token counts up rapidly.

+-------------------------------------------------------------+
| Specification Prompt + Interfaces + Lint Rules (~2.5k tok)  |
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
| Nemotron 3 Ultra (55B Active MoE / Mamba Hybrid)            |
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
| Generated SystemVerilog Module (~1.2k tok)                  |
+-------------------------------------------------------------+
                              |
                              v
+-------------------------------------------------------------+
| Verilator Compilation & Functional Simulation Run           |
+-------------------------------------------------------------+
       |                                              |
   [Passes]                                       [Fails]
       |                                              |
       v                                              v
+--------------+                    +-------------------------+
| Output RTL   |                    | Error Diagnostics Dump  |
+--------------+                    | (~2.8k tok context)     |
                                    +-------------------------+
                                              |
                                              +--> [Feed to Next Loop]

If an agent requires four to six iterations to converge on an edge-case bug in a complex arbiter, the total token expenditure for that single module reaches 25,000 to 40,000 tokens.

This is where Nemotron 3 Ultra's architecture matters. Running a dense 550B model for 40,000 tokens across hundreds of sub-blocks would be commercially impractical. Because Nemotron 3 Ultra uses an MoE architecture with only 55 billion active parameters, and incorporates Mamba state-space layers that process context with linear computational scaling, the generation throughput is significantly higher. NVIDIA reports up to 5.9x higher throughput compared to competing open models like GLM-5.1 or Kimi-K2.6 under long output contexts. In an agent loop, generation speed directly dictates whether an automated run takes five minutes or two hours.

Where 97% Breaks Down: Clock Domain Crossing

The primary flaw in functional benchmarks like CVDP is that they evaluate RTL in an idealized single-clock simulation environment. Real ASICs and complex FPGAs exist in multi-clock environments where clock domain crossing (CDC) correctness is non-negotiable.

Language models consistently struggle with CDC because the functional validity of a synchronizer cannot be proven by standard cycle-accurate functional simulation without injected jitter or specialized formal tooling.

Consider an asynchronous FIFO crossing data between clk_a and clk_b. An LLM can easily write the dual-port memory array and instantiate the read/write pointer logic. Nemotron 3 Ultra will generate valid Gray code conversion logic for the pointers without syntactic errors:

// Standard binary to Gray conversion generated by LLM
always_comb begin
    wptr_gray = (wptr_bin >> 1) ^ wptr_bin;
end

Where models fail is in the structural synchronization details:

Multi-bit bus synchronization without Gray coding: When asked to pass a multi-bit control bus across domains, models frequently attempt to place a simple two-flip-flop synchronizer (always_ff @(posedge clk_b) sync_reg <= {sync_reg[0], async_sig};) on every bit of a wide binary bus. In simulation, this works perfectly because all signals transition simultaneously without routing skew. On silicon, skew between bits causes intermediate states to be sampled, corrupting the transfer.
Missing synchronizer attributes: Modern synthesis and CDC sign-off tools (like Synopsys SpyGlass or Questa CDC) require specific attributes ((* async_reg = "true" *) in Vivado, or set_false_path / set_max_delay -datapath_only constraints in SDC files) to prevent the synthesis tool from placing synchronizer flops far apart or optimizing them into a single multi-input cell. Models virtually never generate these integration hooks unless explicitly prompted with foundry-specific templates.
Pulse synchronizers and handshake protocols: For control signals, models often generate edge detectors on the destination domain without ensuring the source pulse is stretched long enough to be sampled by the slower destination clock. If clk_a runs at 800 MHz and clk_b runs at 100 MHz, a single-cycle pulse on clk_a disappears completely. Simulated in a benchmark where both clocks are tied to the same base period or not stressed with randomized phase relationships, the test passes. In hardware, it fails intermittently.

Reset Trees and Asynchronous Deassertion Hazards

Another systemic blind spot in agent-generated RTL is reset design. Benchmarks generally accept either synchronous or asynchronous resets as long as the registers clear on testbench initialization.

In standard digital design, the reset architecture must follow strict rules depending on the target technology:

Asynchronous Assert, Synchronous Deassert: For high-speed ASIC flows, asynchronous reset asserting must be captured, but deassertion must be synchronized to the local clock domain to avoid reset recovery and removal timing violations (rst_n transitioning inside the setup/hold window of the flip-flop).
Synchronous Reset Dominance: Modern FPGA architectures (such as AMD UltraScale+ or Intel Agilex) heavily optimize for synchronous resets. Using asynchronous resets wastes dedicated flip-flop control sets, degrades routing density, and prevents the packer from combining logic into unified slice registers.

When Nemotron 3 Ultra generates an always_ff block, it frequently alternates reset styles across different sub-modules unless constrained by a strict system prompt. A module might define:

// Module A (Generated Style 1)
always_ff @(posedge clk or negedge rst_n) begin
    if (!rst_n) begin
        q <= '0;
    end else begin
        q <= d;
    end
end

// Module B (Generated Style 2)
always_ff @(posedge clk) begin
    if (rst_sync) begin
        q <= '0;
    end else begin
        q <= d;
    end
end

While both are individually valid, mixing these paradigms in a single subsystem creates structural chaos for physical design teams. The clock tree synthesis (CTS) and reset tree synthesis scripts must treat these nets differently. An agent optimizing purely for a functional pass on CVDP has no awareness of control set routing limits or reset recovery arcs.

Synthesis Pitfalls: Latches, Blocking Assignments, and Dialects

A common issue with automated RTL generation is code that simulates accurately in an event-driven simulator but creates unroutable or timing-hostile silicon during logic synthesis.

Inferred Latches from Incomplete Combinatorial Paths

Even strong models occasionally generate unintended latches in large combinatorial blocks. In complex state machines with dozens of output signals, missing an assignment in one branch of a case or if-else statement forces the synthesis tool to infer a transparent latch to preserve the previous state.

// Faulty combinatorial assignment leading to latch inference
always_comb begin
    next_state = current_state;
    out_valid  = 1'b0;
    // Missing default assignment for out_data
    case (current_state)
        IDLE: begin
            if (start) begin
                next_state = RUN;
                out_data   = in_data; // out_data not assigned if !start
                out_valid  = 1'b1;
            end
        end
        RUN: begin
            next_state = DONE;
            out_data   = processed_data;
        end
        DONE: begin
            next_state = IDLE;
        end
    endcase
end

In standard SystemVerilog, using always_comb forces the simulator to issue a warning if a latch is inferred, but some simulators allow execution to proceed anyway. An agent running a basic test suite might observe functional passes if the execution path never exercises the unassigned latch condition. In physical synthesis, that latch introduces critical timing paths and creates major testability (DFT) problems.

Blocking vs Non-Blocking Assignment Confusion

While frontier models have largely stopped using blocking assignments (=) inside sequential always_ff blocks, subtle ordering bugs remain in complex pipeline designs.

When updating multi-stage pipeline registers, an agent might occasionally order assignments such that a value propagates across multiple pipeline stages in a single simulated clock cycle if it accidentally mixes assignment types. Modern lint rules flag this instantly, but lightweight agent loops without comprehensive lint integration often miss the violation.

Why Hybrid Mamba-Attention Matters for Hardware EDA

Despite these limitations, Nemotron 3 Ultra's architecture offers a concrete technical advantage over traditional pure-Transformer models when integrated into EDA pipelines: long-context log ingestion.

A typical static timing analysis (STA) report from Synopsys PrimeTime or a gate-level simulation log can span 50,000 to 100,000 lines of text. Standard dense Transformer architectures struggle with long-context inference costs, because self-attention compute scales quadratically ($O(N^2)$) with token length.

Nemotron 3 Ultra utilizes Mamba state-space layers interleaved with attention. The Mamba layers process long sequence histories with linear computational complexity ($O(N)$), maintaining a compressed recurrent state. This allows an RTL agent to ingest entire synthesis log files, multi-thousand-line lint reports, and gate-level trace dumps without running into prohibitive memory walls or stalling inference throughput.

When a timing violation occurs along a critical path, the agent must parse the data arrival time, clock skew, library cell delays, and net fanouts across dozens of hierarchical levels. An MoE model with linear-scaling context layers can ingest that entire path report, pinpoint the specific combinatorial logic cone causing the delay (such as an oversized ripple adder or deeply nested priority multiplexer), and rewrite that specific sub-block into a pipelined tree structure.

At Silicode, we see this exact interface between long-form EDA diagnostics and fast iterative generation as the primary bottleneck in autonomous RTL development, far more than basic syntax completion.

What Works Today vs What Still Needs Human Verification

The 97.1% CVDP pass rate demonstrates that LLM agents have essentially solved small-block syntax and local functional composition. If an engineer gives Nemotron 3 Ultra an explicit, bounded interface specification for an isolated digital block, the model will generate working SystemVerilog that satisfies functional assertions within a few iterations.

Engineering teams should treat agentic RTL models as rapid drafting engines rather than autonomous silicon designers. The real leverage lies in pairing these models with rigid, non-LLM deterministic tooling:

Automated Lint Guardrails: Never let an agent feed code directly into a functional simulator without first passing through a strict linter configured with production waivers. The linter must reject any inferred latches, implicit wire declarations, or non-standard reset constructs before simulation begins.
Formal CDC Verification: Every multi-clock module generated by an agent must be processed by a dedicated CDC tool. Do not rely on functional testbenches to validate asynchronous handshakes.
Power, Performance, and Area (PPA) Feedback Loops: Passing a functional test is the bare minimum. The next generation of RTL agents must integrate synthesis results directly into the loop, allowing the model to optimize register balancing, retiming, and logic depth based on actual standard cell library metrics.

Nemotron 3 Ultra proves that open, efficient architectures can eliminate the manual labor of typing boilerplate SystemVerilog. The job of the digital design engineer is shifting away from writing state machine transitions by hand, and moving toward defining exhaustive architectural constraints, formal properties, and synthesis envelopes that prevent high-probability agent code from failing on real silicon.