For the past several years, each NVIDIA GPU generation invited the same question: how much faster is the chip?
The Vera Rubin NVL72 reframes it.
NVIDIA's unit now is not a die but a rack, and the most important number is not peak FLOPS but sustained tokens per megawatt and cost per token. That shift is deliberate, and it moves the hardest problems onto the data hall where the impacts are felt most - both in performance and in cost.
At the center is Vera Rubin NVL72, a liquid-cooled system that combines 72 Rubin GPUs and 36 Vera CPUs within a rack-wide NVLink domain. Around it, faster Ethernet, context-memory storage, new optical technologies, and changes to power distribution, cooling, and serviceability support the movement of data across an expanding AI factory. This article examines the platform’s architecture and operating model, including the co-designed CPU and GPU stack, memory and bandwidth gains, NVLink connectivity, network and security considerations, and what it takes to deploy and operate the system in production with Radiant.
For enterprises and AI organizations operating at hyperscale, the significance extends beyond higher peak performance. Rubin changes how large-scale inference is measured and delivered: more users served, more complex reasoning completed faster, and agent workflows run with better efficiency, while demanding data center designs that infrastructure teams and decision-makers need to evaluate now.
A System of Seven Co-designed Chips
The NVIDIA Vera Rubin NVL72 platform brings together the Rubin GPU, Vera CPU, NVLink 6 switch, ConnectX-9 SuperNIC, BlueField-4 DPU and Spectrum-6 Ethernet switch. A seventh component, the Groq 3 LPU, extends the platform through separate LPX racks designed for low-latency inference. These components serve distinct roles across the AI factory rather than all residing inside the NVL72 rack.
The building block is the Vera Rubin Superchip, which pairs one Vera CPU with two Rubin GPUs over a coherent NVLink-C2C connection. Two superchips form a compute tray; 18 compute trays and nine NVLink switch trays form the NVL72 system.
This architecture reflects the changing demands of inference. Processing an input prompt, or prefill, often benefits from substantial parallel compute. Generating subsequent tokens, or decode, can be constrained by memory bandwidth and communication latency, with bandwidth often becoming the dominant limiter in decode-heavy workloads. Mixture-of-experts models introduce another dependency by routing tokens between experts distributed across GPUs, so the architecture is designed for both AI training and inference, and changing inference workloads are shaping the system design.
Vera CPU brings 88 cores and 1.5 TB of memory
Vera expands the CPU’s role in supplying and coordinating GPU work. Each processor contains 88 custom Olympus cores and supports 176 hardware threads through Spatial Multithreading, which partitions core resources between threads. Its memory subsystem provides up to 1.5 TB of LPDDR5X capacity and 1.2 TB/s of bandwidth, compared with Grace’s 480 GB and approximately 512 GB/s. Compared with previous generations, more orchestration and memory work shifts onto the CPU side.
The new trick is Spatial Multithreading, which runs two hardware threads per core by physically partitioning resources instead of time-slicing them. That preserves the deterministic latency and strong isolation for multi-tenancy, rather than the noisy-neighbor behavior in traditional multithreading.
Source: NVIDIA
The CPU–GPU NVLink-C2C connection also doubles to 1.8 TB/s, supporting coherent access across CPU and GPU memory. This gives the system more capacity for data preparation, orchestration, memory offload and the CPU work surrounding agentic inference, as well as reinforcement learning.
For infrastructure teams, Vera’s significance is therefore broader than its core count. It increases the resources available to keep GPUs supplied with useful work as applications become more dependent on retrieval, tool execution and coordination.
Rubin GPU delivers 5x Blackwell’s inference
NVIDIA’s technical specifications place Rubin’s peak NVFP4 inference capability at five times the original NVIDIA Blackwell architecture baseline. The Rubin GPU combines two compute dies in a 336-billion-transistor package, compared with Blackwell’s 208 billion.
NVIDIA has mentioned up to 50 PFLOPS of NVFP4 inference performance and 35 PFLOPS of dense NVFP4 training performance per GPU, while the full NVIDIA Vera Rubin NVL72 system delivers 3.6 Exaflops of AI inference performance.
Several enhancements explain how Rubin aims to convert that arithmetic capacity into useful work:
- More efficient Tensor Core execution: A larger K dimension reduces the loop iterations required for matrix multiplication, lowering overhead as operations are distributed across GPUs.
- Simpler expert-data access: Tensor Memory Accelerator improvements allow kernels to update tensor addresses and strides inline, reducing descriptor-management overhead when moving between experts.
- Earlier execution of dependent work: Finer-grained coordination allows consumer kernels to begin when their required inputs become available, reducing idle gaps between operations.
- Faster attention for long-context reasoning: Compression, structured sparsity and improved exponential throughput reduce the work associated with intermediate attention data and softmax.
Smaller Weights with Rubin GPU
Rubin’s most novel feature is a 3-bit lookup-table format. Each weight is stored as a 3-bit index into a small table of representative values, resolved inline by Tensor Cores to reduce weight storage. For a 2.8-trillion-parameter model, that means roughly 1.09 TB of weights versus 1.49 TB with MXFP4, the difference between four HBM4 packages and six, helping cut token cost while maintaining model accuracy. This is particularly helpful in today's memory-constrained infrastructure landscape.
HBM4: A Major Increase in Memory Bandwidth
Rubin provides up to 288 GB of HBM4 per GPU, matching Blackwell Ultra’s physical memory capacity while substantially increasing bandwidth. NVIDIA’s architectural disclosures specify up to 22 TB/s per GPU, compared with 8 TB/s for Blackwell and Blackwell Ultra.
Relative to GB300, Rubin’s memory advantage therefore primarily concerns how quickly data can be accessed. Higher bandwidth can help serve more concurrent requests or reduce token-generation latency when memory traffic is the limiting factor, but it does not automatically increase the amount of model data that fits in HBM.
Across 72 GPUs, the rack contains approximately 20.7 TB of physical HBM4 and supports up to 75TB of fast memory, alongside up to 54 TB of CPU memory. This creates substantial capacity for model parameters, context and supporting workloads, provided software places data appropriately across the hierarchy.
That placement becomes more consequential as context windows grow, because aggregate memory bandwidth is critical for long-context and large scale AI inference. Model weights, active key-value caches and intermediate data compete for finite HBM capacity. Faster memory helps the GPU process that working set for AI inference, while effective cache management determines how many requests the system can sustain. With HBM supply constrained across the industry, Rubin’s higher bandwidth and effective memory management can help increase inference throughput per gigabyte, supporting growing AI demand.
NVLink 6 turns 72 GPUs into one accelerator
NVIDIA NVLink 6 raises architectural scale-up bandwidth to 3.6 TB/s bidirectional per GPU, twice that of NVLink 5. The rack’s switched fabric provides all-to-all connectivity across 72 GPUs, supporting workloads that distribute parameters, experts and computation throughout the system for AI training as well as inference-heavy MoE deployments.
For mixture-of-experts (MoE) inference, this helps address the communication overhead accompanying sparse computation. Activating only a subset of experts reduces arithmetic work, but tokens must still reach the GPUs hosting those experts and return their results. As compute becomes faster, those exchanges can occupy a larger share of execution time.
Beyond MoE, NVLink connects 72 GPUs so larger models and longer contexts can stay in GPU memory, reducing slower CPU-memory offload. Faster exchanges can shorten training and accelerate token generation, reducing GPU-hours per job or per million tokens when communication limits performance.
Rubin also improves the mechanics of communication for training and inference. Counted writes streamline synchronization for GPU-initiated NVLink transfers, while SHARP in-network compute can move supported collective operations into the switching fabric. These features address latency and coordination overhead alongside bandwidth.
Up to 67x better performance per dollar
SemiAnalysis’s verified AgentX results show Vera Rubin NVL72 delivering up to 67x more total token throughput per dollar than GB300 NVL72 on DeepSeek V4-Pro. The economic advantage spans two measures: interactivity, or how quickly tokens stream to each user, and end-to-end latency, the time required to complete a response.
At matched interactivity targets, Rubin serves significantly more traffic for the same expenditure:
Source: SemiAnalysis
The advantage extends into typical serving speeds, while the rental results show that efficiency gains can outweigh higher capacity prices. For infrastructure teams, this means evaluating how much demand a deployment can support at the required responsiveness alongside its hourly cost.
Faster streaming is only part of that experience. At matched token economics, Rubin also reduces end-to-end response times:
Source: SemiAnalysis
Shorter waits can accumulate across agent workflows that depend on successive model responses, reducing delays between reasoning steps and tool calls. Together, these gains create room to serve more concurrent work while helping individual workflows finish sooner.
Up to 30x higher token throughput per megawatt
NVIDIA’s latest results show Vera Rubin NVL72 achieving up to 30x the throughput per megawatt of GB300 NVL72 for agentic workloads on SemiAnalysis’ AgentX benchmark using DeepSeek V4-Pro, at 160 tokens per second per user, and NVIDIA positions the platform for both AI inference and training efficiency at scale. Agentic workloads add another consideration: generated tokens are only one contributor to task completion. Tool execution, retries and sequential dependencies also consume time and resources.
These results illustrate why system-level gains can exceed the improvement in any individual component. Faster memory, more efficient kernels and reduced communication overhead can collectively allow a system to serve more requests within a given latency target.
Vera Rubin is transforming data center design
The NVL72 rack weighs roughly 4,000 lbs, carries about 1,300 chips and 1.3 million components, and draws 190-230 kW without CPX. Blackwell runs 120-130 kW and a Hopper rack is 30-40 kW. A conventional enterprise rack draws 5-10 kW. This is not a refresh cycle you retrofit into. It is a rack you design a building around, and the industry is redesigning three things at once: how power reaches the rack, how heat leaves it, and how the network is wired.
At this density, floor loading, equipment delivery routes and maintenance clearances must accommodate the rack’s weight and dimensions, while electrical distribution and liquid-cooling loops must support concentrated demand at each rack position. These requirements shape data hall layouts, plant capacity and service routes early in design, with redundancy and isolation provisions that allow equipment to be maintained or replaced while workloads remain live.
Vera Rubin NVL72 is now in full production and Nvidia expects Vera Rubin systems to account for 20% of its data center revenue by Q3’ 2027, making it the fastest ramp in the company’s history.
Compatibility with an established rack format helps deployment, but an existing liquid-cooled data hall still needs to be assessed against the selected configuration’s electrical and thermal requirements. The design challenge is to deliver sufficient power and cooling while preserving the ability to maintain, repair and expand infrastructure around live workloads. At higher density, those operational conditions become inseparable from the performance available to customers.
800V DC re-architects the entire power chain
As rack power rises, distributing that power at low voltage requires higher current and increasingly substantial conductors. Higher-voltage DC architectures, including the industry’s move toward 800 VDC, address this pressure by reducing current for a given power transfer and creating opportunities to consolidate conversion stages.
The implications extend beyond the rack. The location of AC-to-DC conversion affects power-room design, distribution equipment, protection systems, backup integration and maintenance boundaries. Solid-state transformers are part of the longer-term direction, while nearer-term designs can retain conventional equipment and introduce DC distribution in stages.
For Vera Rubin planning, 800 VDC should be treated as an infrastructure roadmap consideration rather than a universal NVL72 installation requirement. The practical question is how the facility will support the initial deployment and subsequent density increases without an avoidable redesign of its electrical backbone.
Liquid-only cooling with warm water, fanless trays
NVIDIA specifies a 45°C coolant inlet temperature and describes a fully liquid-cooled MGX design with fanless compute trays. Warmer coolant can extend the conditions under which dry coolers provide heat rejection, reducing dependence on mechanical refrigeration. The opportunity depends on climate and the facility’s thermal design.
That efficiency opportunity still requires a carefully engineered hydraulic system. Coolant distribution units, manifolds, pressure limits, fluid quality and leak detection must support the rack’s heat load under normal operation and during maintenance.
The important design question is whether the cooling system can sustain the required flow and temperature while individual components are isolated or serviced. At this density, cooling availability is directly connected to compute availability.
Co-packaged optics cut network power 5x
Vera Rubin’s internal NVLink fabric connects GPUs within the rack. Outside the rack, NVIDIA Spectrum with Spectrum-X Ethernet Photonics addresses a different problem: moving data efficiently across the wider cluster.
Conventional optical switches drive high-speed electrical signals between the switch ASIC and faceplate transceivers. Co-packaged optics (CPO) brings optical engines onto the switch package, shortening that electrical path and reducing associated power consumption. NVIDIA reports 5x better power efficiency for its CPO technology compared with pluggable transceivers; this should not be interpreted as a fivefold reduction in the entire cluster’s networking power.
At large scale, lower optical-link power can reduce the networking share of the facility’s energy budget. It also changes the physical implementation and service model of the fabric, making fiber routing, connector handling, cooling and replacement procedures part of deployment planning.
CPO is therefore both a networking choice and an operational one. Its benefits should be evaluated against the required topology, availability targets and maintenance approach.
Higher density and sharper power ramps
AI workloads create changing power demands as GPUs move between computation, communication and waiting. Average rack consumption does not capture the effect of synchronized transitions across a cluster.
Vera Rubin incorporates local energy storage and power smoothing to reduce those transients. For operators, the opportunity is to coordinate workload power management with facility limits, making more effective use of available capacity while preserving application performance.
The redesigned compute tray also simplifies assembly and servicing through a PCB midplane and integrated connections. NVLink switch maintenance capabilities and hardware health checks are intended to reduce disruption and recovery overhead.
These improvements matter because denser systems concentrate more productive capacity within each maintenance boundary. Faster component replacement creates value when it shortens the time between detecting a fault and returning validated capacity to service.
KV-cache storage and full-rack confidential computing
Long-running agents accumulate context through conversation history, retrieved documents, tool outputs and intermediate reasoning. Retaining the corresponding key-value cache avoids repeating earlier computation, but consumes memory that could otherwise support more requests.
BlueField-4 supports NVIDIA’s Inference Context Memory Storage architecture, extending the hierarchy with a shared tier for context data. The purpose is to retain and reuse more state without keeping every cached interaction in scarce GPU memory, and separating active GPU memory from shared context tiers can improve large-scale AI inference efficiency while lowering token cost.
Its value depends on the access pattern. Context retained for later reuse presents a different requirement from an active attention working set. As MinIO explains in its analysis of KV-cache offload, moving context beyond GPU memory requires coordinated scheduling, networking and storage, which is one of the key objectives of NVIDIA’s Vera Rubin co-design strategy.
The security boundary expands with the system as well. Vera Rubin extends confidential computing across CPUs, GPUs and their interconnects within the rack-scale domain. For regulated and sovereign deployments, protecting distributed execution becomes increasingly relevant as sensitive models and data span multiple devices.
Turn Vera Rubin Performance into Lasting Value with Radiant
Vera Rubin creates value by delivering more useful work from the power, capital and time invested. Realizing that value requires the facility, cluster and operations to work together, with efficiency engineered beyond the rack.
Radiant’s hyperscale delivery model brings site development, power, data center engineering and compute into one program. Design aligns electrical distribution, liquid cooling, networking and expansion with Vera Rubin NVL72’s requirements, addressing dependencies that delay commissioning and constrain performance.
Deployment begins before hardware arrives. Radiant’s powered, permitted and fiber-connected sites, combined with prefabricated power and cooling systems, shorten the path to productive capacity. Standardized building blocks reduce integration work and simplify expansion.
In production, Radiant FlightDeck unifies capacity, health, topology and maintenance. Teams can allocate resources to workload requirements, trace faults to affected applications, and coordinate recovery through one platform. Radiant delivers reliable support beyond commissioning, helping sustain performance through maintenance and changing demand.
For Vera Rubin NVL72 deployments, Radiant connects faster delivery with sustained operating efficiency: bringing capacity into service sooner, keeping it productive and preserving its value as infrastructure grows.
.jpg)
