AMD Unveils Helios AI Rack to Take On Nvidia at Microsoft Scale

On July 20, AMD published the full specifications for its Helios AI rack and announced that Microsoft plans to begin deploying the systems in its Azure data centers in the second half of 2026. Rather than selling individual accelerator cards, AMD is pitching a unified turnkey system designed to rival Nvidia's dominant rack architectures.

Inside the liquid-cooled, OCP-compliant frame are 18 compute nodes housing 72 Instinct MI455X GPUs, EPYC 9006 "Venice" processors built on TSMC's 2 nm process with up to 256 cores, and Pensando data processing units. According to reported figures, the three-ton assembly draws up to 245 kW and carries an estimated price tag around $5.5 million per unit, citing an estimate from CNBC.

The headline metric is memory density. While Nvidia's competing Vera Rubin NVL72 reportedly holds an edge in raw compute—50 FP4 petaflops per GPU against 40 for the MI455X—AMD packs 432 GB of HBM4 per chip. Across the rack, that totals 31 TB of high-bandwidth memory, compared to 20.7 TB on the Rubin design. For serving frontier language models with long context windows, keeping full parameters in unified fast memory can matter more than peak arithmetic throughput.

Interconnect performance remains theoretical. AMD claims 260 TB/s of scale-up bandwidth within the rack and 43 TB/s between racks over UALink and Ultra Ethernet. Yet open interconnect standards are largely unproven at this density compared to Nvidia's proprietary NVLink ecosystem.

The key hurdle remains the software. While AMD's open-source ROCm stack is validated for frameworks like PyTorch, TensorFlow, and vLLM, production deployments at hyperscale often encounter edge cases that require engineering effort to iron out. Hardware deliveries to Azure will reveal whether AMD's software ecosystem can match its ambitious memory footprint.

Source: habr