Cerebras drops CS-4: three giant dinner-plate chips to crush Nvidia

While standard chipmakers take a pristine 300mm silicon wafer and chop it up into hundreds of tiny dies, Cerebras uses the entire uncut silicon disc as one absurdly massive processor. The newly revealed CS-4 server rack runs on three updated WSE-3 Turbo processors manufactured on TSMC's 5nm node.

Packing 4 trillion transistors and 900,000 tensor cores across 46,225 square millimeters, each wafer is essentially a computing continent. Since making a defect-free wafer of that size is chemically impossible, the system relies on hardware redundancy to dynamically route connections around flawed silicon sections without missing a beat.

Instead of watching AI calculations choke while waiting for data from external memory chips over slow buses, Cerebras baked 44 GB of SRAM directly onto the silicon wafer. That gives each chip an aggregated bandwidth of 43.2 PB/s, completely bypassing the PCIe and NVLink traffic jams that plague traditional GPU clusters.

Inside the rack, the three massive wafers link via proprietary direct interconnects that shave inter-chip latency down to a mere 2 microseconds. That architectural setup allows the system to spit out up to 4,400 tokens per second per user on hefty 120-billion-parameter models, performing inferencing workloads in one second that would tie up a typical GPU rack for half a minute.

Tossing out decades of modular chip design in favor of monolithic dinner-plate computing proves that when modern software hits a hardware wall, the only brute-force solution left is throwing an entire manufacturing line into a single chassis.

Source: cerebras.ai