GDDR6, Ethernet, and the Economics of Open AI Infrastructure

GDDR6 vs HBM: A Compile-Time Trade-Off

High-bandwidth memory (HBM) dominates frontier GPU designs because it delivers extraordinary bytes-per-second — essential when runtime scheduling cannot predict access patterns. The cost is BOM, packaging complexity, and thermals: HBM stacks raise unit price and often push systems toward liquid cooling at scale.

Tenstorrent's counter-thesis for Blackhole: when TT-Forge knows the graph, memory fetches become predictable. Prefetch instructions hide GDDR6 latency; the chip spends less die area on HBM controllers and more on compute. For sustained inference — not one-off training spikes — the economics can favour cheaper memory plus smarter software.

Ethernet-on-Die and Open Networking

Proprietary scale-up interconnects excel at tightly coupled multi-GPU training pods — but they create fabric lock-in: switches, cables, and firmware from a single vendor ecosystem, complicating sovereign audits and spare-parts diversification.

Blackhole integrates Ethernet on die, enabling Galaxy servers to scale horizontally over the same 100G/400G Ethernet plant used for storage and management traffic. For AGICY's Vasilikos campus — and any EU facility prioritising auditable, multi-vendor networking — that openness reduces operational dependency on a single link-layer supplier.

This aligns with Phase 1 design: sovereign compute is not only about chip ISA but about whether the entire stack — from tensor compiler to top-of-rack switch — can be inspected, replaced, and certified under EU procurement rules.

Galaxy at Datacenter Scale

Scaling via standard Ethernet means AGICY can add Galaxy racks incrementally — matching Phase 1's 1,801-unit ramp without redesigning facility cooling or procuring proprietary switch ASICs.