GDDR vs HBM: bandwidth, capacity, cost, and the packaging fork

GDDR and HBM answer the same question — how do you feed a compute-starved GPU enough data? — with opposite strategies, and almost every difference between them (cost, capacity, power, and which product each ends up in) follows from that one fork. GDDR keeps memory as discrete DRAM chips soldered onto the circuit board, connected by a narrow-but-very-fast per-chip interface. HBM instead stacks DRAM dies and places the stack a few millimeters from the processor on a silicon interposer, connected by an enormously wide-but-modestly-clocked interface. This unit assumes the width-versus-clock mechanism established in HBM’s overview and contrasts the two families head to head.

Two ways to build bandwidth

GDDR — narrow interface, extreme per-pin speed, aggregate across many chips. A GDDR device presents a narrow interface (32 bits per chip for GDDR6/6X) and wins bandwidth by clocking each pin very hard and by wiring many chips in parallel across a wide board-level bus. Three signaling generations trace the escalation (Wevolver; Rambus):

• GDDR6 uses NRZ (two-level) signaling — one bit per symbol — at roughly 14–16 Gbps per pin, reaching ~18–24 Gbps in advanced parts.
• GDDR6X switches to PAM4 (four voltage levels, two bits per symbol) to push effective throughput up at ~19–21 Gbps per pin.
• GDDR7 adopts PAM3 (three levels, ~1.5 bits per cycle, a ~50% gain over NRZ) and starts at 32 Gbps per pin with a roadmap toward 48 Gbps.

Aggregate bandwidth is then the per-pin rate times the total bus width. NVIDIA’s RTX 4090 puts twelve GDDR6X devices on a 384-bit bus at 21 Gbps for about 1,008 GB/s across 24 GB (Runpod; ITCreations), and the GDDR7-based RTX 5090 reaches roughly 1.8 TB/s (Spheron) — the top of what a board-mounted, discrete-chip design delivers today.

HBM — a very wide interface at a modest clock, on-package. An HBM stack exposes a 1024-bit interface and runs each pin comparatively slowly: HBM3 at 6.4 Gbps per pin over 1024 bits is about 819 GB/s per stack, and HBM3E at ~9.2–12.4 Gbps per pin reaches roughly 1.2–1.3 TB/s per stack (Wevolver). A single HBM3E stack thus rivals or beats an entire high-end consumer card’s memory system. Accelerators mount several stacks side by side, so aggregate device bandwidth is the per-stack figure multiplied across stacks: NVIDIA’s H100 carries 80 GB of HBM3 at about 3.35 TB/s (Runpod), the H200 141 GB of HBM3E at about 4.8 TB/s (Runpod), and the B200 192 GB of HBM3E at about 8.0 TB/s (Jarvislabs); AMD’s MI300X carries 192 GB of HBM3 at about 5.3 TB/s (AMD). Datacenter HBM systems therefore run several times to an order of magnitude ahead of the fastest consumer GDDR on aggregate bandwidth.

Capacity: on-package stacking vs. board real estate

HBM’s stacking also lifts the capacity ceiling. HBM3/HBM3E stacks reach up to about 36 GB each (Wevolver), and shipping accelerators carry 80–192 GB of it on-package (H100 80 GB, H200 141 GB, B200 and MI300X 192 GB). GDDR capacity is bounded by how many discrete devices fit around the die and route cleanly on the board; consumer cards land in the ~24–48 GB range (the RTX 4090’s 24 GB is twelve 2 GB devices on its 384-bit bus). Capacity, not just bandwidth, is a first-class reason datacenters reach for HBM — see the segmentation below.

The packaging fork (where cost and supply are decided)

The strategies diverge physically. GDDR uses standard surface-mount (BGA) packaging on the printed circuit board: mature, high-volume, cheap assembly, with the design burden falling on high-speed board routing. HBM cannot be built that way — a stack’s more than a thousand I/O connections cannot be routed on a conventional organic substrate because of trace length, loss, and crosstalk, so the stacks and the compute die are co-mounted on a passive silicon interposer (2.5D integration, e.g. TSMC’s CoWoS) whose fine metal routing and dense through-silicon vias make the wide bus feasible (Wevolver). That interposer step is exactly what makes HBM expensive and supply-constrained: it adds advanced packaging on top of an already hard stacked-die process (TSV etch and copper fill, thinning, and bonding a dozen-plus dies with micrometer alignment), and it is produced by only a handful of firms (Vik’s Newsletter). Advanced-packaging capacity — CoWoS in particular — has been a binding, allocation-gated constraint on accelerator supply, with the largest buyers securing much of it (Introl). The practical consequence: HBM is markedly more expensive per gigabyte than GDDR, but the exact premium is hard to state — HBM is never sold on the open market, so per-GB figures are analyst estimates rather than posted prices. The market dynamics of that premium are tracked as analysis in the HBM pricing unit; this unit deliberately states the cost difference qualitatively rather than as a multiplier.

Power, and a genuinely contested latency picture

On energy, the wide-and-slow strategy helps HBM: shorter on-package signal paths and a lower-clocked interface move each bit for less energy, which is why HBM is generally credited with better bandwidth-per-watt (ByteBridge). A concrete data point from the HBM2 generation put HBM2 near 65 GB/s per watt against GDDR6’s ~45 GB/s per watt (Wevolver) — a real advantage, though that specific figure is an earlier-generation comparison and should not be read as the current HBM3E-vs- GDDR7 gap.

Latency, by contrast, is not something to declare a winner on. Sources genuinely disagree, and in opposite directions: GDDR6 raw access latency is often quoted at roughly 20–30 ns (Jarvis Labs), yet other comparisons report HBM as the lower-latency part and still others report the reverse, apparently because they measure different things (raw device access vs. effective latency under load, and different clock assumptions). The honest framing is not “X is faster” but that the two make different bets: GDDR trades latency tolerance for cheap, high per-pin streaming bandwidth, while HBM buys maximum aggregate bandwidth and capacity at a packaging cost. The raw-latency comparison between them is contested across the available sources and is left unresolved here.

Why datacenters take HBM and consumers take GDDR

The split is now sharp. Datacenter accelerators — NVIDIA H100/H200/B200, AMD MI300X — use HBM3/HBM3E; consumer and gaming cards — GeForce RTX, Radeon — use GDDR (Spheron; AMD). The fork above explains why. Consumer graphics workloads are not starved for the last increment of bandwidth or capacity, so the cost-efficient, board-mountable GDDR path is the right one. Datacenter inference is the opposite regime, and the KV cache is the clearest driver: large-model inference is memory-bandwidth-bound during decode, and the KV cache is a large, sequence-length-dependent consumer of both capacity and bandwidth. Its scale is what forces the HBM choice — the worked example in the KV-cache unit (320 KiB per token for a Llama-3-70B-class model) implies on the order of 40 GiB of KV cache for a single 128K-token request, before model weights are counted. A workload like that does not fit, or stream fast enough, on a 24–48 GB GDDR card, but sits comfortably inside an 80–192 GB HBM accelerator. Capacity to hold large models plus their KV cache, and the aggregate bandwidth to stream all of it every decode step, are precisely what HBM’s wide-and-stacked strategy buys — and precisely what justifies its cost only when a workload is bandwidth- or capacity-bound.

Confidence 0.65: the structural spine — the narrow-fast-discrete vs. wide-slow-stacked fork, the packaging basis (BGA-on-PCB vs. interposer/CoWoS), the capacity and bandwidth ordering, and the datacenter-vs-consumer segmentation — is corroborated across multiple independent sources and is internally consistent with this base’s HBM and KV-cache units. It is held in the mid range because many of the exact figures (per-stack bandwidths, accelerator aggregates, the HBM2-era power ratio) come from secondary trade sources rather than primary datasheets, because the per-GB cost premium is deliberately left qualitative (no resolvable source states a defensible multiplier, and HBM is not openly priced), and because the raw-latency comparison is contested and left open. Confidence should rise as figures are pinned to vendor datasheets and JEDEC specs in the HBM-generations unit.