TraviaTechPie Review

The Story

For a couple of years, the story of AI hardware was a story about one company and one kind of chip. Nvidia builds the GPU, everyone lines up to buy it, the stock goes to the moon, repeat. The GPU was the thing you couldn’t get enough of. If you wanted to train or run a frontier model, you needed more compute, and compute meant Nvidia silicon.

관련해서 the next bottleneck: heat and power도 함께 참고하시면 좋습니다.

For a different angle on the same wall — putting memory on top of the GPU — see Samsung’s zHBM concept.

That story is quietly changing. The scarce thing isn’t really the GPU anymore. It’s the memory bolted onto it.

The memory in question is HBM — high-bandwidth memory. If you haven’t run into the term, here’s the plain version: it’s DRAM, the same basic stuff in your laptop, but stacked vertically into towers of chips and wired directly next to the GPU with a firehose-wide connection. The whole point is bandwidth — how fast you can shovel data into the processor. And in 2026, the industry moved to the next generation of it, HBM4. That’s the headline. But the more interesting thing is why HBM suddenly matters this much, and what happened the moment it did.

Let me set the stage with the actual products, because they’re real and shipping. Nvidia’s next platform, Vera Rubin, entered full production — Jensen Huang confirmed it at the company’s GTC Taipei keynote on June 1, with first shipments heading to cloud partners in the second half of the year. Rubin is built around HBM4. Each Rubin GPU carries eight stacks of it, for 288GB of memory and an aggregate bandwidth of up to 22 TB/s. That’s not a spec-sheet flex for its own sake. Bandwidth is the resource these chips are hungry for, and Rubin is basically a machine designed to be fed by HBM4 as fast as physics allows.

On the supply side, the three companies that can actually make this stuff have been racing. SK hynix showed the world’s first 16-high HBM4 at CES 2026 — sixteen DRAM dies stacked into a single package for 48GB per stack, clearing 2 TB/s of bandwidth over a 2,048-bit interface, with mass production targeted for the third quarter. Samsung started shipping samples of the faster follow-on generation, HBM4E, in late May — 12-layer stacks pushing per-pin speeds toward 16 Gbps and about 3.6 TB/s per stack. Micron is the third player, smaller but in the mix. All three are qualified suppliers for Rubin.

Quick caveat before anyone over-reads those numbers: a “16-high” stack is genuinely hard to build. You’re thinning each DRAM die to around 30 micrometers — thinner than a sheet of paper — and stacking sixteen of them without warping the wafer or botching the bonding, all inside a strict height limit. A demo at CES is not the same as high yield at volume. When you see “world’s first,” read it as “we can make one,” not “we can make a million cheaply.” The gap between those two is where this whole story actually lives.

Because here’s what happened the instant HBM became the thing everyone needed. It became the thing nobody can get enough of.

In July, SK hynix CEO Kwak Noh-jung said something that should stop you. He forecast that 2027 will be the worst year in the industry’s history from a supply perspective — worst meaning demand so far ahead of what the factories can produce that customers are already reserving memory years in advance. And he didn’t frame it as a one-year blip. He said demand is expected to outstrip supply beyond 2030. Wafer capacity is projected to grow only around 12% a year, and most of that new capacity is expected to go to HBM — which eats several times the wafer area per bit that standard DRAM does. Samsung has echoed the same warning. So has, in effect, the whole industry.

Read that again. The people who make the memory are the ones telling you there isn’t going to be enough of it for years.

The Takeaway

The phrase worth putting quotes around here is the “memory wall.” It’s an old idea in computing, and it’s back with a vengeance. The short version: for decades, processors got faster much quicker than memory did. The chip could do the math, but it kept stalling out waiting for data to arrive. Compute raced ahead; the pipe feeding it lagged behind. That gap is the wall.

Modern AI slams into that wall harder than almost anything before it. A frontier model is a mountain of weights — hundreds of billions of numbers — and running it means streaming those weights through the processor over and over. The bottleneck usually isn’t whether the GPU can do the arithmetic. It’s whether you can get the numbers to it fast enough. Which means the real currency of AI performance has quietly shifted from raw compute to memory bandwidth. HBM4 is the industry’s answer to the wall — a wider, faster, taller pipe. And it’s a genuinely good answer.

But — and this is the part I keep coming back to — building a better pipe doesn’t help if you can’t build enough pipes. That’s the twist. HBM was supposed to be the solution to the bottleneck. Instead, HBM supply has become the new bottleneck. AI’s growth used to be gated by how many GPUs Nvidia could ship. Now it’s increasingly gated by how much HBM three companies can physically manufacture — and by their own accounts, that’s not going to be enough for years. The scarcity just moved one layer down the stack, from the processor to the memory feeding it.

This connects to a thread we’ve been pulling on for a while, and it’s worth naming. When we wrote about Thinking Machines shrinking a model to a quarter of its size, the point was that a smaller, more efficient model reasoning well matters more than a bigger one that scores slightly higher — because efficiency, not brute scale, is where the leverage is. And when we looked at the notepad trick for giving LLMs real memory, the lesson was that clever use of a small scratchpad beat throwing a giant context window at the problem. Same theme, over and over: the field keeps discovering that the constraint isn’t more compute, it’s using memory smarter. The HBM crunch is that same lesson written in steel and silicon. When the thing you’re short on is memory bandwidth, the models that win are the ones that need less of it.

So what does this actually mean if you’re not buying data-center chips? More than it looks like. The pace and price of AI — how fast the good models get better, how much it costs to run the thing answering your questions — is increasingly set not in Nvidia’s design labs but in a handful of memory fabs in South Korea and the US. If HBM stays scarce through 2027 and beyond, the cost of AI compute stays high, which trickles into everything built on top of it. And there’s a geopolitical edge to it too: the world’s AI ambitions now rest on the output of maybe three companies’ factories, most of that capacity concentrated in one country. That’s a narrow foundation for something the entire tech industry is betting its future on.

Let me be honest about the other side, though, because doom is easy and usually wrong. Shortages are the best thing that ever happens to supply. SK hynix, Samsung, and Micron are pouring capital into new fabs and packaging lines precisely because the demand is there and the margins are extraordinary. Capacity will grow. New approaches to memory and packaging are in the labs. The wall gets pushed back every generation — that’s the whole history of this industry. HBM4 pushes it back meaningfully. The question was never whether the industry adapts; it always does. The question is how long the crunch lasts and how much it costs while it does.

My read: HBM4 is a real advance and a real relief valve, and the supply crunch is a real problem, and both are true at once. The tell isn’t the shiny 288GB spec sheet — it’s the CEO of the biggest HBM maker on earth telling you, plainly, that next year is going to be the tightest in the industry’s history and it won’t fully ease before 2030. When the people selling the shovels warn you there aren’t enough shovels, believe them. The bottleneck in AI has moved. It’s not the chip anymore. It’s the memory — and, more precisely, the small number of factories that can make it.

This article is for informational purposes only.

Photo: Igor Omilaev / Unsplash