NVIDIA adopts full-stack approach to ease AI memory pressure
NVIDIA Blog: More Performance From Every Byte: How Full-Stack Codesign Solves AI’s Memory Challenge
Larger models, longer contexts and agentic applications are increasing GPU memory pressure while supply tightens and prices rise — a trend expected to continue through the near term. NVIDIA works closely with memory partners to rapidly adopt the latest memory technologies. NVIDIA was first to adopt HBM3e and HBM4 technologies into an accelerated computing stack. But the approach isn’t to simply pack more memory onto the GPU. It’s full-stack codesign. NVIDIA intentionally designs GPUs, CPUs, net
Get smart on it
AI models are consuming more memory as they grow larger and handle longer conversations with users, while memory hardware remains expensive and in short supply. Rather than simply adding more memory to individual processors, a full-stack approach designs processors, supporting chips, networking, storage, and software together to make more efficient use of the entire memory system. This strategy involves reducing the size of model data, placing different types of data on appropriate storage tiers from fastest to slowest, and retrieving information back to the fastest tier only when needed, which allows AI systems to deliver significantly more output per dollar spent despite only modest increases in the fastest memory capacity.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
We Visited China’s Stargate in Inner Mongolia
Stargate Abilene has become the symbol of America’s AI buildout. It is a gigawatt-scale campus on the Texas plains built for OpenAI. China has campuses on a similar scale, though few people outside the country have had a look inside. In September, we visited one, and we found the speed, scale, ultra low cost, and deep vertical integration astonishing.During the trip, we toured Inner Mongolia, China’s fastest-growing AI datacenter region. It's the “West Texas” of China, bu
Elon Is Doing It Again: SpaceX Will Be The World’s Biggest AI Cloud
We wrote at length about how SpaceX’s plans for 10GW in 2027 was feasible, paving the way for +300b+ ARR. We received a lot of pushback from folks calling it impossible. Now, just two months later, we’re already seeing 3.6GW under construction across multiple sites, both existing and new. SpaceX has even revised its estimate up to ~13GW in 2027. This would be the largest amount of datacenter capacity ever built by any one company in that timeframe, roughly equivalent to all of Google
Data centers lost the midterms
Data center developers are poised to face a more restrictive building environment after voters across the country voted in candidates who favor moratoriums and ballot measures that put more obstacles on their expansion, delivering a stinging blow to a central pillar of the AI industry. Of voters polled by NBC News on Election Day, 76 percent said they opposed data centers in their communities. Dozens of local communities voted directly to restrict development through ballot mea
Also reported by Wired AI and TechCrunch
Crusoe sells troubled Wyoming data center project to Google
The deal highlights the challenges of developing massive AI data centers.
What Huawei’s Horinger data center reveals about China’s compute gap
Huawei’s AI data center in Horinger, Inner Mongolia, uses Ascend chips and has an estimated AI computing capacity that is just 7% that of the world’s largest AI data center, SpaceX’s Colossus 2 in Tennessee.
Protesters break into Microsoft data center site in Amsterdam
Members of the action group Geef Tegengas put a banner on a crane during a protest. | Image: Getty Images Protesters have broken into a construction site of a new Microsoft data center in Amsterdam this morning. Members of the Geef Tegengas group helped organize the protest against the construction of three 85-meter data center towers in the outskirts of Amsterdam. Reuters reports that around 100 people made it into the construction site, with some scaling a crane to place a "fu