Industry Research · AI Compute Stack Series, Part 2. Part 1 looked at how value migrates down the stack. This one returns to the stack's brightest node—Nvidia—and asks a question everyone cares about but few seriously dissect: why does it earn so much, and how long can that last?
1. First, Correct a Misconception: The Moat Isn't the Chip
When talking about Nvidia's moat, most people's first reaction is "its is the best." That understanding is both right and dangerous.
Right, because Nvidia's chips are indeed ahead. Dangerous because if the moat is just "chip performance lead," it's actually fragile—chip performance can be caught up. AMD is catching up. Broadcom's custom chips for customers are catching up. The performance gap is narrowing. If Nvidia's only advantage were "my chip is a bit faster," its high margins wouldn't last long.
But Nvidia's 75% gross margins and near-monopoly status clearly can't be explained by "a slightly faster chip." Its real moat is a four-layer structure, with the chip being only the outermost, most replicable layer. Dissecting these four layers reveals how deep its moat really is and where the cracks lie.
I'll rank these four layers from shallow to deep.
2. The Four-Layer Moat Structure
Layer 1 (Shallowest): Chip Performance. This is the visible layer—Nvidia's GPU leads in raw compute. But as noted, it's the most vulnerable. It's part of the moat, but not the core.
Layer 2: Full-Stack System Capability. Nvidia stopped selling just chips long ago—it sells the entire rack. With products like the GB200 , it integrates 72 GPUs, CPUs, networking, interconnects, and cooling into a single "turnkey supercomputer" sold as a product. Behind this is its 2019 acquisition of Mellanox giving it high-speed networking, and NVLink, a private high-speed interconnect between GPUs. While competitors are still competing on single chips, Nvidia competes on "system efficiency of the entire cluster"—and system-level integration is far harder to replicate than a single chip.
Layer 3 (Core): CUDA Software Ecosystem. This is Nvidia's deepest moat. CUDA is a software platform Nvidia started building in 2007—for fifteen years, nearly every AI researcher, every deep learning framework, every algorithm library has been built on CUDA. An AI engineer has used CUDA since grad school; all their tools, code, and experience are tied to it. This means even if a competitor's chip were half the price with equivalent performance, the "software migration cost" for a customer to switch would be prohibitively high—rewriting code, retraining teams, bearing compatibility risk. This is classic ecosystem lock-in: the moat isn't in hardware, but in the software habit that makes everyone dependent. This is precisely the highest level I described in the moat piece—from product barrier to network/ecosystem barrier.
Layer 4: Iteration Cadence. Nvidia has compressed product iterations to "one generation per year" (Hopper → Blackwell → Vera Rubin...). This pace leaves competitors gasping—by the time you catch up to one generation, the next is already out. When a leader can still run faster than those chasing it, the chasers are forever chasing, forever missing the leading-edge profits. This state of "leading and accelerating" is the key move that separates Nvidia from its rivals.
Stack these four layers together—the best chip + full-stack system + software ecosystem lock-in + annual iteration cadence—and you understand the source of its 75% gross margin. It doesn't sell a chip; it sells a "compute solution + ecosystem" that no one can replicate wholesale in the short term.
3. But Every Layer Has Real Cracks
A deep moat doesn't mean no cracks. I respect Nvidia, but must honestly point out four real, ongoing threats. Note—these won't cause a "collapse," but they could erode its most precious asset: monopoly-level margins.
Threat 1 (Biggest): Its Biggest Customers Are Becoming Its Rivals.
This is the sword hanging over Nvidia. Its largest customers—Google, Amazon, Microsoft, Meta—are all developing their own AI chips: Google's TPU, Amazon's Trainium, Microsoft's Maia, Meta's MTIA. These hyperscalers are Nvidia's biggest revenue sources, but they don't want to pay Nvidia a toll forever. They have money, data, and engineering talent; their motivation to build custom chips is extreme—even if their own chips only replace some Nvidia usage in specific scenarios, the savings are astronomical. When your top customers are also your most motivated and capable potential gravediggers, that's a structural risk any moat should worry about.
Threat 2: AMD and Software Ecosystem's "Slow Encroachment."
AMD's chips are already competitive in hardware; their weakness has always been software—ROCm vs CUDA, the gap remains large. But the gap is slowly closing, and the entire industry (especially customers wanting to avoid Nvidia lock-in) has strong incentives to support a "second supplier." The software ecosystem moat is deep, but it's something that "time plus industry collaboration" can gradually fill—as long as customers want an alternative badly enough.
Threat 3: The Shift from Training to Inference Weakens CUDA's Lock-In.
This point is critical and most underestimated. Nvidia's moat is strongest in training (where full-stack performance, ecosystem, and iteration matter most). But as AI goes mainstream, workloads are shifting significantly from training to inference (running already-trained models). Inference scenarios are more cost-sensitive, less dependent on the CUDA ecosystem, and easier to replace with cheaper custom chips. As the center of gravity of compute demand moves from training to inference, Nvidia's "full-stack + ecosystem" premium gets diluted—because inference customers want "good enough and cheap," not "the best and most expensive." The bottleneck is moving; Nvidia's strongest position (training) may be declining as a share of total demand.
Threat 4: Customer Concentration.
Nvidia's revenue is heavily concentrated among a handful of mega-customers. This concentration is sweet in boom times (big orders roll in) but bitter in downturns (if a few big customers slow capex or pivot to custom silicon, the impact is concentrated and sharp). High customer concentration magnifies its exposure to the "AI capex cycle"—which brings us back to the master switch for the entire series.
4. My Take: It's Not About Whether It Collapses, but Whether Margins Hold
Putting the moat and cracks together, my judgment is a layered conclusion, not a simple "bullish" or "bearish."
First, the probability of Nvidia being "disrupted" (losing leadership) is low. The CUDA ecosystem lock-in plus full-stack capability plus annual iteration cadence—these three combined cannot be replicated wholesale in three to five years. Saying Nvidia will decline as quickly as Intel did underestimates the depth of its moat. It will likely remain the long-term leader in AI compute.
Second, however, the probability of its "monopoly-level margins" being eroded is high. What will actually change is not "is Nvidia still there?" but "can Nvidia maintain 75% gross margin?" When customer custom chips divert some demand, when AMD offers a credible second option, when demand shifts from training to cheaper inference—none of this kills Nvidia, but it will gradually weaken its pricing power. An Nvidia that still leads but sees margins slowly fall from 75% to 60% to 50% is still a great company, but for a stock that has priced in "perpetual monopoly profits," this gradual decline alone could be a poor outcome.
Third, therefore, the real risk lies in "valuation" rather than "fundamentals." This is my core view on Nvidia: its fundamentals are likely fine (it will remain a giant), but its stock price embeds a perfect assumption—that the moat will never be eroded, margins will never compress, and AI capex will grow forever. The four cracks I've outlined suggest this perfect assumption will likely be discounted somewhere. The biggest risk in owning Nvidia isn't that it becomes worse; it's that it's "less perfect than the market expects"—and when a stock prices in perfection, "less than perfect" is enough for a severe correction.
This is entirely the discipline I've internalized from reading This Time Is Different and The Most Important Thing—the most dangerous thing isn't bad companies; it's extrapolating a good company's goodness infinitely and paying too much for that extrapolation.
5. Closing Thoughts
Nvidia is one of the greatest companies of this era, and I have no doubt about that. Its moat—CUDA ecosystem, full-stack systems, iteration cadence—is real, deep, and among the strongest in business history.
But "great company" and "great investment" have never been the same thing (a point I've made repeatedly). A company with a deep moat, if its price already assumes the moat will never be breached, can still be a mediocre or even poor investment. Will Nvidia's moat hold? It will likely hold its position, but probably not its current monopoly-level margins—because its largest customers are building their own chips, because alternatives are closing in, and because demand is shifting toward cheaper inference.
So when looking at Nvidia, I'm not watching for "will it be disrupted?" (probably not). Instead, I'm focused on three more granular signals: how much demand do custom chips divert? how high does inference share rise? and when does its gross margin inflection point appear? These three are the real variables determining how long its "monopoly profits" can last.
If I'm left with one sentence—
Nvidia's moat is deep enough that it won't be disrupted; but not deep enough that it can maintain 75% gross margins forever. Betting against its leadership is wrong, but extrapolating its perfection infinitely—and paying any price for it—is equally wrong.
Next time, we look at an interesting new force in this chain—the Neocloud: NBIS, CoreWeave, and others—companies that specialize in renting GPUs to others. Are they the shovel sellers of AI compute democratization, or another leverage-driven bubble?
—
Risk disclaimer: This is an industry chain analysis. The companies mentioned are examples for research only and do not constitute investment advice. Markets are risky; invest with caution.
On investment analysis, market insight, and asset allocation — not chasing short-term noise, but understanding what truly drives long-term returns.
