In a recent two-hour debate, NVIDIA CEO Jensen Huang fielded questions about competitors, geopolitical pressures, and the future of AI hardware. His responses revealed a company that, despite facing headwinds from export controls and rising rivals like Google’s TPU and Huawei’s Ascend chips, remains supremely confident in its market position. But this confidence isn’t mere bravado—it’s grounded in a deep understanding of the AI ecosystem’s structural advantages.
The core of Huang’s argument rests on NVIDIA’s CUDA ecosystem, a software platform that has become the de facto standard for AI development. Since its launch in 2006, CUDA has accumulated over 4 million developers and supports nearly every major AI framework, from TensorFlow to PyTorch. This creates a massive switching cost for any company considering an alternative. When Google introduced its TPU in 2015, it required developers to use TensorFlow exclusively—a limitation that slowed adoption. By contrast, NVIDIA’s hardware works seamlessly across frameworks, making it the path of least resistance for AI researchers.
Huang’s calm regarding export controls, particularly those targeting China, stems from a similar logic of dependency. While Huawei’s Ascend 910B chip, announced in 2019, has shown competitive performance in benchmark tests—achieving 256 TFLOPS in FP16 calculations—it lacks the software maturity of CUDA. Chinese companies like Baidu and Alibaba have invested heavily in NVIDIA’s ecosystem, training AI models on A100 and H100 GPUs. Switching to Huawei’s hardware would require re-optimizing entire workflows, a process that could take months and cost millions of dollars. As Huang noted, “The moat isn’t just hardware—it’s the years of software investment that competitors can’t replicate overnight.”
The real battle isn’t chip against chip, but ecosystem against ecosystem.
To understand NVIDIA’s advantage, consider the case of Graphcore, a UK-based AI chip startup that raised over $700 million but filed for bankruptcy in 2023. Despite designing a chip that outperformed NVIDIA’s V100 in certain workloads, Graphcore failed to build a comparable software stack. Developers found its IPU (Intelligence Processing Unit) difficult to program, and adoption stagnated. This pattern repeats with other would-be challengers: Cerebras, SambaNova, and Groq have all struggled to gain traction beyond niche applications. NVIDIA’s CUDA, combined with its cuDNN library for deep neural networks, has become the lingua franca of AI computing.
However, the rise of large language models (LLMs) presents a potential disruption. OpenAI’s GPT-4, trained on 25,000 NVIDIA A100 GPUs, required approximately $100 million in compute costs. As models grow larger—Meta’s LLaMA 3 used 16,000 H100 GPUs—the demand for specialized hardware intensifies. Google’s TPU v5p, released in 2023, offers 95% of the performance of NVIDIA’s H100 at 80% of the cost per chip, according to internal benchmarks. Yet Huang remains unfazed, pointing out that TPU’s cost advantage diminishes when factoring in the software overhead. “A cheaper chip that takes twice as long to debug isn’t cheaper at all,” he argued.
In AI, time-to-market often outweighs hardware cost-per-chip.
Export controls add a geopolitical dimension. The US government’s October 2022 rules restricted sales of A100 and H100 GPUs to China, creating an opening for Huawei. Huawei’s Ascend 910B, built on 7nm process technology despite US sanctions, has been adopted by Chinese cloud providers like Tencent and JD.com for internal AI workloads. Yet these deployments remain limited in scale. A 2023 report from the Semiconductor Industry Association estimated that Chinese companies still rely on NVIDIA for 80% of their AI compute needs, with domestic alternatives accounting for less than 15%. The remaining gap is filled by older NVIDIA chips, like the V100, or through gray-market imports.
Huang’s strategy for countering Huawei involves accelerating NVIDIA’s product cycle. The company now releases a new GPU architecture every two years, down from three previously. The Blackwell architecture, announced in March 2024, promises a 30x performance improvement over Hopper for LLM training. This relentless pace means that even if Huawei matches NVIDIA’s current generation, NVIDIA will already be two steps ahead. As Huang put it, “We don’t run from competition—we run faster.”
Innovation velocity is the ultimate defense against commoditization.
The debate also touched on the threat from custom chips. Companies like Amazon (Trainium) and Microsoft (Maia) are designing their own AI accelerators, raising questions about whether NVIDIA will be displaced in the long term. Yet these custom chips face a chicken-and-egg problem: they only run on their respective cloud platforms, limiting portability. A startup training a model on Amazon’s Trainium can’t easily switch to Microsoft Azure without rewriting code. NVIDIA’s hardware, by contrast, runs across AWS, Azure, Google Cloud, and on-premise data centers, offering flexibility that custom chips cannot match.
Huang’s confidence, however, is not without risks. The US government’s tightening of export controls in November 2023, which included a ban on NVIDIA’s H800 chip for China, could push Chinese companies to invest more aggressively in domestic alternatives. Huawei’s recent partnership with SMIC to produce the Kirin 9000s chip for smartphones suggests that Chinese innovation is accelerating. If Huawei can replicate this momentum in AI chips, the gap might narrow faster than NVIDIA anticipates.
Complacency is the only competitor that NVIDIA cannot afford.
For now, NVIDIA’s dominance appears secure. The company’s data center revenue reached $47.5 billion in fiscal 2024, up 217% year-over-year, driven by AI demand. Its gross margins of 72% dwarf competitors like AMD (46%) and Intel (40%). Huang’s message is clear: while rivals may nibble at the edges, the core of the AI computing stack—CUDA, cuDNN, TensorRT—remains under NVIDIA’s control. As one analyst noted, “Jensen isn’t afraid because he knows that in AI, the software defines the hardware, not the other way around.”
The two-hour debate ended with a question about the future. Huang paused, then offered a prediction: “In five years, every major company will have an AI infrastructure team. And 80% of them will still be using NVIDIA chips—not because we’re the only option, but because we’re the easiest one to trust.” It’s a bold claim, but one backed by a decade of ecosystem investment that no competitor has yet matched.