Page 39 - EE Times Europe Magazine – June 2024

EE|Times EUROPE 39

Nvidia GTC 2024: Why Nvidia Dominates AI

As AI became the driving force behind GPU improvements, the tran- given the company a dominant share of the GPU market for over a
sistor count rose to more than 28 billion for the Ampere GPU in 2020 decade. Nvidia retained its leadership during the graphics card era
and to 80 billion for the Hopper GPU in 2022. More parallel and spe- and through the video processing era as well. Now, the CUDA software
cialized computing functions are key to improving performance for AI base and parallel processing capabilities are the leaders for AI model
software. The Hopper GPU microarchitecture had especially extensive development and AI application deployments. This AI software base
improvements for accelerating AI computing. Hopper was the first GPU advantage will persist for some time, even with increased competition
with a transformer engine for acceleration of AI training models. in the future.
In March 2024, Nvidia announced Blackwell as the latest GPU The question is whether Nvidia can extend its software base advan-
generation, with 208 billion transistors. Product shipment is due tages to new AI applications and other new application segments as
later this year. The 2024 Blackwell GPU chip will more than double they emerge. Nvidia has embraced and transitioned to new CUDA
the Hopper performance and transistor count. However, much of the opportunities before and is likely to do so again. Another question
transistor increase is due to the use of two chip dies connected by a is whether the Unified Acceleration (UXL) Foundation will have an
10-TB/s chip-to-chip interconnect in a unified single GPU. Each die impact. The UXL Foundation was launched in September 2023 to
has 104 billion transistors—a 30% increase over Hopper’s 80 billion develop a CUDA alternative and has an impressive membership roster,
transistors. The Blackwell architecture can have up to 576 GPU including Arm, Fujitsu, Google Cloud, Intel, Qualcomm, Samsung and
processors on a chip. Blackwell includes a second-generation trans- others.
former engine. The formation of UXL looks like an acknowledgement that CUDA is
TSMC has manufactured
The AI GPU trend the vast majority of Nvid- a clear leader and that no single company can compete with CUDA’s
software base and momentum. Only a foundation with major company
accelerated Nvidia’s ia’s chips, though IBM and participation can or may build a CUDA competitor.
market value and Samsung have manufactured HOW NVIDIA BECAME THE LEADER IN AI
some chips for Nvidia in the
revenue, and it is now past 20 years. Samsung was We have already discussed two reasons for Nvidia’s rise to AI domi-
a fab partner from 2016 to
nance: GPU technology advances for AI calculations and the CUDA
one of the three most 2022. SGS-Thomson software platform for writing code for parallel execution of many
valuable companies Microelectronics was an early streams of information. The technology advances of GPUs have been
very rapid—from 1 million transistors on a GPU in 1995 to 208 billion
Nvidia fab partner.
in the world based on CUDA PLATFORM transistors in 2024. This functionality and performance growth was key
to enabling the rapid expansion and complexity of AI models. There
market capitalization. GPUs were originally designed are at least two more factors: building momentum in graphics- and
for image manipulation video-related GPU applications, and understanding and reacting to the
and the calculation of local potential of AI as a major future GPU market.
image properties. The mathematical foundations of neural networks On all counts, Nvidia’s management gets credit for crafting and
and image manipulation are similar, which created a big opportunity implementing a successful strategy.
for GPUs as AI chips. In the mid-2010s, GPUs evolved to enable deep Nvidia made a foresighted investment in the late 2000s to set up
learning for training and inference in many applications, including CUDA programming classes at most of the major universities
autonomous vehicles. worldwide—at Nvidia’s cost. The bold gambit paid off; it was part of the
Nvidia defined a GPU programming interface called CUDA (which creation of a market concept and enabled the later growth of the CUDA
stands for Compute Unified Device Architecture, though Nvidia uses software base.
the acronym only) and released the first version in mid-2007. CUDA Nvidia’s early realization that AI had great potential was critical.
version 12.4 was released in March 2024. CUDA is a parallel computing The company first focused on AVs as an AI opportunity and included
platform and application programming interface that allows software GPU functionality for speeding up AI calculation. The AV market was
to use GPUs for accelerated general-purpose processing. Nvidia has delayed, but generative AI came along to take AVs’ place—and Nvidia
used “accelerated computing” to define its product strategy for about a was ready to ride this market growth. The rest is a great history for
decade. Nvidia.
The CUDA API is an extension of the C programming language that Another key factor is that Nvidia became as much a software
adds the ability to specify software-level parallelism in C and to spec- company as a chip design company. Today, Nvidia is more of a soft-
ify GPU-device–specific operations. CUDA is also a software layer that ware company with highly skilled AI GPU chip designers. Nvidia has
gives direct access to the GPU’s virtual instruction set and parallel developed a large AI software base, including software development
computational elements. CUDA is designed to work with program- tools, AI libraries and AI foundational models for a large spectrum of
ming languages like C, C++, Fortran and Python. This accessibility AI applications.
makes it easier to do parallel programming via the many processors Through its AI software development activity, Nvidia gained an
and accelerators provided in GPU chips. Many corporations program understanding of what GPU hardware architecture and processing
their AI applications in CUDA to get maximum performance from accelerators were needed for AI performance, especially for the AI
Nvidia GPU systems. This has built a broad CUDA software base and training phase.
a large population of programmers with expertise in developing soft- This knowledge base was reinforced and amplified by building AI
ware for Nvidia’s GPUs. systems for cloud system operators and similar customers. All of this
Intel’s continued leadership in processor chips for the PC market has given Nvidia the expertise to rapidly improve its AI-focused GPUs
is attributable to the software base that was built around IBM’s PC and the systems for AI training and inferencing. In the last two GPU
standard in the 1980s and 1990s. Intel has leveraged that software architecture upgrades, the focus was on improving AI performance for
to maintain dominance in PC chips for more than 40 years, but it did training and inferencing. AI application improvements will be the focus
not transfer this software advantage to later computer segments like of future GPU chip architectures.
smartphones and tablets. Apple is known for gaining major advantages from designing both
Nvidia has a similar software base, built around CUDA, that has its hardware and software into a better system experience for its users.

www.eetimes.eu | JUNE 2024