Page 18 - EE Times Europe Magazine – November 2023

18 EE|Times EUROPE

GREENER ELECTRONICS | PROCESSING
Could IBM’s AI Chip Reinvent Deep Learning

Inference?

By Rebecca Pool
A novel analog in-memory chip from IBM demonstrates relocated into the memory, eliminating the

that both performance and energy efficiency are possible separation between memory and processor
and thereby raising energy efficiency far
for AI operations. beyond that of digital chips.
IBM has embraced this approach in its
n August, a 30-strong team of IBM Research scientists revealed a radically latest analog AI chip. The chip is designed
and fabricated in 14-nm CMOS and comprises
new chip, designed to drastically raise the energy efficiency of power-hungry 64 compute cores, each with its own PCM
deep learning inference while maintaining operation precision. The analog array as well as a light digital processor for
Iin-memory chip, dubbed Hermes, merges digital circuits with phase-change additional neural network operations, such as
activation and accumulation. On-chip
memory (PCM) so neural network calculations can take place within the mem- communications network the cores together,
ory circuits. and a further digital processing unit at the
center of the chip can tackle more demanding
This processor-memory pairing removes example, Nvidia has gotten a lot of mileage neural network calculations.
the need to endlessly shuffle vast swaths of out of its GPU technology; Google developed Though its use of PCM technology sets it
data between memory and processing units, its Tensor Processing Unit; Intel offers myriad apart, IBM is not alone in its analog AI chip
as takes place in the typical AI accelerator solutions, including FPGAs, that can be pro- endeavors. For example, Intel Lab researchers
chip. And critically, analyses indicate the grammed for AI calculations; and Qualcomm have been working with static random-access
chip is as adept as its digital counterparts at has designed AI accelerator chips for mobile memory and other technologies; U.S. startup
computer-vision AI tasks while consuming a devices. Mythic has focused on multi-level flash mem-
lot less power. Still, the power-hungry demands of AI, with ory; and researchers from Stanford University,
In their recent blog, Abu Sebastian, leader its ever-increasing number of operations, call University of Notre Dame, University of
of the IBM Research Europe team that pio- for a more energy-efficient approach, and California San Diego and Tsinghua
neered the chip, and colleagues Manuel Le that is where analog in-memory computing University developed a neuromorphic chip,
Gallo-Bourdeau and Vijay Narayanan were appears set to play a key role. dubbed NeuRRAM, that uses resistive
clear that they have now demonstrated many Analog in-memory chips can circumvent random-access memory.
of the building blocks needed to realize an the von Neumann bottleneck by performing PCM was a logical choice for IBM, as the
architectural vision for a fast, low-power ana- computations directly inside the memory. company has spent at least a decade develop-
log AI chip. And as an IBM scientist from the Analog computing schemes need only a ing these materials for storage applications,
team told EE Times Europe, “We’re still at the few resistors or capacitors, so they can be including a multi-level PCM chip. PCM itself
research stage with the chip, but we’re now
going to build as much hardware as we can
to understand exactly how it functions and
to see the best ways to move forward.” (The
scientist spoke with us on the record but asked
not to be identified.—Ed.)

POWER-HUNGRY AI
The AI power problem is rooted in a phenom-
enon known as the von Neumann bottleneck,
named after John von Neumann, who
pioneered the architecture behind modern
computers. Here, the CPU and memory are
separate entities, so data and calculations
must continually shuttle back and forth
between the two. This process creates a
bottleneck, as processing speeds are typically
faster than the rate at which data can be
retrieved or written to memory.
Designers have long battled the
bottleneck using workarounds like memory
caching, pipelining and hierarchical memory
schemes. More recently, chip giants have
created accelerator chips specifically to take
on the burgeoning task of AI processing. For A rendering of IBM’s analog AI chip (Source: IBM Research)

NOVEMBER 2023 | www.eetimes.eu