Page 16 - EE|Times Europe Magazine - December 2020

16 EE|Times EUROPE — The Memory Market

SPECIAL REPORT: MEMORY TECHNOLOGY
Memory Technologies Confront Edge AI’s

Diverse Challenges

By Sally Ward-Foxton

ith the rise of artificial intelligence at the edge comes a
whole host of new requirements for memory systems.
Can today’s memory technologies live up to the stringent
Wdemands of this challenging new application, and what do
emerging memory technologies promise for edge AI in the long term?
The first thing to realize is that there is no standard “edge AI” appli-
cation; the edge in its broadest interpretation covers all AI-enabled
electronic systems outside the cloud. That might include “near edge,” An Intel Optane 200 Series module. Intel says Optane is already
which generally covers enterprise data centers and on-premise servers. being used to power AI applications. (Source: Intel)
Further out are applications like computer vision for autonomous
driving. Gateway equipment for manufacturing performs AI inference
to check for flaws in products on the production line. 5G “edge boxes” LRDIMM, and highly available persistent memory like NVDIMM.”
on utility poles analyze video streams for smart-city applications such Gupta sees Intel Optane, the company’s 3D-Xpoint nonvolatile mem-
as traffic management. And 5G infrastructure uses AI at the edge for ory whose properties are between DRAM and flash, as a good solution
complex but efficient beamforming algorithms. for server AI applications.
At the “far edge,” AI is supported in devices such as mobile phones “Both Optane DIMMs and NVDIMMs are being used as AI acceler-
— think Snapchat filters — voice control of appliances, and IoT sensor ators,” he said. “NVDIMMs provide very low-latency tiering, caching,
nodes in factories performing sensor fusion before sending the results write buffering, and metadata storage capabilities for AI application
to another gateway device. acceleration. Optane data center DIMMs are used for in-memory
The role of memory in edge AI systems — to store neural network database acceleration, where
weights, model code, input data, and intermediate activations — is hundreds of gigabytes to terabytes
the same for most AI applications. Workloads must be accelerated of persistent memory are used in
to maximize AI computing capacity in order to remain efficient, so combination with DRAM. Although
demands on capacity and bandwidth are generally high. However, these are both persistent memory
application-specific demands are many and varied and may include solutions for AI/ML acceleration
size, power consumption, low-voltage operation, reliability, thermal/ applications, they have different
cooling considerations, and cost. and separate use cases.”
Kristie Mann, Intel’s director
EDGE DATA CENTERS of product marketing for Optane,
Edge data centers are a key edge market. The use cases include medical told EE Times that Optane is
imaging, research, and complex financial algorithms, in which privacy gaining applications in the server
prevents uploading to the cloud. Another is self-driving vehicles, in AI segment.
which latency prevents it. “Our customers are already using Intel’s Kristie Mann
These systems use the same memories found in servers in other Optane persistent memory to power
applications. their AI applications today,” she said. “They are powering e-commerce,
“It is important to use low-latency DRAM for fast, byte-level main video recommendation engines, and real-time financial analysis usages
memory in applications where AI algorithms are being developed successfully. We are seeing a shift to in-memory applications because
and trained,” said Pekon Gupta, of the increased capacity available.”
solutions architect at Smart Mod- DRAM’s high prices make Optane an attractive alternative. A server
ular Technologies, a designer and with two Intel Xeon Scalable processors plus Optane persistent mem-
developer of memory products. ory can hold up to 6 TB of memory for data-hungry applications.
“High-capacity RDIMMs [registered “DRAM is still the most popular, but it has its limitations from a
dual-in-line memory modules] or cost and capacity perspective,” said Mann. “New memory and stor-
LRDIMMs [load-reduced DIMMs] are age technologies like Optane persistent memory and Optane SSD are
needed for large datasets. NVDIMMs [emerging] as an alternative to DRAM due to their cost, capacity, and
[nonvolatile DIMMs] are needed for performance advantage. Optane SSDs are particularly powerful, caching
system acceleration — we use them HDD and NAND SSD data to continuously feed AI applications data.”
for write caching and checkpointing Optane also compares favorably with other emerging memories that
instead of slower SSDs.” are not fully mature or scalable today, she added.
Locating computing nodes close
Smart Modular to end users is the approach taken GPU ACCELERATION
Technologies’ Pekon Gupta by telecommunications carriers. For high-end edge data center and edge server applications, AI com-
“We’re seeing a trend to make pute accelerators such as GPUs are gaining traction. As well as DRAM,
these [telco] edge servers more capable of running complex algo- the memory choices here include graphics double data rate (GDDR),
rithms,” Gupta said. Hence, “service providers are adding more memory a special DDR SDRAM designed to feed high-bandwidth GPUs, and
and processing power to these edge servers using devices like RDIMM, high-bandwidth memory (HBM), a relatively new die-stacking

DECEMBER 2020 | www.eetimes.eu