Page 44 - EE Times Europe Magazine | April2019

44 EE|Times EUROPE — Boards & Solutions Insert

SPECIAL REPORT: AI AT THE EDGE
Seeing the AI Inference Market with

2020 Vision: Five Predictions

By Geoff Tate
A to get more throughput for the same power
new class of artificial-intelligence chips is coming to market in 2020,
and price as the solutions they are using today.
and the entries will be optimized for inference — not for graphics,
The only way to get higher prediction accuracy
training, or digital signal processing. In fact, according to Nvidia in
will be to get higher throughput per dollar and
its recent quarterly earnings call, the AI inference market exceeded AI
training spending in the data center in 2019. higher throughput per watt.
4. SELECTING THE RIGHT SOLUTION FOR
What is surprising here is that very few startups introduced new products this AUTOMOTIVE
past year, considering the number of startups funded and the number of compa- The automotive market for inference will be
in the millions of dollars, but deployment
nies that have been in business for five or more years. Flex Logix expects that to
takes time. Car companies and their suppli-
change this year. Below are our top five predictions based on what we see in the ers have already selected solutions for the
market and the many conversations we are having with customers. 2024–2025 model years and will evaluate
alternatives in 2020 for the 2026–2027 model
year. A year ago, all of the car companies/
1. THROUGHPUT OVER MEANINGLESS ily correlate with higher throughput; what suppliers planned their own chips, but almost
BENCHMARKS customers really want is high throughput per all have dropped those plans and will use
A year ago, customers were asking about dollar. As an example, if you compare the Flex merchant market solutions instead.
tera-operations per second (TOPS) and Logix InferX X1 device with a market-leading
ResNet-50 at various batch sizes. Today, graphics processor, the GPU may offer 3× to 4× 5. ADVANTAGE FOR SOLUTIONS WITH
leading customers have developed models the throughput, with 10× the TOPS, but it also BF16
that work for their applications, and how a uses 8× the number of DRAMs. The InferX X1 While 8-bit integer (INT8) offers the highest
solution runs their model is what’s import- architecture is a lot more resource-efficient. throughput per dollar and throughput per
ant to them. The thing that matters most is watt, winning solutions will need to have
throughput on megapixel images (not mean- 3. PREDICTION ACCURACY REQUIRES a BFloat16 (BF16) option because it allows
ingless benchmarks), and more companies are HIGHER THROUGHPUT PER DOLLAR AND customers to quickly ramp production. For
going to figure this out in 2020. PER WATT many customers, the cost and complexity of
Applications are being starved for inference quantizing may never be economical.
2. INFERENCE THROUGHPUT AT LOWEST throughput and have been getting by with It’s going to be an exciting year in the AI/
COST down-sampling their native megapixel images inference market. There is no doubt that the
In the server market, some customers will and processing only a fraction of the frames availability of high-throughput inference
want to get more inference per PCIe slot, but per second. In the market for systems that capabilities will change the way people live,
the path to expanding the market will be to need 5- to 30-W chips/modules with heat sinks work, and play. ■
deliver inference throughput at lower price and no fans (now primarily served by Nvidia
points. Having more TOPS doesn’t necessar- Xavier AGX and now NX), customers will want Geoff Tate is the CEO of Flex Logix.

IMAGE: SHUTTERSTOCK

APRIL 2020 | www.eetimes.eu