Page 48 - EE Times Europe Magazine | April2019

48 EE|Times EUROPE — Boards & Solutions Insert

Putting AI into the Edge Is a No-Brainer; Here’s Why

use about 95% less power than the whole SoC does. What if someone storage and analysis is costly and complex. Putting machine-learning
built a chip that had only the edge AI portion (along with a few other processors on the endpoints, whether sensors or cameras, can solve
required functions, such as memory) and that cost less, used less elec- this problem. Cameras, for example, could be equipped with vision-
tricity, and was smaller? processing units (VPUs) — low-power SoC processors specialized
Well, they have. In all, as many as 50 different companies are said to for analyzing or pre-processing digital images. With edge AI chips
be working on AI accelerators of various kinds. The standalone edge embedded, a device can analyze data in real time, transmit only what is
AI chips available in 2019 were targeted at developers, who would buy relevant for further analysis in the cloud, and “forget” the rest, reduc-
them one at a time for about US$80 each. In volumes of thousands or ing the cost of storage and bandwidth.
millions, these chips will likely cost device manufacturers much less
to buy: some as little as US$1 (or possibly even less), some in the tens Power constraints
of dollars. We are, for now, assuming an average cost of about US$3.50, Low-power machine-learning chips can allow even devices with small
using the smartphone edge AI chip as a proxy. batteries to perform AI computations without undue power drain. For
Besides being relatively inexpensive, standalone edge AI processors instance, Arm chips are being embedded in respiratory inhalers to
have the advantage of being small. They are also relatively low-power, analyze data, such as inhalation lung capacity and the flow of medicine
drawing between 1 and 10 W. For comparison, a data-center cluster into the lungs. The AI analysis is performed on the inhaler, and the
(albeit a very powerful one) of 16 GPUs and two CPUs costs US$400,000, results are then sent to a smartphone app, helping health-care profes-
weighs 350 pounds, and consumes 10,000 W. sionals to develop personalized care for asthma patients. In addition
With chips such as these in the works, edge AI can open many new to the low-power edge AI NPUs currently available, companies are
possibilities for enterprises, particularly with regard to IoT applica- working to develop “tiny machine learning”: deep learning on devices
tions. Using edge AI chips, companies can greatly increase their ability as small as microcontroller units. Google, for instance, is developing a
to analyze — not just collect — data from connected devices and con- version of TensorFlow Lite that can enable microcontrollers to analyze
vert the analysis into action while avoiding the cost, complexity, and data, condensing what needs to be sent off-chip into a few bytes.
security challenges of sending huge amounts of data into the cloud.
Issues that AI chips can help address include the following: Low-latency requirements
Whether over a wired or wireless network, performing AI computations
Data security and privacy at a remote data center means a round-trip latency of at least 1 to 2 ms
Collecting, storing, and moving data to the cloud inevitably exposes an in the best case and tens or even hundreds of milliseconds in the worst
organization to cybersecurity and privacy threats, even when compa- case. Performing AI on-device using an edge AI chip would reduce that
nies are vigilant about data protection. This immensely important risk to nanoseconds — critical for applications in which the device must col-
is becoming even more critical to address as time goes on. Regulations lect, process, and act upon data virtually instantaneously. Autonomous
about personally identifiable information are emerging across juris- vehicles, for instance, must collect and process huge amounts of data
dictions, and consumers are becoming more cognizant of the data that from computer-vision systems to identify objects, as well as from the
enterprises collect, with 80% of them saying that they don’t feel that sensors that control the vehicle’s functions. They must then convert
companies are doing all they can to protect consumer privacy. Some this data into decisions immediately — when to turn, brake, or acceler-
devices, such as smart speakers, are starting to be used in settings such ate — in order to operate safely. To do this, autonomous vehicles must
as hospitals, where patient privacy is regulated even more stringently. process much of the data they collect in the vehicle itself. Low latency
By allowing large amounts of data to be processed locally, edge AI is also important for robots, and it will become more so as robots
chips can reduce the likelihood that personal or enterprise data will emerge from factory settings to work alongside people.
be intercepted or misused. Security cameras with machine-learning
processing, for instance, can reduce privacy risks by analyzing the video THE BOTTOM LINE: EDGE AI WILL BE VITAL
to determine which segments of the video are relevant and sending FOR DATA-HEAVY APPS
only those to the cloud. Machine-learning chips can also recognize The spread of edge AI chips will likely drive significant changes for
a broader range of voice commands so that less audio needs to be consumers and enterprises alike. For consumers, edge AI chips can
analyzed in the cloud. More accurate speech recognition can deliver make possible a plethora of features — from unlocking their phone to
the additional bonus of helping smart speakers detect the “wake having a conversation with its voice assistant or taking mind-blowing
word” more accurately, thus preventing it from listening to unrelated photos under extremely difficult conditions — and without the need for
conversation. an internet connection.
But in the long term, edge AI chips’ greater impact may come from
Low connectivity their use in the enterprise, where they can enable companies to take
A device must be connected for data to be processed in the cloud. In their IoT applications to a whole new level. Smart machines powered
some cases, however, connecting the device is impractical. Drones are by AI chips could help expand existing markets, threaten incumbents,
an example. Maintaining connectivity with a drone can be difficult and shift how profits are divided in industries such as manufacturing,
depending on where they operate, and both the connection itself and construction, logistics, agriculture, and energy. The ability to collect,
uploading data to the cloud can reduce battery life. In New South interpret, and immediately act on vast amounts of data is critical for
Wales, Australia, drones with embedded machine learning patrol many of the data-heavy applications that futurists predict will become
beaches to keep swimmers safe. They can identify swimmers who have widespread: video monitoring, virtual reality, autonomous drones and
been taken by riptides or warn swimmers of sharks and crocodiles vehicles, and more.
before an attack, all without an internet connection. That future, in large part, depends on what edge AI chips make pos-
sible: bringing the intelligence to the device. ■
(Too) big data
IoT devices can generate huge amounts of data. For example, an Airbus Duncan Stewart and Jeff Loucks are with Deloitte’s Center for
A-350 jet has more than 6,000 sensors and generates 2.5 terabytes of Technology, Media and Telecommunications. This article is based on
data each day it flies. Globally, security cameras create about an article originally published by Deloitte for its “TMT Predictions 2020”
2,500 petabytes of data per day. Sending all this data to the cloud for report.

APRIL 2020 | www.eetimes.eu