Page 29 - 23_EETE_03

EE|Times EUROPE 29

Can We Trust AI in Safety-Critical Systems?

standard that accepts proof by example,”
he said. “There’s a section that says you can
prove not by design but by testing—or by
real-life usage—so if you can show in real life
that there is a certain level of non-failure, you
can say that this works. … The only problem
is that this is not transferrable, so for the next
system, you would have to prove it again.”
In practice, the amount of test data
required without the ability to bring certified
subsystems to new designs may mean this
route to certification is not economical, he
added.

SAFETY CONCEPTS
General safety concepts like redundancy are
not mutually exclusive with AI, according to
Ryan Zhao, general manager for motor drives
and robotics at TI. “We can do some redun-
dancy in the design,” he said. “We can use
multiple chips, multiple cores, for redundancy Renesas/Reality AI’s tool offers automatic feature extraction and can provide some
not only at the silicon level but also at the explainability. (Source: EE Times)
software level.”
EXPLAINABLE AI Nalin Balan, head of sales at Reality AI,
Mohammed Umar Dogar, vice president of gave the example of an unbalanced load in
the IoT and infrastructure business unit at a dryer drum and the conditions this creates
Renesas, said the overall impact of AI on in the motor. Reality’s tool shows which
safety-critical systems is positive, particularly feature—in this case, a frequency feature—
on the factory floor. correlates to the prediction of an unbalanced
“Real-time analytics is where I see a lot of load. Designers can then use their physics
growth, and that’s why we’re investing into knowledge to understand why an unbalanced
this area very heavily,” he said. “But one of load might correspond to that frequency.
the big problems with AI in general is the In an automotive example, this tech might
explainability. … The model is a black box. A be used to monitor motors in braking or
lot of the companies can develop the model steering systems to look for fault conditions,
itself, but if I’m an OEM, I need to know what Balan said. But the tool also applies to audio
Texas Instruments’ Ryan Zhao it’s doing.” processing applications like See With Sound,
a proprietary AI that can detect pedestrians
or cyclists near a car from the sounds they
TI’s TDA4, part of the Jacinto processor make. In this case, a variety of features can be
series, can be set up with a safety island: used—from bicycle tire sounds to footsteps—
Isolated cores on chip can monitor or cross- but Reality’s tool can tell you exactly what the
check each other without having to execute AI is listening for.
full lock-step operation. The safety island “We can tell the R&D team what features
uses a separate clock and memory. we’re picking up in the targets that allow us
The TDA4 has dual ARM Cortex-R5F cores, to detect them,” Balan said. “For a vehicle, it
plus a C7X DSP and an in-house–developed might be a combination of features—
8-TOPS matrix-multiplication accelerator. perhaps engine noise and tire noise. But we
Partitioning can be done not only between can reliably show you what features we have
cores but also at the virtual level on the same extracted in the data that correlate to that
core. prediction.”
“You might also have the redundancy While this level of explainability may not
more at the sensor level, rather than at the AI be sufficient for certification in a safety-
model level,” said Matthias Thoma, robotics Texas Instruments’ Matthias Thoma critical application, it may have an indirect
systems manager at TI. Thoma’s robotics effect, giving designers confidence that they
example was a warehouse robot with radar Renesas gained significant AI capabilities have some insight into how the AI arrives at
and camera, with camera data used to grab with the acquisition of Reality AI last year. its answer. That helps crack open the black
a package, or both camera and radar used to Reality AI’s tool can help provide explain- box, if just a tiny bit. ■
detect a person walking into the robot’s safety ability for AI models. The tool performs
zone. Today’s industrial robots, however, usu- automated feature extraction from sensor Sally Ward-Foxton is a senior reporter at EE
ally have non-AI technologies, such as light data, then shows the designer what those Times. This article ran as part of the EE Times
grids, to detect a person entering the safety features are and how they correlate to the Special Report: Embedded in the IoT Era,
zone, he said. prediction. which may be read at bit.ly/3lxWhwV.

www.eetimes.eu | MARCH 2023