Page 35 - EETimes Europe June 2021

EE|Times EUROPE 35

AV Safety-Report Scorecard Reveals Gaps in Information

they could about safety, and that would be either ‘conform’ or the less Weast isn’t alone. Given that the industry itself spent significant
buzzwordy ‘follow.’ If they make a weaker claim than either of those engineering resources to create these standards, Koopman asked, “Why
two words, they shouldn’t expect to get credit for a stronger claim.” wouldn’t they follow them?”
Asked to clarify the company’s statement, an Argo AI spokesman Among all the AV companies filing VSSAs, Nvidia’s presence on
told EE Times, “Our intention is to conform, although our safety case the list came as a mild surprise. This suggests Nvidia’s ambition to
is not complete yet, and that’s why we wrote the statement in the way develop an AV platform (complete with its powerful SoC and AV soft-
that we did.” ware stack) that automakers can simply pick up and drop into vehicles
Is there something about the existing standards that makes AV for an AV launch.
industry players believe that they are not relevant to the “autonomy” In its safety report, Nvidia wrote, “The Nvidia Drive architecture
they seek in their L4 and L5 cars? Koopman made it clear: “ISO 26262 enables vehicle manufacturers to build and deploy self-driving cars and
addresses functional safety and deserves a place in an L4/L5 vehi- trucks that are functionally safe and can be demonstrated compliant to
cle. What you need for L4/L5 is additional coverage for Safety of the international safety standards such as ISO 26262 and ISO/DIS 21448,
Intended Function (SOTIF/ISO 21448), and system-level safety NHTSA recommendations, and global NCAP requirements.”
(ANSI/UL 4600). You need all these pieces to be covered.” But Nvidia’s bow to ISO 26262 does not necessarily mean that an AV
With no oversight, no verification, and no validation currently containing Nvidia’s chip will conform to ISO 26262. As Koopman put it,
required in the testing of autonomous vehicles, a fundamental question enabling customers to build AVs that conform to the standards leaves
— how to ensure the safety of autonomous vehicles before they prolif- it “up to the customer to actually conform.” He added that if Nvidia’s
erate on the road — becomes a riddle inside a conundrum. approach lets a client conform to 21448 “in simulation,” that still begs
Despite the existence of competing and misaligned AV standards, the question of whether the final product meets 21448 in the real world.
“a solution to this problem exists: Embrace the approach already “Beyond Nvidia, we’ve seen scattered claims of ISO 26262 confor-
taken by experts — scholars, engineers, automakers, and industry mance, but they are really only talking about the chips, and not even all
representatives — who came together to develop a technology-neu- the hardware,” said Koopman.
tral safety standard,” Intel fellow Jack Weast recently wrote (bit.
ly/2QTcqy6). Weast chairs the effort for the forthcoming IEEE P2846 MARKETING SAFETY
standard, Assumptions for Models in Safety-Related Automated Reading 24 safety reports conveys the inevitable impression that these
Vehicle Behavior. are, essentially, marketing brochures. They are “completely inter-
changeable,” asserted Barnden. “If I was sent the raw text from, say,
the Motional, Waymo, and Zoox brochures with the company name
removed, I couldn’t tell them apart.”
In the main, the reports go heavy on glossy pictures and light on
technical substance.
“It wasn’t clear to me what metric could measure progress made in
the last 12 months, or what information would materially change in a
new safety report issued next year” said Barnden. “Endlessly repeating
the word ‘safe’ doesn’t, in itself, make a product or process safe.”
Consider Waymo. “In 2020, Waymo reported to the California DMV
21 disengagements over 628,839 miles driven, a rate of about one
disengagement every 30,000 miles,” said Barnden. “But as we’ve seen in
a recent video, a ‘fully driverless’ Waymo vehicle was completely bam-
boozled by a single static traffic cone. At one point, the [AV] reversed
on the highway and entirely blocked a lane, forcing human drivers to
steer around it. This took place in Waymo’s primary testing area around
Chandler, Arizona.” (The video [bit.ly/3vrmaPn] is by YouTuber JJRick,
who shot the footage while riding as a customer in a Waymo driverless
taxi in Chandler.)
“Disengagement reports are meaningless,” said Barnden. “These
vehicles are clearly not yet safe enough to be operated on public roads
without a trained human safety driver.”
Considering that Waymo has been on the road since 2015, it is sur-
prising that its robotaxi is still confused by traffic cones, which hardly
seem to qualify as an “edge case.” Once perceived as a clear AV industry
leader, Waymo lost six high-level executives — its CEO, CFO, treasurer,
head of manufacturing, chief safety officer, and system safety chief —
in recent months. The exodus has stirred concern and energized the
Silicon Valley gossip mill.
That brings us back to our original point: How much do we really
know about the progress Waymo Driver has made? Waymo’s safety
reports are notably unrevealing, leaving observers to rely on a
YouTuber as a key source of insight into Driver’s real-road perfor-
mance. Not exactly reassuring.
It’s also past time for AV developers to come clean about how long
it will be before they can launch fully autonomous vehicles. The safety
reports should include not just the developers’ safety claims but the
(Source: Nvidia Self-Driving Safety Report) challenges they’re facing and how they plan to deal with them.

www.eetimes.eu | JUNE 2021