Page 36 - EETimes Europe June 2021

36 EE|Times EUROPE

AV Safety-Report Scorecard Reveals Gaps in Information

Kodiak is one company professing that higher test mileage does not
necessarily equal more safety. In its safety report, the company wrote:
“Of course, this disciplined approach means we will probably never
log as many test miles as some of our competitors. We see our lower
mileage count not as a risk but as a sign of our commitment
to safety.”
Aurora agrees, writing, “[We] treat real-world testing as a mechanism
for validating and improving the fidelity of more rapid offline testing.
This strategy has allowed us to contain the size of our on-road testing
fleet. We limit the distance our test vehicles travel by pursuing mileage
quality over quantity; that is, we seek out interesting miles rather than
just pursuing large quantities of miles.” IMAGE: SHUTTERSTOCK

‘HEAVY TAIL’
On the importance of edge-case research, Koopman noted that the
problem isn’t just about the frequency of surprises — unsafe events —
that occur in real-world driving and require corrective actions.
What matters even more is what the population of fixes looks like. the FAA had grounded the entire Boeing 737 MAX fleet when funda-
Surprises are all different. If there’s a huge population of problems, it mental flaws in the design of the MAX’s Maneuvering Characteristics
will take a very long time to make progress fixing them all, said Koop- Augmentation System (MCAS) were exposed. “The safety questions
man. Most likely, you’d never get there. here are clear, so why the free pass for AV development? It is untenable
Koopman wrote in his paper: for an industry to wail ‘saving lives’ while actually endangering them.
Creating safe autonomous vehicles will require not only extensive The political risks are enormous, and the Administration needs to wake
training and testing against realistic operational scenarios, but up and show decisive leadership right now.”
also dealing with uncertainty. The real world can present many There’s already a precedent. An SAE spokesman noted that “a little
rare but dangerous events, suggesting that these systems will need more than a year after” pedestrian Elaine Herzberg was killed by an
to be robust when encountering novel, unforeseen situations. Uber test AV, the SAE took action, updating SAE J3018, Guidelines for
Safe On-Road Testing of SAE Level 3, 4, and 5 Prototype Automated
Generalizing from observed road data to hypothesize various Driving Systems (first published in March 2015). This was “to better
classes of unusual situations will help. However, a heavy tail dis- reflect the needs of safe on-road performance testing.”
tribution of surprises from the real world could make it impossible The updated J3018 standard “incorporates lessons learned based
to use a simplistic drive/fail/fix development process to achieve on accumulated field experience in testing prototype ADS-operated
acceptable safety. vehicles on public roads and is now compatible with related SAE Inter-
Autonomous vehicles will need to be robust in handling novelty, national documents,”the spokeswoman said.
and will additionally need a way to detect that they are encoun- In parallel, a year after the fatal Uber accident, an invitation-only
tering a surprise so that they can remain safe in the face of auto industry group, the Automated Vehicle Safety Consortium (AVSC),
uncertainty. started to “document and make publicly available best practices
associated with in-vehicle fallback test drivers based on the types of
measures and processes the members use,” said Ed Straub, director
NEW ADMINISTRATION, NEW NHTSA? of the SAE Office of Automation. He described the AVSC’s mission as
NHTSA’s laissez-faire approach to the auto industry is notorious. For “generating public trust in SAE L4 and L5 autonomous vehicles.” All
decades, the agency’s interest has squarely aligned with automakers AVSC members are doing on-road ADS testing for various applications.
and tech companies. But here’s the thing: When AV companies started seeking permission
Thus far, the Biden Administration has given no clues about its from cities and states to test drive AVs on public roads, why didn’t local
approach to safety issues in autonomous vehicles. Will new Transpor- regulators demand that AV operators conform to J3018?
tation Secretary Pete Buttigieg steer NHTSA in a new direction and, for Straub said, “I can’t speak to the minds of different states and cities.”
example, make tech companies and carmakers’ AV safety claims more He speculated that awareness of the standard might not have trickled
transparent and accountable? down to the “right people who could potentially use it as a reference.”
The agency is reviewing the public comments it collected for But he also defended the auto industry’s position, saying, “SAE
the Advance Notice of Proposed Rulemaking (ANPRM) on auton- International standards are voluntary and have been for over a hundred
omous driving systems issued by the previous administration. The years, unless explicitly cited in regulations such as the FMVSS [Federal
comment period ended on April 1. An NHTSA spokeswoman told EE Motor Vehicle Safety Standards].”
Times in May that the agency might have more to say after it finishes He pointed out that “the technology and testing associated with
reviewing the public comments and determining its next steps. automated driving changes at a pace we haven’t seen before. Because of
“NHTSA’s regulatory plans for the next 12 months will be published this, it can be very difficult for open industry standards to keep pace …
later this spring [as part of] the Semi-Annual Unified Agenda of Because voluntary industry standards invite all interested stakeholders
Regulatory Actions,” she said. to develop them, they can take a longer time to develop. Regulations
that would ‘require’ compliance [like FMVSS] can take even longer.”
ONE CATASTROPHIC EVENT AWAY Seriously, though, what sort of industry spends time and resources
Many AV industry observers are aware that the fledgling industry is developing a safety standard for all its members but shows no interest
only one news event away from the government finally deciding to in enforcement? How is it that the “rules of the road” apply to every-
enforce stricter rules. body but the people who make the cars? ■
As Barnden observed, “If an AV test-level vehicle kills a single pedes-
trian or a child, that leads straight to President Biden.” He noted that Junko Yoshida recently retired as global editor-in-chief of AspenCore.

JUNE 2021 | www.eetimes.eu