Nissan's Sierhuis: AVs Useless Without Humans

TRAVERSE CITY, MI – While it contradicts the definition of the word, Nissan believes there is no worth in an “autonomous” system that lacks human involvement.

“Show me an autonomous system without a human in the loop, and I’ll show you a useless system,” Maarten Sierhuis, chief technology officer at Nissan’s Silicon Valley research lab, tells attendees during Wednesday’s keynote speech at the 2018 CAR Management Briefing Seminars.

Nissan, striving to achieve eyes-off fully autonomous urban driving in its vehicles sometime in the 2020s, wants “socially acceptable” autonomy, Sierhuis says, noting humans are social creatures and want the comfort of human involvement.

To illustrate this point, he shows a difficult situation that autonomous vehicles most certainly will encounter and which artificial intelligence programmed into future AVs may not readily solve. Sierhuis plays a 20-second video from a 4-way-stop intersection in a California town, reflecting how pedestrians and those behind the wheel give cues with nods or gestures about how and who proceeds. Sometimes pedestrians go first, sometimes vehicles do.

Nissan is in a collaboration with NASA, Sierhuis’ former employer, to develop what is billed as the “world’s safest traffic control system,” to get autonomous vehicles through busy intersections where humans and vehicles confusingly mix, or through pop-up construction zones where how to proceed may be unclear.

Informing Nissan’s traffic-control system is NASA’s Mars Control center and the U.S. Federal Aviation Admin. air-traffic control system.

With the Mars Control Center, plans are uploaded for the Mars Rover to perform when it wakes up. This is similar to what Sierhuis says Nissan wants to achieve with its AVs.

With NASA, Nissan has devised what it calls a tele-operation system consisting of a tele-operator console environment, 4G or 5G communication and a tele-up system that communicates with the autonomous system.

Data coming back from sensors on an autonomous vehicle will be transmitted to an operator to create a situational awareness of what’s going on.

“The operator can now create a new plan for the vehicle (to get around an impediment in the roadway) and send that to the vehicle via the tele-up system,” Sierhuis says. “And the autonomy system executes that plan. So we’ve created an environment where the onboard autonomy system is always fully responsible for driving the vehicle. And the human operator at a distance does not drive with a joystick or a steering wheel – it sends commands.”

He says Nissan still is studying how to train workers for, and make secure, a tele-operation center, saying it is just starting to research those issues. He says he received an email from a London cabbie who believes he and his fellow cab drivers are the perfect group to work in such a center because cab drivers know the city they work in like nobody else. Nissan employees have visited with the cabbie and are working with him to learn more.

Meanwhile, Sierhuis notes Nissan is not working with municipalities on programming AVs for infrastructure. “The answer is really no (to working with municipalities) because everybody knows the last thing to change, especially in this country, is the infrastructure. If we’re waiting for infrastructure, we won’t have driverless vehicles for the next 20 years.”

As an example, he cites a particular road in California’s Bay Area that has been under construction since he moved there in 1998.

Later, Sierhuis tells WardsAuto Nissan has built a cloud-based environment where it can communicate all the data from the vehicle to the cloud, and now it is talking with government agencies about using infrastructure data, but not in the U.S.

“In Europe they have way better infrastructure and systems, so we’re actually working with people in Europe in connecting (our cloud-based environment) to a traffic-management center that has every traffic light’s data.

“And now we can have 2-way communications, because once you have that you can now start optimizing your system at a higher level,” he says.

Sierhuis says he tried to do the same in Silicon Valley but found two small traffic centers in neighboring towns didn’t communicate with each other.

Nissan also built a couple of traffic lights around its Silicon Valley R&D center using dedicated short-range communication. A third light it wanted to integrate belonged to a municipality that didn’t want to give the automaker access to its data.

“So in the end, it was a year and a half and a lot of money and a lot of effort. And that was just three traffic lights,” he says. Vehicle-to-infrastructure communication “would be great, but I see it as just another data stream.”

A Nissan spokesman says the automaker is involved in V2I research outside Silicon Valley, including as part of the Crash Avoidance Metrics Partnership in Farmington Hills, MI.