On-Vehicle Cameras Watch Out for Drivers by Watching Them

BRIMLEY, MI – Modern premium cars come with outside-aimed detection cameras to help prevent front-end collisions, back-over accidents and the like.

But German auto supplier Continental is preparing to go to market with advanced- safety equipment that includes interior cameras pointed at the driver. The goal: to detect distraction, inattentiveness and drowsiness.

If such lapses occur, the car would warn the driver and, if that doesn’t work, take emergency braking and maneuvering actions on its own. 

Continental demonstrates interior-camera systems as well as five other pre-production safety technologies during a media preview at its 540-acre (219-ha) test track in Michigan’s remote Upper Peninsula.

It’s often a tough sell for suppliers to convince cost-conscious automakers of the need for advanced-safety equipment. The prototype camera systems present a new set of challenges extending to potential end-user push-back. Could consumers perceive the cameras as surveillance devices keeping watch on them?

“We’ve heard that many times,” says Jeremy McClain, Continental’s engineering manager-Advanced Technology Systems & Technology, Chassis & Safety Div.

On the face of it, some drivers might bridle at the idea of cameras pointed their way, he says. “We have to show the benefit.”

Adds Steffen Linkenbach, who heads a Continental North American systems and technology business unit: “It’s a question of getting more than you are giving.”

At the test facility, engineers try to demonstrate drivers would get life-saving equipment that looks out for them in the course of looking at them.

Engineers also note the unobtrusive positioning of the cameras; they’re virtually hidden to the naked eye. “The point of technology like this is that you are not supposed to know about it unless you need it,” says Ibro Muharemovic, a lead engineer. “It’s like airbags. How often do you think about those under normal circumstances?”

Head, Eye Movement Tracked

Continental shows two camera systems. Both work in concert with emergency braking and collision-avoidance systems that are in cars today and come with their own sets of sensing devices, including cameras and radar.

Closer to production, the simpler of the two interior camera systems detects head movement. The other more futuristically tracks precise eye movement. 

In a test-track car, Continental demonstrates the less-complicated system carrying a lugubrious working title: “Adaptive Autonomous Emergency Braking Plus Driver Attention Monitoring (Driver Focus).” It’s likened to a co-pilot.

“The critical point is recognizing what the driver is doing,” Muharemovic says. “Is he distracted?” 

The camera is encased atop the steering column. The car is equipped with light strips running across the inside of the front doors just below the side window. When activated, the moving lights function as an early-warning system in the event of an impending emergency situation the driver doesn’t readily sense but the car does.

“If the driver is looking forward, the camera sees that, and we make the assumption he is attentive,” Muharemovic says. “So we give him the benefit of the doubt until the last possible moment.”

To show that in action, the engineer whose work centers on autonomous-vehicle development drives about 15 mph (25 km/h) eyes-forward toward an obstacle representing anything from another car to a pedestrian. Because Muharemovic doesn’t brake for it, the car does, harshly and at the last moment, but stopping just inches in front of the object.

In a second run-through, Muharemovic this time looks to the left rather than ahead as the car closes in on the obstacle. The collision-avoidance system detects an impending crash, while the camera detects inattentive driving. The autonomous brakes kick in earlier than before. This time, the car stops sooner and less abruptly than when the camera sensed Muharemovic had his eyes on the roadway and presumably knew what was ahead.

“We calculate attention and inattention levels,” he tells WardsAuto. “There is a good algorithm model for those calculations.”

If a distracted or inattentive driver is looking right or left, the blue-light strips on the doors activate in a forward-motion pattern to signal the driver of an accident in the making. An audible alarm goes off, too.

“The light motion is important, but we’re still deciding ultimately how it should move and what color to use,” McClain says. “So there’s work left to be done with that.”

Then there’s the issue of selling the pre-production safety feature to automakers and the marketplace. “Using it as part of overall ambient lighting might be a way of getting this system in the car,” he says.

The second, more involved camera system is not developed to the point that Continental demonstrates it in a moving car on the track. Instead, the supplier uses an indoors simulator that includes a driver’s seat, steering wheel and instrument panel.

Three dashboard-embedded cameras follow the eyes of the person in the driver’s seat. The cameras can pinpoint exactly where the driver is looking. A warning is activated if the driver is looking one way and collision-avoidance sensors detect an imminent danger in another direction.

“We can localize the warning to wherever the driver is looking,” Brian Saloka, a senior software engineer, says of possibilities.

He emphasizes the camera system is designed to avoid or mitigate an accident, not admonish drivers for taking their eyes off the road.

“It can also detect drowsy driving,” he says. “We can apply algorithms like blink rate and head bobbing to gauge whether the driver is falling asleep.”

If nod-off occurs, the system would send an alert. It could take many forms. The one on the simulator uses driver-seat-embedded air bladders. Those typically are intended to massage backs of premium-car owners.

“But we can use the air bladders to ‘tap’ you on the back, basically saying, ‘Wake up,’” Saloka says. He cites studies indicating such pressure-point stimulation also increases circulation in the head and extremities, thus helping rouse sleepy drivers.

“It’s hard to believe, but these nice creature comforts can add to vehicle safety,” he says.

Insurance companies and NHTSA like the safety system using multiple cameras aimed at the driver, Saloka says. “But consumers might say, ‘Why am I paying for something to watch me?’ So we have to come up with something that will get them to say, ‘Yeah, I really want this system.’”  

Toward that end, Continental touts its potential ability to use facial recognition to personalize seat, mirror, steering-wheel and radio-station preferences for various drivers.

“There’s also theft detection,” Saloka says. “You can take a snapshot of authorized drivers’ faces and have the system recognize as many people as you want. If they’re not registered by facial recognition, the engine won’t start.”

McClain adds, “You have to show the added benefits and find ways to get this in the car. Once it’s there, it’s about safety.”

Step Toward Autonomous Vehicles

Many of Continental’s demonstrated safety technologies use existing equipment on the market, but the supplier is integrating them more, he says. “There are ways to continually enhance systems, especially by bringing them together.”            

Also shown during the so-called Winter Tech Show Event (although it took place on the second day of spring):

With all of the demonstrated technology, the driver can override the systems and regain control of the car. That reserved right is important, McClain says.

“We think we are on to something with a lot of this stuff,” he says.

Such technology represents more movement toward fully autonomous cars. Continental estimates 2025 as the advent of those.

In the meantime, McClain addresses a current issue: driver irritation with warning systems that, while well-intended, are viewed as jarring and sometimes unnecessary, such as the activation of lane-drifting beeps when an alert driver merely is changing lanes.       

Effective human-machine interfacing is holistic, he says. “It makes sure the driver understands the alert. It needs to tell the driver when to intervene, get back in the loop or look here. There’s an optimal way to offer that without a lot of nuisances and disorienting information.”

[email protected]