Camera Technology Monitors Where Driver is Looking

TRAVERSE CITY, MI – Lane-departure warnings are one of the most unused active-safety features, frequently disabled by drivers even though they can save lives, says the Continental Automotive Systems engineer in charge of addressing that problem.

“If you swerve out of your lane to give a bicyclist more room, you don’t want a buzzer warning,” says Zachary Bolton, a mechanical engineer who learned software development at Continental and now leads the Driver Focus Vehicle project at the supplier’s Auburn Hills (MI) Technical Center.

Driver Focus positions a camera in the car to watch the driver’s face and see whether he or she is distracted.

Conti introduced the camera project at the Chicago auto show and brings it to the CAR Management Briefing Seminars here to attract more interest from the industry.

Continental says the technology is designed to bring the driver’s attention to where it needs to be. With a camera in the car, lane-departure warnings become adaptive. If there is no danger, and the driver is looking at the road, no warning buzz or vibration need be activated.

Algorithms written by Bolton and his team control several hundred light-emitting diodes that circle the interior under the windows and light up where the driver is looking, such as toward the back-seat child carrier, and moving them forward to where a car is pulling out of a driveway.

Continental is showing the system in two units of the Cadillac XTS. One is touring the U.S., and the other is in Germany for testing and display at September’s IAA auto show in Frankfurt.

The real innovation is not the strip of LEDs, but the camera in the car watching the driver.

“I wasn’t sure about that at first,” says Bolton. “I read Orwell’s 1984 and Big Brother, and at first I wanted to improve the exterior sensors,” radar and camera and the data that comes from their blending. “But with a camera, you can do so much.”

The inexpensive black-and-white camera, using infrared imagery, identifies the curve below the eyes and jawline to identify a face and follow it. With that knowledge, the car could light up the floor to help a driver find the phone he dropped.

It could use the ambient lighting already in some General Motors products instead of LEDs to call the driver to attention. It could use the driver’s look at the display screen on the center console to change the buttons that appear on the console, depending on the situation.

The camera could remember the driver’s face and adjust the car’s seats and lights and preferences for adaptive-cruise-control settings. One complaint about adaptive cruise control is that it can leave too large a safety cushion, letting too many other drivers cut in.

Future cameras, Bolton says, will know where the pupils are looking, within 0.5 degrees, so the system would detect which button on the screen is being considered.

About 200 people have driven vehicles equipped with the system, and none have worried that a camera is watching them. The images are used only within the car and nothing is recorded, he says, so “if your girlfriend gives you a peck on the cheek,” nobody else could know.

People testing the Cadillac XTS prototype knew about the camera but ignored its presence. The testing also helped fine-tune the camera’s software, including the algorithms, says Bolton. The camera has learned more face patterns, and the designers of the actuator concept learned they could count on the driver’s peripheral vision when calculating where the light alert should begin.

Continental is one of the suppliers most active in developing autonomous vehicles and is licensed in Nevada for public testing of self-driving vehicles. Some sort of driver observation will be essential for such cars, because control from time to time will change between the driver and vehicle, and the vehicle will have to know the driver is ready to take control.

Thus, the camera technology and its acceptance by customers is critical for autonomous vehicles‘success.