Industry Voices | The Road to 99.999%: Eliminating Blind Spots in Automotive Visual AIIndustry Voices | The Road to 99.999%: Eliminating Blind Spots in Automotive Visual AI

As the automotive industry races toward a future of increasingly autonomous and smarter vehicles, visual AI is paramount in shaping this transformation. And even as we try to instill faith in this burgeoning technology, the reality is that most AI projects fail – to the tune of up to 80% – and data is often at the heart of this problem.

Harnessing visual AI to successfully compete in this space requires eliminating the blind spots around data and the AI models it fuels. When we have a clear view of what’s going wrong, we can quickly iterate for success to cut the failure rate and deliver an ever-improving experience to human drivers and those sharing the road with us.

The AI challenges around data are not unique to the automotive industry. That said, the consequences of AI failures in automotive applications can be a matter of life and death. Autonomous cars and driver-assist features must meet the five nines of accuracy and reliability to avoid being judged as failures – which means surpassing human behavior by performing safely and accurately 99.999% of the time. 

With 70% of the accidents at intersections with stop signs caused by failure to fully stop, for example, anyone with experience on the road has likely shaken their fists at someone who rolled through or didn't even slow down. An autonomous car that stops at only 95% of stop signs is a marked improvement over human performance, and yet it still presents a clear hazard. 

To meet this high bar for success and hit the five nines, we must train systems to handle the broadest possible range of scenarios – including those we cannot anticipate. In addition, by amassing the right visual data, and maintaining a clear view of that data and how it impacts our AI models, we are poised to quickly and efficiently find a solution when something goes wrong instead of blindly throwing more data into the darkness.

 Shining Headlights on Visual AI’s Data Problem

As a graduate student researching AI in the early 2000s, I dedicated an entire year to testing the impact of changes to algorithm parameters against a dataset of 60 to 80 images. Flash forward to today and datasets are huge, reaching hundreds of millions or even billions of samples. 

At these scales, relying on manual labor to work with all that data is simply not a viable solution. For example, we are far past the tipping point of being able to rely solely on human data annotation, let alone manually manage how data interacts with an algorithm, or manually comb through data to find a match for what we’re looking for. The result is that when projects take a wrong turn, so to speak, we are often left scrambling in the dark, unable to identify the cause. 

Take, for example, a situation faced by a visual AI innovator whose lane-following software started identifying phantom potholes. The model hadn’t changed, and no one knew why cars were suddenly swerving to avoid something that wasn’t there. To uncover the root cause of the issue, the team had to get their hands on the visual data – all 3 million images of it.

Without a unified view of what was happening, this blind spot had the potential to bring the project to a screeching halt. The company struggled to collect and process this vast amount of data, handing it from one harried team to the next, and then repeatedly reviewing and testing data against the model in a complicated array of steps to pinpoint the exact data causing the problem.

Cruising Through the Data Challenge in Visual AI

Improving data quality and adjusting model parameters is an iterative and lengthy process. Adding to its complexity, AI development is often siloed with separate teams, tools and processes managing data and models. Tack on outside vendors and this creates a bottleneck as teams are left in the dark, struggling to shine light on the cause of each new problem. 

Ultimately, if the data culprit cannot be identified, it’s difficult or impossible to improve the model to meet performance requirements, leaving new projects stuck in R&D and making maintenance for active models like the one involving the nonexistent potholes above complex and costly. When teams have visibility into how data interacts with a model, they can also help fill the gaps and further strengthen that model with synthetic visual data that is generated algorithmically to represent real-world scenarios. 

Driving Success for Visual AI Projects

An integrated approach to building models with data allows teams to take control of the development process, removing the obstacles that slow them down – and the blind spots that stall or kill projects. Beyond simply seeing data, teams armed with the tools to query it, analyze it, refine it and shine a light on how it is impacting model performance can quickly iterate and improve both the data and algorithm to bring powerful AI models into production. 

In the case of the phantom potholes, the organization had its teams, data and models all in the same place. They worked quickly to check for anomalies and easily recognized that an unusual string of storms had thrown branches onto the ground, casting shadows. Because this stormy scenario had not been in the training data used to create the model, it interpreted the shadows as potholes.

Instead of stalling, the teams rapidly iterated their models to address this problem – which empowered them to continue to hit the five nines and maintain customers' faith in their autonomous offering. To create and maintain AI systems that meet the five-nines level and reach the promise of autonomous cars and intelligent driver-assist features, we must iterate ever faster to take control of the data and algorithms that power visual AI. By replacing fragmented, inflexible tools and clunky manual processes with modern technology that unlocks the insights that come from connecting data and models, we can begin to cut the failure rate and, in turn, make traveling on the road safer and easier.