A vehicle driving through a tunnel of blurred lights.
During a visit to Austin, Texas in August 2023, the buzz around Cruise’s self-driving cars piqued my interest. Eager to experience this cutting-edge technology, I accepted an invitation for a ride. As my friend and I settled into the back seat, the car’s automated voice reminded us to buckle up, and we set off on our journey.
The drive was initially smooth, with the car navigating carefully through the streets. However, our excitement turned to apprehension when the vehicle abruptly halted. Peering through the rain-soaked windows, we discovered a massive tree trunk obstructing our path. Expecting the car to maneuver around the obstacle, we were surprised when it hesitated, unsure of how to proceed. After what felt like an eternity, the car slowly began to navigate around the impediment, but not without considerable delay.
Months later, I seized the opportunity to delve deeper into the incident by speaking with USC Viterbi professors Rahul Jain and Jyo Deshmukh, director and co-director of the USC Center for Autonomy and AI (AAI).
Established in 2021, AAI serves as a hub for industry professionals, faculty, and students dedicated to solving challenges in intelligent autonomous technologies, including robotics, energy systems, and autonomous vehicles. In close collaboration with industry giants such as Toyota and Siemens, the center conducts cutting-edge research, fueled by both funding and industrial insights.
“The goal of AAI is to bring USC and industry researchers together to solve problems. By engaging with companies, our research today might be their solution tomorrow,” said Jain.
Rahul Jain (left), Pierluigi Nuzzo (middle), and Jyotirmoy Deshmukh (right) at the AAI Fall 2023 Workshop
To foster collaboration, AAI hosts an annual workshop where professionals, faculty, and students convene to exchange ideas and showcase their latest research. At the Fall 2023 workshop, representatives from Toyota and Siemens engaged in a panel discussion with USC professors, shedding light on industry obstacles and critical needs within the autonomous vehicle sector. In particular, the industrial professionals stressed one bottleneck that has been preventing them from bringing more autonomous vehicles on the road: “corner cases.”
During my meeting with Jain and Deshmukh, I recounted my encounter with the obstructed tree trunk, and Jain quickly identified it as a “corner case,” a scenario the car hadn’t been explicitly trained to handle, revealing a limitation of the current approach based on an AI expert system methodology.
The expert system relies on engineers to anticipate and program for every conceivable scenario, but its effectiveness is limited by the infinite variability of the real world. As Deshmukh explained, autonomous vehicles must emulate the way humans interact with their environment, continuously processing perceptual inputs to make informed decisions. As a result, the perception system remains as “the toughest challenge for engineers to break through,” said Deshmukh.
“Ideally, a self-driving car works like how a human unconsciously interacts with the environment,” said Deshmukh. “To make a decision, the cars need to have all the perception inputs because the environment is constantly evolving.”
The perception systems, which rely on data from cameras, light sensors, and radar, continue to face challenges when navigating complex real-world scenarios. Currently, Deshmukh and his lab are working on correcting the reasoning behind sensor fusion techniques to help them better predict “scenarios that are out of distribution of training data.”
On the other hand, Jain and other AAI researchers are developing an approach that promises fully autonomous cars in the future. Through reinforcement learning, a proven machine learning technique in generative AI, AI systems can learn from feedback and improve decision-making over time by generalizing from what it has seen to what it has not yet encountered. With this approach, every corner case serves as a valuable lesson, empowering the car to make accurate decisions, even in novel situations. With the integration of safety and operational protection, this approach holds the potential to enable future autonomous vehicles to drive themselves safely, reliably, and autonomously.
Thus far, the reinforcement learning method is still in the simulating phase and requires a robust perception system as the cornerstone.
“Occasional errors may be tolerated in simulations, but real-world deployment requires hard guarantees,” Jain emphasized.
Currently, Jain and Deshmukh are spearheading efforts in their lab to enhance perception system accuracy and refine reinforcement learning methods to improve vehicles’ response for corner cases and reach full autonomous vehicles in five years. Moving forward, AAI aims to deepen its industry partnerships and bridge the gap between academic research and real-world applications.
Published on April 10th, 2024
Last updated on May 16th, 2024
