Technology helps self-driving cars learn from their own “memories” – ScienceDaily

Researchers at Cornell University have developed a way to help autonomous vehicles create “memories” of past experiences and use them in future navigation, especially in adverse weather conditions, when the car cannot safely rely on its sensors.

Cars using artificial neural networks have no memory of the past and are in a constant state of seeing the world for the first time – no matter how many times they have driven a certain way before.

Researchers have created three simultaneous articles to overcome this limitation. Two were presented at the IEEE Conference on Computer Vision and Image Recognition (CVPR 2022), held June 19-24 in New Orleans.

“The fundamental question is, can we learn from repeated tours?” said senior author Killian Weinberger, a professor of computer science. “For example, a car can confuse a tree with a strange shape for a pedestrian when the laser scanner detects it from a distance, but once it is close enough, the category of the object will become clear. So, the second time you pass the same tree, even in fog or snow, you would hope that the car has learned to recognize it correctly. “

Led by PhD student Carlos Diaz-Ruiz, the group compiled a data set by driving a car equipped with LiDAR light detection and range sensors, repeatedly on a 15-kilometer circuit in and around Ithaca, 40 times over 18 months. The tours capture different environments (highway, city, campus), weather conditions (sunny, rainy, snow) and hours of the day. This resulting data set has more than 600,000 scenes.

“This deliberately reveals one of the key challenges in self-driving cars: bad weather,” Diaz-Ruiz said. “If the street is covered in snow, people can count on memories, but without memories, the neural network is at a great disadvantage.”

HINDSIGHT is an approach that uses neural networks to calculate object descriptors as the car passes them. Then compress these descriptions, which the group called SQuaSH? (Spatially Quantized Diluted History) and stores them on a virtual map, as a “memory” stored in the human brain.

The next time the self-driving car passes through the same place, it can search the local SQuaSH database at each LiDAR point on the route and “remember” what it learned last time. The database is constantly updated and shared between vehicles, thus enriching the available information for performing identification.

“This information can be added as features to any LiDAR-based 3D object detector;” said PhD student Urong Yu. “Both the detector and the presentation of SQuaSH can be trained together without additional supervision or human annotation, which is time consuming and time consuming.”

HINDSIGHT is a precursor to the additional research the team is conducting, MODEST (Discovering Mobile Objects with Ephemerality and Self-Learning), which will go even further, allowing the car to learn the whole pipeline of perception from scratch.

While HINDSIGHT still suggests that the artificial neural network is already trained to detect objects and complements it with the ability to create memories, MODEST recognizes that the artificial neural network in the vehicle has never been exposed to any objects or streets at all. Through multiple tours of the same route, he can learn which parts of the environment are stationary and which are moving objects. You are slowly learning what other traffickers are and what is safe to ignore.

The algorithm can then detect these objects reliably – even on roads that were not part of the original multiple rounds.

Researchers hope that the approaches could drastically reduce the cost of developing autonomous vehicles (which currently still rely heavily on expensive annotated human data) and make such vehicles more efficient by learning to navigate. in the places where they are used the most.

Source of history:

Materials provided by Cornell University. Original, written by Tom Fleischmann, courtesy of the Cornell Chronicle. Note: Content can be edited for style and length.

Leave a Comment

Your email address will not be published.