Tesla Autopilot to Avoid Potholes via HD Maps

Tesla Autopilot Pothole Detection

Elon Musk famously said during the 2019 Tesla Autonomy Day that Tesla will not rely on HD Maps for its Autopilot system, saying they are a “really bad idea,” resulting in a “system [that] becomes extremely brittle” by being too dependent on the map and not being able to quickly adapt changes in the environment like humans do.

Instead, Tesla Autopilot relies primarily on visual perception using the eight cameras around the vehicle plus additional input from a forward-facing radar unit and ultrasonic sensors around the vehicle. This is in contrast to many other self-driving companies that use high-resolution LiDAR to provide precise distance information to other objects (see Cameras vs LiDAR for Self-Driving Cars).

That said, it appears that Elon is not against using HD Maps (or “micro maps”) to help enhance the precision by which Autopilot can navigate the environment.

Crowdsourced HD Maps

A one-word tweet from Elon Musk on Feb. 3, 2020, confirmed that Tesla will likely utilize HD Maps (or “micro maps”) to help the Autopilot system better navigate with precision and avoid obstacles like potholes.

Reto Siegrist of Switzerland asked Musk this question on Twitter: “Is it possible that Tesla will be able to create a “micro map” of every road with all the details (stop sign, potholes, etc.) that can be used by other Teslas when they drive along the same road?” The Tesla Motors CEO answered, “Yes.”

Having the ability to identify and navigate potholes without human driver assistance moves Tesla’s Autopilot-equipped vehicles closer to a true self-driving system. In a tweet dated April 9, 2019, Musk referenced adding an anti-pothole component as an Autopilot feature.

Even with its latest updates and more seamless Navigate feature, Autopilot requires human drivers to look out for and dodge potholes. However, Musk’s latest tweet shows the electric vehicle team is advancing in its quest to reach Full Self-Driving (FSD) capability for Tesla Model 3.

How HD Maps Work with Self-Driving Cars

For the Autopilot pothole detection system to work, cameras in Tesla vehicles will collect data and create “micro maps” of roads and upload that information so it can be shared with the rest of the Tesla fleet. This system will provide valuable information that drivers do not find in typical maps. The technology will allow fellow Tesla owners access to the information as they drive along the same road. Streets with lots of activity require data updates more frequently than do long stretches of highway.

Some autonomous vehicles use HD maps, which update data constantly, monitoring traffic signals, potholes, street signs, and curb heights with to-the-centimeter precision. Uber’s former VP of maps, Brian McClendon, worked on Google Maps before joining Uber. “If you thought creating regular maps for Google was hard, creating self-driving maps is much, much harder,” McClendon said. “The amount of data involved is huge. The precision required is very hard as well,” he added.

The Challenge of Pothole Detection

Unlike other road hazards, potholes are depressions on the road. Holes can appear differently to a self-driving car’s camera, depending on factors such as the time of day, weather, and lighting. Tesla’s electric car camera system will compare the pothole’s interior texture to pavement nearby, and its irregular surface with other parts of the road. They’ll leverage Tesla’s sophisticated Hardware 3 and the Tesla Neural Network to make it happen.

Here’s how Tesla describes the neural networks function in Autopilot: “Our networks learn from the most complicated and diverse scenarios in the world, iteratively sourced from our fleet of nearly 1M vehicles in real-time. A full build of Autopilot neural networks involves 48 networks that take 70,000 GPU hours to train. Together, they output 1,000 distinct tensors (predictions) at each timestep.”

Why Mapping Potholes Matters

Potholes are a big problem throughout the United States and in locations including Quebec, Canada. Changing weather conditions and government disputes about infrastructure projects are among the reasons why drivers on two-lane blacktops and major highway systems are constantly watching for road hazards, including potholes. Problems can be as simple as a flat tire or progress to a broken axle, bent chassis, damaged suspension or a ruined drivetrain. Striking potholes can cause drivers to lose control of their vehicles and cause serious accidents.

Repairing pothole damage to vehicles costs American drivers an average of $3 billion per year, according to a study by AAA. Sixteen million drivers across the United States have experienced vehicle pothole damage in the last five years. Their average repair bill is $300, which vehicle owners often pay out of pocket. Frequently, the damage is not covered by insurance or the policyholders’ deductibles are higher than their repair costs, notes the study. In the United States, about a third of major urban road conditions are substandard, according to a national non-profit transportation research group, TRIP.

One YouTuber had some very expensive damage to his performance Model 3 when hitting a pothole:

MIT Pothole Project

In pursuing its pothole mapping task, Tesla might consider getting a little help from friends at MIT’s Concrete Sustainability Hub. A collaboration between MIT faculty and students, UMass Dartmouth, Birzeit University in the West Bank, and the American University of Beirut has produced an app called Carbin. The app measures and tracks road quality while driving and aims to help drivers find the smoothest road on which to travel. Combining GPS navigation and software from Mapbox, a third-party GPS and maps provider, Carbin monitors and records how rough the roads are. The app’s measurement gauge is a familiar one: the accelerometer. A component in most smartphones, it measures velocity changes along one axis. So when the phone shakes a lot, it means the road is bad. Once the app’s servers get repeated data from several drivers on the same lane or road, they analyze the information. Results are uploaded to FixMyRoad.us, which tracks international road quality.

Carbin launched in February 2019, and several hundred users have contributed data for 3,000 recorded trips as of November 2019. Obtaining near real-time information is a boon to mapping road conditions accurately. Apps like Carbin help conserve finances, human resources, and time when compared to the two years a government’s DOT may take to evaluate one section of road using a $250,000 laser-measuring device.

Other Automakers Working on Pothole Avoidance

Tesla is not the only automaker employing HD Maps to potentially avoid potholes. Ford is working on a system to help drivers avoid such road hazards and take action to keep damage to a minimum.

Google’s self-driving car division, Waymo, has been working for years to develop the mapping necessary to fully automate its vehicles. The car cameras and sensors gather data, showing items such as fire hydrants and driveways and report it to the mapping crew. Crew members adjust the maps to match the changing environment, such as road work or potholes.

Toyota is working with CARMERA to develop an Automated Mapping Platform (AMP) to support full self-driving using HD maps created with LiDAR plus RGB scans. Uber is also in the race to gather mapping data.

Finally, Mobileye, Intel’s self-driving technology division and one of the leaders in the field, is using what it’s calling Road Experience Management (REM) to constantly collect road data from the fleet and update its maps.

When Are HD Maps Coming to Tesla?

While timelines for Autopilot are constantly shifting and somewhat elusive, we hope to see the use of HD Maps / Micro Maps soon as avoiding potholes is one of the few reasons why many current Tesla owners have to disengage Autopilot temporarily. Hitting potholes is not only uncomfortable, it’s also costly. This technology can’t come soon enough!