Elon Musk famously said during the 2019 Tesla Autonomy Day that Tesla will not rely on HD Maps for its Autopilot system, saying they are a “really bad idea,” resulting in a “system [that] becomes extremely brittle” by being too dependent on the map and not being able to quickly adapt changes in the environment like humans do.
Instead, Tesla Autopilot relies primarily on visual perception using the eight cameras around the vehicle plus additional input from a forward-facing radar unit and ultrasonic sensors around the vehicle. This is in contrast to many other self-driving companies that use high-resolution LiDAR to provide precise distance information to other objects (see Cameras vs LiDAR for Self-Driving Cars).
We’re now getting a glimpse of how Autopilot may use maps thanks to prolific Tesla hacker Green and this tweet where he looks at developer data in the Full Self Driving program.
In it he shows the following image where the Source is clearly listed as MAP and indicates upcoming stop signs.
It’s clear that Tesla uses a combination of vision and maps to navigate the world. However, the debate is whether these are considered “HD Maps” or not. As Elon pointed out earlier, the system needs to rely on vision primarily, since relying solely on HD maps when the world changes in real-time would be dangerous.
For example, as debris shows up in the road, here’s how the Full Self-Driving Beta handles it:
While this is happening in real-time via cameras, it appears that Elon is not against using some form of maps (or “micro maps”) to help enhance the precision by which Autopilot can navigate the environment. This will be critical as obstacles and potholes can cause damage and, worse, potential accidents.
Since obstacles and potholes come and go over time, once a vehicle sees them, letting other vehicles know of them, could help future drivers traveling down that same path avoid them, increasing safety and reducing potential damage.
One could argue that the precision to avoid something like potholes would make these maps closer to HD Maps.
Crowdsourced HD Maps
A one-word tweet from Elon Musk on Feb. 3, 2020, confirmed that Tesla will likely utilize high-precision maps (or “micro maps”) to help the Autopilot system better navigate with precision and avoid obstacles like potholes.
Reto Siegrist of Switzerland asked Musk this question on Twitter: “Is it possible that Tesla will be able to create a “micro map” of every road with all the details (stop sign, potholes, etc.) that can be used by other Teslas when they drive along the same road?” The Tesla Motors CEO answered, “Yes.”
Having the ability to identify and navigate potholes without human driver assistance moves Tesla’s Autopilot-equipped vehicles closer to a true self-driving system. In a tweet dated April 9, 2019, Musk referenced adding an anti-pothole component as an Autopilot feature.
Even with its latest updates and more seamless Navigate feature, Autopilot requires human drivers to look out for and dodge potholes. However, Musk’s latest tweet shows the electric vehicle team is advancing in its quest to reach Full Self-Driving (FSD) capability for Tesla Model 3.
How HD Maps Work with Self-Driving Cars
For the Autopilot pothole detection system to work, cameras in Tesla vehicles will collect data and create “micro maps” of roads and upload that information so it can be shared with the rest of the Tesla fleet. This system will provide valuable information that drivers do not find in typical maps. The technology will allow fellow Tesla owners access to the information as they drive along the same road. Streets with lots of activity require data updates more frequently than do long stretches of highway.
Some autonomous vehicles use HD maps, which update data constantly, monitoring traffic signals, potholes, street signs, and curb heights with to-the-centimeter precision. Uber’s former VP of maps, Brian McClendon, worked on Google Maps before joining Uber. “If you thought creating regular maps for Google was hard, creating self-driving maps is much, much harder,” McClendon said. “The amount of data involved is huge. The precision required is very hard as well,” he added.
Use Case: Potholes and HD Maps
Unlike other road hazards and obstacles, potholes are depressions in the road and can be harder to detect. Holes can appear differently to a self-driving car’s camera, depending on factors such as the time of day, weather, and lighting. Tesla’s electric car camera system will compare the pothole’s interior texture to pavement nearby, and its irregular surface with other parts of the road. They’ll leverage Tesla’s sophisticated Hardware 3 and the Tesla Neural Network to make it happen.
Here’s how Tesla describes the neural networks function in Autopilot: “Our networks learn from the most complicated and diverse scenarios in the world, iteratively sourced from our fleet of nearly 1M vehicles in real-time. A full build of Autopilot neural networks involves 48 networks that take 70,000 GPU hours to train. Together, they output 1,000 distinct tensors (predictions) at each timestep.”
According Elon, the Pothole detection should arrive with the FSD Rewrite happening in late 2020 or early 2021.
Why Mapping Potholes Matters
Potholes are a big problem throughout the United States and in locations including Quebec, Canada. Changing weather conditions and government disputes about infrastructure projects are among the reasons why drivers on two-lane blacktops and major highway systems are constantly watching for road hazards, including potholes. Problems can be as simple as a flat tire or progress to a broken axle, bent chassis, damaged suspension or a ruined drivetrain. Striking potholes can cause drivers to lose control of their vehicles and cause serious accidents.
Repairing pothole damage to vehicles costs American drivers an average of $3 billion per year, according to a study by AAA. Sixteen million drivers across the United States have experienced vehicle pothole damage in the last five years. Their average repair bill is $300, which vehicle owners often pay out of pocket. Frequently, the damage is not covered by insurance or the policyholders’ deductibles are higher than their repair costs, notes the study. In the United States, about a third of major urban road conditions are substandard, according to a national non-profit transportation research group, TRIP.
One YouTuber had some very expensive damage to his performance Model 3 when hitting a pothole:
MIT Pothole Project
In pursuing its pothole mapping task, Tesla might consider getting a little help from friends at MIT’s Concrete Sustainability Hub. A collaboration between MIT faculty and students, UMass Dartmouth, Birzeit University in the West Bank, and the American University of Beirut has produced an app called Carbin. The app measures and tracks road quality while driving and aims to help drivers find the smoothest road on which to travel. Combining GPS navigation and software from Mapbox, a third-party GPS and maps provider, Carbin monitors and records how rough the roads are. The app’s measurement gauge is a familiar one: the accelerometer. A component in most smartphones, it measures velocity changes along one axis. So when the phone shakes a lot, it means the road is bad. Once the app’s servers get repeated data from several drivers on the same lane or road, they analyze the information. Results are uploaded to FixMyRoad.us, which tracks international road quality.
Carbin launched in February 2019, and several hundred users have contributed data for 3,000 recorded trips as of November 2019. Obtaining near real-time information is a boon to mapping road conditions accurately. Apps like Carbin help conserve finances, human resources, and time when compared to the two years a government’s DOT may take to evaluate one section of road using a $250,000 laser-measuring device.
Other Automakers Working on HD Maps
Tesla is not the only automaker employing dynamic HD Maps to potentially avoid things like potholes. Ford is working on a system to help drivers avoid such road hazards and take action to keep damage to a minimum.
Google’s self-driving car division, Waymo, has been working for years to develop the mapping necessary to fully automate its vehicles. The car cameras and sensors gather data, showing items such as fire hydrants and driveways and report it to the mapping crew. Crew members adjust the maps to match the changing environment, such as road work or potholes.
Toyota is working with CARMERA to develop an Automated Mapping Platform (AMP) to support full self-driving using HD maps created with LiDAR plus RGB scans. Uber is also in the race to gather mapping data.
Finally, Mobileye, Intel’s self-driving technology division and one of the leaders in the field, is using what it’s calling Road Experience Management (REM) to constantly collect road data from the fleet and update its maps.
Bottom Line
When Elon Musk says Tesla does not rely on HD Maps, what he really means is that the primary means to navigate the world cannot use HD Maps since the world is dynamic and changes, hence why he deems those maps to be too “brittle.” That said, Tesla Autopilot certainly uses some form of high-precision maps to help navigate surroundings will likely update those maps in almost real-time from the fleet over time, as indicated by his reply around potholes and “micro-maps.”
Learn More
- Should Tesla use LiDAR?
- Tesla Vision – No More Radar!
- Check out Tesla Full Self-Driving Beta
- See what’s coming from Tesla Autonomy Day
- Get a peek at Tesla Autopilot Developer Mode