Tesla currently offers two autopilot features: standard Autopilot and Full Self-Driving (FSD). Autopilot offers basic driver assistance functions including distance cruise control and lane centering functions, while Full Self-Driving, an optional paid feature, has more sophisticated “semi-autonomous” features like automated lane changes and Navigate on Autopilot, with the promise of fully autonomous driving in the future.
You can read more about the differences in detail in our Autopilot vs Full Self-Driving review.
Tesla Autonomous Driving Progress
In this article we chronicle the progress of Tesla’s Full Self-Driving capabilities towards full autonomy. Elon Musk has pushed his team to make rapid progress in the area and even claimed during the Autonomy Investor Day back in April 2019 that autonomous Full Self-Driving would be ‘feature complete’ by the end of that year. While that didn’t happen, they did release a Full Self-Driving ‘sneak preview’ that showcased how the Full Self-Driving Computer (aka Hardware 3) can now recognize and visualize road markings.
That said, Tesla has consistently pushed the envelope in consumer autonomous driving technology while auto manufacturers have struggled to determine the best path forward, who to partner with for technology or even whether to use LiDAR vs cameras for perception. Yes there are cars with autopilot features, but no one to date, has matched Tesla’s progress.
Autonomy Day Full Self-Driving Demo
Back in April, Tesla treated the media and investors to the first glimpse of a fully autonomous drive in a Model 3 using pre-release development code. The vehicle performed a loop around their Palo Alto headquarters covering both freeway and city routes while the driver had the hands fully off the wheel. They even released a video (see below).
Autonomy Day Demo vs Reality
While the 2019 Autonomy Day self-driving demo is impressive, we decided to drive the same route using the Full Self-Driving ‘sneak preview’ provided by Tesla at the end of December 2019 (software update 2019.40.50.1) to see how far the public release has come, particularly in comparison to the demo.
Here are the results with summary below:
Full Self-Driving Autonomous Driving Capabilities
Detailed below is the current state of some key elements needed for autonomous full self-driving and their status with Tesla’s Full Self-Driving as available today.
FSD is NOT Autonomous nor Driverless
First off, none of the current Tesla driver assistance features such as Autopilot or Full Self-Driving are actually driverless. They require that the driver pay attention and keep hands on the wheel at all times. This is especially important since the system may at times make sudden lane adjustments that are not anticipated so hands on the wheel to detect and correct these movements is critical.
Autosteer – Excellent but Not Autonomous
First off, the good news. On the freeway, and even city streets, Tesla’s dynamic cruise control, lane centering (Autosteer), traffic control handling, and automated lane changes are far beyond what other cars with autopilot offer. With the new visualizations and Autopilot improvements released in 2019, closed-access freeway driving is very close to the Tesla Autonomy Day driving demo, especially if you have the Full Self-Driving package with Hardware 3. That said there are still too many rough edges where a human still needs to pay attention, including lane changes, merges, construction zones, and sudden stationary objects appearing in the road (see Autopilot Crashes and Causes).
Point-to-Point Navigation – Freeway Only for Now
The Navigate on Autopilot feature included with the Full Self-Driving option allows drivers to navigate from one point to another, fully autonomously with driver supervision, as long as the route is closed-access (i.e. no cross traffic), such as freeways. It will navigate freeway interchanges and automatically exit the freeway as well.
City Driving – Tesla’s Big Challenge
As Elon already stated during Autonomy Day, autonomous city driving is the next frontier that Tesla is focused on to achieve true Full Self-Driving. City driving is especially challenging given the variety of road, signals, traffic and pedestrians, etc. occupying the same space in an almost infinite number of situations. We break down various components of city driving below.
Perceiving the Environment
Critical to autonomous driving is environmental perception and understanding various objects. As shown here, there are a multitude of objects the Tesla neural network must perceive and understand, including both stationary and moving objects.
Drivable Space Perception
The first step in city driving has been to confidently understand lane and curb markings to ensure the vehicle knows what is drivable space. This is fairly good and far better than other cars with autopilot can achieve.
Example of Drivable Space visualization (from Tesla’s demo vehicle on right):
Perceiving Road Markings, Street Signs and Signals
Critical to autonomous driving in an urban environment is the perception and reaction to road marketing, street signs and signals. With the Full Self-Driving ‘Sneak Preview’ we get a first glimpse into how the Hardware 3 (the Full Self-Driving Computer) is utilizing the neural network to classify and display the road markings and signs it recognizes.
Now, with the Traffic Light and Stop Sign Control feature released in mid-2020, Tesla vehicles with the optional Full Self-Driving package can respond to traffic controls on city streets such as traffic lights and stop signs.
Perceiving Other Moving Objects
Another crucial step in city driving is the perception of surrounding objects and correctly categorizing them, including other vehicles, cyclists, pedestrians, etc. In addition to identifying those objects the system must also predict their path in order to react appropriately, requiring a notion of how those objects are moving in time.
Currently the Tesla system seems to recognize objects fairly consistently, including cyclists, pedestrians, etc. However, it does not yet react to them and expects the driver to take control most of the time, especially in city driving. Additionally, Tesla Autopilot will not take action if there is an object within the lane at highway speeds (see Autopilot Crashes and Causes).
Here’s a great example of a autonomous driving startup, called Zoox, and how they recognize objects and pedestrians:
Navigating Intersections
Being able to navigate through an intersection is already hard enough for humans, and even more so for autopilot and self-driving systems, currently. While the Tesla Autonomy Day demo seemed to handle very basic and simple intersections, current Full Self-Driving doesn’t handle them at all and it still remains to be seen what sorts of intersections Tesla can navigate in 2020.
Below you can see how the Tesla Autonomy Day vehicle sees the intersection via drivable space.
Taking Sharp Turns and Roundabouts
Handling turns, especially sharp turn and those turns that tighten as they progress can be challenging for autopilot systems as they must anticipate the turn and slow the car down appropriately. Most cars with autopilot do not handle this situation well, however, Tesla’s Autopilot does a very admirable job, doing far better than most, however, still not perfect and requiring driver attention.
Here’s a particularly tricky turn that was used during Autonomy Day. This turn is taken from highway 280 North in California onto Sand Hill Road West (current Autopilot versions often disengage on the tail end of this turn).
Parking Lot Navigation
Another very challenging situation is parking lots. These areas are often congested with cars and pedestrians in addition to having very tight maneuvering spaces.
Currently Tesla offers the “Smart Summon” feature of the Full Self-Driving package that allows Tesla vehicles to drive themselves to the owner who is standing nearby.
Tesla also showcased parking lot navigation way back in 2016 with its first autonomous driving video, the famous “Paint it Black” video shown below (see more Tesla autonomous demonstration videos here):
It remains to be seen how well parking can be integrated into the autonomous experience by bridging the current Smart Summon feature with Navigate on Autopilot.
Our Take
Tesla and Elon Musk have placed a big target on delivering autonomous Full Self-Driving and even a Tesla RoboTaxi service. By most accounts, that’s extremely ambitious and it doesn’t seem close-at-hand… yet.
That said, the latest Full Self-Driving Beta is showing some impressive improvements. We expect the full stack version (V11), launching in 2023 to continue helping drivers relieve the stress of driving and improve safety along the way.
Learn More
