Tesla Autonomous Full Self-Driving Capabilities in 2020

Tesla currently offers two autopilot features: standard Autopilot and Full Self-Driving (FSD). Autopilot offers basic driver assistance functions including distance cruise control and lane centering functions, while Full Self-Driving, an optional paid feature, has more sophisticated “semi-autonomous” features like automated lane changes and Navigate on Autopilot, with the promise of fully autonomous driving in the future.

You can read more about the differences in detail in our Autopilot vs Full Self-Driving review.

Tesla Autonomous Driving Progress

In this article we chronicle the progress of Tesla’s Full Self-Driving capabilities towards full autonomy. Elon Musk has pushed his team to make rapid progress in the area and even claimed during the Autonomy Investor Day back in April 2019 that autonomous Full Self-Driving would be ‘feature complete’ by the end of that year. While that didn’t happen, they did release a Full Self-Driving ‘sneak preview’ that showcased how the Full Self-Driving Computer (aka Hardware 3) can now recognize and visualize road markings.

That said, Tesla has consistently pushed the envelope in consumer autonomous driving technology while auto manufacturers have struggled to determine the best path forward, who to partner with for technology or even whether to use LiDAR vs cameras for perception. Yes there are cars with autopilot features, but no one to date, has matched Tesla’s progress.

Autonomy Day Full Self-Driving Demo

Back in April, Tesla treated the media and investors to the first glimpse of a fully autonomous drive in a Model 3 using pre-release development code. The vehicle performed a loop around their Palo Alto headquarters covering both freeway and city routes while the driver had the hands fully off the wheel. They even released a video (see below).

Autonomy Day Demo vs Reality

While the 2019 Autonomy Day self-driving demo is impressive, we decided to drive the same route using the Full Self-Driving ‘sneak preview’ provided by Tesla at the end of December 2019 (software update 2019.40.50.1) to see how far the public release has come, particularly in comparison to the demo.

Here are the results with summary below:

Full Self-Driving Autonomous Driving Capabilities

Detailed below is the current state of some key elements needed for autonomous full self-driving and their status with Tesla’s Full Self-Driving as available today.

FSD is NOT Autonomous nor Driverless

First off, none of the current Tesla driver assistance features such as Autopilot or Full Self-Driving are actually driverless. They require that the driver pay attention and keep hands on the wheel at all times. This is especially important since the system may at times make sudden lane adjustments that are not anticipated so hands on the wheel to detect and correct these movements is critical.

Autosteer – Excellent but Not Autonomous

First off, the good news. On the freeway, and even city streets, Tesla’s dynamic cruise control, lane centering (Autosteer), traffic control handling, and automated lane changes are far beyond what other cars with autopilot offer. With the new visualizations and Autopilot improvements released in 2019, closed-access freeway driving is very close to the Tesla Autonomy Day driving demo, especially if you have the Full Self-Driving package with Hardware 3. That said there are still too many rough edges where a human still needs to pay attention, including lane changes, merges, construction zones, and sudden stationary objects appearing in the road (see Autopilot Crashes and Causes).

Tesla Autopilot on Freeway
Tesla Autopilot on Freeway (Autonomy Day demo screen on right)

Point-to-Point Navigation – Freeway Only for Now

The Navigate on Autopilot feature included with the Full Self-Driving option allows drivers to navigate from one point to another, fully autonomously with driver supervision, as long as the route is closed-access (i.e. no cross traffic), such as freeways. It will navigate freeway interchanges and automatically exit the freeway as well.

City Driving – Tesla’s Big Challenge

As Elon already stated during Autonomy Day, autonomous city driving is the next frontier that Tesla is focused on to achieve true Full Self-Driving. City driving is especially challenging given the variety of road, signals, traffic and pedestrians, etc. occupying the same space in an almost infinite number of situations. We break down various components of city driving below.

Perceiving the Environment

Critical to autonomous driving is environmental perception and understanding various objects. As shown here, there are a multitude of objects the Tesla neural network must perceive and understand, including both stationary and moving objects.

Tesla Neural Network Environment Tags
Tesla Neural Network Environment Tags (Source: Tesla)

Drivable Space Perception

The first step in city driving has been to confidently understand lane and curb markings to ensure the vehicle knows what is drivable space. This is fairly good and far better than other cars with autopilot can achieve.

Example of Drivable Space visualization (from Tesla’s demo vehicle on right):

Tesla Autopilot Drivable Space
Tesla Autopilot Drivable Space (Autonomy Day demo screen on right)

Perceiving Road Markings, Street Signs and Signals

Critical to autonomous driving in an urban environment is the perception and reaction to road marketing, street signs and signals. With the Full Self-Driving ‘Sneak Preview’ we get a first glimpse into how the Hardware 3 (the Full Self-Driving Computer) is utilizing the neural network to classify and display the road markings and signs it recognizes.

Tesla Autopilot Street Sign Visualizations
Tesla Autopilot Street Sign Visualizations

Now, with the Traffic Light and Stop Sign Control feature released in mid-2020, Tesla vehicles with the optional Full Self-Driving package can respond to traffic controls on city streets such as traffic lights and stop signs.

Perceiving Other Moving Objects

Another crucial step in city driving is the perception of surrounding objects and correctly categorizing them, including other vehicles, cyclists, pedestrians, etc. In addition to identifying those objects the system must also predict their path in order to react appropriately, requiring a notion of how those objects are moving in time.

Currently the Tesla system seems to recognize objects fairly consistently, including cyclists, pedestrians, etc. However, it does not yet react to them and expects the driver to take control most of the time, especially in city driving. Additionally, Tesla Autopilot will not take action if there is an object within the lane at highway speeds (see Autopilot Crashes and Causes).

Here’s a great example of a autonomous driving startup, called Zoox, and how they recognize objects and pedestrians:

Navigating Intersections

Being able to navigate through an intersection is already hard enough for humans, and even more so for autopilot and self-driving systems, currently. While the Tesla Autonomy Day demo seemed to handle very basic and simple intersections, current Full Self-Driving doesn’t handle them at all and it still remains to be seen what sorts of intersections Tesla can navigate in 2020.

Below you can see how the Tesla Autonomy Day vehicle sees the intersection via drivable space.

Tesla Autopilot Intersection Recognition
Tesla Autopilot Intersection Recognition (Autonomy Day demo screen on right)

Taking Sharp Turns and Roundabouts

Handling turns, especially sharp turn and those turns that tighten as they progress can be challenging for autopilot systems as they must anticipate the turn and slow the car down appropriately. Most cars with autopilot do not handle this situation well, however, Tesla’s Autopilot does a very admirable job, doing far better than most, however, still not perfect and requiring driver attention.

Here’s a particularly tricky turn that was used during Autonomy Day. This turn is taken from highway 280 North in California onto Sand Hill Road West (current Autopilot versions often disengage on the tail end of this turn).

Tesla Autopilot on Tight Turns
Tesla Autopilot on Tight Turns (Autonomy Day demo screen shown on right)

Parking Lot Navigation

Another very challenging situation is parking lots. These areas are often congested with cars and pedestrians in addition to having very tight maneuvering spaces.

Currently Tesla offers the “Smart Summon” feature of the Full Self-Driving package that allows Tesla vehicles to drive themselves to the owner who is standing nearby.

Tesla also showcased parking lot navigation way back in 2016 with its first autonomous driving video, the famous “Paint it Black” video shown below (see more Tesla autonomous demonstration videos here):

It remains to be seen how well parking can be integrated into the autonomous experience by bridging the current Smart Summon feature with Navigate on Autopilot.

Our Take

Tesla and Elon Musk have placed a big target on delivering autonomous Full Self-Driving and even a Tesla RoboTaxi service in 2020. By most accounts, that’s extremely ambitious and it doesn’t seem close-at-hand… yet.

Most likely we’ll continue to see gradual improvements from Tesla Autopilot and Full Self-Driving that help drivers relieve the stress of driving and improve safety along the way.

Learn More