Tesla Hardware 3 (AP3) Powers Full Self-Driving Computer via Custom AI

Tesla Custom AI Chip Hardware 3.0

For years it was rumored that Tesla might be building their own AI hardware to power their full self-driving technology after hiring several prominent chip architects, including Jim Keller from AMD (who has since left).  Now with the recent Tesla Autonomy Day for investors, we have the full details around the hardware specs, which we detail below.

Full Self-Driving Computer (aka Hardware 3)

Tesla launched its efforts to build the Full Self-Driving (FSD) Computer, also known as ‘Hardware 3’ (or AP3) back in early 2016 when they realized there were no other good solutions on the market. Two years later, in late 2018, the FSD Computer went into production – amazingly fast for such a project.

Tesla FSD Computer
Tesla FSD Computer (aka Hardware 3)

Previous (AP 2) Hardware Powered by NVIDIA

Previously, up until early 2019, Tesla vehicles, including the Model 3, S and X all used NVIDIA hardware to power the neural network used for Full Self-Driving (or previous Enhanced Autopilot) features, specifically the NVIDIA DRIVE PX 2 AI computing platform.  This was introduced in late 2016 and coined, ‘Hardware 2.0’ or AP2. There was a small update since then that some call ‘Hardware 2.5’ (AP2.5), but Tesla disputes its significance.  Prior to AP2, the original ‘Hardware 1.0’ AP1 cars with first-generation Autopilot capabilities utilized the Mobileye EyeQ 3 platform. Learn more about Autopilot features in this article.

Swappable Computers

Tesla wisely made the AP2 and AP2.5 NVIDIA computing controllers swappable for future upgrades since the chip technology is rapidly advancing, but the sensors, like cameras and radar will remain fairly stable over time. In fact, Tesla is already working on an updated FSD Computer (we’re calling ‘FSD Computer 2’) that should be released in 2021 and will be three times as powerful as the one released in 2019.

Requirements for the FSD Computer

When Tesla was building the new FSD Computer, they had the following requirements:

  • Had to be power efficient and less than 100W
  • Could be retrofitted into AP2 and AP2.5 vehicles
  • Lower part costs to allow redundancy
  • Batch size of one (minimum input batch job)
  • At least 50 TOPs of neural network performance
  • GPU for post-processing
  • Security

Fully Redundant

One important tenant of the FSD Computer, and design of the Tesla cars in general, is that they have fully redundant systems in case of system fails. The FSD Computer actually has two computers running, and is able to immediately shift to over to another if one fails. That’s in addition to redundant power supplies, steering controls throughout the car, etc.

Tesla Full Self-Driving Computer Redundancy
Redundant Computers

Core Components

The FSD Computer has three major sections:

  • Video
  • Computer
  • Power

The FSD Chip and Hardware

Tesla Full Self-Driving Chip

The Tesla Full Self-Driving Chip is manufactured by Samsung for Tesla in the US. It’s a 14-nanometer FinFET CMOS processor that measures 260 millimeters squared and has over 6 billion transistors. They use LPDDR4 RAM that boasts a peak bandwidth of 68 GB/s.

Overall the chip and computer was designed to be somewhat middle-of-the-road in terms of design and performance to both minimize costs and reduce power load. For example it’s using 14nm technology while newer systems use 10nm technology (something Tesla said they’ll use in FSD Computer 2 in the future).

It also has tone mapping (for things like shadows) and noise reduction. In addition, it houses the Neural Network Accelerator, of which there are two per chip running at 36 TOPS each or 72 TOPS combined.

There’s a H.265 video encoder that’s used for the backup camera, dashcam and Sentry Mode.

Security is another place where Tesla has spent a good deal of time, since obviously, having your car hacked could lead to an extremely dangerous situation. There is built-in hardware on the FSD Computer that checks to ensure that only software that is cryptologically signed by Tesla can run.

FSD Computer Power Consumption

As mentioned earlier, Tesla had to be extremely careful around power use and consumption as that produces heat and reduces range, a particular issue for the upcoming Tesla Network / Robo Taxis. While the FSD Computer is much more powerful, it only produces a bit more heat.

Tesla FSD Computer Power Consumption

Sensor Inputs

The inputs from the sensors (perception) come into the FSD as shown here:

Tesla FSD Driving Computer Perception Inputs

Image Processing

Camera and image processing is much more powerful with the FSD Computer than with with Hardware 2 or 2.5. The FSD Computer can process a stunning 2,300 frames-per-second versus 110 frames-per-second with Hardware 2.5.

Compared to NVIDIA

Compared to the competition, in terms of sheer computing power, the comparable NVIDIA Xavier system runs at 21 TOPS, whereas the FSD Computer runs at 144 TOPS, 7X faster.

“Accelerated Mathematical Engine”

Patents filed by Tesla around an “Accelerated Mathematical Engine” say that the hardware required to power self-driving systems require “high-computational-throughput systems and methods that can perform matrix mathematical operations quickly and efficiently.” This is important to ensure the neural network used by Tesla to perform self driving, called a convolutional neural network (CNN), can run efficiently.

The Full Self-Driving Computer (Hardware 3.0 / AP3) improves the processing speed by 10 times, from 200 frames per second to 2,000 frames per second from the car’s onboard cameras. This is critical as Tesla has opted to use a vision-based camera system rather than lidar (see lidar vs cameras) to power its self-driving system.  While cameras are cheaper to deploy and easier to integrate in the vehicle, they require much more processing power to understand the images and environment, which is one of the reasons Tesla had to create their own custom AI hardware. Tesla is pushing the boundaries of AI and self-driving and some consider it more of a software company than a car company in that regard, so to move quickly, in many cases they need to build versus buy technology.

In a recent interview with ARK Invest, Musk said:

If you want to have a complex neural network, you need to have a combination of software and hardware. And your software needs to be that much better in order to compensate for hardware’s (limitations). Sort of like, you have video games and how they’ve progressed — it’s a combination of software and hardware. No amount of clever software could produce a video game on old hardware that you have today. It doesn’t matter, you know. It’s the same thing with neural nets.

Elon also elaborated in that same interview:

So right now, we can process on the order of 100 frames a second and we really need to do a lot of work in terms of cropping the frames, and sort of bending the pixels, and not going to full resolution on all cameras, that kind of thing with the current hardware. We’re at full frames, full resolution with the Tesla hardware. All cameras, at full resolution, full frames, and it still hasn’t tapped out.

Same Cost to Produce

Tesla said that the cost of the new hardware is the same as the previous version despite the huge processing gains. Once the Full Self-Driving Computer (Hardware 3.0 / AP3) arrives in 2019, owners of older models can swap out for new computers if they have paid for the Full Self-Driving option. It’s a relatively simple process to update the boards as all the connectors are the same.

When is It Coming?

As of mid-April, all new Tesla vehicles are shipping with the FSD Computer. For those with current vehicles, looking to upgrade Elon tweeted that it may be some time for upgrades since the software must be updated to take full advantage of it.

The Future – ‘FSD Computer 2’

Elon mentioned that they are working on a future FSD Computer (what we’re calling ‘FSD Computer 2’) that will be roughly three times as powerful and the current one and will use 10nm technology (vs. 14nm in the current FSD driving computer). They are about halfway through the four year development cycle on the FSD Computer 2 and it should be ready by 2021.

Our Take

We’re excited to see Tesla continuing to innovate and develop novel software and hardware solution in order to bring the vision of true autonomous Full Self-Driving to consumers. 2019 and 2020 should be an exciting years for Tesla self-driving capabilities!

Learn More