Tesla released its own AI self-driving hardware in 2019, dubbed Hardware 3.0 (or AP3) to power its Full Self-Driving (FSD) technology after hiring several prominent chip architects, including Jim Keller from AMD (who has since left). At the Tesla Autonomy Day 2019 for investors, Tesla detailed what’s now called the Full Self-Driving Computer (or FSD Computer for short), including all the hardware specs, which we outline below. In addition, we’ll also touch on the future Hardware 4.
We condensed the Hardware portion of Autonomy Day, which is fascinating, to 11 minutes here:
Full Self-Driving Computer (aka Hardware 3)
Tesla launched its efforts to build the Full Self-Driving (FSD) Computer, also known as ‘Hardware 3’ (or AP3) back in early 2016 when they realized there were no other good solutions on the market. Two years later, in late 2018, the FSD Computer went into production – amazingly fast for such a project.
Previous (AP 2) Hardware Powered by NVIDIA
Previously, up until early 2019, Tesla vehicles, including the Model 3, S and X all used NVIDIA hardware to power the neural network used for Full Self-Driving (or previous Enhanced Autopilot) features, specifically the NVIDIA DRIVE PX 2 AI computing platform. This was introduced in late 2016 and coined, ‘Hardware 2.0’ or AP2. There was a small update since then that some call ‘Hardware 2.5’ (AP2.5), but Tesla disputes its significance. Prior to AP2, the original ‘Hardware 1.0’ AP1 cars with first-generation Autopilot capabilities utilized the Mobileye EyeQ 3 platform.
For more on the Tesla Autopilot hardware history and how to tell the difference between cars see: Tesla Autopilot AP1 vs AP2 vs AP3.
Swappable Computers
Tesla wisely made the AP2 and AP2.5 NVIDIA computing controllers swappable for future upgrades since the chip technology is rapidly advancing, but the sensors, like cameras and radar will remain fairly stable over time (although Tesla has since removed radar, relying only on cameras now, called Tesla Vision).
In fact, Tesla is already working on an updated FSD Computer (we’re calling ‘FSD Computer 2’) that should be released in late 2021 and will be three times as powerful as the one released in 2019.
Requirements for the FSD Computer
When Tesla was building the new FSD Computer, they had the following requirements:
- Had to be power efficient and less than 100W
- Could be retrofitted into AP2 and AP2.5 vehicles
- Lower part costs to allow redundancy
- Batch size of one (minimum input batch job)
- At least 50 TOPs of neural network performance
- GPU for post-processing
- Security
Fully Redundant
One important tenant of the FSD Computer, and design of the Tesla cars in general, is that they have fully redundant systems in case of the systems fails. The FSD Computer actually has two computers running, and is able to immediately shift over to another if one fails. That’s in addition to redundant power supplies, steering controls throughout the car, etc.
Core Components
The FSD Computer has three major sections:
- Video
- Computer
- Power
The FSD Chip and Hardware
The Tesla Full Self-Driving Chip is manufactured by Samsung for Tesla in the US. It’s a 14-nanometer FinFET CMOS processor that measures 260 millimeters squared and has over 6 billion transistors. They use LPDDR4 RAM that boasts a peak bandwidth of 68 GB/s.
Overall the chip and computer was designed to be somewhat middle-of-the-road in terms of design and performance to both minimize costs and reduce power load. For example it’s using 14nm technology while newer systems use 10nm technology (something Tesla said they’ll use in FSD Computer 2 in the future).
It also has tone mapping (for things like shadows) and noise reduction. In addition, it houses the Neural Network Accelerator, of which there are two per chip running at 36 TOPS each or 72 TOPS combined.
There’s an H.265 video encoder that’s used for the backup camera, dashcam and Sentry Mode.
Security is another place where Tesla has spent a good deal of time since obviously, having your car hacked could lead to an extremely dangerous situation. There is built-in hardware on the FSD Computer that checks to ensure that only software that is cryptologically signed by Tesla can run.
FSD Computer Power Consumption
As mentioned earlier, Tesla had to be extremely careful around power use and consumption as that produces heat and reduces range, a particular issue for the upcoming Tesla Network / Robo Taxis. While the FSD Computer is much more powerful, it only produces a bit more heat.
Sensor Inputs
The inputs from the sensors (perception) come into the FSD as shown here:
Image Processing
Camera and image processing is much more powerful with the FSD Computer than with Hardware 2 or 2.5. The FSD Computer can process a stunning 2,300 frames-per-second versus 110 frames-per-second with Hardware 2.5.
Compared to NVIDIA
Compared to the competition, in terms of sheer computing power, the comparable NVIDIA Xavier system runs at 21 TOPS, whereas the FSD Computer runs at 144 TOPS, 7X faster.
“Accelerated Mathematical Engine”
Patents filed by Tesla around an “Accelerated Mathematical Engine” say that the hardware required to power self-driving systems require “high-computational-throughput systems and methods that can perform matrix mathematical operations quickly and efficiently.” This is important to ensure the neural network used by Tesla to perform self-driving, called a convolutional neural network (CNN), can run efficiently.
The Full Self-Driving Computer (Hardware 3.0 / AP3) improves the processing speed by 10 times, from 200 frames per second to 2,000 frames per second from the car’s onboard cameras. This is critical as Tesla has opted to use a vision-based camera system rather than lidar (see lidar vs cameras) to power its self-driving system. While cameras are cheaper to deploy and easier to integrate in the vehicle, they require much more processing power to understand the images and environment, which is one of the reasons Tesla had to create their own custom AI hardware. Tesla is pushing the boundaries of AI and self-driving and some consider it more of a software company than a car company in that regard, so to move quickly, in many cases they need to build versus buy technology.
In an interview with ARK Invest, Musk said:
If you want to have a complex neural network, you need to have a combination of software and hardware. And your software needs to be that much better in order to compensate for hardware’s (limitations). Sort of like, you have video games and how they’ve progressed — it’s a combination of software and hardware. No amount of clever software could produce a video game on old hardware that you have today. It doesn’t matter, you know. It’s the same thing with neural nets.
Elon also elaborated in that same interview:
So right now, we can process on the order of 100 frames a second and we really need to do a lot of work in terms of cropping the frames, and sort of bending the pixels, and not going to full resolution on all cameras, that kind of thing with the current hardware. We’re at full frames, full resolution with the Tesla hardware. All cameras, at full resolution, full frames, and it still hasn’t tapped out.
Same Cost to Produce
Tesla said that the cost of the new hardware is the same as the previous version despite the huge processing gains. Once the Full Self-Driving Computer (Hardware 3.0 / AP3) arrives in 2019, owners of older models can swap out for new computers if they have paid for the Full Self-Driving option. It’s a relatively simple process to update the boards as all the connectors are the same.
Which Vehicles Get Hardware 3?
As of mid-April 2019, all new Tesla vehicles started shipping with the FSD Computer. For those with older vehicles (Hardware 2 or greater) who have purchased FSD, upgrades began rolling out in earnest in Q1 of 2020.
Hardware 4 (aka ‘FSD Computer 2’)
During Tesla AI Day in 2021, Elon mentioned that they are working on a future FSD Computer (called ‘FSD Computer 2’ or Hardware 4) that will be roughly three times as powerful and the current one and will use 10nm technology (vs. 14nm in the current FSD driving computer).
Tesla is now rolling out Hardware 4, which not only includes FSD Computer 2 but also a new camera suite in addition to a new upgraded radar unit Tesla previously removed to focus on camera-only Autopilot, dubbed Tesla Vision.
Our Take
We’re excited to see Tesla continuing to innovate and develop novel software and hardware solution in order to bring the vision of true autonomous Full Self-Driving to consumers. 2022 and beyond should be exciting years for Tesla self-driving capabilities, especially as Hardware 4 is widely released in 2023 with the Cybertruck.
Learn More
- See Tesla Autonomy Day for a deep-dive into the tech
- Tesla Autonomous Full Self Driving Capabilities
- AP1 vs AP2 vs AP3 – What are the Differences?
- Hardware 3 Upgrades for Older Vehicles