Fairly Comparing Autonomous Vehicle Systems Isn’t Really Possible

Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!

If you’ve been following autonomous vehicle development, you’ve probably come across this chart that tries to compare the systems (which the poster on X/Twitter was re-using as a joke, so don’t go heckle him!). It was seriously posted by the people mentioned in this article, though. Using vague descriptions like “execution” and “strategy,” the team of experts tried to sort the autonomous vehicle (AV) space into leaders, contenders, challengers, and followers. Tesla was placed in the “followers” category.

Tesla FSD is gonna be dead last, they will most likely kill the program anytime now. pic.twitter.com/rQS6Drz2Nb

— faizur rehman (@faizur_rah) March 29, 2024

If you’re an FSD Beta user, the chart made by these “experts” might be making blood shoot out of your eyes, especially as impressed as owners seem to be with the latest release. But, seeing that reaction from people made me realize that there’s a fundamental problem with the chart itself: a lack of objective criteria. And, when I tried to mentally sort the players myself using objective criteria, I realized that this was impossible.

So, in the end, my conclusion is that it’s really not possible to make fair comparisons at this point, but it’s going to take an article to explain why.

Problem #1: Vastly Differing Design Goals

If you’re of the opinion that Tesla is the far-and-away leader of the pack, that’s probably because you can see its value as a general purpose system. FSD Beta works everywhere, even if it’s not perfect yet. You can take the system on nearly any street, road, or highway (or stroad) and it will perform roughly the same as it does anywhere. So, if it’s a “works almost anywhere” system that we choose to use as a goalpost, Tesla is closer to getting the ball in than anybody else.

But, if we look at systems like Ford’s Blue Cruise and GM’s Super Cruise, there are some things that they beat Tesla at. Blue Cruise can’t drive unsupervised, but it can drive without you needing to put your hands on the wheel all the time (or cheating and using an orange). The problem is that these systems have a big limit: they only work on certain highways that have been mapped by Ford or GM. But, in these limited areas, they work quite well. So, if you’re looking for a system that works very well on the highway, you could argue that Ford and GM are ahead of Tesla.

There’s a similar issue trying to compare systems from Cruise, Waymo, and Baidu. These systems really suck in that they can’t be used anywhere and there’s no national coverage along major highways, either. If you could buy a Waymo van, it wouldn’t be able to take you on a road trip, so that means it sucks compared to Tesla, Ford, GM, and almost everyone else. But the Waymo van can operate without a driver at all in the small geographic areas that it was designed to operate in. So, if it’s a truly driverless, unattended system you’re looking for, Waymo is the clear winner.

Given the very different development tracks these systems are all on, it’s hard to really compare them. Who’s ahead really depends on what you want and what you expect. Depending on what your priorities are, anybody could be the winner and anybody could be the loser. You could easily cherry-pick criteria to make anyone be the winner or loser, too (and people do that).

Chip in a few dollars a month to help support independent cleantech coverage that helps to accelerate the cleantech revolution!

Problem #2: Lack of Predictable Development

The version of the chart that was shared above has another problem: it was made before GM’s Cruise division ran into big problems last year. Not long after they declared Cruise to the a leader, a cute little Cruise car got into an accident that wasn’t really its fault. Some guy in a pickup hit a pedestrian and threw them into the path of the Cruise vehicle, which then wasn’t able to tell that it had gone over someone. It then dragged the pedestrian under the vehicle for several yards before coming to a safe stop.

Overnight, Cruise went from a clear winner in the local geofenced and mapped robotaxi market to being banned in California. We’ve seen similar things happen to other companies. The most notable example was Uber’s self-driving vehicle efforts that ended suddenly in 2018 when a woman trying to cross a dark section of road was struck by a development vehicle. Who was really to blame was never firmly established, as Uber managed to evade responsibility and the safety driver took a plea deal for probation, but there are no Uber self-driving test vehicles now.

So at any time, a company that is in the lead even in a restricted set of criteria could suddenly go away. It’s like running a race with a sniper taking aim at the track. Sure, someone may appear to be winning, but they could very easily fall over dead at any moment, so you don’t know that for sure.

At the same time, we never really know who might pull ahead from the rear suddenly and win. The nature of artificial neural networks and development of AV systems is such that even Elon Musk is constantly saying solving it is just around the corner. For years, he even promised big things that year, only to do it again the next year and the next.

Eventually, one of these companies is going to make a breakthrough and solve the problem well enough to make a car that’s vastly safer than human drivers and perfect enough for people to trust with their lives. More importantly, a system will eventually come that a company is willing to take legal responsibility for (that’s the real point when someone “wins”) and that works in enough places to be useful.

We don’t know whether the geofenced approach will break out of the geofence. Or, will the general purpose approach improve everywhere at the same time enough to beat everyone? Or, will Blue Cruise break out of the highways and onto the city streets and win first? Anybody claiming to know the answer to this is kidding themselves.

An Unpredictable Game

Given that the different systems are taking such different roads to the finish line, with the possibility of losing badly or suddenly skipping to the end always there, nobody can really say who’s winning right now. Nobody can say that everyone will finish the race, or even say that anyone will achieve it.