In November 2017, Waymo GOOG announced it was testing autonomous cars in Phoenix, without drivers behind the steering wheels but with safety engineers in the back seats. Next, the company plans to commercialize a ridesharing service.
According to data from the California Department of Motor Vehicles,1 Waymo’s cars are improving rapidly enough to support a commercial launch within the next two years. The same data, however, also suggest that Waymo will face challenges, perhaps explaining its decision to put engineers in the back seat. Nonetheless, competition is brewing, reason enough to launch early to achieve first mover advantage in a market that ARK estimates will generate trillions of dollars in revenue by the 2030s.
ARK classifies autonomous car failures in two categories:
(1) System Identified Failures (SIFs), or incidents during which the autonomous system is confused and knows it’s confused, signaling to a remote operator for help; and
(2) Unexpected Failures (UFs), or incidents during which the autonomous system is confused but does not know it’s confused, continuing to drive and potentially crashing in the absence of human intervention.
ARK envisions that while a remote network of third party operators will be able to direct autonomous taxis in the event of System Identified Failures, it will face significant challenges in anticipating and responding to Unexpected Failures. To prevent or address UFs, remote operators would have to monitor each autonomous taxi continuously which, even if possible, would add substantially to the per-mile cost of an autonomous service. Network latency and other technical issues would prevent even the most diligent operators from intervening in time to prevent most if not all failures. Consequently, Waymo and other service providers must drive down the UF rate substantially before autonomous taxi networks commercialize.
What is an Acceptable Error Rate for Autonomous Cars?
To estimate the rate at which passengers will tolerate autonomous taxi errors, we analyzed the manually driven car statistics to set the hurdle. On average human driven cars break down roughly once every 50,000 miles and crash once every 240,000 miles,2 thus offering perspective on acceptable tolerance rates for autonomous vehicle SIFs and UFs, respectively, as shown below.
To benchmark the progress of autonomous technology, ARK examined Waymo’s California fleet which, as measured by failure rates,3 is the most advanced autonomous system in the US, if not the world.
Today Waymo’s autonomous test cars can drive for roughly 5,000 miles on average with no need for human intervention. If Waymo were to commercialize its autonomous efforts today, therefore, its riders probably would face and balk at a service with two or more breakdowns per year, especially because their personal cars are ten times better by that measure. From 2015 to 2016, however, Waymo tripled the number of miles between interventions, a promising rate of improvement. If the progress were to continue at that rate, and even factoring in a slight decay to account for the increasing difficulty of unsolved issues, Waymo’s cars should be ready for commercial deployment by 2019, as shown below.
Why Has Waymo’s Error Rate Stalled Recently?
During the last eighteen months, the improvement in Waymo’s disengagement rate has stalled. According to data submitted to the California DMV, UFs—the errors most difficult to manage—have not improved in line with SIFs, signaling perhaps that Waymo’s car could be further from commercialization than the disengagement rate’s longer-term trajectory had suggested in mid 2016. Supporting this hypothesis, its cars seem to have had difficulty making left turns.4 One possible explanation is that it has chosen not to vertically-integrate, outsourcing vehicle production to partners like Fiat Chrysler and then taking engineering shortcuts by integrating its sensor suite into a product manufactured away from its controls. In contrast, Tesla’s and Cruise Automation’s (GM) manufacturing operations are vertically-integrated, which could become an important source of competitive advantage.
While Waymo could hit the 50,000 mile SIF rate in 2018, it probably will miss the more critical 240,000 mile UF parity car crash rates, as shown in the two charts below. In other words, based on recent evidence alone, critics could conclude that autonomous taxis will not be ready for prime time until the early 2020’s, if then.
Note: The dip in miles system identified failures may be due to Waymo testing cars in new, unfamiliar geographies. Waymo started testing in Kirkland, Washington in early 2016 and expanded testing efforts in Arizona throughout 2016.5
We are skeptical of that negative conclusion for a number of reasons. Today, Waymo probably is trying to maximize its failure rate to identify faults and root them out. Some stretches of road are trickier and some intersections more difficult to navigate than others. In Los Angeles, for example, roughly a quarter of pedestrian collisions take place at only 1% of its intersections.6 By testing vehicles in the most challenging venues, Waymo probably is experiencing failure rates much higher than would otherwise be the case. Moreover, only in California does the Department of Motor Vehicles require the reporting of interventions. Waymo is testing cars in Washington, Arizona, and Texas, where the geographies and test results could be much different. Finally, Waymo is likely to launch its commercial service in safer geographies and terrains, gathering lots of data on miles traveled with which to train and improve the service. Chandler, Arizona, a suburb of Phoenix, seems ideal for a pilot test, with good weather, simple roads, and limited government oversight.7
Without a doubt, the stakes are high. Tesla,TSLA BaiduBIDU and GMGM all have plans to launch autonomous services within the next two years. The company first to market could accrue an insurmountable data advantage which, given a market opportunity worth trillions of dollars, makes the risks worth taking sooner rather than later.
Cover Image: Uploaded by Grendelkhan is licensed under CC BY-SA 4.0
- https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous+/testing ↩
- http://www.vtti.vt.edu/PDFs/Automated%20Vehicle%20Crash%20Rate%20Comparison%20Using%20Naturalistic%20Data_Final%20Report_20160107.pdf http://www.telegraph.co.uk/motoring/road-safety/8510497/One-in-three-cars-will-break-down-over-the-next-12-months.html http://www.nhtsa.gov/cars/rules/rulings/TPMS_FMVSS_No138/part5.5.html ↩
- Note that Tesla has collected five billion miles worth of data from customer cars compared to Waymo’s millions of miles worth of data. Tesla uses this data to validate and test autopilot instead of using a fleet of test cars. Tesla’s data is likely much less detailed than Waymo’s as Waymo has been using LiDAR sensors on its test vehicles. ↩
- https://www.theinformation.com/waymos-foes-left-turns-and-the-mean-streets-of-phoenix ↩
- https://techcrunch.com/2016/02/03/google-expands-its-self-driving-car-pilot-to-kirkland/ https://www.bizjournals.com/phoenix/news/2016/12/15/ducey-test-drives-waymos-self-driving-car-in.html ↩
- http://graphics.latimes.com/la-pedestrians/ ↩
- https://www.wired.com/story/waymo-google-arizona-phoenix-driverless-self-driving-cars/ ↩
ARK's statements are not an endorsement of any company or a recommendation to buy, sell or hold any security. For a list of all purchases and sales made by ARK for client accounts during the past year that could be considered by the SEC as recommendations, click here. It should not be assumed that recommendations made in the future will be profitable or will equal the performance of the securities in this list. For full disclosures, click here.