What happened
On Monday 13 November 2023 Rivian pushed over-the-air update 2023.42 to its vehicles. Ars Technica reported that the update stalled before finishing, taking out both the infotainment and main instrument display screens, and that Rivian was still working on a fix.
Wassym Bensaid, Rivian's vice president of software engineering, posted on the company's subreddit that it had made 'a fat finger where the wrong build with the wrong security certificates was sent out'. He said the campaign had been cancelled and would be restarted with software that had gone through the different beta-testing campaigns. A later update said the issue hit the infotainment system, the rest of the vehicle was still operational in most cases, and a reset or sleep cycle would not fix it.
Bensaid wrote that service would contact impacted customers about resolution options, which might require physical repair in some cases. 'This is on us - we messed up', he said, thanking customers for patience.
What it cost
Infotainment and instrument screens failed on affected vehicles; some needed physical repair. Number of vehicles not disclosed.
Why it happened
The wrong build, signed with the wrong security certificates, went out instead of the version that had completed beta testing, in Rivian's own words.
Once installed, the build stalled before finishing, so affected trucks lost their screens rather than receiving a working update.
A vehicle reset or sleep cycle could not undo it, which pushed some repairs to physical servicing.
The lesson
An over-the-air release path needs a gate that checks the build and certificates before it reaches customers. A fleet that can be fixed from home can also be broken from home.
Aftermath
Rivian cancelled the 2023.42 campaign, planned to restart it with the correctly tested software and had its customer support team prioritise affected customers. Ars Technica reported that Rivian was still working on a fix; the article does not say how many vehicles were affected or how long repairs took.
FOLLOW THE EVIDENCE