The problem
Sunswift 7's embedded system is a centralized network: an internet-connected central computer (an ADLink ROSCube running ROS2) makes all the decisions, and 8+ STM32G4-based CAN nodes spread throughout the car act purely as sensors and actuators. Easy to extend, but it does mean that changing any node's behavior, even something as small as how often a sensor sends readings, means reflashing its firmware.
And here's the catch: at the time, reflashing meant physically digging the node out of some hard-to-reach corner of a car, cracking open its housing, hooking up an ST-LINK programmer and flashing it from a laptop. On a car under active development, that was quite the hurdle. Basically impossible during a race. So for my honors thesis I built a fully automated FOTA (firmware over the air) system. Push code to GitHub, and it ends up running on a microcontroller in the car with no human touching anything in between.
The build pipeline
The cloud side is a pretty classic pipeline: push to the firmware repo, GitHub Actions kicks off AWS CodeBuild, and CodeBuild compiles every node's STM32CubeIDE project inside a Docker image with the toolchain preinstalled. A build script loops through each project, builds it headlessly, and collects the binaries.
Alongside the binaries, the build spits out a build_info.json with
each binary's MD5 hash and its node ID. The ID gets scraped straight out of the
bootloader config in the source code, because it would otherwise be lost in the build.
Everything lands in an S3 bucket organized by Git commit hash, so every binary
traces back to the exact commit that produced it. Fun side quest: bumping the
build machine from 2 to 8 virtual cores cut the six-project build from 3m40s to
2m23s, a 35% improvement for changing one setting.
Delivery over the CAN bus
The microcontrollers are only connected together on a singular CAN bus. Bolting a radio onto every node would've meant hardware changes, power budget, and extra mass on a vehicle obsessed with efficiency. So updates ride the CAN bus that already connects every node.
Updates are pull-based. A Python "Firmware Manager" I wrote for the car's central
computer resolves a Git branch or tag to a commit hash, grabs that commit's
build_info.json from S3, and compares each binary's MD5 hash against
what was last flashed. Only changed binaries get downloaded and flashed, since
reflashing identical firmware would just waste the bus. The actual flashing goes
through LibOpenBLT, the driver for an open source bootloader, driving the bootloader on the target node frame by frame over
CAN. The manager is exposed to the rest of the car's software as a ROS2 Action,
but underneath it's deliberately ROS-free, so it survives if the team ever ditches
the framework.
Addressing reuses the same 7-bit node IDs the network uses for CANopen. Each
node's bootloader listens on CAN ID 0x800 + node ID
and transmits on 0x880 + node ID. That range was
picked to sit completely clear of CANopen's own message IDs, so update traffic can never
collide with live control messages. And since lower IDs win arbitration in CAN,
bootloader traffic always yields the bus to control frames. The important stuff
always wins.
The bootloader
On the receiving end is OpenBLT, an open-source bootloader that lets each node be reprogrammed in place instead of pulled and flashed. Why OpenBLT? ST's stock STM32G4 bootloader has no CAN support at all, so a third-party option was the only way. OpenBLT has an official STM32G4 port, CAN support, and an open codebase. I ported its example projects to Sunswift's hardware: new CubeIDE projects targeting our microcontroller variant, reconfigured for the 8 MHz crystal on Sunswift boards, with the node ID moved into the bootloader config file so each board knows who it is.
The bootloader gets its own region of flash. The application is linked 12 KB in to leave it room, carries a checksum tacked onto its interrupt vector table so the bootloader can verify there's a valid program before handing over control, and explicitly relocates the vector table on startup. The app's only other job is to listen for the bootloader's CAN message and reset into the bootloader when an update kicks off.
What broke
After flashing, nodes would run the bootloader for about a second and then just... hang. No output, nothing. After a lot of debugging, the culprit turned out to be the oscillator. The bootloader initializes the external crystal, and then the user application tried to initialize it again, which fails and crashed the program. The fix was simply deleting the automatic oscillator init from the application template, since the bootloader has always done it by the time the app runs. Hours of debugging just for one line.
The bigger retreat was automatic updates. The dream was for the Firmware Manager to run on every car boot-up, but it fell apart for three separate reasons: the update Action server claimed it was ready before it actually was, so the car's init code fired requests that just failed; the car's "Safe State" mode (when the car is in "Park") powers the central computer but not the CAN bus, as the high voltage system is disabled, so there was literally nothing to talk to. Silent updates are a scary idea in a car anyways. If the driver doesn't know the pedal box controller is mid-update, they can't accelerate. So updates became a manually triggered command.
Proven end to end
The payoff test ran on the actual car. I flashed the pedal box controller with
firmware that transmits 0x0, then changed the source to send real
pedal position and pushed it to GitHub, deliberately not touching the board
again. Once CodeBuild finished, I triggered the Firmware Manager on the car: it
spotted the changed MD5 hash for that one node, pulled the new binary from S3,
and reflashed it over CAN. Watching candump on the bus, the node's
messages flipped from 0x0 to live pedal values.
What I'd change
Plenty. Every commit rebuilds every node, because each CodeBuild run only sees a detached commit with no history to diff against, so change-based builds are the obvious next step. The Firmware Manager flashes nodes one at a time, but the car has up to six separate CAN buses, so nodes on different buses could easily be updated in parallel. The literature also points at double-buffered transfer (streaming the next chunk while the previous one writes to flash) as good for up to ~40% faster flashing. Longer term: the firmware repo really belongs inside the main software monorepo with its build and versioning infrastructure, the pipeline needs hardening against someone injecting a malicious binary, and it'd be great if the car could just tell you which firmware version each node is actually running.