Automated firmware testing in CI

Run STM32 firmware in Sim86, assert GPIO behavior, inject a disconnected input, and keep test evidence in GitHub Actions.

A firmware build can succeed while the device does the wrong thing. This guide uses automated firmware testing to check behavior: press a button, verify that the firmware drives an output high, release it, and verify that the output returns low.

The simulator executes an STM32F405 ELF. The test controls the surrounding circuit and reads recorded pin transitions. You can use the same approach for production firmware, replacing the sample circuit and assertions with your product’s behavior.

This guide demonstrates automation with the TypeScript SDK, which is primarily for custom dashboards and web integrations. For pytest workflows, we recommend the CLI. CLI access is currently limited to a closed group of users; book a call to request access.

What you will run

For an overview of where simulation fits in your team’s testing workflow, see firmware testing with Sim86. For guidance on choosing a fault and defining the expected recovery, see fault injection testing.

The sample firmware mirrors input PC7 onto output PC1 on the Adafruit STM32F405 Express. It has no simulator-specific branches. Board pin 5 is PC7; 13 is PC1.

Connection	Purpose
MCU `3.3V (1)` → button `L2`	Supply for the pressed button
Button `L1` → MCU `5`	Input; firmware enables an internal pull-down
MCU `13` → LED `+`	Output under test
LED `-` → MCU `GND`	Simulated LED return

The SDK builds this circuit for each run. It does not require you to edit the project’s saved diagram. The LED wiring above describes the simulator component; use a suitable current-limiting resistor if you add a physical LED on a bench.

Get the example

This TypeScript SDK example does not require CLI access.

You need Node.js 22 or later, make, and the Arm bare-metal GCC toolchain (arm-none-eabi-gcc). Create a project you own in the editor, then create an API key under Cloud → API keys. Cloud runs use your account’s available simulation capacity.

Download and extract the complete firmware CI example. It contains a directory named firmware-ci. Individual files are also available:

main.c, memory.ld, and Makefile
test-firmware.mjs and assertions.mjs

cd firmware-ci
npm init -y
npm install --save-exact @sim86/sdk@0.1.2
make

Keep package.json and package-lock.json with the example so CI installs the same SDK version. make creates firmware.elf at flash address 0x08000000. Keep this ELF: do not replace it with a binary built for a different board.

The download also includes a prebuilt ELF and its hash and validation record. The compiled sample was checked in the local native simulator with both healthy and disconnected wiring. The authenticated cloud SDK workflow has not been run as part of that validation.

/* STM32F405: mirror PC7 (board pin 5) onto PC1 (board pin 13).
 * No simulator hooks: this ELF also runs on the physical board.
 */
#include <stdint.h>
#define REG32(address) (*(volatile uint32_t *)(address))

void reset_handler(void);
extern uint32_t _stack_top;
__attribute__((section(".vectors"), used))
const uintptr_t vectors[] = {(uintptr_t)&_stack_top, (uintptr_t)reset_handler};

void reset_handler(void) {
    REG32(0x40023830) |= 1u << 2; /* RCC_AHB1ENR: GPIOC */
    (void)REG32(0x40023830);
    REG32(0x40020800) = (REG32(0x40020800) & ~((3u << 2) | (3u << 14))) | (1u << 2);
    REG32(0x4002080c) = (REG32(0x4002080c) & ~(3u << 14)) | (2u << 14); /* PC7 pull-down */
    for (;;) {
        const uint32_t high = (REG32(0x40020810) >> 7) & 1u;
        REG32(0x40020818) = high ? (1u << 1) : (1u << 17); /* PC1 set/reset */
    }
}

Run the behavioral test

Set SIM86_API_KEY and SIM86_PROJECT_ID in your shell or secret manager. The project ID is the UUID in your project URL. Then run:

node test-firmware.mjs

The run lasts five simulated seconds, with a 60-second server wall-time limit. The button is pressed at 1,000 ms and released at 3,000 ms. We allow time for the input to propagate before checking stable output intervals. These generous windows test functional behavior; they are not a claim about interrupt latency or timing accuracy.

import { createClient, Components } from '@sim86/sdk';
import { createHash } from 'node:crypto';
import { readFile, mkdir, writeFile } from 'node:fs/promises';
import { assertPinWindow, assertRunCompleted } from './assertions.mjs';

const key = process.env.SIM86_API_KEY;
const projectId = process.env.SIM86_PROJECT_ID;
if (!key || !projectId) throw new Error('Set SIM86_API_KEY and SIM86_PROJECT_ID');
const disconnected = process.argv.includes('--disconnect-input');
const firmwarePath = process.env.FIRMWARE_PATH ?? './firmware.elf';
const firmware = new Uint8Array(await readFile(firmwarePath));
const client = createClient(key);
const project = await client.getProject(projectId);
const graph = project.graph;
const mcu = graph.addComponent(Components.STM32F405_EXPRESS, { id: 'mcu' });
const button = graph.addComponent(Components.BUTTON, { id: 'button' });
const led = graph.addComponent(Components.LED, { id: 'led' });
graph.connect(mcu.pin('3.3V (1)'), button.pin('L2'));
if (!disconnected) graph.connect(button.pin('L1'), mcu.pin('5'));
graph.connect(mcu.pin('13'), led.pin('+'));
graph.connect(mcu.pin('GND'), led.pin('-'));
mcu.record('pins');
await mcu.setFlash(firmware);

await mkdir('artifacts', { recursive: true });
const run = await graph.run({ duration: 5000, realtimeDuration: 60000, seed: 42 });
let completed = false;
// A client/network failure must not leave CI waiting indefinitely.
const watchdog = setTimeout(() => {
  console.error(`Timed out waiting for run ${run.id}; server wall-time limit is 60s`);
  run.detach();
  process.exit(1);
}, 90000);

try {
  await run.at(1000).press(button);
  await run.at(3000).send(button, 'released');
  const info = await run.done;
  completed = true;
  const logs = await run.logs({ from: 'mcu:@pins' });
  await writeFile('artifacts/run.json', JSON.stringify({
    runId: run.id, seed: 42, disconnected,
    firmwareSha256: createHash('sha256').update(firmware).digest('hex'),
    info, logs,
  }, null, 2));
  assertRunCompleted(info);
  assertPinWindow(logs, '13', 0, 500, 900);
  assertPinWindow(logs, '13', 1, 1500, 2500);
  assertPinWindow(logs, '13', 0, 3500, 4500);
  console.log(`PASS: button response and release verified; run ${run.id}`);
} finally {
  if (!completed) await run.stop().catch(() => {});
  run.detach();
  clearTimeout(watchdog);
}

The assertion helper checks the state entering each interval and every recorded transition inside it. A missing log, an unknown pin state, or a simulator failure cannot count as a pass. A successful execution prints PASS: button response and release verified, followed by the run ID.

artifacts/run.json contains the firmware SHA-256, seed, run ID, terminal status, and pin logs. Save it with your CI result so a failure can be investigated later.

Prove that the test catches a fault

Run the same binary with the button disconnected from the input:

node test-firmware.mjs --disconnect-input

This fault injection testing example removes one connection from the simulated circuit. The firmware’s pull-down keeps PC7 low, so PC1 should fail the high-state assertion. The command should exit nonzero. If this fault passes, investigate the test before trusting its healthy result.

This is a wiring fault, not an injected processor exception or a firmware patch. For radio inputs, the SDK’s Wi-Fi frame injection and RF Environment provide different controls.

Run it in GitHub Actions

Commit the example directory, including its npm lockfile. Add a repository secret called SIM86_API_KEY and a repository variable called SIM86_PROJECT_ID. Save this workflow as .github/workflows/firmware-ci.yml in your firmware repository:

name: Firmware simulation
on:
  push:
    branches: [main]
  workflow_dispatch:
permissions:
  contents: read
jobs:
  firmware:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    defaults:
      run:
        working-directory: firmware-ci
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v7
        with:
          node-version: '22'
      - run: sudo apt-get update && sudo apt-get install -y gcc-arm-none-eabi
      - run: npm ci
      - run: make
      - run: node test-firmware.mjs
        env:
          SIM86_API_KEY: ${{ secrets.SIM86_API_KEY }}
          SIM86_PROJECT_ID: ${{ vars.SIM86_PROJECT_ID }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: firmware-simulation
          path: firmware-ci/artifacts/
          if-no-files-found: warn

This adds firmware testing to your continuous integration workflow. It runs on pushes to main and on manual dispatch. The assertion failure fails the job; the artifact step still saves diagnostics. The intentional disconnected case is for checking the test itself and is not enabled in the healthy workflow.

Apply it to your production firmware

Keep your existing build. Set FIRMWARE_PATH to its output and change the circuit, stimulus, and assertions to match the actual product. The included assertions are only valid for the sample’s button-to-output behavior.

Start with one requirement you can observe: an input produces the expected output, a device responds on a bus, or a failed connection is retried. Add one failing scenario for every assertion. For OTA update testing, define how you will observe the active firmware version, update outcome, and recovery after interruption; the button example does not establish those properties.

Use the component guides for board setup before porting a test. Contact us about specific peripheral requirements. The SDK reference covers logs, firmware formats, and run lifecycle. For accuracy measurements, follow the hardware comparison procedure.

Troubleshooting

Symptom	Check
Authentication or project error	The API key is valid and the project belongs to your account
No output-pin logs	Firmware format, the PC1 mapping, and `mcu.record('pins')`
Healthy circuit stays low	PC7 pull-down/input setup, button contacts, and supply wiring
Run stops before five simulated seconds	Terminal reason and available simulation capacity; a timeout is not a pass
Fault case passes	Confirm that the input connection was removed and the high-state assertion is still present