Flaky Vitest tests on GitHub Actions.
Set up, retried right, quarantined.
On GitHub Actions, Vitest needs one reporter and one secret: add @mergifyio/vitest to your reporters and give the test step a MERGIFY_TOKEN. The reporter uploads every result, applies quarantine in every project of the run, and reruns new tests before the pull request can merge. The one GitHub Actions detail to get right is the reporter list, because setting it drops the annotations Vitest writes on pull requests.
Setup
Run Vitest with Test Engine on GitHub Actions
Create a Mergify application key with the ci scope and store it as the MERGIFY_TOKEN secret. Then:
vitest.config.js: Add the Mergify reporter and keep the default and github-actions reporters, which an explicit list replaces.import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
reporters: [
"default",
...(process.env.GITHUB_ACTIONS === "true" ? ["github-actions"] : []),
"@mergifyio/vitest",
],
},
});.github/workflows/ci.yml: Pass the MERGIFY_TOKEN secret to the step that runs Vitest.name: CI
on:
push:
branches: [main]
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: 24
cache: npm
- run: npm ci
- name: Run tests
env:
MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
run: npm test # vitest run- Setting
reportersreplaces Vitest's defaults. Keep"default"for the console and"github-actions"on GitHub Actions: that reporter is what turns a failure into an annotation on the pull request's diff. - The reporter needs Node.js 22 or later in the job. With an older Node.js it reports nothing and prints no Mergify line.
- Running Vitest across a matrix? Every leg reports the same
GITHUB_JOB, so setMERGIFY_TEST_JOB_NAME: test (${{ matrix.node-version }})on each leg. Without it, the legs' results merge into one test, and a test that only fails on one version looks flaky instead of broken. - No
continue-on-errorand no upload step: quarantine is applied inside the run, so Vitest's exit code is already the right one.
Full reference: Vitest on docs.mergify.com.
Retries
Re-run failed jobs versus retrying the test
Re-run failed jobs repeats npm ci and the whole Vitest run for one test that may have been unlucky, and the attempt that passes is the one the pull request shows.
Vitest's own retry option is cheaper: it retries the failing test inside the run and reports it as passed when an attempt succeeds. The test is still flaky, and nothing outside that log says so.
@mergifyio/vitest reruns with a rule instead of a blanket count. On a pull request, once the run is over, it starts a Vitest run of its own on the files holding tests the target branch does not have yet, repeats those tests within the Prevention budget, and fails the run if any attempt fails. On the default branch it reruns the tests Test Engine knows as unhealthy without changing the result. The reporter's Mergify CI section in the log names what it caught.
Runners
Why Vitest tests flake on GitHub Actions
Fewer workers than on your machine
Vitest sizes its pool from the CPUs it sees. A private-repository runner has 2, a public one 4, your laptop maybe 12. Fewer workers means each one runs more files back to back, which exposes state a file leaves behind for the next one in the same worker, and lengthens each file's wall time against the default 5-second
testTimeout. Reproduce with the runner's worker count.npx vitest run --maxWorkers=2Shards that change with every pull request
--shard=1/3across matrix legs splits the suite by file. Adding a file moves others to a different shard, so a test that depends on what ran before it in the same worker can start failing on a pull request that never touched it. Keep the shard index out ofMERGIFY_TEST_JOB_NAME: a test moves between shards, and a name per shard would split its history.A Node.js version that moves under you
node-version: 24takes whichever 24.x the runner image has cached, and GitHub updates the images every week. Two attempts on the same commit can run on different images and different Node.js patch releases. When a failure only shows on one of them, pin the exact version (node-version-file: .nvmrc) so a rerun runs the same runtime.Runners in UTC
Hosted runners run in UTC. A test that formats
new Date()as a calendar day fails on CI in the hour around midnight UTC and nowhere else. Set the clock withvi.setSystemTime()in the test.
The missing layer
GitHub Actions doesn't know what a test is.
GitHub Actions tells you a job failed. Test Engine tells you which Vitest test failed, whether it failed on the same commit before, and whether it should block the merge. Read what Test Engine adds to GitHub Actions, or the Vitest guide for the patterns behind flaky Vitest suites on any CI: thread vs fork pool, mock hoisting, and shared-state snapshot flakiness.
Frequently asked questions
- Why did my Vitest failures stop showing on the pull request diff?
- Annotations come from the github-actions reporter, which Vitest only adds by default when you have not set reporters yourself. Once you list reporters to add the Mergify one, add "github-actions" back for GitHub Actions runs, as in the configuration above.
- Does @mergifyio/vitest work with Vitest projects and browser mode?
- Quarantine applies to every project of the run, browser mode and custom pools included. If two projects run the same test files, run them in separate jobs or steps, each with its own MERGIFY_TEST_JOB_NAME, so each test keeps one history.
- Do I need vitest run in CI, or is vitest enough?
- GitHub Actions sets CI=true, and Vitest runs once instead of watching when it sees it. vitest run says the same thing explicitly, which is clearer to whoever reads the workflow.