Skip to content

Flaky Vitest tests on GitHub Actions.
Set up, retried right, quarantined.

On GitHub Actions, Vitest needs one reporter and one secret: add @mergifyio/vitest to your reporters and give the test step a MERGIFY_TOKEN. The reporter uploads every result, applies quarantine in every project of the run, and reruns new tests before the pull request can merge. The one GitHub Actions detail to get right is the reporter list, because setting it drops the annotations Vitest writes on pull requests.

By Julien Danjou, Co-founder & CEO of MergifyUpdated

Setup

Run Vitest with Test Engine on GitHub Actions

Create a Mergify application key with the ci scope and store it as the MERGIFY_TOKEN secret. Then:

vitest.config.js: Add the Mergify reporter and keep the default and github-actions reporters, which an explicit list replaces.
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    reporters: [
      "default",
      ...(process.env.GITHUB_ACTIONS === "true" ? ["github-actions"] : []),
      "@mergifyio/vitest",
    ],
  },
});
.github/workflows/ci.yml: Pass the MERGIFY_TOKEN secret to the step that runs Vitest.
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - name: Run tests
        env:
          MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
        run: npm test # vitest run
  • Setting reporters replaces Vitest's defaults. Keep "default" for the console and "github-actions" on GitHub Actions: that reporter is what turns a failure into an annotation on the pull request's diff.
  • The reporter needs Node.js 22 or later in the job. With an older Node.js it reports nothing and prints no Mergify line.
  • Running Vitest across a matrix? Every leg reports the same GITHUB_JOB, so set MERGIFY_TEST_JOB_NAME: test (${{ matrix.node-version }}) on each leg. Without it, the legs' results merge into one test, and a test that only fails on one version looks flaky instead of broken.
  • No continue-on-error and no upload step: quarantine is applied inside the run, so Vitest's exit code is already the right one.

Full reference: Vitest on docs.mergify.com.

Retries

Re-run failed jobs versus retrying the test

Re-run failed jobs repeats npm ci and the whole Vitest run for one test that may have been unlucky, and the attempt that passes is the one the pull request shows.

Vitest's own retry option is cheaper: it retries the failing test inside the run and reports it as passed when an attempt succeeds. The test is still flaky, and nothing outside that log says so.

@mergifyio/vitest reruns with a rule instead of a blanket count. On a pull request, once the run is over, it starts a Vitest run of its own on the files holding tests the target branch does not have yet, repeats those tests within the Prevention budget, and fails the run if any attempt fails. On the default branch it reruns the tests Test Engine knows as unhealthy without changing the result. The reporter's Mergify CI section in the log names what it caught.

Runners

Why Vitest tests flake on GitHub Actions

  1. Fewer workers than on your machine

    Vitest sizes its pool from the CPUs it sees. A private-repository runner has 2, a public one 4, your laptop maybe 12. Fewer workers means each one runs more files back to back, which exposes state a file leaves behind for the next one in the same worker, and lengthens each file's wall time against the default 5-second testTimeout. Reproduce with the runner's worker count.

    npx vitest run --maxWorkers=2
  2. Shards that change with every pull request

    --shard=1/3 across matrix legs splits the suite by file. Adding a file moves others to a different shard, so a test that depends on what ran before it in the same worker can start failing on a pull request that never touched it. Keep the shard index out of MERGIFY_TEST_JOB_NAME: a test moves between shards, and a name per shard would split its history.

  3. A Node.js version that moves under you

    node-version: 24 takes whichever 24.x the runner image has cached, and GitHub updates the images every week. Two attempts on the same commit can run on different images and different Node.js patch releases. When a failure only shows on one of them, pin the exact version (node-version-file: .nvmrc) so a rerun runs the same runtime.

  4. Runners in UTC

    Hosted runners run in UTC. A test that formats new Date() as a calendar day fails on CI in the hour around midnight UTC and nowhere else. Set the clock with vi.setSystemTime() in the test.

The missing layer

GitHub Actions doesn't know what a test is.

GitHub Actions tells you a job failed. Test Engine tells you which Vitest test failed, whether it failed on the same commit before, and whether it should block the merge. Read what Test Engine adds to GitHub Actions, or the Vitest guide for the patterns behind flaky Vitest suites on any CI: thread vs fork pool, mock hoisting, and shared-state snapshot flakiness.

Frequently asked questions

Why did my Vitest failures stop showing on the pull request diff?
Annotations come from the github-actions reporter, which Vitest only adds by default when you have not set reporters yourself. Once you list reporters to add the Mergify one, add "github-actions" back for GitHub Actions runs, as in the configuration above.
Does @mergifyio/vitest work with Vitest projects and browser mode?
Quarantine applies to every project of the run, browser mode and custom pools included. If two projects run the same test files, run them in separate jobs or steps, each with its own MERGIFY_TEST_JOB_NAME, so each test keeps one history.
Do I need vitest run in CI, or is vitest enough?
GitHub Actions sets CI=true, and Vitest runs once instead of watching when it sees it. vitest run says the same thing explicitly, which is clearer to whoever reads the workflow.

Catch the flaky Vitest tests in your GitHub Actions runs.

Mergify merges more than half a million pull requests a month without breaking main.