---
title: "Flaky Minitest tests on GitHub Actions: setup, retries and runner pitfalls"
description: "Run Minitest and Rails tests with Mergify Test Engine on GitHub Actions: the minitest-mergify setup, minitest-retry versus Re-run failed jobs, and why Rails suites flake on hosted runners."
url: https://mergify.com/learn/flaky-tests/github-actions/minitest
---

[Learn](https://mergify.com/learn)/[Flaky tests](https://mergify.com/learn/flaky-tests)/[GitHub Actions](https://mergify.com/learn/flaky-tests/github-actions)/Minitest

# Flaky Minitest tests on GitHub Actions.  
Set up, retried right, quarantined.

On GitHub Actions, Minitest needs a gem, one require and one secret: add `minitest-mergify` to the test group, require `minitest/mergify` from your test helper, and give the step that runs the tests a `MERGIFY_TOKEN`. The plugin uploads every test's result when the run ends, applies quarantine inside Minitest, and reruns new tests before the pull request can merge. Rails test suites run on Minitest, so the same setup covers `bin/rails test`.

By [Julien Danjou](https://mergify.com/about), Co-founder & CEO of MergifyUpdated November 5, 2026

Setup

## Run Minitest with Test Engine on GitHub Actions

Create a Mergify application key with the `ci` scope and store it as the `MERGIFY_TOKEN` secret. Then:

`Gemfile`: Add minitest-mergify to the test group of the Gemfile. In a Rails app, minitest is already there.

```
source 'https://rubygems.org'

group :test do
  gem 'minitest'
  gem 'minitest-mergify'
  gem 'rake'
end
```

`test/test_helper.rb`: Require the plugin in the test helper, before any test is defined. Minitest 6 does not load it otherwise.

```
require 'minitest/autorun'
require 'minitest/mergify'
require 'calc'
```

`.github/workflows/ci.yml`: Pass the MERGIFY\_TOKEN secret to the step that runs Minitest. In a Rails app, keep bin/rails test.

```
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: "3.4"
          bundler-cache: true
      - name: Run tests
        env:
          MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
        run: bundle exec rake test
```

-   Minitest 6 loads only the plugins a suite asks for, so without `require 'minitest/mergify'` the run goes as usual and nothing reaches Test Engine. Keep the line on Minitest 5 too: there the plugin would load only after the tests are defined, and spec tests with no body lose their line number.
-   No `continue-on-error` and no `test_step_outcome`: there is no upload step to hand the result to. A quarantined test that fails counts as a skip, so the step's exit code is already the job's result. A test file that fails to load still fails the step.
-   Running Minitest across a matrix? Every leg reports the same `GITHUB_JOB`, so set `MERGIFY_TEST_JOB_NAME: test (${{ matrix.ruby-version }})` on each leg. Without it, the legs' results merge into one test, and a test that only fails on one version looks flaky instead of broken.
-   CRuby 3.1 to 4.0, with Minitest 5.19 or later, or Minitest 6 on Ruby 3.2 and later. A JRuby or `ruby-head` leg runs its tests but reports nothing; give that leg the JUnit reporter from `minitest-reporters` and the `mergifyio/gha-mergify-ci` upload step instead.
-   The gem runs on `mergify-ci-core`, precompiled per platform. If your `Gemfile.lock` was written on a Mac, run `bundle lock --add-platform x86_64-linux` and commit it, so Bundler can install the Linux build on the runner.
-   [bin/rails test leaves system tests out](https://guides.rubyonrails.org/testing.html#running-tests-in-continuous-integration-ci). Run `bin/rails test:all` in the step if you want them reported too.

Full reference: [Minitest on docs.mergify.com](https://docs.mergify.com/test-engine/test-frameworks/minitest).

Retries

## Re-run failed jobs versus retrying the test

GitHub Actions retries at the job level. Re-run failed jobs, or `gh run rerun --failed`, starts the failed jobs again with the same `GITHUB_SHA` and `GITHUB_REF`, up to 30 days after the run and up to 50 times per run. Each attempt repeats the whole job: checkout, toolchain setup, dependency install, then every test, including the ones that passed.

Re-run failed jobs replays the job from the top: setup-ruby, the bundle restored from cache, the database setup if you have one, then the whole suite. Minitest also picks a new seed for that attempt (the log prints it as `Run options: --seed`), so the attempt that passes did not even run the tests in the same order as the one that failed.

[minitest-retry](https://github.com/y-yagi/minitest-retry) moves the retry inside the run: a failing test is retried up to 3 times by default, and it counts as passed if one attempt passes. The `[MinitestRetry] retry` lines in that job's log are the only trace of the flake.

minitest-mergify reruns with a narrower rule. On a pull request, once the suite is over, it reruns the tests the target branch does not have yet, within the Prevention budget, through Minitest itself, so a test that ran in Rails' parallel workers is rerun there. If one attempt fails, the test is flaky and the job fails. On pushes to the default branch it reruns the tests Test Engine knows as unhealthy, within the Detection budget, without changing the result.

Runners

## Why Minitest tests flake on GitHub Actions

1.  ### A worker count that follows the runner
    
    `parallelize(workers: :number_of_processors)` forks one worker per core, which is [2 on a private-repository runner](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) and 4 on a public one, against often 10 or more on a laptop. [Rails also skips parallelization below 50 tests](https://guides.rubyonrails.org/testing.html#parallel-testing), so the test file you run locally usually runs in one process, while CI runs the full suite across workers, each with its own share of the tests. Reproduce with the runner's worker count and the seed from the failed attempt's log.
    
    ```
    PARALLEL_WORKERS=2 bin/rails test --seed 39840
    ```
    
2.  ### Workers that share everything but the database
    
    [Active Record gives each worker its own database](https://guides.rubyonrails.org/testing.html#parallel-testing-with-processes), suffixed with the worker's number (`test-database-0`, `test-database-1`). The rest is yours to split. Active Storage's Disk service folder or a Redis service container is shared by every worker of the job, so one worker's teardown can delete the files another worker's test is still reading. [The Active Storage guide](https://guides.rubyonrails.org/active_storage_overview.html#discarding-files-created-during-tests) gives each worker its own folder in `parallelize_setup`.
    
    ```
    class ActionDispatch::IntegrationTest
      parallelize_setup do |i|
        ActiveStorage::Blob.service.root = "#{ActiveStorage::Blob.service.root}-#{i}"
      end
    end
    ```
    
3.  ### A new seed on every attempt
    
    [Test methods run in random order](https://guides.rubyonrails.org/testing.html), and Minitest prints the seed before the first one. Every attempt of the job draws a new seed, so a test that depends on what ran before it fails on one attempt and passes on the next, and Re-run failed jobs reads that as a fluke. Rerun with the failed attempt's seed (`--seed`, or `SEED=` with Rake) before calling the test flaky.
    
    ```
    SEED=39840 bundle exec rake test
    ```
    

The missing layer

## GitHub Actions doesn't know what a test is.

GitHub Actions tells you a job failed. Test Engine tells you which Minitest test failed, whether it failed on the same commit before, and whether it should block the merge. Read [what Test Engine adds to GitHub Actions](https://mergify.com/learn/flaky-tests/github-actions), or the [Minitest guide](https://mergify.com/learn/flaky-tests/minitest) for the patterns behind flaky Minitest suites on any CI: parallelization races, fixture loading, and test isolation in Rails suites.

## Other frameworks on GitHub Actions

-   [pytest](https://mergify.com/learn/flaky-tests/github-actions/pytest)
-   [Vitest](https://mergify.com/learn/flaky-tests/github-actions/vitest)
-   [Playwright](https://mergify.com/learn/flaky-tests/github-actions/playwright)
-   [RSpec](https://mergify.com/learn/flaky-tests/github-actions/rspec)
-   [Jest](https://mergify.com/learn/flaky-tests/github-actions/jest)
-   [Go](https://mergify.com/learn/flaky-tests/github-actions/go)
-   [Cypress](https://mergify.com/learn/flaky-tests/github-actions/cypress)
-   [JUnit](https://mergify.com/learn/flaky-tests/github-actions/junit)
-   [TestNG](https://mergify.com/learn/flaky-tests/github-actions/testng)
-   [MSTest](https://mergify.com/learn/flaky-tests/github-actions/mstest)
-   [NUnit](https://mergify.com/learn/flaky-tests/github-actions/nunit)
-   [Pest](https://mergify.com/learn/flaky-tests/github-actions/pest)
-   [PHPUnit](https://mergify.com/learn/flaky-tests/github-actions/phpunit)
-   [Rust](https://mergify.com/learn/flaky-tests/github-actions/rust)

## Frequently asked questions

Why do my Rails tests pass locally and fail on GitHub Actions?

Usually the runner, not the test: 2 parallel workers on a private repository, a new seed on every attempt, and a full suite that runs in parallel while the one file you ran locally did not (Rails only parallelizes 50 tests or more). Run the suite with PARALLEL\_WORKERS=2 and the seed printed in the failed attempt's log.

Why does Minitest report 0 failures when the job failed?

On a pull request, minitest-mergify reruns new tests after the suite. Minitest's summary and your other reporters only show your own run, so a new test that failed on a rerun still counts as passed there. The exit code and the plugin's # Mergify CI: section in the log say it is flaky.

Do I still need minitest-reporters or minitest-junit on GitHub Actions?

Not for Test Engine: minitest-mergify uploads the results itself. Keep the JUnit reporter only for a leg the plugin cannot report from, such as JRuby. Moving from a JUnit upload keeps your quarantine entries, while the tests' history starts again under their new names.

Does minitest-mergify work with Rails parallel tests?

Yes, whether the tests run in Minitest's threads (parallelize\_me!) or in Rails' parallel workers, and its reruns go through those workers too. Quarantine applies the same way as in a single process.

## Catch the flaky Minitest tests in your GitHub Actions runs.

Mergify merges more than half a million pull requests a month without breaking main.

[Start a free 14-day trial](https://dashboard.mergify.com/)[Read the setup docs](https://docs.mergify.com/test-engine/test-frameworks/minitest)
