---
title: "Flaky tests on GitHub Actions: the missing test layer"
description: "It stores, reruns and reports jobs. The tests inside a job are lines in its log. Test Engine adds the missing test layer: a history for every test, flaky detection, quarantine and in-job retries, for every framework you run."
url: https://mergify.com/learn/flaky-tests/github-actions
---

[Learn](https://mergify.com/learn)/[Flaky tests](https://mergify.com/learn/flaky-tests)/GitHub Actions

# Flaky tests on GitHub Actions.  
GitHub Actions doesn't know what a test is.

It stores, reruns and reports jobs. The tests inside a job are lines in its log. Test Engine adds the missing test layer: a history for every test, flaky detection, quarantine and in-job retries, for every framework you run.

By [Julien Danjou](https://mergify.com/about), Co-founder & CEO of MergifyUpdated October 28, 2026

[Start a free 14-day trial](https://dashboard.mergify.com/)[Set up GitHub Actions](https://docs.mergify.com/ci-insights/setup/github-actions)

Open a failed run on GitHub Actions and you get a red job and a log. Which test failed, whether it also failed yesterday, whether it fails one run in twenty: GitHub Actions keeps none of it. The unit it stores, reruns and reports is the job.

So the one tool it gives you against a flaky test is [Re-run failed jobs](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs). It starts the failed jobs again on the same commit, every test in them included, and the pull request keeps whichever attempt comes back green. The flaky test is still in the suite, nothing recorded it, and the next pull request pays for it again.

Test Engine records every test of every run, on pull requests and on the default branch. A test that passes and fails on the same commit is flagged as flaky. A test you already know is flaky is quarantined: it keeps running and reporting, and its failure stops blocking the merge. On a pull request, the native plugins rerun the new tests inside the job, within a budget you set, so a new flaky test is caught before it lands. Test Engine is the test layer GitHub Actions is missing, and the CI layer is next.

Re-run failed jobs

GitHub Actions retries at the job level. Re-run failed jobs, or `gh run rerun --failed`, starts the failed jobs again with the same `GITHUB_SHA` and `GITHUB_REF`, up to 30 days after the run and up to 50 times per run. Each attempt repeats the whole job: checkout, toolchain setup, dependency install, then every test, including the ones that passed.

## Pick your framework

The GitHub Actions setup for each framework, how its retries compare with rerunning the job, and what makes it flake on GitHub Actions runners.

[

pytest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/pytest)[

Vitest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/vitest)[

Playwright on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/playwright)[

RSpec on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/rspec)[

Jest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/jest)[

Go on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/go)[

Cypress on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/cypress)[

JUnit on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/junit)[

TestNG on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/testng)[

Minitest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/minitest)[

MSTest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/mstest)[

NUnit on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/nunit)[

Pest on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/pest)[

PHPUnit on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/phpunit)[

Rust on GitHub Actions

Read the setup →](https://mergify.com/learn/flaky-tests/github-actions/rust)

## Frequently asked questions

Does GitHub Actions detect flaky tests?

No. GitHub Actions records a result per job and keeps the log. It has no notion of an individual test, so it cannot compare one test across runs of the same commit, which is what detecting a flaky test takes. Test Engine records each test's result on every run and flags the ones that both pass and fail on the same code.

How do I rerun only the failed tests on GitHub Actions?

GitHub Actions cannot: Re-run failed jobs repeats the whole job. Retrying a single test has to happen inside the job, in the test framework. Test Engine's plugins for pytest, Vitest, Playwright, RSpec and Jest rerun new tests on pull requests and known unhealthy tests on the default branch, inside the job that ran them.

How do I quarantine a flaky test on GitHub Actions?

Without a tool, you skip it in code and open a pull request to do it, then another one to bring it back. With Test Engine, the quarantine list is fetched at the start of the run. A quarantined test still runs and reports, but its failure no longer fails the job, and quarantine lifts when the test is healthy again on the default branch.

Does it work on pull requests from forks and from Dependabot?

Those runs get no repository secrets, so MERGIFY\_TOKEN is empty. The native plugins then upload nothing and the tests run as usual. With the JUnit upload action, skip the upload step when the secret is missing, as the Mergify docs show.

## Give GitHub Actions a test layer.

Mergify merges more than half a million pull requests a month without breaking main.

[Start a free 14-day trial](https://dashboard.mergify.com/)[Read the setup docs](https://docs.mergify.com/ci-insights/setup/github-actions)
