---
title: "Flaky PHPUnit tests on GitHub Actions: setup, retries and runner pitfalls"
description: "Run PHPUnit with Mergify Test Engine on GitHub Actions: the JUnit upload workflow, why --retry hides flaky tests, and what makes PHPUnit flake on GitHub-hosted runners."
url: https://mergify.com/learn/flaky-tests/github-actions/phpunit
---

[Learn](https://mergify.com/learn)/[Flaky tests](https://mergify.com/learn/flaky-tests)/[GitHub Actions](https://mergify.com/learn/flaky-tests/github-actions)/PHPUnit

# Flaky PHPUnit tests on GitHub Actions.  
Set up, retried right, quarantined.

On GitHub Actions, PHPUnit reaches Test Engine as a JUnit report: `--log-junit junit.xml` writes it, and the `mergifyio/gha-mergify-ci` action uploads it and sets the job's result with quarantine taken into account. PHPUnit writes the report itself, so `phpunit.xml` stays as it is. Below: the workflow, why PHPUnit's own retry hides exactly what Test Engine needs to see, and the runner details that make PHPUnit suites flake on GitHub Actions.

By [Julien Danjou](https://mergify.com/about), Co-founder & CEO of MergifyUpdated November 5, 2026

Setup

## Run PHPUnit with Test Engine on GitHub Actions

Create a Mergify application key with the `ci` scope and store it as the `MERGIFY_TOKEN` secret. Then:

`.github/workflows/ci.yml`: Write junit.xml with PHPUnit's --log-junit, then upload it with the Mergify action, which sets the job's result.

```
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      HAS_MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN != '' }}
    steps:
      - uses: actions/checkout@v7
      - uses: shivammathur/setup-php@v2
        with:
          php-version: "8.5"
          coverage: none
      - run: composer install --no-interaction --no-progress
      - name: Run tests
        id: tests
        continue-on-error: true
        run: ./vendor/bin/phpunit --log-junit junit.xml
      - name: Mergify CI Upload
        if: (success() || failure()) && env.HAS_MERGIFY_TOKEN == 'true'
        uses: mergifyio/gha-mergify-ci@v28
        with:
          token: ${{ secrets.MERGIFY_TOKEN }}
          report_path: junit.xml
          test_step_outcome: ${{ steps.tests.outcome }}
      - name: Fail when tests failed and no report was uploaded
        if: steps.tests.outcome == 'failure' && env.HAS_MERGIFY_TOKEN != 'true'
        run: exit 1
```

-   `continue-on-error: true` on the PHPUnit step hands the job's result to the upload step, which leaves out the failures of quarantined tests. `test_step_outcome` passes PHPUnit's own outcome along, so a run that crashed but left a clean-looking report still fails the job.
-   Pull requests from forks and from Dependabot get no `MERGIFY_TOKEN`. `HAS_MERGIFY_TOKEN` skips the upload there, and the last step fails the job when PHPUnit failed and nothing was uploaded. Without that step, `continue-on-error` would turn those red runs green.
-   [setup-php's README](https://github.com/shivammathur/setup-php) notes that Xdebug is enabled by default on Ubuntu GitHub Actions images and recommends turning it off for PHP performance. `coverage: none` disables both Xdebug and PCOV. If you need coverage, collect it in a job of its own.
-   PHPUnit 13 [requires PHP 8.4.1 or later](https://github.com/sebastianbergmann/phpunit/blob/13.4/composer.json), so `php-version` has to match the PHPUnit your lock file installs. Testing several PHP versions in a matrix? Every leg reports the same `GITHUB_JOB`, so add `job_name: test (${{ matrix.php }})` to the upload step. Without it, a test that only fails on one PHP version looks flaky instead of broken.
-   Running [paratest](https://github.com/paratestphp/paratest)? It takes the same `--log-junit` option. Swap the command in the test step and keep its `id`, and the upload step stays as it is.

Full reference: [PHPUnit on docs.mergify.com](https://docs.mergify.com/test-engine/test-frameworks/phpunit).

Retries

## Re-run failed jobs versus retrying the test

GitHub Actions retries at the job level. Re-run failed jobs, or `gh run rerun --failed`, starts the failed jobs again with the same `GITHUB_SHA` and `GITHUB_REF`, up to 30 days after the run and up to 50 times per run. Each attempt repeats the whole job: checkout, toolchain setup, dependency install, then every test, including the ones that passed.

[Re-run failed jobs](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs) starts the job over from the checkout, `composer install` included, and runs every test again to find out whether one of them was unlucky. If the second attempt passes, the pull request goes green, and the failure stays in the log of an attempt nobody opens.

PHPUnit 13.3 added a retry of its own, `--retry N` and the `#[Retry]` attribute ([changelog](https://github.com/sebastianbergmann/phpunit/blob/13.3.6/ChangeLog-13.3.md)). It reruns a failing test inside the run and stops at the first success. The console summary still mentions the failed attempt, but the [JUnit logger](https://github.com/sebastianbergmann/phpunit/blob/13.4/src/Logging/JUnit/JunitXmlLogger.php) writes a single testcase from the final attempt, and the failed attempts only add to its time. A test saved by a retry reaches Test Engine as a pass. Leave `--retry` off the job that uploads.

With the upload in place, every attempt sends its report, on pull requests and on the default branch, and each one adds to the test's history. A test that fails on one attempt and passes on another of the same commit is flagged as flaky. Once it is quarantined, the upload step leaves its failure out when it sets the job's result, and the test keeps running and reporting until it is healthy again.

Runners

## Why PHPUnit tests flake on GitHub Actions

1.  ### paratest at the runner's width
    
    [paratest](https://github.com/paratestphp/paratest/blob/7.x/src/Options.php) starts one process per CPU core unless you pass `--processes`. A [GitHub-hosted](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) `ubuntu-latest` runner has 2 vCPUs and 8 GB on a private repository and 4 vCPUs and 16 GB on a public one, so the same suite runs on two workers in the company repository and four in the open source one, with different tests sharing each worker's static state. Workers that share one database collide on top of that. paratest sets a `TEST_TOKEN` that is different for every running process: build the test database name from it.
    
    ```
    # Reproduce a failure from a private-repository runner
    vendor/bin/paratest --processes=2
    ```
    
2.  ### A new random order on every attempt
    
    With `--order-by random` (or the same setting in `phpunit.xml`), each attempt of the job draws a new order, so a test that depends on what ran before it fails on one attempt and passes on the rerun. PHPUnit [prints the seed](https://github.com/sebastianbergmann/phpunit/blob/13.4/src/TextUI/Application.php) as `Random Seed:` at the top of a randomized run. Copy it from the failed attempt's log and replay that order with [\--random-order-seed](https://github.com/sebastianbergmann/phpunit/blob/13.4/src/TextUI/Help.php) before calling the test flaky.
    
    ```
    ./vendor/bin/phpunit --order-by random --random-order-seed 1762367891
    ```
    
3.  ### A database container that is started but not ready
    
    A `services:` database starts before your steps run, but started is not ready. Without a health check, the migrations or the first test that opens a connection can reach a PostgreSQL that is still initializing, and the failure lands on whichever test PHPUnit runs first. [GitHub's own example](https://docs.github.com/en/actions/tutorials/use-containerized-services/create-postgresql-service-containers) adds the health check that makes the job wait for the server.
    
    ```
    services:
      postgres:
        image: postgres
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432
    ```
    
4.  ### A library tested without its lock file
    
    Without a committed `composer.lock`, `composer install` resolves the dependencies again on every run. A release that lands between two attempts on the same commit makes the second attempt run different code than the first, and the difference reads as a flaky test. [Composer's docs](https://github.com/composer/composer/blob/main/doc/02-libraries.md) recommend committing the lock file for libraries too, and testing against fresh releases in a separate `composer update` job. Give that job its own name, so its failures do not land in the history of the job that runs on the lock file.
    

The missing layer

## GitHub Actions doesn't know what a test is.

GitHub Actions tells you a job failed. Test Engine tells you which PHPUnit test failed, whether it failed on the same commit before, and whether it should block the merge. Read [what Test Engine adds to GitHub Actions](https://mergify.com/learn/flaky-tests/github-actions), or the [PHPUnit guide](https://mergify.com/learn/flaky-tests/phpunit) for the patterns behind flaky PHPUnit suites on any CI: test isolation, database transactions, and dataProvider non-determinism.

## Other frameworks on GitHub Actions

-   [pytest](https://mergify.com/learn/flaky-tests/github-actions/pytest)
-   [Vitest](https://mergify.com/learn/flaky-tests/github-actions/vitest)
-   [Playwright](https://mergify.com/learn/flaky-tests/github-actions/playwright)
-   [RSpec](https://mergify.com/learn/flaky-tests/github-actions/rspec)
-   [Jest](https://mergify.com/learn/flaky-tests/github-actions/jest)
-   [Go](https://mergify.com/learn/flaky-tests/github-actions/go)
-   [Cypress](https://mergify.com/learn/flaky-tests/github-actions/cypress)
-   [JUnit](https://mergify.com/learn/flaky-tests/github-actions/junit)
-   [TestNG](https://mergify.com/learn/flaky-tests/github-actions/testng)
-   [Minitest](https://mergify.com/learn/flaky-tests/github-actions/minitest)
-   [MSTest](https://mergify.com/learn/flaky-tests/github-actions/mstest)
-   [NUnit](https://mergify.com/learn/flaky-tests/github-actions/nunit)
-   [Pest](https://mergify.com/learn/flaky-tests/github-actions/pest)
-   [Rust](https://mergify.com/learn/flaky-tests/github-actions/rust)

## Frequently asked questions

Why do my PHPUnit tests pass locally but fail on GitHub Actions?

On GitHub Actions, look first at the runner's 2 vCPUs on a private repository, which changes how paratest spreads tests across workers, and at random order, which draws a new seed on every attempt. A database container that was not ready and dependencies resolved again without a lock file come next. Replay the failed attempt's seed and run paratest --processes=2 locally before you call the test flaky.

Does PHPUnit's --retry work with Test Engine?

It runs, but it hides what Test Engine needs. PHPUnit's JUnit report keeps only the final attempt of a retried test, so a test that failed and then passed on retry is uploaded as a pass and never flagged. Drop --retry and #\[Retry\] from the job that uploads, and let quarantine keep the known flaky tests from blocking merges.

Why does the PHPUnit step have continue-on-error?

So quarantine can work. The upload step decides whether the job passes, and it ignores the failures of quarantined tests. If PHPUnit's failing exit code failed the job first, a quarantined test would still turn the pull request red.

Can I upload a report from paratest?

Yes. paratest accepts PHPUnit's --log-junit option, so replace the command in the test step, keep the step's id, and leave the upload step unchanged.

## Catch the flaky PHPUnit tests in your GitHub Actions runs.

Mergify merges more than half a million pull requests a month without breaking main.

[Start a free 14-day trial](https://dashboard.mergify.com/)[Read the setup docs](https://docs.mergify.com/test-engine/test-frameworks/phpunit)
