Test results in CircleCI
CircleCI shows your tests once your runner writes JUnit XML and a store_test_results step uploads it. Here is a config that works, what the Tests tab and Test Insights do with those files, how CircleCI flags and reruns flaky tests, and how to send the same results to Mergify.
In one paragraph
Make your test runner write JUnit XML into a directory, and add store_test_results with that directory as its path. The step runs even when the tests failed. CircleCI then lists each job's tests in the Tests tab, uses their durations to split tests across parallel containers when your test command splits by timing, and builds Test Insights over time, where a test that failed and passed on the same commit within 14 days is labeled FLAKY. Rerunning only the failed tests goes through Smarter Testing's circleci testsuite run.
store_test_results in .circleci/config.yml
This config runs a pytest suite and stores its results:
version: 2.1
jobs:
test:
docker:
- image: cimg/python:3.13
steps:
- checkout
- run: pip install -r requirements.txt
- run:
name: Run tests
command: |
mkdir -p test-results
pytest --junitxml=test-results/junit.xml
- store_test_results:
path: test-results
workflows:
main:
jobs:
- testpath is relative to the job's working directory and can be a directory (CircleCI reads every JUnit XML file in it and in its subdirectories) or a single file. Three details trip people up: the directory must not be hidden, so .test-results does not work; CircleCI does not expand variables in it, so $CIRCLE_WORKING_DIRECTORY/test-results is read literally; and nothing is stored if the runner writes no XML, which most runners do not do by default. pytest takes --junitxml, Vitest and Playwright have a built-in junit reporter, Jest needs jest-junit, RSpec needs rspec_junit_formatter, and Go needs gotestsum. The JUnit XML guide has the exact command for each.
You do not need when: always on store_test_results. A failed run step skips the run steps after it, but store_test_results and store_artifacts always run, so a failed build still has its results. They are skipped only when the job is canceled or times out.
What CircleCI does with your results
| Tests tab | The passing and failing tests of each job, on the job page. |
|---|---|
| Test splitting by timing | With parallelism and a test command that splits by timing, such as circleci tests run --split-by=timings, CircleCI uses the durations in your stored results to spread tests across containers. parallelism on its own runs the whole suite in every container. |
| Test Insights | Across runs of the project: the most failed tests, the slowest ones, and tests labeled FLAKY. Organizations whose slug starts with circleci/ (rather than github/ or bitbucket/) do not get it. |
| Insights API | The same test data over CircleCI's API v2, for your own dashboards. |
From CircleCI's test data, parallelism and Test Insights documentation, as of November 2026.
Flaky tests in CircleCI: the label and the rerun
Test Insights labels a test FLAKY when it failed and passed on the same commit within a 14-day window. In practice that is usually a test someone reran: the workflow failed, an engineer clicked rerun, the test passed. The label is useful for finding those tests. It does not change what they do to your build: a FLAKY test keeps failing the jobs it fails in until someone fixes it.
For the rerun itself, CircleCI's current answer is Smarter Testing. You describe how to discover and run your tests in .circleci/test-suites.yml, replace your test command with circleci testsuite run, and from then on Rerun workflow from failed runs only the tests that failed instead of the whole suite, at no extra charge. The older Rerun failed tests button, built on circleci tests run, is deprecated and only shows on projects that still use that command.
Smarter Testing can also retry failed tests within the same run: set max-auto-rerun in test-suites.yml, and if every failed test passes on a retry, the job succeeds. That is charged in credits above your plan's allowance. It is also where flaky tests start to disappear from view: a job that went green on its second attempt looks like any other green job, so keep an eye on what got retried. The rerun only failed tests and auto rerun pages have the setup.
Send the same results to Mergify Test Engine
Test Engine keeps every test's result from every run, classifies each test as healthy, flaky or broken from that history, and can quarantine the flaky ones so they keep running without failing your build. It reads the same results CircleCI does, and your store_test_results step stays.
Start with a Mergify application key with the ci scope, stored as MERGIFY_TOKEN in a project environment variable (every job gets it) or in a context (add context to each test job in workflows). Never put it in the config file.
With a test plugin (pytest-mergify, @mergifyio/vitest, @mergifyio/playwright, @mergifyio/jest, rspec-mergify), install it and turn it on in your test runner, as its setup page shows: pytest-mergify works once installed, Vitest needs MergifyReporter in its reporters, Playwright and Jest wrap their configuration in withMergify, and RSpec needs --require rspec_mergify. The CircleCI config above does not change: CircleCI sets CI and the CIRCLE_* variables the plugin reads, and each job reports under its own name.
With any other framework, upload the JUnit XML it already writes with the Mergify CLI, in the same run step as the tests:
version: 2.1
jobs:
test:
docker:
- image: cimg/go:1.25
steps:
- checkout
- run:
name: Install gotestsum and the Mergify CLI
command: |
go install gotest.tools/gotestsum@v1.13.0
curl -fsSL https://raw.githubusercontent.com/Mergifyio/mergify-cli/main/install.sh | sh
- run:
name: Run tests and upload them to Mergify
command: |
mkdir -p test-results
exit_code=0
gotestsum --junitfile test-results/junit.xml ./... || exit_code=$?
if [ -n "${MERGIFY_TOKEN:-}" ]; then
mergify ci junit-process --test-exit-code "$exit_code" test-results/junit.xml
else
exit "$exit_code"
fi
- store_test_results:
path: test-results
workflows:
main:
jobs:
- testCircleCI runs each step with bash -eo pipefail, so the test command on its own line would end the step before the upload. || exit_code=$? records the exit code instead, --test-exit-code passes it on so a runner that crashed is not read as a clean report, and junit-process exits last, so its status becomes the step's. The if covers builds without a token: there the step exits with the tests' own code, so failing tests still fail the job.
Builds of pull requests from forks are those builds: they get no project variables or contexts unless you enable Pass secrets to builds from forked pull requests. The plugins then upload nothing and your tests run as usual, and junit-process, which would fail the step without a token, is skipped.
CircleCI publishes no base branch for a pull request build either, so the plugins and junit-process ask Mergify which open pull request has the build's commit as its head, and read the build as one into that pull request's base. There is nothing to configure, and Prevention, which reruns the tests a pull request adds to catch the flaky ones, works on those builds. A build Mergify knows no pull request for, such as one pushed before the pull request was opened, is read like a branch build, and so is a build of your default branch. This lookup ships with version 2026.11.1 of the plugins and the CLI. The CircleCI setup docs have the details.
FAQ
What does store_test_results do in CircleCI?
It uploads the JUnit XML files your tests wrote, from a directory or a single file, so CircleCI can parse them. That feeds the Tests tab on the job page, timing-based test splitting, Test Insights and its flaky test detection. CircleCI does not collect test results on its own: without the step and a JUnit reporter, it only knows whether the job passed.
Does store_test_results run when the tests fail?
Yes. store_test_results and store_artifacts always run, even after a failed step, so you do not need a when: always on them. They do not run when the job is canceled or hits its timeout; CircleCI's teardown steps cover that case.
Why does the Tests tab say no test results were found?
The path points at nothing CircleCI can parse. Check that the path is relative to the working directory, is not a hidden directory (.reports does not work), and that your runner actually wrote JUnit XML there: pytest needs --junitxml, Jest needs jest-junit, RSpec needs rspec_junit_formatter. CircleCI does not expand variables in the path either.
How does CircleCI detect flaky tests?
Test Insights labels a test FLAKY when it failed and passed on the same commit within a 14-day window. That usually means someone reran a workflow and the test passed the second time. The label is informational: a FLAKY test still fails the jobs it fails in.
How do I rerun only the failed tests in CircleCI?
Run your tests through Smarter Testing's circleci testsuite run, configured in .circleci/test-suites.yml, then use Rerun workflow from failed: only the tests that failed run again, at no extra charge. The older Rerun failed tests button, built on circleci tests run, is deprecated. Auto rerun retries failed tests within the same run, and is charged in credits above your plan's allowance.
Last updated: