Skip to content
Guide

JUnit XML: the format, a reporter per framework, and what to do with it

JUnit XML is the closest thing CI has to a common language for test results. Your test runner writes it, and Jenkins, CircleCI, GitLab and every GitHub Actions report action read it. Here is what is in the file, the one flag or reporter that writes it for each framework, and where to send it once you have it.

By Julien Danjou, Co-founder & CEO of MergifyUpdated

In one paragraph

A JUnit XML file lists every test of a run as a testcase with a classname, a name and a time, plus a failure, error or skipped child when it did not pass. There is no official schema, only a shared core every tool reads. pytest, Vitest and Playwright write it with a flag, Jest and RSpec with a small reporter package, Go through gotestsum, Maven and Gradle by default. The file describes one run. Reading it across runs, to see which test fails on some commits and passes on others, is the part you have to add.

What is in a JUnit XML file

This is a real one, written by pytest on a run where one of four tests failed:

<?xml version="1.0" encoding="utf-8"?>
<testsuites name="pytest tests">
  <testsuite name="pytest" errors="0" failures="1" skipped="0"
             tests="4" time="0.010" timestamp="2026-10-30T12:26:30+00:00">
    <testcase classname="tests.test_calc" name="test_add" time="0.000" />
    <testcase classname="tests.test_calc" name="test_add_negative" time="0.000" />
    <testcase classname="tests.test_calc" name="test_multiply" time="0.000" />
    <testcase classname="tests.test_calc" name="test_flaky" time="0.000">
      <failure message="assert 0.3812248539328298 &gt; 0.4">def test_flaky():
    # Deliberately flaky: fails about 40% of the time.
&gt;   assert random.random() &gt; 0.4
E   assert 0.3812248539328298 &gt; 0.4

tests/test_calc.py:20: AssertionError</failure>
    </testcase>
  </testsuite>
</testsuites>

Four elements carry nearly all the information:

  • testsuites and testsuite group the tests and carry totals. Some tools write one suite per file or class (Maven and Gradle write one file per class), others one suite for the whole run, as pytest does here.
  • testcase is one test. Its classname and name together are its identity: every tool that tracks a test across runs keys on them, so renaming either one starts a new history.
  • failure and error both mark a test that did not pass, and most tools show both in red. Where the line falls depends on the runner: Maven Surefire writes failure for an assertion that did not hold and error for any other exception, while pytest writes failure for anything that breaks inside the test body and error for a fixture that crashed in setup or teardown, or a file that failed to collect.
  • skipped marks a test that did not run. A testcase with no child passed.

There is no official schema. JUnit, the Java framework, never published one for this file: it grew out of the Ant junit task's output, and each tool writes its own dialect. That is why some readers want a file attribute that pytest leaves out by default, as in the file above. The core above is the part every reader agrees on.

How to write it, framework by framework

Most runners need one flag. The pytest, Jest, Vitest, Playwright, RSpec and Go commands below each ran against a small suite with one deliberately flaky test, and wrote a junit.xml next to it. Maven and Gradle need no flag at all.

FrameworkCommandNotes
pytestpytest --junitxml=junit.xmlBuilt in.
JestJEST_JUNIT_OUTPUT_FILE=junit.xml npx jest --ci --reporters=default --reporters=jest-junitAdd jest-junit to devDependencies. Keep default in the list, or the console output disappears.
Vitestnpx vitest run --reporter=default --reporter=junit --outputFile.junit=junit.xmlBuilt in. Same remark about default.
PlaywrightPLAYWRIGHT_JUNIT_OUTPUT_FILE=junit.xml npx playwright test --reporter=line,junitBuilt in. Or set the junit reporter in playwright.config.
RSpecrspec --format progress --format RspecJunitFormatter --out junit.xmlAdd the rspec_junit_formatter gem.
Gogo run gotest.tools/gotestsum@v1.13.0 --junitfile junit.xml -- ./...go test has no JUnit output; gotestsum wraps it.
Mavenmvn -B -fae testSurefire writes target/surefire-reports/TEST-*.xml, one file per class.
Gradle./gradlew test --continueWrites build/test-results/test/*.xml, one file per class.

Two habits save time later. Write the file to a fixed path, so the CI step that reads it never has to guess. And make sure the step that reads it still runs when tests fail, since a failed run is the one you want the report for: post { always { ... } } in Jenkins and if: ${{ !cancelled() }} on GitHub Actions. CircleCI's store_test_results runs after a failed step without being asked.

What each CI does with it

  • Jenkins reads it natively with the junit step (the "Publish JUnit test result report" post-build action in freestyle jobs), draws a trend graph and marks the build unstable on a failed test. See JUnit test results in Jenkins.
  • CircleCI reads it with store_test_results and shows it in the job's Tests tab and in Test Insights. See CircleCI test results.
  • GitHub Actions does not read it at all. A test step is a command that exits 0 or not. You add an action to turn the file into a check run or a job summary (compared in GitHub Actions test report), or write the summary yourself.

Upload JUnit XML on GitHub Actions

A report action turns the file into a check run, a job summary or a pull request comment, and that report stays with the run. EnricoMi's action also compares its totals with an earlier commit. None of them keeps each test's result from run to run, which is what it takes to tell a flaky test from a broken one, so for that the file has to go to a service that stores it. This workflow runs a Go suite through gotestsum and uploads the result to Mergify Test Engine:

name: Tests

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      HAS_MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN != '' }}
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-go@v7
        with:
          go-version: stable
      - name: Run tests
        id: tests
        continue-on-error: true
        run: go run gotest.tools/gotestsum@v1.13.0 --junitfile junit.xml -- -count=1 ./...
      - name: Upload to Mergify Test Engine
        if: (success() || failure()) && env.HAS_MERGIFY_TOKEN == 'true' && hashFiles('junit.xml') != ''
        uses: mergifyio/gha-mergify-ci@v27
        with:
          token: ${{ secrets.MERGIFY_TOKEN }}
          report_path: junit.xml
          test_step_outcome: ${{ steps.tests.outcome }}
      - name: Fail when tests failed and nothing was uploaded
        if: steps.tests.outcome == 'failure' && (env.HAS_MERGIFY_TOKEN != 'true' || hashFiles('junit.xml') == '')
        run: exit 1

Three details make it work. continue-on-error: true on the test step lets the upload run after a failure, and test_step_outcome hands it the test step's result, so the upload step decides whether the job passes. It fails when a test failed, unless that test is quarantined, and it still fails when the runner crashed but left a clean-looking report. HAS_MERGIFY_TOKEN covers the runs that get no secret: pull requests from forks, and Dependabot runs unless you also store the token in Dependabot's own secrets. The upload would fail there without a token, so it is skipped, and the last step fails the job when the tests failed, which continue-on-error would otherwise hide. It does the same when the tests broke before writing junit.xml. This is the upload step's output on a run where the flaky test failed:

Run ID: cf28ed9881208cea
    โ˜๏ธ 1 report uploaded
    ๐Ÿงช 4 tests (1 failure)
โŒ Unquarantined (1):
    โ”‚      calc_test.go:29: unlucky draw: 0.14
    โ”‚  --- FAIL: TestFlaky (0.00s)
โŒ FAIL โ€” 0/1 failures quarantined

Test Engine stored that run with 4 tests and 1 failure, under the name github.com/mergify-sandbox/te-demo-go.TestFlaky: gotestsum's classname (the package path), a dot, and its name. For pytest, Vitest, Playwright, Jest and RSpec you can skip the XML entirely: their Mergify plugins send results on their own once MERGIFY_TOKEN is set. Every other framework goes through this upload. The Test Engine docs have the upload for each.

What one file cannot tell you

A JUnit file is a snapshot. The one above says test_flaky failed on that run, and it is right. It cannot say that the same test passed on the same commit five minutes earlier, which is the only fact that matters when you are deciding whether to fix the code or the test.

That answer needs the files from many runs, kept, and lined up test by test. Jenkins keeps some of it (the age of a failure, within the builds it retains). GitHub Actions keeps none. Test Engine keeps every test's result from every run it receives, uploaded files included, and lines them up test by test. A test you quarantine keeps running and reporting, and the upload step above stops failing the job on it while someone fixes it. The flaky tests guide covers what usually causes them, framework by framework.

FAQ

What is JUnit XML?

An XML file that lists every test a run executed, grouped into suites, with a time, and a failure, error or skipped element when the test did not pass. It is named after the Java framework whose Ant task first wrote it, but pytest, Jest, Vitest, Playwright, RSpec, Go (through gotestsum) and nearly every other runner can write it, and nearly every CI system can read it.

Is there an official JUnit XML schema?

No. JUnit itself never published one for this file: the format grew out of the Ant junit task's output and each tool writes its own dialect of it. The common core (testsuites, testsuite, testcase, failure, error, skipped) is what everyone reads; extra attributes such as file, line or properties vary from tool to tool.

What is the difference between failure and error in JUnit XML?

It depends on the runner. Maven Surefire writes failure when a test threw an AssertionError and error for any other exception. pytest writes failure for anything that goes wrong inside the test body, assertion or not, and keeps error for a fixture that crashed in setup or teardown and for a file that failed to collect. Most CI tools count both as a failed test, so read the two through your runner's rules before deciding whether the code or the test harness broke.

How do I generate a JUnit XML report with pytest?

Run pytest --junitxml=junit.xml. It is built in, with no plugin to install. pytest names each testcase with classname set to the dotted module path (tests.test_calc) and name set to the function (test_flaky).

How do I upload JUnit XML on GitHub Actions?

GitHub Actions does not read JUnit XML on its own. Either add a report action that turns the file into a check run or job summary (dorny/test-reporter, EnricoMi/publish-unit-test-result-action, mikepenz/action-junit-report), upload it as an artifact, or send it to a service that keeps it, such as Mergify Test Engine with mergifyio/gha-mergify-ci.

Why does renaming a test lose its history?

Because a test's identity in JUnit XML is its classname plus its name. Rename the function or move it to another module and every tool that tracks history sees a new test.

Last updated:

Send your JUnit XML somewhere it is remembered

Test Engine keeps every test's result from every run, and quarantined tests stop failing the job. Free for up to 5 users.