Skip to content
Jenkins

JUnit test results in Jenkins

Jenkins reads JUnit XML natively: the "Publish JUnit test result report" post-build action in a freestyle job, the junit step in a Pipeline. Here is a Jenkinsfile that works, why a failed test turns your build red instead of yellow and how to change that, the options worth knowing, and what Jenkins remembers about a test from one build to the next.

By Julien Danjou, Co-founder & CEO of MergifyUpdated

In one paragraph

Have your tests write JUnit XML and call junit 'junit.xml' in a post { always { } } block, so it runs when tests fail. With a plain sh test step, a failed test makes the build FAILURE; run the tests with returnStatus: true and the build becomes UNSTABLE instead, which is what the JUnit plugin was designed around. Jenkins then keeps a trend graph and, per test, how many builds in a row it has been failing. It does not notice a test that fails one build and passes the next on the same code.

"Publish JUnit test result report" in a freestyle job

In a freestyle job, open Configure, scroll to Post-build Actions, and add Publish JUnit test result report. The one field that matters is Test report XMLs: an Ant glob relative to the workspace root, such as junit.xml or **/target/surefire-reports/TEST-*.xml. A post-build action runs whether or not the build step failed, so the report is there on the builds you need it for.

Keep the pattern tight. The publisher reads every file the glob matches. Well-formed XML with no tests in it, like Maven Failsafe's failsafe-summary.xml, is skipped. A matched file that is not XML fails the step, and a report it cannot parse, such as one cut off when the test run was killed, is recorded as a failed test named [failed-to-read].

The junit step in a Jenkinsfile

In a Pipeline, the same publisher is the junit step. This Jenkinsfile runs a pytest suite and records its results:

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                sh 'python3 -m venv .venv && .venv/bin/pip install -q -r requirements.txt'
                sh '.venv/bin/pytest --junitxml=junit.xml'
            }
            post {
                always {
                    junit 'junit.xml'
                }
            }
        }
    }
}

The post { always { } } block is the part people get wrong. Put junit in steps after the test command and it never runs on a failed build, because the failed sh step ends the stage first. Any test runner works the same way once it writes a JUnit XML file: Maven and Gradle write one per test class by default, so their pattern is **/target/surefire-reports/TEST-*.xml or **/build/test-results/**/*.xml.

FAILURE or UNSTABLE

Jenkins has a status for "the build worked, some tests failed": UNSTABLE, shown in yellow. The JUnit plugin sets it when it finds a failed test. With the Jenkinsfile above you will rarely see it, though. In our runs, every build where the flaky test failed came out FAILURE: pytest exited 1, the sh step failed the stage, and junit only recorded the results afterwards.

To let the tests decide, capture the exit code instead of failing on it:

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                sh 'python3 -m venv .venv && .venv/bin/pip install -q -r requirements.txt'
                script {
                    // pytest exits 1 when tests fail: let junit mark the build
                    // UNSTABLE. Any other non-zero code means pytest itself broke.
                    def status = sh(script: '.venv/bin/pytest --junitxml=junit.xml', returnStatus: true)
                    if (status > 1) {
                        error "pytest exited with ${status}"
                    }
                }
            }
            post {
                always {
                    junit 'junit.xml'
                }
            }
        }
    }
}

Now a failed test gives UNSTABLE and a passing run gives SUCCESS, which is what we got on two builds of this file. The status > 1 check keeps the build FAILURE when pytest itself breaks (exit code 2 for an interruption, 4 for a usage error), so a crashed run still reads as broken instead of as a test result. Other runners use other codes: Maven, for example, exits 1 for both failed tests and a broken build, so there mvn test -Dmaven.test.failure.ignore=true is the usual way to hand the decision to junit.

Whether you want this is a team decision. UNSTABLE makes "the tests found a problem" visibly different from "the build is broken". It also means a pipeline that only gates on FAILURE keeps going past failed tests, so check what your later stages and your branch protection treat as passing.

The junit options worth knowing

testResultsThe report files, as an Ant glob relative to the workspace (**/target/surefire-reports/TEST-*.xml). Several patterns are separated by commas. It is the positional argument in junit 'junit.xml'.
allowEmptyResultsBy default the step fails the build when no file matches or the files hold no test. true turns that off, which also hides a job that silently stopped writing reports.
skipMarkingBuildUnstableKeeps the build green even with failed tests. The stage is still marked unstable unless skipMarkingStageUnstable is set too.
skipPublishingChecksStops the step from publishing the results as a GitHub check run. Publishing only happens with the GitHub Checks plugin and GitHub App credentials.
stdioRetentionHow much test stdout and stderr to keep: 'ALL', 'FAILED' or 'NONE'. It replaces the deprecated keepLongStdio. Keeping everything costs Jenkins memory.
healthScaleFactorHow much test failures weigh in the job's health score (default 1.0, 0.0 to ignore them).

From the JUnit plugin's documentation and source, as of October 2026. In a Jenkinsfile: junit allowEmptyResults: true, testResults: 'junit.xml'.

What Jenkins remembers about a test

Jenkins keeps more than GitHub Actions does. Each build's test result page lists every test, the job page draws a trend of passed and failed tests over the builds Jenkins keeps, and each test carries an age: how many builds in a row it has been failing. Here is our flaky test over five builds of the first Jenkinsfile, with no change to the test code:

BuildBuild resulttest_flakyAge
#1SUCCESSPASSED0
#2FAILUREFAILED1 (failed since #2)
#3FAILUREFAILED2 (failed since #2)
#4FAILUREFAILED3 (failed since #2)
#5SUCCESSPASSED0

Read build #4 and you would think the test broke at #2 and has stayed broken. On build #5 its age drops back to 0. The three failures are still on the test's History page, which charts its result over the builds Jenkins keeps, so the pattern is there for anyone who opens it. You have to go looking: the age counts a streak of failures, and nothing in Jenkins compares results on the same commit to call a test flaky. What describes a flaky test is its failure rate on the same code: that rate is what tells you whether to fix the code, fix the test, or quarantine it while you find out.

Add flaky detection: Mergify Test Engine on Jenkins

Test Engine keeps every test's result from every build, classifies each test as healthy, flaky or broken from that history, and can quarantine the flaky ones so they keep running without failing your build. On Jenkins, a test plugin reports by itself once two variables are set on the stage that runs the tests:

pipeline {
    agent any
    stages {
        stage('Test') {
            environment {
                MERGIFY_TOKEN = credentials('MERGIFY_TOKEN')
                // Jenkins does not set CI, and the test plugins only report when it is set.
                CI = 'true'
            }
            steps {
                // requirements.txt lists pytest and pytest-mergify
                sh 'python3 -m venv .venv && .venv/bin/pip install -q -r requirements.txt'
                script {
                    def status = sh(script: '.venv/bin/pytest --junitxml=junit.xml', returnStatus: true)
                    if (status > 1) {
                        error "pytest exited with ${status}"
                    }
                }
            }
            post {
                always {
                    junit 'junit.xml'
                }
            }
        }
    }
}

Store a Mergify application key with the ci scope as a Jenkins credential of kind Secret text with the ID MERGIFY_TOKEN. CI = 'true' is not optional on Jenkins: the plugins only report when CI is set, GitHub Actions and CircleCI set it for you, and Jenkins does not. The junit step stays, so the Jenkins test pages and trend keep working.

pytest-mergify, @mergifyio/vitest, @mergifyio/playwright, @mergifyio/jest and rspec-mergify work this way. Any other framework uploads the JUnit XML it already writes with mergify ci junit-process, from the Mergify CLI installed on the agent. In a multibranch pipeline, each branch and each pull request keeps its own test history. The Jenkins setup docs cover both, and the scheduled job that re-runs your suite on the default branch to find flaky tests faster.

FAQ

What is 'Publish JUnit test result report' in Jenkins?

The post-build action the JUnit plugin adds to freestyle jobs. You give it the path of the JUnit XML files your tests write, and Jenkins parses them after every build: a test result page per build, a trend graph on the job, and the build marked unstable when a test failed. In a Pipeline the same publisher is the junit step.

Why is my build FAILURE and not UNSTABLE when tests fail?

Because the test command failed first. When pytest, Maven or npm test exits non-zero, the sh step fails the stage and the build is FAILURE before junit runs; junit only adds UNSTABLE on top. To get UNSTABLE for test failures, run the tests with returnStatus: true and let junit decide, while still failing on exit codes that mean the runner itself broke.

Why does the junit step say no test report files were found?

The pattern matched nothing in the workspace. Check that it is relative to the workspace root, that the tests actually wrote the file (a crash before any test ran writes nothing), and, for Maven, that you match TEST-*.xml rather than every XML file in the reports directory. allowEmptyResults: true silences it, but then a job that stopped writing reports looks green.

Does the junit step run if the tests fail?

Only if you put it where a failure does not skip it. In a declarative Pipeline that is post { always { junit '...' } } on the stage. A junit step placed after the test command in steps is skipped as soon as the test command fails.

Can Jenkins show JUnit results on GitHub?

Yes, with the GitHub Checks plugin and a GitHub App credential: the junit step then publishes the results as a check run on the commit. Without them it logs 'No suitable checks publisher found' and carries on. skipPublishingChecks: true turns it off.

Can Jenkins detect flaky tests?

Not on its own. The JUnit plugin tracks how many consecutive builds a test has been failing, and resets that count to zero the first time it passes. A test that fails on two builds out of five reads as a fresh failure each time. Detecting flakiness needs results kept across builds and compared on the same commit, which is what Mergify Test Engine does from the same JUnit files or test plugins.

Last updated:

Your Jenkins test results, with a failure rate

Keep the junit step. Add flaky detection and quarantine to the same Jenkinsfile. Free for up to 5 users.