Skip to content

CI Visibility That Takes Action

Datadog can find and quarantine a flaky test. Mergify does that too, then carries the same signal into what gets merged.

Feature comparison

Datadog

Mergify

Flaky test detection

Test quarantine

Quarantine or disable

Auto-retry on failure

Test-level, in the tracer
Job-level, configurable rules

Pipeline traces

Test-focused analytics

CI framework support

Broad (APM-based)
15+ test frameworks

Merge queue integration

–
Native

GitHub-native workflow

Separate platform

Merge Protections (freezes, dependencies)

–
Full product

Setup complexity

Agent + library instrumentation
JUnit upload or GitHub App

Used by the platform teams that ship fast and keep CI green

From test health to merge decisions

Datadog's flaky test tooling ends at the test run. Mergify's keeps going until the pull request is on main.

Flaky tests quarantined automatically. No more blocked pipelines.
CI-native. Built for GitHub Actions and Jenkins, not bolted onto APM.
Action rules with no human in the loop: retry, skip, or quarantine on a condition.
Complete merge platform. CI insights feed merge queue decisions.
No Datadog dependency. Standalone CI intelligence, no existing spend required.
Predictable pricing. No per-committer add-on billing surprises.

Before CI Insights, we were wasting hours debugging CI issues. We now cut triage time in half within a week.

Liam Hartman

Liam Hartman

Platform Engineer at Vertex

CI Insights is just the beginning

Mergify connects test intelligence directly to your merge workflow. Detect problems, act on them automatically.

Detect

Flaky tests, slow jobs, failure patterns

Act

Quarantine, auto-retry, skip known failures

Merge

Smart queue decisions using CI health data

Evaluating CI visibility tools?

We'll walk you through how Mergify compares for your specific setup. No pitch, just answers.

Book a discovery call

What Datadog CI Visibility doesn't cover

Datadog gives you visibility. These are the gaps teams notice when they need action.

Quarantine stops at the test

A quarantined test no longer fails the build, which is the point. Whether the pull request is then safe to merge against today's main is a question Datadog leaves to someone else.

Retries live inside the test run

Auto Test Retries re-run a failing test case through the language library. Re-running a whole CI job that died on a runner or a network timeout is not part of it.

No merge queue

Datadog observes your CI but doesn't touch your merge process. PRs still pile up and conflict.

Requires Datadog platform

CI Visibility is an add-on to the broader Datadog platform. You need an existing Datadog contract to use it.

Per-committer pricing

CI Visibility bills per committer on top of your existing Datadog spend. Costs grow with your team, fast.

We had 40 known-flaky tests and no system to deal with them. Engineers just re-ran the suite and hoped. Test Insights quarantined the worst offenders on day one, and within a month we actually fixed most of them because we could finally see which ones mattered.

Lucas Meier

Lucas Meier

Staff Software Engineer at Norde

The pricing question

Datadog CI Visibility is a per-committer add-on to the broader Datadog platform. You need an existing Datadog contract, then pay extra for CI visibility on top. Costs scale with every developer who pushes code.

Mergify includes CI Insights in every plan. Predictable per-seat pricing, no surprise line items. Free for up to 5 users on private repos.

Your CI pipeline should fix itself.

CI Insights, test quarantine, merge queue, and merge protections. All included, up to 5 users free.