CI/CD Pipeline
A CI/CD pipeline is the automated sequence of steps that turns a code change into running software. Build, run tests, package an artifact, deploy. Every team that ships software has one, whether it is a fifty-line GitHub Actions YAML or a multi-stage Jenkins setup that nobody fully understands. The shape is always roughly the same.
In one paragraph
A pipeline runs automatically on every code change. It builds the code, runs static checks and tests, produces an artifact (container image, binary, deploy bundle), and ships it through environments toward production. The CI half gives you confidence the change is safe. The CD half gives you the artifact, in a known good state, ready to deploy. The pipeline is defined as code in a YAML file inside the repository, which means changing the pipeline is a normal pull request like any other.
The canonical pipeline
flowchart LR Push["Push / PR"] Build["Build"] Lint["Lint & static checks"] Unit["Unit tests"] E2E["Integration<br/>+ E2E tests"] Package["Package /<br/>artifact"] Deploy["Deploy to<br/>staging"] Promote["Promote to<br/>production"] Push --> Build --> Lint --> Unit --> E2E --> Package --> Deploy --> Promote style Push fill:#E6F0FF,stroke:#43A7E5,color:#1A1D24 style Promote fill:#E6F8F2,stroke:#1CB893,color:#1A1D24
The standard shape of a CI/CD pipeline. Most pipelines are subsets of this, not supersets.
The order is not arbitrary. Each stage filters the work so the slow, expensive ones only run on changes that already cleared the fast checks.
Build
Compile the code. Resolve dependencies. Produce the binary, bundle, or container image. The build step is the gate that says "this change is at least syntactically real." A failure here usually means a typo or a missing import, and you want to find that out before any test ever runs.
Static checks
Linters, type checkers, formatters. These are fast (seconds, not minutes) and catch a real class of bugs that tests do not: unused imports, missing nullability handling, code style drift. They belong second because they cost almost nothing to run.
Unit tests
Fast tests with no I/O. Pure functions, business logic, data transformations. The unit suite should run in a couple of minutes for a healthy project. When it gets slower, the suite has usually drifted into integration territory and needs to be split.
Integration and end-to-end tests
Tests that touch a real database, a real browser, real external services. These are the slow, expensive part of the pipeline. Playwright tests live here. So do tests that hit a containerized database, an HTTP test server, or a docker-compose stack of the whole app. This stage is where time and money are usually spent, and where the question of when to run it becomes interesting.
Package
Build the artifact that will actually be deployed. For most modern apps, this is a container image pushed to a registry, tagged with the commit SHA. The package step happens after tests, on the same commit, so the image you ship is provably the image you tested.
Deploy to staging
Push the artifact to a staging environment that looks like production. Run smoke tests against the running deployment. This is the last check before production, and the first chance to catch deployment-level bugs (config errors, secrets misconfigured, database migrations that take longer than the deploy budget).
Promote to production
The same artifact moves from staging to production. In continuous delivery this is a manual click. In continuous deployment it is automatic on green. The choice between the two comes down to the team's tolerance for unattended pushes, not to any technical limitation in the pipeline.
The two stages teams should keep separate
Most pipelines that work in practice have two distinct shapes for two distinct moments: the per-push check and the pre-merge check.
| Stage | Runs when | What it does | Target time |
|---|---|---|---|
| Fast PR check | Every push to a PR | Build, lint, type-check, unit tests | Under 10 minutes |
| Heavy merge check | When the PR is queued to merge | Integration tests, E2E, security scans | Under 30 minutes |
| Deploy stage | After merge to main | Package, deploy to staging, smoke test, promote | Under 15 minutes per environment |
This is the basic shape of two-step CI. Engineers iterating on a PR get fast feedback. The expensive checks run once per merge, on a commit that is already trusted to be reviewable. The team pays for the slow stage on every merge, not on every push, and that is usually the difference between a CI bill that is reasonable and one that gets a meeting.
Where CI/CD pipelines quietly stop working
flowchart LR Build["Build ✓"] Lint["Lint ✓"] Unit["Unit ✓"] E2E["E2E ✗"] Block["Block merge"] Notify["Notify author"] Build --> Lint --> Unit --> E2E E2E --> Block E2E --> Notify style E2E fill:#FDECEA,stroke:#E53935,color:#1A1D24 style Block fill:#FDECEA,stroke:#E53935,color:#1A1D24
A failing pipeline blocks the merge. The harder question is what happens when the failure is not a real one.
Pipelines fail in obvious ways (test crashes, build errors) and in less obvious ways that compound over months.
Slow stages no one fixes
A pipeline that runs 35 minutes is not necessarily slower than the same pipeline a year ago. Stages add tests, tests add fixtures, fixtures add I/O. Each individual change is small. The aggregate effect, over a year of normal product work, is that PR feedback time doubles. Without a metric on pipeline duration, the slowdown is invisible until somebody notices that "CI takes forever" has become a permanent state.
Flaky tests that mask real failures
A test that fails on one run and passes on the next teaches engineers to click "rerun." Once that habit forms, a real failure goes through the rerun reflex first, and the pipeline's signal degrades into a question. The fix is to detect flakes systematically and quarantine them out of the required check until they are fixed. See the flaky tests guide for the patterns.
Pipeline drift
CI configurations age badly. Versions pin, then drift, then break when something upstream changes. Security patches lag. Cache invalidation logic falls behind the build graph. None of this shows up as a failing pipeline today, but it shows up as a 90-minute incident the week before launch. Treating the pipeline file as production code (review, test changes in branches, monitor) is what keeps this in check.
Two green PRs that break main together
The pipeline runs on each PR independently. If PR #1 renames a function and PR #2 adds a caller, both pass CI in isolation. When they land back to back, main is red. The pipeline cannot catch this by design; the fix is a merge queue that tests each PR against the actual future state of main before merging.
The pipeline tools landscape
The choice of pipeline tool is settled by the choice of forge in most modern setups.
- GitHub Actions. The default for GitHub repositories. YAML-based, large marketplace of community actions, fast PR feedback.
- GitLab CI. Native to GitLab. Integrated with merge requests, container registry, and the rest of GitLab's surface.
- CircleCI. Independent CI provider. Strong macOS support, fast on the right configuration, used heavily by mobile teams.
- Buildkite. Hosted control plane, self-hosted runners. The pattern of choice when CI needs to run on the team's own hardware.
- Jenkins. Long-standing, plugin-rich, still pervasive in enterprise. Powerful, operationally heavy.
- Bazel-driven pipelines. Inside large monorepos, the CI tool is often a thin shell around a build tool that does selective testing and remote caching. The CI tool runs Bazel; Bazel decides what runs.
FAQ
What is a CI/CD pipeline?
A CI/CD pipeline is the automated sequence of steps that runs on every code change, taking the change from a pushed commit through build, test, package, and deploy. The continuous integration half ensures the change builds cleanly and tests pass. The continuous delivery (or deployment) half ensures the resulting artifact reaches staging and production.
What are the stages of a CI/CD pipeline?
The common stages, in order, are: clone the repository, install dependencies, build, run static checks (linters, type checks), run unit tests, run integration and end-to-end tests, package the artifact (container image, binary, deploy bundle), deploy to staging, and promote to production. Most pipelines have a subset of these, not all.
What is the difference between CI and CD?
Continuous integration is about the code: every change is built and tested on a shared branch. Continuous delivery is about the artifact: the build output is always in a state that could be deployed to production. Continuous deployment goes further and actually ships every green commit. Most pipelines marketed as CI/CD cover the first two and leave the final promote-to-production step as a click.
What is a CI/CD pipeline example?
On GitHub, a typical pipeline is defined in .github/workflows/ci.yml: a workflow that runs on every push, with jobs for lint, test, build, and deploy. On GitLab, the same shape lives in .gitlab-ci.yml. On CircleCI, .circleci/config.yml. The structure is the same across tools, the syntax differs.
Why are CI/CD pipelines important?
Because they replace human checklist work with automated guarantees. Without a pipeline, somebody has to remember to run the tests, build the image, push it to the registry, and deploy it. Each of those steps gets skipped under pressure. A pipeline removes the option of skipping by making the automation the only path to production.
How long should a CI/CD pipeline take?
For the parts an engineer waits on (PR checks), the practical ceiling is 10 to 15 minutes. Past that, engineers context-switch and the pipeline becomes a tax on productivity. Heavy stages (E2E suites, integration tests, security scans) belong in a second stage that runs at merge time, not on every push.
What tools are used to build CI/CD pipelines?
The CI runner is one of GitHub Actions, GitLab CI, CircleCI, Buildkite, Jenkins, or (in monorepos) a build-tool-driven setup around Bazel. The deploy step usually talks to a container registry (ECR, GCR, Docker Hub) and an orchestrator (Kubernetes, ECS, Cloud Run) or a platform-as-a-service (Vercel, Render, Fly.io).
What is the difference between a CI/CD pipeline and a build pipeline?
A build pipeline produces an artifact: compile the code, package it, push the image. A CI/CD pipeline does that and also runs the tests, gates the merge on success, and pushes the artifact to staging and production. The build pipeline is one stage of a CI/CD pipeline.
A pipeline is only useful if it is fast, trustworthy, and never quietly broken.
CI Insights tells you which stages are slowing the pipeline down, which tests are unreliable, and where the build is failing. The merge queue tests every PR against the actual future state of main before it merges, so two green PRs cannot break it together.