A reliable CI/CD pipeline is more than a sequence of build commands. It is a controlled path from a code change to a production release, with clear evidence, deliberate approvals, and a tested way back when something fails. This guide presents a reusable workflow for designing continuous integration and continuous delivery pipelines across common CI/CD tools and deployment environments.
Overview
Pipeline design starts with a simple question: what must be true before a change can move to the next environment? A good pipeline makes those conditions visible and repeatable. It builds the same artifact once, verifies it through increasingly realistic checks, and promotes that artifact rather than rebuilding it for each environment.
Continuous integration focuses on combining small changes frequently and finding defects early. Continuous delivery extends that process by preparing a verified release for deployment, often with an approval or policy decision before production. Continuous deployment goes one step further by deploying qualifying changes automatically. These models can share the same stages, but the required controls should match the risk of the application and the organization’s operating model.
A practical pipeline usually separates these concerns:
- Fast feedback: formatting, linting, unit tests, and basic validation run early.
- Build reproducibility: the pipeline creates a versioned package, container image, or deployment bundle.
- Confidence: integration, security, and configuration checks test the release more broadly.
- Controlled promotion: environments receive the same artifact in a defined order.
- Recovery: rollback or forward-fix procedures are documented and executable.
For a broader foundation, use this CI/CD pipeline best practices guide alongside the workflow below. The specific CI/CD tool matters less than the clarity of the stages, the quality of the handoffs, and the evidence retained for each decision.
Step-by-step workflow
1. Define the change and trigger conditions
Begin by deciding which events start the pipeline. Common triggers include a pull request, a merge into a protected branch, a version tag, or a manually approved release. Keep validation close to the change: pull requests should receive fast checks before merge, while release workflows can perform longer tests and deployment actions.
Make branch and trigger rules explicit. A deployment pipeline should not run accidentally because a file changed in an unrelated directory, and a production release should not depend on an undocumented manual command. If path-based triggers, schedules, or manual inputs are used, record their purpose in the repository.
2. Run fast checks first
The first stage should find inexpensive, high-frequency problems. Typical checks include dependency installation, formatting, linting, static type checks, unit tests, and validation of pipeline configuration. Parallelize independent checks when the CI/CD platform supports it, but avoid excessive parallelism that makes logs difficult to interpret or overloads shared services.
Fail with useful messages. A red job should identify the command, relevant file or test, and the next diagnostic step. Keep flaky tests out of the required path until their causes are understood; allowing intermittent failures to pass can hide genuine regressions.
3. Build and identify one immutable artifact
After the initial checks pass, create the deployable output once. Give it a traceable identifier such as a commit digest, release version, or build number, and retain the metadata needed to connect it to source code and dependencies. For container workloads, image optimization can reduce transfer time and simplify scanning; see the Docker image optimization checklist for build-focused considerations.
Do not rebuild the application separately for testing, staging, and production if the goal is artifact promotion. Environment-specific values should be supplied through configuration or deployment mechanisms, not by changing the application package after it has passed validation.
4. Test the artifact in a realistic environment
Deploy the artifact to a development or test environment and run integration, contract, end-to-end, and migration checks appropriate to the system. Test dependencies that are easy to overlook, including databases, queues, object storage, identity providers, and external APIs. Where a complete environment is expensive, use focused test doubles but reserve a path for validating real integration behavior.
Infrastructure changes belong in the same reviewable process as application changes. Validate infrastructure as code, preview its impact where possible, and check for unexpected drift before applying changes. The infrastructure drift detection guide can help shape that control.
5. Add security and policy gates
Security checks should be proportionate and placed where they provide actionable feedback. Useful controls may include dependency and license checks, secret detection, static analysis, container scanning, infrastructure scanning, and policy validation. A critical finding can block promotion, while lower-severity findings may create a tracked task depending on the application’s risk model.
Never place long-lived credentials directly in repository files or unredacted pipeline logs. Use the CI/CD platform’s secret storage or an external secrets manager, grant jobs only the permissions they need, and separate credentials by environment. For a practical control list, review the DevSecOps checklist for CI/CD pipelines.
6. Promote through environments deliberately
Promotion should be a controlled handoff, not a new build. A typical path is test, staging, and production, although the names and number of environments vary. Require an approval when the potential impact justifies human review, and make the approver’s decision auditable. Automated promotion is reasonable when the tests, permissions, monitoring, and rollback path provide sufficient confidence.
For production deployment, choose a strategy that matches the service’s risk profile. Rolling releases replace instances progressively; blue-green releases maintain two environments and switch traffic; canary releases expose a controlled portion of traffic before broader rollout. Compare these approaches in the deployment strategy guide.
7. Verify, observe, and recover
A successful deployment command does not prove that the service is healthy. Add smoke tests, health checks, error-rate monitoring, latency checks, and application-specific signals after deployment. Define a short observation period or automated analysis step before declaring the release complete.
Rollback should be a known operation with clear limits. It may mean returning traffic to a previous version, redeploying a prior artifact, disabling a feature flag, or applying a compatible database change. Database migrations deserve special care: prefer backward-compatible changes when releases may overlap, and document how a failed migration is handled. If a release cannot be safely rolled back, make the forward-fix process equally explicit.
Tools and handoffs
Most teams assemble a pipeline from several categories of tools rather than selecting one product for every concern. A source control platform receives changes and records reviews. A CI runner executes jobs. A package or container registry stores immutable artifacts. Infrastructure as code manages the target environment. Deployment tooling applies the release, while observability tools show whether the service remains healthy.
Choose integrations based on ownership and failure behavior. Every handoff should answer four questions:
- What input is being passed, such as a commit, artifact digest, configuration set, or approval?
- Who or what is authorized to perform the next action?
- What evidence is retained, including logs, test results, scan reports, and deployment records?
- What happens if the receiving system is unavailable or returns an unexpected result?
A tool directory can help compare categories without treating a product list as a pipeline design: browse the DevOps tools directory for CI/CD, security, Kubernetes, and observability options. For Kubernetes deployments, keep cluster access narrow and review service accounts and roles regularly; the Kubernetes RBAC guide provides a useful access-control framework.
Keep pipeline definitions close to the application when possible, and make reusable components versioned and reviewable. Shared templates can reduce duplication, but they also create a dependency: changes to a central template may affect many repositories. Test template updates in a representative project before broad adoption.
Quality checks
Evaluate a pipeline by its outcomes, not by the number of stages it contains. A useful review asks whether the workflow is fast enough for everyday development, strict enough for production risk, and understandable during an incident.
- Change traceability: can a production artifact be connected to its commit, review, build, tests, and deployment?
- Repeatability: can another engineer run the same process without undocumented local steps?
- Least privilege: can a compromised job reach only the resources required for its task?
- Failure clarity: do logs show where the pipeline stopped and what action is appropriate?
- Rollback readiness: has recovery been exercised rather than merely described?
- Environment consistency: are differences between test and production intentional and documented?
- Operational visibility: do deployment events appear alongside service health and incident data?
Track a small set of pipeline health measures over time. Useful signals include time from commit to deployable artifact, queue and execution duration, failure rate, rerun frequency, flaky-test rate, deployment frequency, and time required to recover from a failed release. Use these measures to locate friction, not to encourage teams to bypass safety checks.
When to revisit
Review the pipeline whenever the application, platform, or release process changes. A new runtime, deployment target, repository structure, identity provider, database migration approach, or security requirement can invalidate assumptions that were previously safe.
Set a regular review cadence as well as event-based reviews. During the review, inspect failed and manually rerun jobs, unused secrets, permissions, dependency caches, approval rules, artifact retention, test duration, and rollback evidence. Remove stages that no longer provide useful information, and split stages that have become too broad to diagnose.
When a production incident involves a release, update the pipeline while the lesson is clear. Add a regression test, strengthen a policy gate, improve deployment verification, or document a missing handoff. When a CI/CD platform changes its runner behavior, permissions model, caching features, or deployment integrations, revalidate the pipeline in a non-production environment before relying on the change.
To put this guide into practice, map your current workflow from commit to production on one page. Mark every build, test, approval, credential, artifact, and rollback action. Then choose one improvement with a measurable outcome—such as reducing feedback time, removing a rebuild, limiting a production credential, or rehearsing rollback—and review the result after the next few releases.
