Standard : CI pipelines are fast, reliable, and block on quality gates

Purpose and Strategic Importance

This standard ensures Continuous Integration (CI) pipelines are engineered for speed, reliability, and enforce essential quality gates automatically. Fast, stable pipelines shorten feedback loops, reduce the cost of change, and ensure defects are caught early without slowing down delivery.

Aligned to our "Automate Everything Possible" and "Engineering Excellence First" policies, this standard improves team velocity, delivery confidence, and technical quality. Without it, delivery slows down, errors creep into production, and engineering teams lose confidence in their ability to release safely at speed.

Strategic Impact

Clearly defined impacts of meeting this standard include faster feedback cycles, higher quality releases, reduced rework, improved engineering confidence, and accelerated time to market.

Risks of Not Having This Standard

  • Long or unreliable builds that delay feature delivery
  • Higher defect rates due to missed quality checks
  • Reduced engineering trust in pipeline results
  • Increased manual intervention and rework
  • Difficulty scaling or sustaining continuous delivery practices

CMMI Maturity Model

Level 1 – Initial

  • People & Culture

    • Developers rely on manual builds or scripts.
    • No shared ownership of pipeline health.

  • Process & Governance

    • Builds are inconsistent across teams and environments.
    • No standard for pipeline quality gates.

  • Technology & Tools

    • Minimal or no use of CI tools.
    • Build failures are diagnosed manually.

  • Measurement & Metrics

    • Build success/failure rates are not tracked.
    • No visibility into average build or feedback time.

Level 2 – Managed

  • People & Culture

    • Teams begin to adopt basic CI practices.
    • Ownership of build scripts starts to emerge.

  • Process & Governance

    • Pipelines are manually triggered and inconsistently maintained.
    • Some manual checkpoints for quality (e.g., basic linting or tests).

  • Technology & Tools

    • CI tools are used but lack standardisation across teams.
    • Build status dashboards are partially adopted.

  • Measurement & Metrics

    • Build pass rates and durations are captured sporadically.

Level 3 – Defined

  • People & Culture

    • Teams treat pipeline health as critical to delivery flow.
    • Peer support and reviews include pipeline reliability.

  • Process & Governance

    • CI pipelines are triggered automatically on code changes.
    • Quality gates (unit tests, static analysis, code coverage) are enforced.

  • Technology & Tools

    • Standardised CI platforms and templates are adopted.
    • Build failures trigger automated notifications to responsible teams.

  • Measurement & Metrics

    • Time from commit to feedback is measured and targeted.
    • Pipeline reliability (e.g., flakiness rates) is tracked.

Level 4 – Quantitatively Managed

  • People & Culture

    • Teams are proactive in optimising pipelines for speed and resilience.
    • Build health is reviewed regularly in retrospectives.

  • Process & Governance

    • Quality gate thresholds are continuously tuned based on risk and goals.
    • Failed builds block merges until resolved.

  • Technology & Tools

    • Parallelisation, caching, and selective test execution are leveraged.
    • Automated rollback and retry mechanisms are in place for flaky tests.

  • Measurement & Metrics

    • Targets for feedback times (e.g., <10 minutes) are tracked and met.
    • Flaky tests are measured, prioritised, and eliminated.

Level 5 – Optimising

  • People & Culture

    • Pipeline performance and quality are ingrained in engineering culture.
    • Teams continuously improve pipelines based on live telemetry.

  • Process & Governance

    • Pipelines adapt automatically based on risk profile (e.g., faster validation for smaller changes).

  • Technology & Tools

    • Advanced pipeline analytics and optimisations (e.g., ML-assisted test selection) are used.
    • Self-healing pipelines detect and remediate issues without human intervention.

  • Measurement & Metrics

    • Build-to-feedback times continue to improve year over year.
    • Incidents caused by missed quality gates approach zero.

Key Measures

  • Average time from commit to feedback (build duration)
  • Build success rate (pass/fail percentages)
  • Flaky test rate (number of intermittent failures)
  • % of builds enforcing critical quality gates
  • % of blocked merges due to failing pipelines
  • Developer satisfaction with CI feedback speed and reliability