Standard : Model iteration cycles are measured and continuously shortened
Purpose and Strategic Importance
This standard requires that the end-to-end cycle time for a model iteration — from identifying a need to improve to deploying a validated improvement to production — is measured, visible to the team, and subject to continuous improvement. It supports the policy of shipping AI incrementally to learn faster by treating iteration speed as a strategic metric rather than an incidental outcome of project planning. Teams that can iterate on models quickly compound their learning faster and respond to production signals before they become serious problems.
Strategic Impact
- Creates a competitive advantage in AI-driven products by enabling more learning cycles per time period than slower-moving competitors
- Shortens the window between identifying a performance problem and deploying a fix, reducing the duration of user impact
- Builds engineering discipline around the full model iteration lifecycle rather than treating it as unpredictable project work
- Encourages incremental improvement culture that favours many small validated improvements over rare large-batch releases
- Provides the organisation with data to make evidence-based investments in tooling and process improvements that accelerate delivery
Risks of Not Having This Standard
- Model iteration cycles grow progressively longer as technical debt, process overhead, and approval friction accumulate unnoticed
- Teams cannot predict delivery timelines for model improvements, undermining product planning and stakeholder confidence
- Slow iteration cycles prevent the organisation from responding to model degradation before it significantly impacts users
- Engineers become demoralised by the gap between identifying an improvement and seeing it in production
- Competitors with faster iteration cycles learn and adapt faster, eroding AI-driven product advantages
CMMI Maturity Model
Level 1 – Initial
| Category |
Description |
| People & Culture |
- Model iteration cycle time is unknown; teams have no sense of how long a typical improvement takes from start to deployment |
| Process & Governance |
- No tracking of model iteration lifecycle; each improvement is treated as a unique project with ad hoc planning |
| Technology & Tools |
- No tooling measures cycle time across the model development lifecycle stages |
| Measurement & Metrics |
- Cycle time is not measured; the team cannot identify where time is being lost in the iteration process |
Level 2 – Managed
| Category |
Description |
| People & Culture |
- Teams begin tracking the start and end dates of model improvement work; awareness of cycle time is growing |
| Process & Governance |
- Model iteration lifecycle stages are defined (identify, experiment, train, evaluate, deploy); time spent in each stage is recorded |
| Technology & Tools |
- A simple tracking board or project management tool captures model improvement items with stage transitions and timestamps |
| Measurement & Metrics |
- Mean cycle time per model iteration is calculated; the team discusses outliers in retrospectives |
Level 3 – Defined
| Category |
Description |
| People & Culture |
- Reducing cycle time is an explicit team goal; engineers are aware of where friction and delays occur in the iteration pipeline |
| Process & Governance |
- Cycle time targets are set per model type; improvements to cycle time are a standing item in engineering reviews |
| Technology & Tools |
- MLOps pipeline instrumentation captures time spent at each lifecycle stage automatically; flow metrics are visible on a live dashboard |
| Measurement & Metrics |
- Cycle time is broken down by stage (data prep, training, evaluation, deployment approval, deployment); bottlenecks are identified and addressed |
Level 4 – Quantitatively Managed
| Category |
Description |
| People & Culture |
- Teams are accountable for cycle time targets; iteration speed is reviewed alongside quality and reliability in engineering governance |
| Process & Governance |
- Process improvement initiatives are prioritised based on data showing where the most cycle time is being lost |
| Technology & Tools |
- Automated pipeline optimisation tools identify parallelisation opportunities and resource allocation improvements |
| Measurement & Metrics |
- Cycle time trend, throughput (iterations per month), and lead time distribution are tracked and compared to targets quarterly |
Level 5 – Optimising
| Category |
Description |
| People & Culture |
- Cycle time improvement is a shared organisational discipline; teams publish their improvement stories and tooling contributions |
| Process & Governance |
- Cycle time standards are continuously updated based on advances in tooling and the organisation's growing operational maturity |
| Technology & Tools |
- Automated retraining pipelines trigger, execute, and validate model iterations without manual intervention for defined change classes |
| Measurement & Metrics |
- Cycle time data drives strategic roadmap decisions about automation investment and pipeline architecture |
Key Measures
- Mean end-to-end model iteration cycle time (from improvement identification to production deployment) per model
- Cycle time broken down by stage: data preparation, training, evaluation, deployment approval, and deployment execution
- Cycle time trend per model over rolling 12-month period
- Percentage of model iterations completed within the target cycle time for their model risk tier
- Number of process improvement actions taken in the last quarter specifically to reduce model iteration cycle time