Standard : AI tooling is selected with developer experience as a primary criterion
Purpose and Strategic Importance
This standard requires that when AI tooling — including IDEs, ML platforms, experiment tracking tools, data labelling environments, and deployment infrastructure — is selected or evaluated, developer experience is treated as a primary criterion alongside capability and cost. It supports the policy of making AI work sustainable for the people who build it by recognising that tools shape the working lives of AI practitioners profoundly. Poor tooling creates friction, increases cognitive load, and drives burnout and attrition in teams already operating in a demanding discipline.
Strategic Impact
- Reduces the cognitive overhead that poor tooling imposes on AI practitioners, freeing capacity for higher-value technical work
- Decreases time-to-productivity for new team members who must learn the tooling ecosystem alongside the domain
- Reduces attrition by removing a common source of frustration cited by experienced AI engineers when leaving organisations
- Accelerates delivery by removing workflow friction that slows model development, experimentation, and deployment cycles
- Creates a virtuous cycle in which good tooling investment attracts quality talent who expect a mature engineering environment
Risks of Not Having This Standard
- AI engineers waste significant working time navigating poor tooling rather than solving domain problems
- Tool selection is driven by vendor relationships or infrastructure convenience rather than the needs of the practitioners who use the tools daily
- Experienced AI engineers leave organisations where tooling is chronically inadequate; the cost of attrition vastly exceeds the cost of better tooling
- Poor tooling increases error rates by forcing engineers into workarounds that introduce manual steps and reduce reproducibility
- The organisation falls behind in talent attraction as engineer communities share tooling quality signals on forums and in hiring conversations
CMMI Maturity Model
Level 1 – Initial
| Category |
Description |
| People & Culture |
- Tool selection is made by IT procurement or senior management without consulting the engineers who will use the tools daily |
| Process & Governance |
- No developer experience criterion in tool procurement; decisions are made on cost and vendor relationship alone |
| Technology & Tools |
- The tooling estate is fragmented and inconsistent; engineers maintain personal setups outside the standard environment |
| Measurement & Metrics |
- Developer satisfaction with tooling is not measured; frustration is expressed informally in team meetings |
Level 2 – Managed
| Category |
Description |
| People & Culture |
- Engineers are consulted during tool selection; developer experience feedback is gathered informally |
| Process & Governance |
- Tool selection requests include a section on expected developer experience impact; engineer input is documented |
| Technology & Tools |
- A common tooling baseline is agreed per discipline; deviations are discussed and documented |
| Measurement & Metrics |
- Developer satisfaction with the tooling environment is included in team health surveys |
Level 3 – Defined
| Category |
Description |
| People & Culture |
- Developer experience is a named evaluation criterion with a defined weight in the tool selection framework |
| Process & Governance |
- A formal tooling evaluation process includes developer trials, structured feedback collection, and DX scoring before procurement decisions are made |
| Technology & Tools |
- A curated AI tooling catalogue is maintained; recommended tools have been validated by practitioners and meet defined DX standards |
| Measurement & Metrics |
- Developer experience scores per tool are tracked; tools that consistently receive low DX scores are reviewed for replacement |
Level 4 – Quantitatively Managed
| Category |
Description |
| People & Culture |
- Tooling investment decisions are informed by data on developer time saved, error rate reduction, and satisfaction improvement |
| Process & Governance |
- Tooling review cycles are defined; tools that fall below the defined DX threshold are scheduled for replacement regardless of contract status |
| Technology & Tools |
- Developer experience metrics (time-to-first-experiment, onboarding time, friction event rate) are instrumented in the tooling environment |
| Measurement & Metrics |
- DX score trends, tooling-related time waste, and correlation between tooling quality and delivery throughput are reported quarterly |
Level 5 – Optimising
| Category |
Description |
| People & Culture |
- The organisation contributes to open source AI tooling improvements and shares DX learnings with the community |
| Process & Governance |
- Tooling standards are continuously updated based on practitioner feedback and advances in the AI tooling ecosystem |
| Technology & Tools |
- Engineers have budget and process to experiment with and advocate for emerging tools that could improve DX before organisation-wide adoption |
| Measurement & Metrics |
- DX investment ROI is calculated by correlating tooling improvements with delivery throughput, attrition reduction, and onboarding time reduction |
Key Measures
- Developer experience score per AI tool measured in quarterly tooling surveys
- Mean time-to-productivity for a new AI engineer using the standard tooling stack
- Percentage of AI tooling procurement decisions where a developer experience trial and scoring process was completed
- Number of tooling-related friction events reported per sprint (wasted time, workarounds, outages)
- Voluntary AI engineer attrition rate and tooling satisfaction cited as a factor in exit interviews