Draft:Test Maturity Model integration

TMMi

3. Background and History

Starting in 2005 a group of volunteers who were testing professionals and organisations with expertise in software testing and quality assurance got together to create the TMMi Foundation initiative. Brian Wells then officially launched the TMMi Foundation, a non-profit organisation at the ICSTest-UK conference in 2005. The conference dedicated a track to TMM (since TMMi did not exist then), including two case studies presented by the retailer Marks and Spencer and IT service provider Improve QS and an afternoon tutorial on TMM.

The TMMi model released in 2010 is aligned with international testing standards, and the syllabi and terminology of the International Software Testing Qualifications Board [lSTQB]. The TMMi Foundation has consciously not introduced new or their own terminology but reuses the ISTQB terminology. TMMi is an objective and business-driven model. Testing is never an activity on its own and the purpose behind TMMi was to introduce a structured staged approach to improve testing practices within software development by providing detailed guidance on test processes, techniques and management. The model contains stages or levels through which an organisation passes as its testing processes evolves. It was evident that whilst software development maturity models like CMMi existed, there was no such maturity model for software testing and its processes. Organisations using the TMMi framework could make incremental improvements by following the five maturity levels when assessing the test organisations maturity. TMMi rapidly gained acceptance within the industry mainly due to its alignment with the CMMi model and the model's ability to address software testing challenges. The TMMi model is considered a complementary practice to CMMi and has been translated into many languages. Over the years the foundation has benefited from a broad contribution from members and organisations to develop the Foundation based on the following principles:

1. Develop a model that is
   • Progressive and data-driven
   • Enables continious improvement
   • Inspired by CMMi
   • Integrated with project lifecycle
   • Measurable outcomes
   • Encourages institutionalised learning
2. The model was made available so that anyone could download and use the model as a reference for making improvements to testing.

5. Scope of the TMMi v1.3

The Test Maturity Model integration (TMMi) is a detailed and comprehensive framework for assessing and improving the test process maturity of an organization. The scope of the TMMi model encompasses various aspects of testing processes, aiming to provide a structured path for organizations to follow in order to achieve higher levels of testing maturity.

TMMi supports testing and test process improvement in systems and software engineering, encompassing both hardware and software aspects. TMMi addresses all test levels, from static to dynamic, including component, integration, system, and acceptance tests. It covers the four key areas of structured testing: lifecycle, techniques, infrastructure, and organization. While TMMi functions as an independent test improvement model, it can complement CMMI. It references CMMI practices, especially for supporting and management processes (like Configuration Management), without duplicating details.

TMMi is lifecycle-independent and can be applied to sequential, Agile, and DevOps models. A survey shows 78% of TMMi users utilize it in Agile contexts, and 57% in DevOps. The TMMi Foundation provides a detailed guideline to support Agile use, offering Agile alternatives for traditional practices. Additionally, a white paper explains how TMMi supports the DevOps approach . See https://www.tmmi.org/tmmi-documents/ for details. .

Organizations use TMMi for test process assessments to benchmark and identify improvements. The TMMi framework is a reference for these assessments, guided by the TMMi Assessment Method Application Requirements based on ISO 15504. TMMi can also complement CMMI assessments by integrating development and testing evaluations.

While TMMi offers a framework for test process improvement, it does not prescribe a specific approach like the IDEAL model. Successful improvement often starts with strong organizational support and the formation of a skilled test process group.

6. TMMi Maturity Levels

A maturity level within the TMMi represents as a degree of organizational test process quality. It is defined as an evolutionary plateau of test process improvement. Each level progressively develops a critical aspect of the organization’s test processes. TMMi framework has five maturity levels. Each maturity level defines what to implement in order to achieve the given level. Note that all organizations doing testing usually posess a minimum set of activities required by the model already on TMMi level 1, even if this level does not contain any goals that must be satisfied. The higher the maturity level the organization achieves, the more mature the test processes of organization are. The Process Areas associated to the maturity levels are the following:

Maturity Level 2 – Managed

• 2.1 Test Policy and Strategy
• 2.2 Test Planning
• 2.3 Test Monitoring and Control
• 2.4 Test Design and Execution
• 2.5 Test Environment

Maturity Level 3 – Defined

• 3.1 Test Organization
• 3.2 Test Training Program
• 3.3 Test Lifecycle and Integration
• 3.4 Non-functional Testing
• 3.5 Peer Reviews

Maturity Level 4 – Measured

• 4.1 Test Measurement
• 4.2 Product Quality Evaluation
• 4.3 Advanced Reviews

Maturity Level 5 – Optimization

• 5.1 Defect Prevention
• 5.2 Quality Control
• 5.3 Test Process Optimization

8. TMMi and CMMi

The TMMi model has positioned itself as being complementary to CMMI (Capability Maturity Model Integration ^[1]). While TMMi concentrates on testing processes, CMMI was conceived to help improve the entire software development process.

Testing within the SDLC (Software Development Life Cycle) is an important activity, emphasized by CMMI as needed during the entire software development. However, testing appears explicitly in the CMMI model in a few Practice Areas, and these provide a high-level guidance, without entering into details. Practice areas in CMMI connected to testing are: VER (Verification) and VAL (Validation). Elements of the integration testing can be found in PI (Product Integration). Peer Reviews, being a separate Process Area previously, became a Specific Practice of the VER Process Area. As testing is addressed at a rather high level in CMMI, organizations that wish to concentrate on improving their testing processes benefit from using TMMi. At the same time, having detailed testing processes will increase the maturity of a software development organization. Therefore, an organization that complies with the TMMi level 2 requirements is highly probable that will, at the same time, largely or fully fulfil the requirements for the CMMI practice areas Verification and Validation.

TMMi v1.3 shares a common structure with CMMI v1.3. The basic elements of the two models are the same: Specific Goals and Generic Goals are required model components, Specific Practices and Generic Practices are expected components, and both models have informative components (sub-practices, example work products, notes, examples, and references).Although declaring complementarity with CMMI (having both staged and continuous representations), TMMi was developed basically as a staged model. The Five Maturity Levels of TMMi are the same as CMMI Maturity Levels.

The TMMi does not emphasize Equivalent Staging, which has a major role in CMMI V1.3. Instead, the TMMi model descriptionemphasizes the importance of the Generic Goals and Generic Practices by describing them separately for each Process Area, giving hints on applying these generic practices / principles for each and every Process Area in a way that is best for the Process Area in case. Although TMMi can be used in isolation for the testing processes within a software development organization, it is often used as a complementary model to the CMMI. Practice areas from CMMI are generally not repeated within TMMi model description, but they are often referenced. Entire chapters from TMMi model description point out the interconnections between CMMI and TMMi practices. Here are some examples of such interconnections (the list is not exhaustive):

• Test planning can be addressed as part of the CMMI practice area Planning, and it can help implementing GP 2.2 (Plan the Process).
• CMMI Configuration Management practice area can implement GP 2.6 (Manage configurations) in full for all project-related process areas as well as some of the organizational process areas.

CMMI Process Quality Assurance practice areacan implement GP 2.9 (Objectively evaluate adherence) in full for all process areas.

• Stakeholder involvement for test planning can be addressed as part of the CMMI practice area Planning. The Test Planning process area may support generic practice GP 2.7 (Identify and involve the relevant stakeholders) for all project-related process areas by planning the involvement of identified stakeholders and documenting those in the test plan.
• CMMI Monitor and Controlpractice area provides support for the implementation of the TMMi process area Test Monitoring and Control. Project management practices can be re-used for test management.

CMMI Managing Performance and Measurement practice area provides support for theimplementation of the SG 3 Establish Test Performance Indicators of the TMMi process area Test Policy and Strategy.

• Requirements Development and Management: the implementation of this CMMI practice area is a mechanism for managing derived (work) products, such as the product risk analysis and test designs, and keeping them up-to-date. The practice regarding maintaining traceability possibly can be re-used within the Test Design and Execution TMMi process area. Practices from Requirements Development and Management can be re-used when developing test environment requirements within the TMMi process area Test Environment.
• CMMI Risk and Opportunity Management practice area can be re-used for identifying and controlling product risk and test project risks within the TMMi process areas Test Planning and Test Monitoring and Control.

The list of examples could be continued with ML[KY1] 3, ML4 and ML5 practices. For details see TMMi model description ( TMMi Reference model v1.3)

The world-wide surveys from 2021([10]) and 2023 ([11]) indicate that while initially positioned as a complementary model to CMMI, today TMMi is also widely used independently: around 40 %. of its users do not use CMMI - taking the average of both user surveys.

The actual version of TMMi is v1.3, which used the concepts of CMMI V1.3. In the meantime CMMI had new version release: CMMI V2.0 – with a major structural change- released on March 28. 2018, and CMMI V3.0 (minor updates compared to CMMI V2.0), released on 6 April 2023. Actually (August 2024) it is a subject of discussion if TMMi should keep its complementarity to CMMI and if yes, in what degree. [KY2]

---