Convergence Analysis of ISO/IEC 12207 and CMMI-DEV: Complementary Result from Systematic Literature Review

The organizations and people are demanding more and better software products and services, which implies adequate processes for its development. In the context of the software industry, there are two models, the CMMI-DEV and ISO/IEC 12207 that are influencing it. Though, they are evolving separately, recurrently they have been compared to determine its coverage (in both directions). In this study is analyzed the results of those comparisons (partials and completed) to determine if the models ISO/IEC 12207 and CMMI-DEV converge at processes level. This study identified eight articles where the comparison is carried out between ISO/IEC 12207 and CMMI-DEV. The results show that technique most used is the mapping comparisons between the models and according to the analyzed studies is not possible to determine whether there is convergence in the time. However, we found some items and criterions for use in comparisons.


Introduction
The development of Information Technology (IT) has revolutionized the world, and the level of use of software has grown significantly, which generates a market pressure for better software products and services [1]. This new context requires that organizations that develop software products or services adopt good practices that allow them to be productive and that their software products or services have the right quality for both its internal and external customers [2]. To achieve this, these organizations must implement processes aligned to their operational and

Background and Related Work
During the previous review, we found some studies about methods or techniques useful for the task of analyzing the coverage of different process models among which may include the following:  In [11], it is determined that the mapping and comparison techniques are widely used in the harmonization of models and standards such as CMMI, ISO 9001, ISO/IEC 15504 and ISO/IEC 12207.
 In [12], the relationship between maturity levels of CMMI-DEV and process capability of ISO/IEC 15504 is researched through a mapping.
In the context of the software industry we found that CMMI-DEV is one of the most popular models for process improvement [13] and ISO/IEC 12207 is the standard for defining processes in the software life cycle [14]. For this reason, there are research studies showing interest in knowing the coverage of both, for example we can cite the following works:  In [15], a comparison between the process areas of CMMI-DEV v1.2 and processes described in ISO/IEC 12207:2008 is performed. Based on these results, the relationship between CMMI-DEV and ISO/IEC 15504-7 is studied to identify the degree of coverage of the maturity levels of CMMI-DEV against ISO/IEC 15504-7.
 In [16], the relationship between CMMI, the Process Assessment Model described in ISO/IEC 15504-2 and the Process Reference Model described in ISO/IEC 12207 AMD 1/2 is analyzed.

Research Protocol
A Systematic Literature Review (SLR) according to [17] and [18] is a technique that aims to identify, critically evaluate and synthesize relevant studies on a topic or research question.

SLR Fundamentals
In the present work, the process proposed by [17] for performing a SLR was used, which involves a series of iterative activities and grouped into three stages: plan, perform and report the SLR. As part of the planning stage of the SLR, the following research questions were defined:  RQ-1. Are there any research studies related to the comparison between ISO/IEC 12207 and CMMI-DEV?
 PI-2. What are the versions and methods used in the comparison of ISO/IEC 12207 and CMMI-DEV?
 RQ-3. What are the elements used in the comparison between ISO/IEC 12207 and CMMI DEV?
 RQ-4. What are the comparison criteria adopted to compare ISO/IEC 12207 and CMMI-DEV?
 RQ-5. What has been the degree of coverage obtained from the comparison of ISO/IEC 12207 and CMMI DEV?
 RQ-6. How have evolved the results of comparisons between ISO/IEC 12207 and CMMI DEV?
 RQ-7. What have been the approaches in comparative analysis regarding the unidirectional or bidirectional comparison?
 RQ-8. What aspects of subjectivity are discussed in comparisons between ISO/IEC 12207 and CMMI?
In order to obtain the search string, the PICO strategy was used (Population, Intervention, Comparison, Outcome) presented by Pai et al. cited in [19]. Our process was carried out as one iterative where some adjustments were made based on the results and databases, as described [17]. Table 1 shows the main search terms based in PICO criteria.

Element Value
Population "ISO/IEC 12207" OR "ISO 12207" OR "12207" Intervention "CMMI-DEV" or "CMM-SW" or "CMM" or "CMMI" or "MATURITY MODEL" Comparator Not Apply Outcome "COMPAR*" or "MAPP*" or "HARMONIZA*" or "COVER*" or "CORRESPONDEN*" or "BENCHMARK*" or "HOMOGENIZA*" or "INTEGRA*" or "ANALIS*" or "STUD*" or "MEASU*" Search Strategy Population, Intervention and Outcome Taking into account the data sources identified by [20] and [18] as the most important in the domain of software engineering and the number of studies obtained in a first search, the following data sources were selected: Search strings used in selected data sources are shown in Table 2. Quality evaluation study was performed using a questionnaire with seven quality criteria based on [18] listed in Table 3. In order to score each criterion, the value of 1 was set in case of an affirmative answer and 0 in case of a negative response. Consequently, each study can be rated between 0 and 7.

SLR Protocol
The protocol developed in this work has been structured following the guidance of Kitchenham [17] which included: the initial selection of studies obtained from the execution of the search strings in the selected data sources. Later stages were related to the application of the criteria for inclusion and exclusion where items were reviewed considering the title of the study, abstract review and content review. The criteria for inclusion and exclusion used in the SLR were:  Inclusion Criteria: papers belonging to indexed Digital Libraries and that have been published in English were accepted. As well as papers developed by prestigious organizations in the subject, such as the Software Engineering Institute (SEI) or CMMI Institute. Papers that present a partial or complete comparison between ISO/IEC 12007 and CMMI in any of its versions, performing a comparison between ISO/IEC 12207 and CMMI based on a process model harmonization method or technique were included.

Results
The results of the SLR are presented next:

Studies Classification
The search procedure was executed in the selected data sources in October 2015. The results of this run are shown in Table 4, the initial results showed a total of 864 papers, after a preliminary review 815 papers were excluded because they were unrelated with the subject of this study and 21 papers were repeated. After abstract review, 4 papers were excluded. Then, after content review, 5 items were excluded. Finally, 19 papers were selected for this study.

Quality Evaluation
To assess the quality of each paper obtained as a result of the execution of the search string, the quality evaluation criteria defined earlier were applied. After this evaluation, 11 papers that obtained a score of zero were removed, finally being selected 8 papers. The results of evaluation of quality of the selected primary studies are shown in Table 5.

Results Analysis
From the results obtained after quality evaluation, it can be observed that the publication of the new version of the standard ISO/IEC 12207 in 2008 led to an increase of studies related to comparisons with CMMI. Instead, release of version v1.3 of CMMI in 2010, only derived a research paper, as shown in Table 6. Additionally, it is observed that most of published papers related to comparison between the standard ISO/IEC 12207 and CMMI were found in Scopus, as shown in Fig. 1.

Discussion
This section presents the answers to each research question presented in Section 3.1.

RQ-1. Are there any research studies related to the comparison between ISO/IEC 12207 and CMMI-DEV?
According to the results, studies related with comparison between ISO/IEC 12207 and CMMI in its different versions were founded. The following studies make a comparison between the two models as the main objective of their research:  In [21], a comparison between CMM (SW-CMM) v.1.1 and IEEE/EIA 12207 is performed in order to serve as a guide to meet the requirements for implementing IEEE/EIA 12207 using CMM.
 In [13], a bidirectional comparison between ISO/IEC 12207:2008 and CMMI v1.2 is performed to detect similarities and differences between the two models and to know the potential of each model to support an improvement process.
 In [15], a comparison between the process areas of CMMI-DEV and the processes described in standard ISO/IEC 12207:2008 is performed. Based on these results, the relationship between CMMI-DEV and ISO/IEC 15504-7 models is investigated, with the aim of identifying the degree of coverage of maturity levels in CMMI-DEV in relation with the ISO/IEC 15504-7.
 In [22], we found information about how related are the practices described in both models, with the aim of helping organizations interested in introducing or improving their procurement practices for products and services using these two models.
 Information is provided in [24] about how the practices described in both models are related.
 In [25] the compatibility of measurement processes in CMMI DEV 1.3 and ISO/IEC 12207: 2008 is verified performing traceability between the two models.
According to [22], studies related to the standard ISO/IEC 15504-2:2004 involve ISO/IEC 12207: 2008 because this is the process reference model (PRM) suggested by ISO/IEC 15504. Having this in mind, the following studies that show a comparison between ISO/IEC 12207 and CMMI as an intermediate step in its goal to compare CMMI and the standard ISO/IEC 15504 were included:  In [16], the relationship between CMMI, the Process Assessment Model described in ISO/IEC 15504-2 and the Process Reference Model described in ISO/IEC 12207 AMD 1/2 is analyzed.
 In [23], a comparison between models of ISO/IEC 15504-7 is developed in order to obtain information about the degree of coverage of maturity levels described in both models. This comparison is done in two stages, i) a mapping between processes of ISO/IEC 12207:2008 and the process areas of CMMI-DEV is developed ii) processes of ISO/IEC 12207:2008 and ISO/IEC 15504-5 are mapped.
From the papers identified, we can say that there are several works of comparison between the two models, showing the interest of the scientific and technological community about their coverage and evolution.

PI-2. What are the versions and methods used in the comparison of ISO/IEC 12207 and CMMI-DEV?
The technique most used in the comparison of models ISO/IEC 12207 and CMMI is the mapping, as detailed in the studies of [21], [16], [13], [15], [23], [22], [24], while traceability is the technique used in [25]. The versions of ISO/IEC 12207 and CMMI used in the comparisons have been different, as shown in Table 7.

Number of Papers by Source
In addition, studies [15], [23], [22], [24], in order to ensure the reliability of the results obtained in comparisons define a procedure and guidelines for making the mapping step by step. This process defines two roles: (i) performers and (ii) Reviewers. Also, 5 activities are defined and are presented below:  Analysis of the models to compare  Design of the comparison  Execution of the comparison  Comparison results presentation.
 Comparison results analysis.
From the papers identified, we can say that the comparison technique most used is the mapping of models between ISO/IEC 12207 and CMMI-DEV.

PI-3. What are the elements used in the comparison between ISO/IEC 12207 and CMMI DEV?
The elements used in the comparison of ISO/IEC 12207 and CMMI have been different depending on the criteria and objective of the researcher:  The key process areas (KPA) of CMM (SW-CMM) v.1.1 and the processes of IEEE/EIA 12207 [21].
 The activities and tasks of ISO/IEC 12207:2008 and the generic practices (GP) and specific practices (SP) by process area (PA) of CMMI v1.2 [13].
 The expected outcomes of "Measurement Process" from ISO/IEC 12207:2008 and specific practices from "Measurement and Analysis" Process Area in CMMI DEV 1.3 [25].

Study Standard used
Baldassarre [15] CMMI From the studies identified it can be seen that the method of mapping is the most used method for model comparison (07 of 08 studies) versus traceability method (only 01 of 08 studies). In addition, from the identified studies, it can be seen that different elements have been used to do the mapping, which can be understood as the level of granularity adopted for comparing models. This level of granularity determines that the central elements of each model are used on an unbundled basis to a greater or lesser extent.

PI-4. What are the comparison criteria adopted to compare ISO/IEC 12207 and CMMI-DEV?
From selected papers, the following comparison criteria have been identified:  In [21] a mapping between requirements to implement IEEE/EIA 12207 that are covered by the key process areas of CMM (SW-CMM) v.1.1.
 In [16] a low level comparison using generic practices from CMMI and the processes of ISO/IEC 12207 Amd 1/2 as process reference model for ISO/IEC 15504-2.
 In [13] characterizes the level of relationship between the elements of ISO/IEC 12207 and CMMI, this study use a confidence factor based on a numerical scale, whose values are shown in Table 8. 100 Strong There is a strong relationship between ISO requirement and the CMMI practice.
The criteria used by [25] is to analyze the similarity between descriptions of each expected outcome of "Measurement Process" of ISO/IEC 12207:2008 and each specific practice of the "Measurement and Analysis" Process of CMMI Dev v1.3.
From selected studies, it can be appreciated that a group (4 of 8) papers use an scheme based on a quantitative rating that has been taken from ISO/IEC 15504; which it's the standard used for evaluation of process capability for ISO/IEC 12207.

PI-5. What has been the degree of coverage obtained from the comparison of ISO/IEC 12207 and CMMI DEV?
 In [21] it is determined that at high-level, the standard ISO/IEC 12207 covers 15 of 18 key process areas from CMM. It also determines that the processes of Supply, Operation and Maintenance have no relationship to any key process area of CMM. But it provides a detailed list of the requirements of IEEE/EIA 12207 which are not covered by CMM. At low level, only shows a list with requirements of IEE/EIA 12207 that are not covered by the elements of CMM.
 In [16] does not include numerically the degree of coverage as a result of the comparison. However it determines that the specific practices of CMMI are addressed in almost all processes of ISO/IEC 12207 Amd 1/2. The degree of processes coverage that is not addressed is shown in Table 9.
In the other way of comparison, the specifics practices of the CMMI process area "Decision Analysis and Resolution" is beyond the scope of ISO/IEC 12207 Amd 1/2.
 The degree of coverage between the processes of ISO/IEC 12207:2008 and CMMI-DEV v.1.2 process areas according to [15], [23] is shown in Table 10.    The degree of coverage between the processes of ISO/IEC 12207:2008 and the sixteen process areas of CMMI-ACQ, which is part of the CMMI framework, are shown in Table 11 [22], [24].
 According to [25], there is a strong coverage between the "Measurement Process" of ISO/IEC 12207:2008 and the process area "Measurement and Analysis" from CMMI DEV v1.3 which agrees with the result obtained by [15], [23].  Comparisons made in the selected studies show that twenty-two (22) process areas (PA) of CMMI-DEV 1.2 are covered by the processes of ISO/IEC 12207:2008. While for the other versions of ISO/IEC 12207 and CMMI there are three (03) CMMI process areas not covered in any way by ISO/IEC 12207 [21]. In this sense, the degree of coverage of ISO/IEC 12207: 2008 and CMMI-DEV 1.2 is very high.

PI-6. How have evolved the results of comparisons between ISO/IEC 12207 and CMMI DEV?
For comparisons between ISO/IEC 12207 and CMMI DEV we have found that different levels of detail are used. Thus a high level comparison of the results obtained was made. These results were obtained from the data available in the selected studies. Table 12 shows these results and indicates the level of coverage obtained. In Fig. 2, the convergence between both models is presented.
Data in Table 12 show that at high-level, the results of the comparisons have evolved convergently to versions of ISO/IEC 12207 and CMMI v1.1, while this decreases for ISO/IEC 12207:2008 and CMMI-DEV v1.2.  In [15], [23], [22], [24] we can observe that in comparisons at low level of abstraction, the degree of relationship founded between ISO/IEC 122207 and Process Areas from CMMI, depends on the chosen direction. That is, it's not bidirectional and this should be taken into account when presenting the results of a comparison. The approach used in [15], [23], [22], [24] for the comparison is from ISO/IEC 12207 to CMMI. The study presented in [13] performs a unidirectional mapping from ISO/IEC 12207 to CMMI. But it considers that performing bidirectional mappings are important for the following reasons:  There are organizations that already have ISO certification and would like to move toward CMMI maturity levels and vice versa.
 There are organizations primarily interested in the impact of using multiple frameworks in improvement processes.
In addition, [13] states that performing bidirectional mappings allows to learn and know each framework individually in detail and it has potential to support an improvement process.
Studies [16], [21] y [25] do not address the effect of directionality of the comparison between ISO/IEC 12207 and CMMI and all of them present a unidirectional mapping. But [21] and [25] state that directionality of the mapping is from ISO/IEC 12207 to CMMI. For [25] the use of this directionality allows to find a set of requirements that a company that already implements CMMI must meet to implement ISO/IEC 12207 as part of their improvement processes. While [16] states that the directionality of the comparison is from CMMI to ISO/IEC 12207 AMD 1/2 in order to analyze the relationship between CMMI and the Process Assessment Model described in ISO/IEC 15504-2 and Process Reference Model described in ISO/IEC 12207 AMD 1/2. Finally, [25] performs a one-way comparison from ISO/IEC 12207 to CMMI, with the aim of analyzing their similarities or differences that allow them to propose a hybrid measurement process for ISO/IEC 15504, ISO/IEC 12207: 2008 and CMMI-DEV v1.3.
All selected studies have performed mappings using a unidirectional approach between ISO/IEC 12207 and CMMI. But only a group of them (05 of 08) discusses the effect of directionality in the mapping performed and the level of abstraction used.

PI-8. What aspects of subjectivity are discussed in comparisons between 12207 and CMMI?
In [13] states that the model harmonization procedure between ISO/IEC 12207 and CMMI-DEV is always imperfect and subjective, so determining the level of granularity between mapping models is also subjective. According to [13], a high-level mapping, for example between the main sections of ISO/IEC 12207 and CMMI-DEV Process Areas could not provide enough detail to determine their similarities or differences. While a very low level mapping can provide a high number of similarities but provide little information about the strength of mapping between ISO/IEC 12207 and CMMI-DEV. As mentioned, the mapping is very subjective and different people can create similar but with very different results mappings.
For this reason, in [13], in order to measure the level of coverage between each element being compared, characterizes each mapping as strong, medium or weak, and to describe the strength of the mapping uses a confidence factor as shown in Table 8.
Given the aspects of subjectivity of the comparison between ISO/IEC 12207 and CMMI-DEV, [15], [23], [22], [24] used a comparison scale similar to [13], but based on a quantitative rating that has been taken from ISO/IEC 15504 and that allows to classify the relationships between the elements compared according to their degree of similarity or difference. But [15], [23], [22], [24], in order to increase the reliability of the results obtained in the mapping, introduce peer reviews to resolve any discrepancies between the performers and thus validate the results of the mapping performed. These activities, according to [15], [23], [22], [24] allow minimizing subjectivity in the comparison.
The studies presented in [16], [21] and [25] do not address the aspect of subjectivity when conducting mapping between ISO/IEC 12207 and CMMI.
Aspects of subjectivity discussed in a group of selected papers (05 of 08) are related to the description of the strength of the comparison between the elements. To characterize this comparison and reduce subjectivity in the mapping, they employ either a scale of quantitative comparison, confidence factors or peer reviews.

Conclusions and future work
While there are studies related to the comparison of two models or standards throughout the years, few studies were found that were related specifically to the comparison between CMMI and ISO/IEC 12207. In the selected studies we found that mapping is the technique most widely used in comparing CMMI models and ISO/IEC 12207 standard, but the degree of similarity or difference obtained it depends on the level of detail of the elements used in the comparison. No detailed comparisons that have used the latest versions of the standard ISO/IEC 12207 and CMMI model were found.
From the obtained studies it cannot be determined the level of convergence of the models CMMI and standard ISO/IEC 12207 throughout its different versions due to the following:  They do not use a common scale for comparing the similarities or differences between the two models that allows to reduce subjectivity in comparison.
 The comparison elements and the level of detail of the comparisons of the two models are different.
From the studies obtained, we conclude that results of a mapping between ISO/IEC 12207 and CMMI DEV depend on the directionality used and the level of detail or granularity of the elements used in the comparison. Therefore, as an initial step is very important to set the direction of mapping being performed and the level of granularity to use. This will always depend on the expected objective of the mapping.
In addition, to reduce the level of subjectivity in the comparison of the elements of ISO/IEC 12207 and CMMI-DEV, it is important to use a comparison scale to provide information of the strength level of similarities/differences between compared elements. And in order to further reduce the level of subjectivity of comparisons, it's possible to perform a peer review that would help resolve any discrepancies that may be found during the process and thus validate the results of the mapping.
As future work we propose the comparison of the standard ISO/IEC 12207:2008 and CMMI-DEV 1.3 following the "procedure for mapping models" proposed by Pino et al. [15], [24]. Thus, in order to have a mapping to use the latest versions of the standards and provide additional data to facilitate the analysis of convergence between the two models. This will help companies that have Implemented CMMI-DEV meet the requirements needed to incorporate ISO/IEC 12207 as part of their processes of software process improvement.