Mapping Process Capability Models to Support Integrated Software Process Assessments

Software process assessments have been used to verify the conformance with quality reference models or standards, usually in a context of software process improvement programs. Most of these assessments are concerned with just one specific model or standard. However, organizations could be interested in more than one model, being necessary to offer an integrated assessment that allows evaluate the conformance with different models at the same time, reducing time and costs. To do this, it is necessary to have some kind of mapping among the models to permit some level of automation. In this context, this paper presents a method to mapping process capability models and standards, including its application. A software tool is also presented to demonstrate how to use the resulting mapping in a real life situation.


INTRODUCTION
During a software improvement process program, it is usual to make assessments to check the current capability of organizational processes in comparison to a reference model.The goal of these assessments is not only to point the adhesion of the organization in relation to a model or standard, but also to follow the evolution of the implemented improvements, identifying its strengths and weaknesses to guide the continuity of the work.
Typically, such assessments consider only one single reference model as the internationally known CMMI-DEV [1], the international standard ISO/IEC 15504 [2] or, in Brazil, the model MPS.BR [3].However, market requirements may demand that the organizations show adjustment to more than one model or standard.In Brazil, for instance, it is common that companies initially assessed in MPS.BR also have interest in CMMI assessment and vice-versa [4].Thus, programs that consider more than one model as a reference for improvement may need integrated assessments.The term "integrated assessment" is used here to define an assessment that considers more than one reference model at the same time.One issue regarding these assessments is that they demand assessors with specialized knowledge upon all the models involved.Furthermore, the complexity involved in the assessment fairly increases, once each observed evidence must be mapped towards one result or practice of each model.This way, it makes sense to try some level of automation to support integrated assessments.
In order to make an integrated assessment attainable, it is necessary to establish a mapping among the considered reference models.In the context of this work, these mappings are made automatically, with no need of an experienced assessor in all considered models.Nevertheless, the final result of an integrated assessment should not be used as the only and absolute truth by the organization, but it should, instead, be understood as an approximate view in accordance with the processes using other models.Another limitation is that it will demand an experienced assessor in one of the models.
One of the problems in mapping process capability models and standards is that the authors of both are not obliged to use the same architecture, syntax and not even the same terminology [6].This way, there is difficulty in understanding how similar contents are described in different formats by the different authors of these standards and models.As a consequence, there is an inherent complexity in the making of mappings among process capability models and standards.This paper presents a method of mapping that was used to establish connections among the models CMMI-DEV, MPS.BR and the part 5 of the standard ISO/IEC 15504.The goal was not to establish a definitive mapping, but to contribute with integrated assessments.From the results obtained with a traditional assessment based on one single model it is possible to demonstrate, with a certain level of reliability, the situation of other processes of the organization, in relation to other reference models.The use of the method is shown, offering a practical application of the method visualization.Furthermore, the paper presents the tool FAPS-INT [7], which supports the execution of integrated assessments on the mentioned models and is also being adapted to allow the making of mapping based on the described method.
The mapping should not be understood as a guideline to the improvement implementation on the organization software process.The goal is to allow an organization to assess the compliance of its processes with more than one reference model at the same time.

RELATED WORKS
In this work, a mapping is any document which represents some sort of comparative analysis among models and software quality standards.Usually, a mapping presents synergisms and differences among models/standards, making possible the visualization of the level of similarity among them.From this visualization, an organization that has reached a model/standard can evaluate the level of additional effort to accomplish expected practices and results from other model/standard, avoiding doubled activities and rework.From these definitions, 5 similar works were identified.It is worth mention that this search has shown the lack of well documented works which demonstrate how mappings were made.

Each work was evaluated under 7 criteria:
1.There is a documented and defined mapping method.2. CMMI-DEV was considered during the mapping.3. MPS.BR was considered during the mapping.4. Item 5 of ISO/IEC 15504 was considered during the mapping.5.The mapping was made for the entire model.6.The mapping was evaluated.7. The mapping can be automatized.
For each criterion, 3 possibilities of correspondence were considered: totally (T), partially (P) and noncorrespondent (N).The table 1 presents a summary of the assessment of the 5 identified works.The work with the best ratings [6] defines a method to the making of mapping used to correspond CMMI-DEV 1.2 with several ISO standards, showing concern with the results validity.Other consistent work is shown in [8].In this research, a new method to map the CMMI 1.1 with the item 5 of the 15504 standard.However, the work does not show how the results of the method implementation were validated.The other works assessed [9] [10] [11] show neither the adequate documentation about the method adopted nor about its assessment.

THE DEVELOPED METHOD
The method presented in this work is an adaptation of two methods adopted for mapping from models of the previous section [6] and [8].This method does not aim to establish statistical coverage comparison, in which we would be able to verify which model is more comprehensive or complete related to the practices of Software Engineering.The focus was to establish the reference among models so that, during the assessment of one of these models it would be possible to identify results that can be compared and show the status of process implementation in an organization in relation to more than one model.
The activities developed for the making of the mapping are accomplished in chronological order, with the first activity being the selection and study of the models considered.After the study, the experiments which are to be part of the mapping scope are selected.The mapping tables are then set to allow the relationships to be identified.Two mapping tables are defined, being one of the model A to model B. The other is named reverse table because it presents a view from model B to model A. The relations in these tables can be made in parallel.Afterwards, a review activity is performed to assess inconsistencies between the tables and establishes the first version of the mapping.From this point, it is possible to go on with the mapping validation in a more comprehensive manner.The Figure 1 shows a general view of the developed method.For each activity, besides the detailing of what has to be done, some guidelines for the method execution were defined.

Activity 1 -Selecting and studying models
In the context of this work, a model can be understood as any process reference model or process capability model.This way, during the method description, the term model is also used to refer to a standard that presents a model (for example, the item 5 of ISO/IEC 15504).This activity is related to the mapping scope definition, which models are to be considered for the mapping.Therefore, it is necessary to choose, at least, two models for the method application.Method selection guidelines are: a) Identify models that are organized in sets of processes (or similar components) b) Identify models that are related among their goals.This relation must be guided by the goals semantic and not only by a descriptive similarity c) The models must describe their processes (or similar components), concerning goals or purposes, and similar outcomes, practices and components.
d) It is mandatory to have access to the original documentation of the models, avoiding biased method conduction.
Once the models are selected, they must be analyzed according to their goals and contents.Still, it is necessary knowledge upon de models structure in order to identify the components used in the comparison.For a better use of the method, some guidelines are defined to establish the components to be considered in the comparison: a) Identify components according to their semantics and not to its descriptive similarity, once the terminology adopted in each model can be different, it is relevant to seek the meaning of each component, and not to base the selection only on the name of the component.
b) Verify if the selected component is present in all the model processes.In this case, not all the model processes could be mapped.
c) Verify the level of detail for each component to avoid comparison among components with very different levels of abstraction.The bigger is the difference in the detailing of the considered components, the bigger will be the mapping subjectivity.This will directly influence the reliability of an established relation.
From the model selection and the identification of the components to be compared during the method mapping, it is possible to start activity 2.

Activity 2 -Selecting and studying processes
This activity is also related to the mapping scope.From the study performed in activity 1, the processes to be considered in the mapping must be selected.Guidelines for process selection to be considered in the comparison are: a) Identify the processes according to their semantics and not to their descriptive similarity.Once the terminology used in each model can be different, it is relevant to seek the meaning of each process, and not to base the selection only on the name of the process.A good reference is the goal description or the purpose of a process.b) Do not restrict the selection to a model process for a process of other model.A model can define a process related to more than a process of other model (considering the previous guideline).In this case, the processes which allow effective comparison must be chosen.
To make mapping possible, at least one process of each model must be selected.The processes must be analyzed in relation to their contents and goals.The proper knowledge will make the mapping relation establishment possible.

Activity 3 -Setting the mapping tables
The mapping tables are set from the crossing among the outcomes/ practices of the selected processes.These tables will form the tool to register the established relations during the mapping.As a way to guarantee different views upon the mapping, it is proposed the use of two tables.The regular table is used in the making of the first mapping, named initial mapping.In this mapping, a model must be selected to be used as the basis of the table, for instance, model A being the base and model B to be related.The reverse table is used in the making of the second mapping, named reverse mapping.In the reverse mapping, the model used in the initial mapping as the basis is then used as the basis is then used as the model to be compared.In the same way, the model that was being compared in initial mapping is then the basis of reverse mapping.As a consequence, model B is then the basis of reversed mapping and model A is the model being compared.Guidelines for the setting of the mapping tables are: a) Select a model to be the base of the mapping.The model will be used to establish the relations from it.
b) Insert the name of the reference model to be used as the base for the mapping in the column on the left end of the table.
c) Insert the selected processes of the base model in the second column from the left end of the table, immediately after this, the outcomes/practices of the selected process(es) of the base model must be listed.
d) Insert the name of the reference model to be compared with the base model in the first line of the table.
e) Insert the selected processes of the model to be compared in the second line of the table, immediately after list all the outcomes/practices of the selected process(es) to be compared.e) In case some outcome/practice requires more than one experiment/result from model B to complete its context, the outcome/ practice which are inserted in the context must be registered.
f) To make a consistent model attainable, the outcomes/practices that do not have equivalence should not be related to the experiments of model B.
It is suggested that the specialists involved in this activity be others than those involved in activity.This way, the objectivity and independence of the mapping are enhanced.

Activity 5 -Relate model B to A
From the reverse mapping table (activity 4), the relation among the outcomes/ practices in model B and the outcome(s)/practice(s) in model A starts.This activity can be executed in parallel with activity 4. The guidelines are the same as the previous activity, considering only the inversion of the models.It is suggested that the specialists involved in this activity be others than the involved in activity 4.This way, the objectivity and independence of the mapping are increased.From the fulfilling of both tables (regular and reverse), the revising activity can be started.

Activity6 -Mapping review
The review is made from the crossing among the results obtained with the initial mapping and the reverse mapping.Each relation established in both mappings must be reviewed in order to verify if the same interpretation has been adopted.The equivalent relations will be automatically approved to make next activity (validation) possible to be executed.The review allows the identification of inconsistencies among the relations established by the teams that executed activities 4 and 5. Guidelines for the execution of mapping reviews are: a) Verify each relation individually, checking whether they have the same connections among the practices/ outcomes in both mapping tables.Having equal connections verified, they must be kept in both tables.
b) When verified the existence of a divergence, a connection that exists only in one of the tables, the documentation of the models should be analyzed, verifying the description of the practice/ outcome again.The interpretation is discussed among the group, whose decision is to keep or to withdraw the relation.c) When observed the need of adding more practices/ outcomes to the models, it is mandatory to return to activity 2 and redo all activities.
d) The connections that might have not been considered must be deleted from both tables, in order to obtain only one mapping among the models.
As a result of this activity, a consistent and revised mapping is obtained.It will then be able to be taken to a set of specialists in order to have the validity of each relation established.

Activity7 -Validation
This activity aims to establish validity for the mapping built until activity 6.This validation is made by a group of independent specialists, who did not have any part in the execution of the previous activities.It aims to reduce the bias during validation.
Using the table with the reviewed mapping (activity 6), the specialist in the quality models selected can characterize each relation established among practices/ outcomes.The specialist must indicate the level of reliability he/ she has on the presented relation.The level of reliability represents how much the specialist believes that the practice of a model really maps the outcome(s) of other models.When the reliability is not complete, the specialist must justify his/ her answer and, occasionally, propose some change for the mapping.IN the context of this work, this activity has been executed through survey available on Internet to the software quality community.The criteria for analysis and interpretation of the outcomes obtained with the survey are still in process of elaboration.

THE METHOD APPLICATION
The models selected were the CMMI-DEV version 1.2 [1], MPS.BR version 2009 [2] and the model presented by the item 5 of the standard ISO/IEC 15504 [3].Having the models that would be mapped selected, the following step was to study each model, aiming its structure.From this study, it was possible to identify which components of each model have a high level of equivalence and similar granularity.Part of the outcomes obtained through this study can be observed in Table 2.

Process Attribute Outcome
In order to assess the method, a subset of each model processes were considered.The processes were chosen according to the following reasons: a) usually, in phased representations, these processes take part in the first level of development; b) they represent precisely the first level of development (G) of the model (which is a subset of level 2 of development of CMMI-DEV); and c) it was necessary to establish an initial scope for method assessment.This way, the result of the high level mapping (considering the processes and not the practices/ results) can be observed in Table 3.

ENG.1 -Requirements elicitation ENG.2 -System requirements analysis ENG.4 -Software requirements analysis SUP.10 -Change request management
Each of the selected processes in each of the models was studied.Even before start the setting of the table for the mapping, the difficulty in mapping the Requirement Management for models CMMI-DEV e MPS.BR for processes of the standard ISO/IEC 15504 had been noticed.The standard does not define any specific process to manage the changes over the requirements.Consequently, the selection had been based on the analysis of the processes goals that resulted in the selection of several processes in the standard.
Based on activity 3 of the presented method, 2 tables for each of the 3 executed mappings were defined: CMMI x MPS.BR, CMMI x 15504 e MPS.BR x 15504.For instance, considering the mapping between CMMI e o MPS.BR, the regular table established the specific practices of CMMI on the lines of the table and the expected outcomes of the process of MPS.BR were placed on the columns.The reverse table established the expected outcomes of the process MPS.BR on the lines and the specific practices of CMMI on the columns.
From the table set, it was then possible to start activities 4 and 5.These activities were executed in parallel by different specialists, as it is defined by the method.Therefore, bias was reduced.Based on activity 4 of the method, the mapping among the models was established.For example, in the case of CMMI-DEV with the MPS.BR, the mapping was nearly direct, once most of the practices are directly related to an outcome.However, in only one case it was necessary to relate two outcomes of the MPS.BR to complete the signification of one practice of the CMMI-DEV.
The practice SP1.5 of the process PMC was related to the outcomes GPR 13 and GPR 14.The practice SP1.5 requires the monitoring of all the project parameters.In MR-MPS, the outcome GPR 13 regards the monitoring of the project in relation to the plan, while the GPR 14 clearly regards the monitoring the project participants.Thus, the practice SP 1.5 is related to the outcomes GPR 13 and 14.
The activity 5 reversed the mapping, but the result was similar.The mapping was nearly direct, once most of the outcomes are directly related to the practice.Only 4 outcomes of the model MPS.BR needed to be related to more than one practice of the model CMMI-DEV.
The outcome GPR 12 was related to 2 practices: SP 3.1 and SP 3.2 of the process PP.The outcome GPR 12 presumes the review of the project plan and the engagement of all the ones interested.The practice SP 3.1 of the process PP regards the review of the plans that affect the project, while the practice SP 3.2 requires conciliation among the projects and the ones involved.This way, both practices cover the result GPR 12.
The outcome GPR 13 was related to 6 practices, SP 1.1, SP 1.2, SP 1.3, SP 1.4, SP 1.5 e SP 1.6 of the process PMC.The outcome GPR 13 presumes the monitoring and documentation of the results of the plans and tasks during the whole lifespan.The practice SP 1.1 of the process PMC regards the monitoring of the project parameter while the practice 1.2 regards the engagement monitoring.The practice SP 1.3 regards the monitoring of the project risks.The practice SP 1.4 presumes the monitoring of the risks documentation.The practice SP 1.5 covers the monitoring of the ones involved in the project and the practice SP 1.6 regards the review of the project progress.This way, these 6 practices are related to the outcome GPR13.
The outcome GPR 14 is related to the practices SP2.6 of the process PP and SP 1.5 of the process PMC.The outcome GPR 14 presumes the management of the ones involved in the project, identifying them, defining their involvement in these stages of the project.The practice SP 2.6 of the process PP regards the identification of the ones involved in the project and in which phases they took part.The practice SP 1.5 of the process PMC regards the monitoring of the ones involved in the project.This way, both practices are related to the result GPR 14.The outcome GPR 17 is related to the practices SP 2.2 and SP 2.3 of the process PMC.The result GPR 17 presumes that for the disagreements found in the monitoring of the project, corrective acts be created and these acts should be followed by the end of the project.The practice 2.2 of the process PMC regards the identification of the corrective acts while the practice 2.3 regards the management of the corrective acts by the conclusion of the project.Thus, these practices are related to the outcome GPR 17.Based on activity 6 the review of the mapping was executed.The review was made crossing the outcomes of the presented mappings (regular and reverse).In the case of CMMI-DEV with the MPS.BR, there were not inconsistencies.The table 4 presents the outcome of the mapping for a specific practice and a general practice of the model CMMI-DEV.The three models were assembled in the same table only to make the visualization easier.By the moment of the elaboration of this work, the activity 7 of validation was still in development.A survey was prepared to be distributed among the specialists who will verify the level of reliability they have on each of the relations.

FAPS-INT
The tool FAPS-INT was developed to support assessments of software processes aligned to more than one reference model, simultaneously.It makes possible the assessment of software processes using as reference the models CMMI-DEV, MPS.BR and the standard ISO/IEC 15504.
The figure 2 partially presents the evidence assessment screen where the assessor can verify the manufactures that the organization registered to one of the reference models mentioned and characterize each of the outcomes or practices.The assessment process is based on the methods SCAMPI [12] and MA-MPS [13].The tool was built in a client-server environment with a simplified installation (direct copy from a directory) reducing the specific necessities like a network infrastructure or Internet access.The assessment can be conducted by a variable number of assessors.There is a means of import/ export which makes easier the management of collected information by different mini-teams (groups of assessors responsible by the analysis of one or more processes) and reduces the possibility of errors in spreadsheet copies.The tool FAPS-INT allows the reuse of collected evidences and assessment outcomes from one of the models and standard supported by the tool.Still, the tool offers a repository of assessment outcomes, allowing an organization to be able to keep a detailed report of its improvement program, identifying and comparing results in different moments.
The structure of FAPS-INT does not present anything specific for integrated assessment, once the assessors should not worry about this possibility.From the view of the assessor, he/she will lead a usual assessment selecting one of the available models.At the end of the assessment, after the outcomes characterization is that the assessor will be able to choose whether he/she will verify the correspondence to the other model to be selected.Consequently, the assessor does not have to be familiar with the second model/standard.
The resulting mapping from the previous section is already available in the tool.For the assessor to have access to the outcomes in other model, it is enough that he/she chooses the correspondent tag.The tool will then, from the mapping, present the conformity coverage.Figure 3 presents this sequence.Currently, the FAPS-INT is becoming developed to support all the method presented in this mapping.This way, the assessors who use the tool will be able to update existing models, as well as insert new models and build new mappings.FAPS-INT is totally free and the latest version is available at http://www.incremental.com.br/faps.

CONCLUSION
The mapping among models and Standards is not a simple task, once it requires specialized knowledge in all involved models.Besides that, the structure of the models is not necessarily similar, making difficult to establish relations without the previous and careful assessment of each one and establishing the granularity to be adopted.
The study of similar works showed the difficulty in finding properly documented researches in this area.In addition to being scarce, most of the studies do not show a well defined method and there is no concern to the built mapping validation.
One difficulty found in the validation stage of the mapping is in the definition of the criteria to assess the outcomes sent by the specialists.There is concern with a classification of the specialists to reduce discrepancies and biases.
The mapping resulting from the method application was inserted in a tool named FAPS-INT.This way, it is possible to lead a traditional assessment and obtain in an automatized way the coverage in relation to other models.
Currently, the tool FAPS-INT is already being used in the context of a process improvement program.Its application in a real environment with real necessities offers an opportunity for adaptation and improvements.
It is worth emphasizing that the tool still has potential to be also used in the training of professionals involved in software quality.Once the tool can be developed with mentoring resources, the ones responsible for process improvement can receive indications about possible examples of products and procedures that might help the organization to reach the outcomes and practices expected from the models.In this case, the organization would know that a certain implementation would be aligned to more than one model.
This work aims to contribute with organizations that make assessments based on reference models, either for other organizations or internal assessments.With the use of integrated assessment, the assessed organization will have an initial idea about the characterization of its processes in other models.

FURTHER WORK
At present, the survey for the validation activity is being made available.However, it is necessary to establish criteria to assess the feedback from the specialists.These criteria are fundamental to take decisions upon alterations in the existing mapping.
The tool FAPS-INT still does not support the mapping method, which should be available in the second semester of 2009.
The tool was built under a stand-alone concept, with no need to be connected to the net.Although this requirement is relevant to allow the making of assessments anywhere, with no need to access the organization network, the assessment outcomes end up dispersed, making difficult the analysis of previous outcomes.The previous register allows the effective tracking on the processes evolution, offering indicators for the organization improvement.
With this purpose, a computer infrastructure is being developed to assemble all the outcomes of already made assessments and make the information available through an organization Web portal.The goal is not only offer the information centralization, but also to create an environment with improvement indicators that may be used to support the effective measurement of the improvement.

Figure 1 :
Figure 1: General view of the developed method.

3. 4
Activity 4 -Relate model A to model B This work considers that a mapping is made of several relations.Relation means the connection from one to many, in which a experiment/result of model A is set and the it is possible to verify which outcome(s)/ practice(s) are equivalent (or show some level of equivalence) in model B. Guidelines to establish the relation among these outcomes/ practices are: a) To guarantee the consistent mapping it is necessary to always consult the models/ guidelines/ standards of the models in use.b) To establish the mapping, not only the names of the practices/results should be considered, but instead their definition, purpose and semantics.During the definition of the outcome/ practice, the requirements of each one must be analyzed.c) Always observe a outcome/ practice of model A in relation to model B. Occasionally, a outcome/ practice in model A may be related to many outcome/ practice in model B.It means that, in an assessment, the experiments/ result in model A would be totally followed only if all the outcome/ practice were totally followed.d) Having found equivalence, an "X" must be registered in the cell that makes the intersection of the outcomes/ practices of model A with the outcomes/ practices of model B.

Figure 2 :
Figure 2: Part of the assessment screen of the evidences in FAPS-INT

Figure 3 :
Figure 3: Mapping and automatic outcome of the integrated assessment in FAPS-INT

Table 1 :
RELATED WORKS ANALYSIS

Table 2 :
MODEL COMPONENTS CONSIDERED FOR THE MAPPING

Table 3 :
PROCESSES CONSIDERED FOR THE MAPPING

Table 4 :
EXCERPT OF THE APPLIED MAPPING