Competency Assessment: Integrating COCOMO II and People-CMM for estimation improvement

‘Human factor’ is one of the most relevant and crucial aspects of software development projects management. Aiming at the performance improvement for software processes in organizations, a new model has been developed to diagnose people related processes. This new model is People-CMM and represents a complementary solution to CMM. On the other hand, existing estimation models in Software Engineering perfectly integrate those aspects related to personnel’s technical and general competence, but fail to integrate competence and performance measurement instruments when it comes to determine the precise value for each of the factors involved in the estimation process. After reviewing the already deployed initiatives and recommendations for competence measurement in the industrial environment and the most relevant estimation methods for personnel factors used in software development projects, this article presents a recommendation for the integration of each of the ‘human factor’ related metrics in COCOMO II with the management tools proposed by People-CMM, which are widely implemented by existing commercial tools.


Introduction.
Presently, software is a critical factor. Failure rate in software projects are high and qualified software engineers pertaining to software development teams are key factors in the software development process and their shortcomings and caveats (Pressman, 2005). These teams are composed by professionals with a heterogeneous training, background and expertise (McConnell, 2003) that Human Resources management must be able to evaluate and provide with a professional view, with the ultimate goal of improving the competences of the workforce and their results (Curtis, Hefley & Miller, 2001).
Individual differences have been identified as one of the paradigms for the research of human factors in software development (Curtis, 2002). Research in this area goes back to the 1960s (Sackman, Erikson & Grant, 1968) and continues actively in the 1980s (Curtis, 1981), (De Marco & Lister, 1985). During that decade, tools and models for software estimation, such as COCOMO (Boehm, 1981), started spreading. Some of the previously quoted models allow to identify productivity differences among human resources calculating time and effort with some parameters provided by the user. Actually, this is very close to the work presented in this paper. The main objective of this paper is to identify a correspondence between the estimation and the evaluation of human resources from the point of view of competence management. This correspondence would allow the integration of both instruments into a unified framework, providing managers highly reliable estimation mechanisms.
The remainder of this paper is organized as follows. Section 2 presents related work and a conceptual model for competency, competency management and competency assessment. Section 3 describes personnel factors in software Project estimation and more concretely, the COCOMO II approach. In Section 4 an integration proposal between competency assessment and COCOMO II personnel factors is presented. Finally, section 5 concludes the paper.

Assessment.
The competency approach to human resources management can be found in early origins. The early Romans practiced a form of competency profiling in attempts to detail the attributes of a "good Roman soldier" (Draganidis & Mentzas, 2006, p.52). In recent history, competence is a concept used by early 20th century scientific management (Taylor, 1911) and later revised and redefined by McClelland, former Hay Group director, in the early seventies.
According to this author competency is comprehended as the relation between humans and work tasks, that is, the concern is not about knowledge and skills in itself, but what knowledge and skills are required to perform a specific job or task in an efficient way (McClelland, 1973). From the sixties, and due to the ever-growing adoption in business environments of the competency paradigm, a number of definitions emerged for the term, to the extent that multiple authors adhere to the opinion of facing a "competency pandemonium" (DeHaro, 2004). In a recent analysis of the term in the scientific literature, Draganidis & Mentzas (2006) state that a competency must be defined in terms of: • Category. A group to which homogeneous and/or similar competencies belong.
• Competency. A descriptive name for the specific competency.
• Definition. Statement(s) that explains the basic concept of this competency.

• Demonstrated behavior. Behavior indicators which an individual
should demonstrate if the specified competency is possessed.
The aforementioned authors point out, based on the survey of a number of competency-based management systems, that competencies are capital in the following employee management applications: Workforce planning, CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 Recruitment management, Learning management, Performance management, Career development and Succession planning. According to People CMM (Curtis, Hefley & Millar, 2001), competency management is a collection of workforce practices used to enhance the capability of the workforce to perform their assigned tasks and responsibilities, and to achieve specific competency growth objectives. Competency management is becoming more and more important in today's organizations. Workers competency levels are important for the achievement of company goals, complimentary to, for instance, core business processes, customer relationships and financial issues (Norton & Kaplan, 1996). Berio & Harzallah (2005) provide a very simplified view, but very intuitive of the Competency Management process which is depicted in the following figure: Co mpe t e n c y Us a ge Co mpe t e n c y Id e n t if ic a t io n Co mpe t e n c y As s e s s me n t Co mpe t e n c y Ac q u is it io n  • Competence identification, i.e. when and how to identify and to define competencies required (in the present or in the future) to carry out tasks, missions, strategies; how competence is represented is included here.
• Competence assessment, i.e. (i) when and how to identify and to define competence acquired by individuals and/or (ii) when and how an enterprise can decide that an employee (or an individual) has acquired specific competencies; how the relationships between individuals and required competencies are represented is included here.
• Competence acquisition, i.e. how an enterprise can decide about how to acquire some competencies in a planned way and when; • Competence usage, i.e. how to use the information or knowledge about the competencies produced and transformed by identification, assessment and acquisition processes; for instance, how to identify gaps between required and acquired competencies, who should attend required training, how finding key employees (i.e. holding key competencies) and so on.
Performance evaluation can be defined as the systematic and periodical process of measuring, in an objective way, the efficiency of an employee or a team in their job (Pereda & Berrocal, 2001). Typically, employees with certain seniority are evaluated yearly by their superior about the set of competences involved in their roles. The evaluation is made under a stable and defined framework applicable to the whole set of human resources of the company. The results of the evaluation can influence the salary and the training plan of the evaluated employee among others. CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 6 Competence-based performance evaluation systems are integrated into global competence management systems that implement competence paradigm to the management of human resources. Competence-based performance evaluation systems assess the possession of generic and technical competences. Generic competences, such as leadership and effective teamwork, are not related with the development of any specific activity. Technical competences are those destined to the development of a specific activity and they are really close to that knowledge put into practice and capabilities. On the market there are several commercial solutions that support both types of competences but, nevertheless, the in-depth management of technical competences is usually made by means of skills management systems that are closer to knowledge management.
• Regression Based (OLS, Robust). CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3,  Due to its relevance and commercial success, our work will analyze and upgrade COCOMO II 2000 . The rest of the estimation techniques, described in (Saliu, Ahmed & AlGhamdi, 2004), will not be tackled in this article and its in-depth analysis is proposed as future work.
COCOMO (COnstructive Cost MOdel) is the most developed algorithmic model since it was first created. Its first version was published in l981 (Boehm, 1981) and updated in 1987 with an extension to ADA language. In 1994 started the development of COCOMO II that was first published in 1995. According to the previous classification of estimation techniques, COCOMO II and COCOMO II 2000  belong to the composite techniques group, due to the use of Bayesian techniques for the calibration of the model.
The effect of personnel factors in algorithmic models is extremely relevant and many models include this kind of factors in their algorithms (SLIM, Checkpoint, PRICE-S, ESTIMACS, SEER-SEM, COCOMO II, …). Nonetheless, the latter is the model that is considered to provide a deeper support.
In COCOMO II, the factors taken into account when capturing the impact of the project's environment in its cost are classified into four big groups: product, platform, project and personnel. Personnel related factors are, just after the size of the product, the most influential factor when determining the required effort for the development of a software product. In particular, six personnel related factors are identified: • ACAP: Analyst capability.
• LTEX: Language and tool experience. To make an estimation based on the principles of Engineering, data should be sufficiently reliable for every factor. Therefore, even when there are available numeric data for factors related to experience and continuity, ¿which are the criteria for the evaluation for the evaluation of the capacity of programmers and analysts?, ¿what should be the reference for the different experience measures?.
Next chapter suggests the utilization of artifacts and methods typical of evaluation and management of competences as a support for the calculation of this kind of factors in software estimation.

Competency assessment & COCOMO II: An integration
proposal.

ACAP.
Analysts, according to the definition provided by the estimation method, are those practitioners related to requirements, high-level design and detailed design. The attributes that should be considered for the valuation of this factor include synthesis and analysis capacity, efficiency, thoroughness, and communication and cooperation capabilities . The experience of the analysts should not be taken into consideration because it is valued by means of APEX, LTEX and PLEX.
Taking into account the provided definition, when determining the value of ACAP, technical competences related to requirements management and design and general competences related to teamwork and synthesis and analysis capacity should be analyzed.

For the integration of the human resources management practices included in
People-CMM and the estimation methods for ACAP, it is regarded as necessary that the organization implements the process areas detailed next: • Level 2.
o Performance management, which includes the assessment of the performance of professionals.
• Level 3. o Concern for quality.
o Oral and written communication.
• Technical competences: o Software requirements.
o Software design.
The provided competence list is a generic proposal. Following the dictates of People-CMM, the determination of this value should be carried on by means of CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 a detailed study of the necessary competences for analysts so that they can develop their duties with excellence in a particular organization.
The value of ACAP is the combination of the performance assessment scores of the analysts with the identified competences involved in the project (EVAL), weighed by the degree of participation in the project (DEDI) and the relative importance of each competence (POND). The following formula shows the summation of the different factors involved in the calculation of ACAP: The numeric value for ACAP obtained after the calculation provides a value within the range of the scores used for the evaluation of competences, which are usually determined by a Likert scale. This value can be automatically converted to a percentiles scale, the unit used to express ACAP.

PCAP.
The capacity of programmers should also be taken into account for estimation.
For this factor, just like ACAP, experience should not be considered but efficiency, thoroughness, and the capacity for communication and teamwork of the programmers should be measured .
PCAP factor requires the same process areas of People-CMM as ACAP.
However, according to the description of the evaluation that should be performed for the determination of PCAP, the set of involved technical competences is reduced to Software Construction. The value of PCAP is obtained from the following formula:

PCON.
The personnel continuity factor (PCON) measures the annual personnel turnover as regards the total number of human resources . It is the easiest factor to calculate, since it is based in a mere historical record of the new staff taking on. Even those organizations in the initial maturity levels should be able to calculate PCON, at least if the staff of the project is formed by internal resources. This is even simpler in Spain where it is compulsory to record the seniority of each worker in their payslip.
However, the effects of subcontracts and the wealth of self-employed practitioners can considerably complicate the calculation of PCON. In this case the relationship between the employees and the organization has not labor character and the traceability is more complex. People-CMM does not include any explicit mention or recommendation for this kind of records, possibly due to its simplicity.
For this reason, organizations are recommended to implement any system to support the assignation of resources and projects to allow their traceability.

APEX.
APEX factor reflects the experience, measured in units of time, of the staff of the project in the type of application being developed .
Experience is approached within People-CMM across several knowledge areas.
The first of the recommendations for the management of experience is provided at level 2, in practice no. 3 at the "Staffing" knowledge area. However, its proper management requires the implementation of techniques closer to Knowledge Management. This circumstance fosters the search of competency within the organization and the determination of the experience of its workers. CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 Having this in mind, the knowledge area that should be considered is "Assets based on competences", placed at level 4. The second practice in this knowledge area recommends the implementation of vehicles for sharing competences. It can be stated that the observance of this second practice is a necessary condition for the successful implementation of this kind of systems. Lindgren and Stenmark (2002) provide a technical guide for the implementation of competence search within organizations.
The adoption of these practices in the environment of software development has been abundantly documented in technical literature (Dingsøyr & Røyrvik, 2003), (Lindgren & Stebnmark, 2002), (Hardless, 2005) and has the support of several commercial tools such as Skillscape and Skillview among others.
In particular, APEX factor can be directly determined from the documented experience stored in the implemented system. The system provides, for each of the project staff members, an expertise report in the functional environment that can be specified as any other kind of knowledge. From this report, that indicates the total labour experience in days (EXPE), and weighing the different dedication of the professionals to the project (DEDI) it can be directly calculated the value for APEX as shown in the following formula: PLEX represents the experience in the technologic platform and reflects the relevance of the understanding and the use of the platform which includes GUI, databases, middleware, etc .
Taking into account the aforementioned considerations about APEX, PLEX can be determined in a similar way with some modifications to the intrinsic CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 characteristics of the factor. PLEX, as pointed out in its definition, involves a set of technical factors that should be reflected in its estimation: knowledge on middleware, database management systems … The proposal for PLEX is to specify each of the elements that make the technological platform up and weigh their relevance. The final estimation will be the combination of the values for the experience of each professional in the different technologies (EXPE), the relative weigh of each one (POND) and the dedication of each professional (DEDI), as expressed in the following formula:

LTEX.
The last of the personnel factors provided by COCOMO II is the experience of the project team in the language and the tools to be used during its development .
The value of LTEX can be determined following the same criteria used for PLEX.
The estimation should include the experience of the professionals in the language and the tools, their relative weigh and the dedication of the professionals to the tasks of the project.

Solution Testing.
To perform an empirical validation of the solution proposed in this paper, it has been implemented in two different environments. The first of them, Company A, represents a software development company focused on out-of-the-box solutions based in Java technologies. This company needs to add an isolated CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3, DECEMBER 2007 functionality to one of the packages for the support of commercial Leads. The second one, Company B, is a company specialized in tailored developments that has received an order to develop a Java-based, integrated traceability control solution for a kettle farm. Both companies have implemented commercial Competence Management Systems that are able to provide all the required data for the implementation of the improvement proposal.
Managers in Company A estimate that the required development will approximately produce 8,000 SLOC, involving lots of references to objects already used in the current version. After providing all the parameters to the COCOMO II model used by the company, the effort estimates (in the interests of simplicity) indicate that the development will require a total effort of 16.01 man-month. On the other hand, after introducing the proposed competential measures, the obtained effort is slightly higher and represents 17.28 man-month.
The tailored development by Company B is estimated to consist of 15,000 SLOC, which using the same model as in the previous case throws an effort estimation of 57.64 man-month. The effort estimation refined with the improvement proposal results in 55.12 man-month.
The actual human resources requirements as well as the different estimates are shown in Table 2.  (Shepperd, 2007). Authors such as Gulezian (1991) and Jeffery & Low (1990) concluded that calibration is essential in models that were to be used outside those environments in which they were developed. This conclusion seems to be paradoxical since COCOMO was intended to be general purpose through its many drivers and parameters (Shepperd, 2007).

COCOMO II Model Reality
In any case, basing estimations on data collection and analysis in accordance to engineering criteria means one step forward for the consideration of Software Engineering as a remarkable discipline.

Conclusions and future work.
This article presents a correspondence between the most outstanding and widespread models for human resources management. In particular, those recommended by People-CMM and COCOMO II, that provide software estimation methods and include personnel related aspects.
This research interest on the improvement of software estimations is based on the specification of conversion criteria between the results of the CLEI ELECTRONIC JOURNAL, VOLUME 10, NUMBER 2, PAPER 3,