Pedagogical Experiences in a Highly Populated Undergraduate Programming Course Teaching How to Develop Medium Size Information Systems

Teaching how to develop medium size information systems in courses with high matriculation rates is a complex task. In Latin America this is an important issue because, due to high matriculation rates and scarce resources, Schools of Engineering have, in general, very low graduation rates and high drop-out rates. This paper presents the experiences of a team of teachers, during a period of eight years. Two strategies were proposed to cope with high drop-out and low graduation rates, and they were implemented and analyzed in two stages. At the initial stage, key roles were defined for each of the members of the teaching staff, focusing on programming assignments and the development and usage of e-learning support systems, such as a course web page and a newsgroup list. At the second stage the main goal was to consolidate the results from the previous stage, but also to make the contents on-line evolve, in order to improve the communication with students and among them. In this second stage it was also important to address the problem of high drop-out/abandon rates, trying to find out the causes and reduce their effect. This work proposes a decentralized form of organization of the teaching staff and shows how this makes a difference to make better use of the resources. It also shows that practical work on a project and the transformation to a semi-distance learning course had the effect of lowering the drop-out rates and increasing the approval rates. Finally this experience and the results have an inspiring effect on teachers and educational institutions to improve courses with similar characteristics.


Introduction
The continuous increase in the enrollment rate of students to the Engineering Programs of the "Universidad de la República" (Uruguay) in the last years has transformed the basic courses into highly populated/massive courses.Following the trend of other Latin American Schools of Engineering, were there are increasing matriculation rates and scarce resources, hence they have low graduation rates and high drop-out rates [1].
For this reason it has become necessary to adopt new strategies in order to adapt the scholar system to this new reality, since it was not conceived for this context of massiveness and does not have the necessary resources to take care of the students in the same way as before.The strategies required range from reviewing the educational strategy/model, the evaluation of the acquired knowledge by the students, and the organization and management of the courses, among others [2].
Bearing the former in mind and having detected, in some of the students attending the final years of the Engineering Program in Computer Science, important shortcomings/deficiencies in their acquired knowledge and capabilities to carry out the tasks involved in the development of medium and large size information systems, the Computer Science Department (InCo -"Instituto de Computación de la Facultad de Ingeniería") decided to address the problem and implement the necessary changes.One of the key actions was to assign more teachers to some of the basic programming courses that are highly populated.
This paper presents the experience of the group of teachers working in the second basic programming course, "Programación 2", since the introduction of the first changes in 2003 until 2010.This time span can be organized in two different stages: the first one spans through 2003-2008 and the second one through 2009-2010.Some of the early results of this experience have already been presented in a local seminar at "Facultad de Ingeniería de la Universidad de la República" (Uruguay) [3] and are reported in [4].Partial results regarding the first four years of work with the new modality, [2003][2004][2005][2006], have also appeared in an earlier paper in CIESC'2006 [5] and [6].Finally, a preliminary version of this paper, in Spanish, appeared in CIESC'2011 [7].
This paper focuses on the description and analysis of two strategies to improve the quality of the acquired knowledge by the students and to lower the drop-out rate and increase the suffice rate, following a methodology research/action.The first strategy was to implement an adequate organization of a highly populated course, with the goal of improving the management of resources, and show how this leads to an improvement in the quality of the acquired knowledge by the students.From 2009 and on, the aim was to consolidate the work done in the period 2003-2008 and to head towards a semi-distance learning.In 2009 the new strategy was to implement a series of measures intended to lower the drop-out rate and increase the approval rate, essentially they were promoting team work among students, without lowering the quality of the course and make the course fully semidistance learning by placing it in a virtual learning environment.
The rest of this paper is organized as follows.Section 2 presents the specific characteristics of the programming course "Programación 2", regarding contents and its place in the Engineering Program in Computer Engineering curricula.Section 3 describes the initial diagnosis of the course made by the team of teachers at the end of 2002.The first strategy is described in Section 4, as well as the implemented measures that were introduced from 2003 and on.Section 5 presents the evaluation results for the students that attended to "Programación 2" course and their evaluation results in some other relevant courses in the period 2003-2008.The second stage and its strategy, 2009 and on, are described in Section 6, stating the new objectives and the additional changes implemented in this stage.The results of the second stage are analyzed in Section 7. Section 8 presents related work in the local context, and finally Section 9 concludes with a summary of our contributions and directions for future work.

Course Description
The actual Engineering Program in Computer Science has been established in 1997.It is a program organized in 10 semesters and it is composed by a series of requirements regarding the credits/points the students must obtain by passing courses in different knowledge areas corresponding to distinct fields of science and technology [8].
"Programación 2" (P2) is the second basic course within the Programming knowledge area and is a mandatory course in the Engineering Program in Computer Science.P2 is imparted in the third semester of the program and is an intermediate course between "Programación 1" (P1) and "Programación 3" (P3), which are lectured in the second and fourth semesters of the program, respectively.
P2 focuses on three main topics: induction and recursion, pointers and dynamic data structures.and abstract data types [9].P1 is an introductory course in imperative programming, which focuses among other topics on: control structures, procedures and functions, and variable scope [10].The core of P3 is the study of data structures and algorithms to solve computational problems, stressing techniques to formulate and implement data abstractions.In particular, some of the most important topics are: analysis of algorithms complexity, sorting algorithms, graph algorithms, and algorithm design techniques [11].
The general goal of P2 course is to teach the students to design and implement medium size systems, for this reason techniques to formulate and implement data abstractions are taught.
The weekly work load for each student is estimated to be: 2 hours of attendance to theoretical lectures, 2 hours of programming exercises, and 2 hours of counseling sessions to get help regarding projects (lab tasks), plus 8 hours of individual work, but students are not forced to attend to classes.
The course evaluation is composed of two written partial exams and a project (study case).Each of the written evaluations is, of course, individual.Projects have been individually assigned in the 2003-2008 period, but in the last two years students were able to join groups of up to 2 students to work on a single project together.The project is based on a study case and is the same for all the students enrolled in the course.It is composed of two different lab assignments that have to be handed-in: one at the middle of the course and the other at the end of the course.The lab tasks are compulsory, which means that the students that do not obtain a passing grade on the lab assignments fail the course, no matter what grades they have obtained in the partial exams.
Based on the results of the partial exams, the students are classified in three groups: exonerate, which means that the student has passed the course; suffice, that indicates that the student has the right to take a written exam; and fail, when the student has to take the course again.From now on the students in the three groups will be said to: exonerate, suffice or fail.For more details regarding the P2 course, refer to [9].

Initial Diagnosis
In 2003, prior to the beginning of lecture year, a careful diagnosis regarding the deficiencies of the course was made by the group of teachers that was to take on the P2 course.In the following we present some of the most remarkable observations.The P2 course is a highly populated course.Since 2001 the enrollment rate to the course is on average 500 students/course.Not all the enrolled students effectively attend to classes, since it is not mandatory, and a growing number of students (between 40% and 50% of the enrollment) prefer e-learning methods.
The course was organized as follows: one professor gave the lectures, and a group of assistants or advanced students handled the exercise and practical classes simultaneously.This organization was inadequate in various aspects.On one hand the exercise-practical classes turned out to be too short and confusing, since the teachers had to discuss very different topics: exercises related to the lectures and guidelines for the practical work on a project.In practice it was very hard and time consuming for the teachers to master both topics and this had a negative impact on the quality of the classes.Since there were no teachers responsible for the practical work alone, deciding on the design and implementation characteristics of the project, there were many open questions regarding the task of the students, neither teachers nor students knew exactly what was expected of them.Coordination meetings were long and inefficient, because all the teachers had to attend so that the exercises were solved similarly, to address students questions concerning the project, and for administrative tasks regarding the course.
On the other hand, there were problems with the content of the exercise classes.They were moderately outdated and did not cover completely the course topics.Another problem was the lack of a repository of solutions for the exercises that could be either used by teachers to prepare classes or partially published at the website to students.
Although there was a webpage with the basic information about the course, it was not enough as a means of communication with the students or as repository of educational material for them.Concerning the project, only the general description of the problem, the assumptions and expected results, were made available to students.
Only the exercises were published on the webpage, no solutions were accessible.Although there were old exams and their solutions, they were not always published.

Strategy Proposed and Implemented in the First Stage
In the year 2003 the number of teachers assigned to P2 was doubled with the purpose of improving the quality of the programming courses in general and P2 in particular.The group of teachers that was originally composed of 6 members was incremented to 12 members.The tasks performed by the group of teachers assigned to P2 from 2003 are described in this section.
Based on the diagnosis regarding the deficiencies of the P2 course and the increase in assigned teachers to the course, the strategy for the first stage was established based on the following general goals: • Define activities and key roles within the teacher group to fully take advantage of the resources in the most relevant tasks and specialize the teachers' work to accomplish more efficiency.• Improve the quality of the education in programming, being more thorough in the programming tasks related to the exercises and mainly the project.
• Exploit the use of technical solutions and new technologies in the communication with the students: webpage and newsgroup, with the aim of improving the ratio teaching/learning, principally for the group of students that do not attend lectures and classes, around 50%.
In the next subsections the strategy for the first stage is presented in terms of the measures that were implemented to achieve these three general goals.

Take advantage of the teaching resources
To fully take advantage of the teaching resources it was considered necessary to put together two work teams with complementary tasks.One work team was appointed responsible for the compulsory project and the other focused on the development of the exercises.Accordingly the following roles were defined: responsible for the course; responsible for the lectures; responsible for the exercises; exercise teacher; responsible for the project; project teacher.
A decentralized work form was chosen as appropriate for the new modality of the course, in which each of the working areas were organized following the structure given by the work teams.Each one of the teams had weekly independent meetings in which the responsible and teachers for the areas participated to discuss relevant questions regarding their topics and administration.Additionally there were weekly coordination meetings with the responsible of all areas and the responsible for the course, which allowed centralizing the information and the critical decision making for the course.This new organization of the course allowed for the confluence of interests of the two dominant aspects of the course giving the students a cohesive image of course.
The decentralized organization and the specialization of teachers' work led to that each teacher, although some had low work load assigned in the course, had deeper and more detailed knowledge about its specific work area and the tasks performed by the work team.The teaching activities, weekly meetings and coordination meetings became more efficient.

Improve the quality of education
To achieve an improvement in the quality of the education in programming, mainly two work lines were followed.The first line was to improve the educational resources available for the students and the creation of new study resources to give support to the students.The second work line was to make the assignments of the practical project a fundamental learning task within the course, since it is through the completion of this project that the students truly develop their skills to design and program medium size systems.And this is the primary goal of the course, although it had not been accomplished so far, according to InCo as stated in Section 1.
Regarding the creation of new study resources, there was a lot of effort put into creating study material for the practical project to give support to the students.The general and particular descriptions of the problem were elaborated; documents that completely specify the system requirements; examples of use and test cases; common mistakes; and additional guidelines about the programming language, the compiler, the programming environment and basic techniques for the tracing and fixing mistakes/bugs.Also there was much work put into automating the grading process of student handouts, followed by reviewing graded handouts with the students, which may lead to a change in the grades.
The resources related to exercises were made more meaningful by adding basic and advanced exercises for all topics covered by the course.Good programming habits were encouraged by offering well-thought pedagogical solutions to selected exercises.Finally a complete revision of the lectures and lecture notes was done.
To make the assignments of the practical project a fundamental learning task within the course, it was necessary to augment the complexity of the requirements of the system and to increase its size, aiming at covering a broader range of essential course topics and integrating them to the development of a medium size system.Aiming at motivating the students in the development of the projects, an effort was made to approaching subjects of greater interest for the students in the definition of the problem.For example the implementation of more or less complex simulators of: operating system file manager, data base manager, spreadsheet program, text/document editor, social networks, etc.Although the projects are fundamental learning tasks within the programming courses, they do not have an effect on the passing grade of the students (if students approve projects), so it was considered necessary to take measures to somehow impact the result of the project in the final grade.For this reason it was planed to include an exercise in the second partial exam that was similar to some aspect of the project, hence students that had a passing grade on the project would do very well in this exercise and would in fact obtain at least a suffice grade.

Improve communication and access to educational material
Even though the direct contact between the teachers and the students was always maintained through lectures, exercise classes, and project classes, there seemed to be an inadequate utilization of the high number of teachers.Therefore, it was crucial to consider alternative means of communication, and this is why the newsgroup was decided on.The newsgroup was thought for answering questions regarding the lectures, exercises and project, and was set to be monitored on a daily basis by some of the assigned members of the work teams, each on answering on its topics.
To give the students better access to the educational resources related to the course, the publication of all the material was systematized through internet (http://www.fing.edu.uy/inco/cursos/prog2/),including the newly generated resources as well as old exams and their solutions and a section for news and general information.

Results of the Implemented Strategy in the First Stage -Period 2003-2008
The presentation and analysis of the observed results for the period 2003-2008 are looked upon from two different angles: internally to the course and projecting the results to estimate the impact that these results will have in the following programming courses.The analysis focuses on the results of the course, not on the final exam which some students earn the right to take; this is because there have not been any significant changes in the passing rate of the exams.

Results of the implemented strategy in the first stage for "Programación 2"
The results for the last 10 years are shown in this section.The results are separated in two relevant periods 1999-2002 and 2003-2008.For each period the average of the enrolled students is given as reference, although the rest of the average results are calculated based on the "active students", those that at least handed in the first project assignment, and there have not been any significant changes in the active student rate, they are around 65% to 70% of the enrolled students.1 shows that in the period 2003-2008 there was an increase in the fail rate.A possible cause for this change is that the course became harder, mainly because of the changes made to the requirements of the practical project, which now was two assignments to hand in and a higher level of complexity and size of the system.
Table 2 presents the results of each one of the evaluations of the practical project in the period 2003-2008.The table shows that the passage rates of the second assignment are better than the first, and that the drop-out rate is very high for the first assignment and significant for the second assignment.In this table, the rate is calculated with respect to the general student population that could hand in the assignments: enrolled in the course, for Assignment 1 and having passed Assignment 1, for Assignment 2. The drop-out rate is calculated in a similar fashion, under the same conditions.The passing rates are calculated regarding the assignments that were handed in.Although there is a visible increase in the number of students that fail in the period 2003-2008 with regard to previous years, it is important to point out that of the students that obtained a passing grade in both assignments, only around 2% did not exonerate or suffice the course, 59% exonerated and 39% sufficed.These results corroborate that the practical project has become a fundamental learning task within the course.

Results of the Impact in the Following Courses of the implemented strategy in the first stage
It is a complex task to analyze the effects of the changes in P2 in the following courses, mainly because the other courses also evolve and change."Programación 3" and "Programación 4" (P3 and P4) were chosen for the analysis because they are also programming courses.The relevant periods defined by the responsible of the courses were considered.P4 follows P3, and its main goal is to introduce object oriented programming to the students.For more details about this course see [12].
Table 3 shows the results for P3, considering two periods: 2001-2002 (previous to the changes in P2) and 2003-2008 (first stage of the changes in P2).For each period the average of the enrolled students is given as reference, although as before the rest of the results are calculated based on the "active students", also the average of the number of "active students" that Failed, Sufficed and Exonerated.The results show that there was an increase in the number of active students and a significant decrease in the failure rate, comparing with previous years before the changes in P2 took place (2001)(2002).These results indicate that exposing the students to the requirement of designing and implementing a medium size system, give them the necessary background to deal with even harder tasks in P3, as they do not face this type of systems for the first time in P3.P3 additionally poses another challenge which is to change the programming language from a strongly typed one (Modula-2 or Pascal) to one that is not (C/C++).As can be seen the changes introduced in P2 have had a positive impact on P3, allowing the students to take advantage of their programming knowledge and focus their learning on the implementation of advanced data structures and algorithms.The statistics show a noteworthy decrease in the percentage of students that fail the course, comparing the year previous to the changes in P2 with the period after that.A possible reason for this improvement is the fact that the changes introduced in P2 allow for the focus of P3 to be on the implementation of advanced data structures and algorithms, taking advantage of their previous programming knowledge, hence having a positive impact in the performance of the student in all the following courses.It seems reasonable to conclude that if students have incorporated better knowledge and good programming practices, they will be able to assimilate the topics of more advanced courses with less effort and better results.Finally it is important to point out that the teachers of P3 and P4 believe there is also a qualitative improvement in the education of the students that finally take these courses.

Strategy Proposed and Implemented in the Second Stage
From 2009 and on the goal was to head towards a semi-distance course and to consolidate the positive changes introduced by the strategy proposed for the period 2003-2008.In order to define the new strategy for the second stage two new ambitious goals were set: • Lower the drop-out rate.
• Increase the suffice and exoneration rates without reducing the quality of the course.
To consolidate the work done in the period 2003-2008 there was much effort put into maintaining the teacher group as stable as possible, trying to reincorporate teachers that had been active in the first stage and that had left for different reasons.It is of outmost importance for the success of the proposed changes to have teachers involved in the course that have consistently been participating in it.
Knowing that the students that enroll in the course attend less and less the lectures and exercise sessions and meeting the terms of the "Comisión Sectorial de Enseñanza de la Universidad de la República" [13], that promotes the use of information and communication technologies (ICTs) in education, it is decided to place the course resources in a virtual learning environment ("Entorno Virtual de Apendizaje", EVA) implemented on the Moodle platform [14].The use of this platform is intended to facilitate the diversification of modalities of the course.Moodle allows including tools that combine the management of courses and users, the creation and publishing of educational content that is reusable in different contexts, and the synchronic and asynchronous communication.The complete set of functionalities available for teachers and students can be seen in [14].Of all these functionalities, the most used one was the forum, for questions regarding the lectures and exercises, but mostly related to the project.There was a big change and a noticeable improvement by using the Moodle forums instead of a newsgroup, because the forums admitted the creation of sub forums and having a better organization of the threads by topics.Through the use of this tool the teachers were able to improve the quality and response time of the answer, it also made it possible for students to interact and answer each other's questions, and finally it encouraged the search for information in general.
Regarding the goal of increasing the suffice and exoneration rates without reducing the quality of the course, using the analysis presented in Section 5.1, it became clear that the projects are the main cause of drop-outs.It is very surprising that 26% of the students do not hand-in the first assignment, failing automatically, because the project is compulsory (Table 2).
Two important measures were proposed and later introduced to increase suffice and exoneration rates: • Students were allowed to work on their projects in groups of up to 2 members • An initial assignment was required, previous to the original two, with the goal of quickly introducing the students to the tools and the language and to persuade them to get involved/interested in the course.
It seems clear that since passing the practical project is crucial to passing the course, then an improvement in the project passing rates had to have a positive effect on the suffice and exonerate rate of the course.Besides the previously stated measures, there were big efforts in the automating of the grading process of the projects handouts, and other teacher tasks related to the project, the goal was to reduce the time spent by teachers rating projects handouts and give them more time for the design and creation of the projects as well as solutions and test cases.

Results of the Implemented Strategy in the Second Stage -Period 2009-2010
This section presents the results of the new modality of the course from 2009 and on, incorporating the measures proposed as part of the strategies of the first and second stages.As before the results are stated for P2 and the following two programming courses: P3 and P4.

Results of the implemented strategy in the second stage for "Programación 2"
Table 5 shows the results of the years corresponding to the second stage (2009)(2010), where the percentage of students that fail the course is reduced while the percentage of students that exonerate increases almost by the same number.Another important fact of this second stage has to do with active students, which raised from 70% to 74% (and 65% to 74% considering the periods 2005-2008 and 2009-2010), this is evidence of a lower drop-out rate in the course.The percentage of students that fail even if they have passed the project is still around 3%, indicating that the practical project is still a fundamental learning task within the course.In Table 6 it is noteworthy that there is a reduction in the drop-out rate in the beginning (Assignment 1), but fundamentally between Assignment 2 and Assignment 3, there is a significant reduction in the later case from 15% to 3% in the period 2009-2010.
To evaluate the effect of the changes introduced in this stage the "group effect" results are presented, this is the impact of the project being done by groups of students on the results of the lab itself and on the course in general.
Tables 7 and 8 present the results of the active students in the practical project and in the P2 course, splitting the population of students in two groups: those that worked in groups of 2 students (WG) and those who did the project individually (NWG).Even though it is a small sample, there seems to be a correlation between being part of a group and the performance in the practical task.
Table 8 shows that the students that did not work in groups performed worse than those that did, with a difference in 4% for all active students.On the other hand, Table 8 shows that being part of a group does not only influence the results of the project but also the global results of the course.Even though the exoneration results remain the same regarding this variable, it stands out that the suffice percentage is higher for students that worked in groups.There are many possible readings for the results observed in Table 7 and Table 8.Nevertheless, it is impossible to determine if the decision of being part of a group is the cause of the improvement or if certain characteristics of the students that influence a better performance, lead them to form groups.There are pedagogical studies supporting the hypothesis that collaborative learning and cooperation between pairs improve the quality of learning, favoring the incorporation of knowledge by the students [15,16].It may very well be that the commitment taken on by the group helps avoid the drop-outs and thus favors that some students that would otherwise fail, now keep coming and have a greater chance to suffice or exonerate.Part of the future work is a necessary in depth analysis of the effects of working in groups.

Local Related Work
This work is part of the policies proposed and adopted by the "Facultad de Ingeniería de la Universidad de la República" to face the problem of highly populated courses.It consists of the implementation of some of the changes proposed in [2].
There are some experiences in the use of new ICTs for the improvement of courses in the local context, mainly addressing two problems [17].The first being the impairment of the students to attend classes on a regular basis, and so it becomes necessary to transform the course to a semi-distance modality [18].The second problem arises in the context of highly populated courses, were there is little contact between teachers and students.In the last case, the primary goal is to carry on the course as normal but increasing the communication between the actors involved [19].The characteristics of the P2 programming course and the actions proposed as part of the first stage strategy performed by the group of teachers during the first stage (2003)(2004)(2005)(2006)(2007)(2008), aim at improving the quality of the communication.Nevertheless, this work also proposes an appropriate organization and management of resources in the context of a highly populated course.Some of the implemented measures in the second stage address the first type of problem [17].
There are several courses in the Engineering Programs of the "Universidad de la República", particularly in Computer Science, that are diversifying their modalities to meet the new requirements and have begun to use EVA [14], for example P1 is now a completely semi-distance course and "Introducción a la Programación Funcional" [20] is a distance learning course.Moreover, there are also some local experiences in basic courses in mathematics and physics, which are developed in a context of massivity, that are related to our approach.In particular, [21,22,23,24].

Conclusions
In the context of highly populated courses and limited resources assigned to teaching, it has been shown to be of outmost importance to adequately organize the teaching staff to make a better use of it.This work proposes and shows the results of a decentralized form of organization, with high specialization of the teachers in the roles required in the P2 course and in the different tasks assigned to the roles.With this type of organization it is possible for all teachers involved in the course to have deeper and more detailed knowledge about their specific work area and the tasks performed by the work team they are a part of, even if they have small workloads assigned in the course.The related teaching activities such as weekly meetings and coordination meetings become more efficient.It is believed that the specialization of the teaching staff has a positive effect on the quality of the programming courses in general and in particular in the level of the P2 course.
Currently, after 8 years of work, the P2 course is at a point where there is significant amount of study resources available for the students and teaching material for the teachers.This allows compensating for the short time the teachers can dedicate to each student, as well as for some teachers having small workloads assigned to the course and the rotation of teachers among other courses.To cope with the problem high students per teacher ratio, initially the newsgroup was considered an appropriate measure and was applied.At a second stage the use of an EVA was instrumented, improving the teaching-learning process by allowing a more fluid exchange of opinions, ideas and knowledge between teachers and students, but fundamentally among students.Both measures also made more study material and resources available for the growing group of students that prefer distance learning and do not attend lectures and exercise classes.
In the P2 course it has been confirmed that the practical work on a project has big incidence on the results of a basic programming undergraduate course, and has direct impact on the results.The experience with P2 shows that the increase in the complexity of the project had the effect of increasing the requirement level of the course for the students, which can be seen in the first years from 2003 and on, were suffice and exoneration rates did not improve.However the result of that first phase was an improvement in the quality of the course, proven by the analysis of the statistic results for the following programming courses (P3 and P4) and the perception of the teachers in those courses, who have observed an improvement in the background knowledge and skills of the students as they start the courses.
The strategy implemented from 2009 and on in P2, on one hand consolidated the work done in the first stage, from 2003 to 2008, and on the other hand provided all the means necessary to accomplish a real semi-distance learning course.The change in the modality of the practical project work, allowing students to work in groups, as well as the establishment of an introductory assignment (about tools and the programming language), had a positive effect in lowering the drop-out rates and increasing the suffice rates.These measures stimulated group work without lowering the requirements for the students; this is shown not only in the results for P2, but also for P3 and P4 during the last years.
For future work we propose that it is both necessary and interesting to perform a qualitative analysis of the changes observed in the following programming courses motivated by the changes in P2, regarding acquired knowledge and skills.
A final conclusion is that the evaluation of this experience and the results has an inspiring effect, stimulating the teachers and the educational institutions involved to keep on working in this direction and to improve courses with similar characteristics.

Table 3 :
Active Students that Failed, Sufficed and Exonerated P3 between 2001 and 2008

Table 4 :
Active Students that Failed, Sufficed and Exonerated P4 between 2003 and 2009

Table 6 :
Results of the first, second and third assignments 2009-2010

Table 7 :
Students that passed or failed the Project of P2 in 2009-2010, classified in worked in group or not In the previous table, arrows indicate the tendency of the values in comparison with the general population.Up arrow (↑) indicates an increase, while down arrow (↓) indicates a decrease.

Table 8 :
Students that Failed, Sufficed and Exonerated P2 in 2009-2010, classified in worked in group or not

Results of the Impact in the Following CoursesTable 9 and
Table 10present the observed results for P3 and P4.In both cases, the results for the second stage are shown in the second line.

Table 9 :
Classified in worked in group or not P3 between 2003 and 2010

Table 10 :
Classified in worked in group or not P4 between 2004 and 2010 Although there are no significant improvements for P3 and P4 (only reduction of 1% in P3 and 3% in P4) compared to those observed in the first stage, it is important to notice that the changes in P2 did not have a negative impact on the following two programming courses, although there had been doubts about a possible negative impact of the group work in the approval rate for P3 and P4.Possibly learning to work in groups may have compensated for the fact that maybe not all students acquired the same practice in programming; nevertheless it is necessary to keep monitoring the results to draw further conclusions.