Design and Experimentation of Activities for CS1: A Competences Oriented Approach (unpacking the Informed Design Teaching and Learning Matrix)

In Introductory Computer Science courses, especially Computer Science 1 (CS1), dropout rates are generally high and results are often disappointing. In order to motivate and engage students to achieve better results in CS1, our teaching strategy is based on designing several activities using a competences oriented approach. This paper describes the use of a framework proposed by Crismond and Adams in order to create pedagogical activities for the CS1 course at Universidad ORT Uruguay. We propose to extend that framework with competences oriented activities. We present a detailed description of each activity. Our thesis is that including this kind of activities helps to obtain better results. Experimentation was done in 2012 and 2013. Teachers of the experimental group referred a high level of motivation of the students. Results show that the inclusion of those activities seems to be helpful for students and the proposed pedagogical design appears to produce better final results.


Introduction
Programming is in the heart of computer science, and therefore most Computer Science (CS) programs globally start with an introductory programming course [1].Programming is certainly a complicated skill to master, and learning to program is correspondingly complex [2,3] and is understandably a key area of education research [4].In Introductory CS courses, especially CS1, dropout rates are generally high [1,3,5,6,7] and results are often disappointing [1,3].Several strategies have been adopted by different institutes in the organization and teaching of these courses to diminish these effects [6].
Competences and their development have acquired a key role in many current teaching and training methods [8,9].Competence means the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development [10].Generic and specific competences are been proposed in Computing Engineering profiles through competency-based curricula models [11].CS1 course at Universidad ORT Uruguay emphasizes teaching problem-solving methodology that uses an Object Oriented Programming approach.
The research question that motivated this study was: What kind of competence-oriented activities can we design to engage students and obtain better results?
We propose unpacking the Informed Design Teaching and Learning Matrix proposed by Crismond and Adams [12], assuming the role of informed design teachers that propose innovative activities (oriented to competences in our case), in order to satisfy learning goals of the course, develop several competences of our students and reduce drop out percentages.This paper is structured as follows: Section 2 refers to competences and learning outcomes.Section 3 includes several teaching strategies at University level.Section 4 reports the Informed Design Teaching and Learning Matrix concepts.Section 5 deals with CS1 course.Section 6 describes our proposal, details the unpacking of the Matrix and explains each of the activities proposed.Section 7 reports the experimentation.Section 8 presents the conclusion and future work.

Competences
Competences and learning outcomes are emerging as a new teaching/learning paradigm where approaches centered on the learner are increasingly important, playing a key role in the teaching and learning process [13].
Competences represent a dynamic combination of knowledge, understanding, skills, and abilities [14].Most of the taxonomies of competences are organized into general and specific.General competences have acquired a special relevance in the last years [15].
Transversal competences are usually forgotten and neglected, however competences in transversal skills are considered by employers thinking about hiring a university graduate as important as technical knowledge [16].
Higher Education must provide advanced knowledge, skills and competences that students need for their professional life [16].
The higher education sector is faced with several strategic decisions in order to maximize quality, impact, and competitiveness.In this context of "engineering" competences and learning outcomes, one core challenge is the inclusion of curriculum stakeholders in prioritizing subject-specific and generic competences in study programs [17].
Tuning Latin America Project (TLAP) [18] refers to 27 generic competences.From the list of generic competences agreed for Latin America we selected those which apply in the context of our CS1 course: capacity for abstraction, analysis, and synthesis (C1), ability to apply knowledge in practice (C2), ability to organize and plan time (C3), capacity for oral and written communication (C4), ability to use information and communication technology (C5), ability to learn and update learning (C6), ability to identify, pose, and solve problems (C7), ability to work as part of a team (C8), interpersonal skills (C9) and ethical commitment (C10).These competences are skills that software engineering graduates must possess [11].

Teaching Strategies
Good teaching is getting most students to use the level of cognitive process needed to achieve the intended outcomes that the more academic students use spontaneously [19].Traditional teaching methods do not seem adequate for many students for different reasons [3].Beyond the mastery of core CS material, good CS educators should also be familiar with a significant body of material that will expand their perspectives on the field, and consequently, enhance the quality of their teaching [20].
Several teaching and learning concepts have been proposed in engineering education [21].Effective teaching requires flexibility, creativity, and responsibility in order to provide an instructional environment able to respond to the learner's individual needs, and one of the ongoing challenges the university teachers are facing is related to matching the teaching strategies with the students' learning styles in order to improve the academic results [22].
Students have different levels of motivation, different attitudes about teaching and learning, and different responses to specific classroom environments and instructional practices, the more thoroughly instructors understand the differences, the better chance they have of meeting the diverse learning needs of all their students [23].Activities that require students to collaborate, share solutions, review each others' work, or create materials have been shown to be beneficial for the students [24].
A survey of literature related to teaching of introductory programming reported by [25] concludes that there is no canonical answer to the question on how to teach introductory programming courses.Several techniques are identified by [26] in this courses: questionnaires, interviews, observations, videos, inventories, tasks, artifacts, tests and formal course assessments.
In this context, our teaching strategy is based on designing several competences oriented activities that motivate and engage students in order to achieve better results.

The Informed Design Teaching Matrix
Crismond and Adams [12] report the Informed Design Teaching and Learning Matrix (IDTLM) for engineering education that describes design strategies, contrasting patterns tiles, statements of how beginning and informed designers do those strategies, relevant goals and instructional approaches that teachers can use.
Crismond and Adams [12] argues that IDTLM acts as a framework for teaching and learning, including teaching strategies that instructors need to know to teach engineering effectively.Design activities are the opportunity to naturally weave together skills, processes, and knowledge that are typically taught separately in the discrete subjects of traditional curricula [27].
The following examples of teaching strategies (TS) are presented by Crismond and Adams [12] supporting design strategies: • TS1: comprehending the problem statement, problem framing, and scoping.
• TS2: focus information searches, study prior art, writing a product history report, research users, product dissections and reverse engineering.• TS3: divergent thinking, brainstorming, constraint relaxation, generative database searches, starter vs. final project challenges.• TS4: thinking with given model, building before sketching, virtual drawing and computational modeling, descriptions and structures reviews of design ideas, artifacts, and gestures as stand-ins for drawings.• TS5: explanation-based designing, decision diagrams, design values and guidelines, emotions and their role in design decision-makings.• TS6: experiment-base design advice, investigate-and-redesign task and product comparisons.
• TS8: design storyboards, project and time management, instruction and scaffolding for systematic design, risk taking and iteration.• TS9: design diaries and portfolios, compare and contrast design cases, computer-supported structures reflections.
All the above teaching strategies could be used in teaching activities and support the design strategies presented in the IDTLM.Nine design strategies (DS1-DS9) are presented in the IDTLM, also contrasting patterns tiles (Informed Design Patterns) and several instructional approaches (TS1-TS9) that teachers can use.
Design knowledge and skills is a core-learning objective in combination with reinforcing fundamental engineering competences [28].Informed design is a pedagogical approach to design, and as a pedagogical strategy, design activities have great potential to: engage students, encourage pluralism thinking, reflect upon, revise and extend internal models [29].
Crismond and Adams [12] focus on teaching and learning design, arguing that IDTLM contains design strategies that help teachers do informed teaching, helping teachers design tasks while developing their own design pedagogical content knowledge.
Crismond [29] argues that The "Informed Design Teaching and Learning Matrix" provides more in-depth descriptions of the design practices, research on misconceptions, and teaching strategies.In this context, unpacking the matrix and extending the IDTML in a pedagogical approach help us to become "informed design teachers".

CS1 Course
Introductory programming courses develop several learning activities and programming projects in an attempt to change negative outcomes, high failure and drop-out rates [1,6,30,31].Various problems experienced by novices were identified relating to basic program design and algorithm complexity.Programming courses suffer from a wide range of difficulties and deficits [30].
Gomez and Mendez [32] argues that programming is a complex subject that requires effort and a special approach in the way it is learned and taught.To become good programmers, students must acquire several abilities and traditional teaching methods do not seem to be adequate for all students' needs.Also, they describe different reasons why learning programming is inherently difficult: teaching is not personalized, teaching strategies do not support all students' learning styles, teachers are more concentrated on teaching programming languages instead of promoting problem solving, programming demands a high level of abstraction and programming languages have complex syntax.
Literature of learning programming contains many research studies that have been proposed to face these problems with different approaches; for example: games [33,34], robots [35], pair programming [36,37].
Our CS1 course at Universidad ORT Uruguay emphasizes teaching problem-solving methodology using an Object Oriented Programming (OOP) approach.The course prepares the learner for constructing simple programs using the OOP paradigm.By the end of the semester, the student will be ready to analyze simple situations, to design possible solutions and to implement them with an OOP approach.Our teaching strategy is based on designing activities that motivate and engage students in order to achieve better results.
The duration of the course is 15 weeks, 4 hours of lectures and 2 hours for lab session per week.The programming language used is Java for all these assignments.A brief description of the 15-week course is shown in Table 1.
The main topics are: pseudo code, variables, and control structures, objects and classes, association, inheritance, aggregation and collections, enumeration, sorting and searching, and advanced use of collections.
The course includes two compulsory programming assignments (done in pairs) and a final evaluation.The first programming assignment has 20 points; the second 30 and the final evaluation 50.If the student achieves 86 points or more passes the course and does not have to take a final exam (AP1).Between 70 and 85 the course is approved but the student must take a final exam (AP2).With less than 70 points the student fail and must retake the course (FL).

Proposal: Unpacking the Matrix
IDTLM proposed by Crismond and Adams [12], suggests pedagogical strategies to help teachers design pedagogical content.IDTLM shows key concepts of what instructors needed to know to use design activities effectively in the classroom.
In this context, we unpack the IDTLM, generating informed design activities based on competences in order to satisfy learning goals and develop selected competences of TLAP.
Our proposal is to design pedagogical activities for CS1 using the IDTLM as a framework.We use several teaching strategies (TS1-TS9) proposed by Crismond and Adams [12] and we integrate a variety types of activities, each one designed with the IDTLM design strategy (DS1-DS9).
In particular, we propose to extend the IDTLM with concrete activities (Table 2, Column 3) incorporating a list of selected competences that students should develop for each one of the proposed activities (Table 2, Column 4).
Learning objectives and goals of each activity have been defined in this context and we ensure that each one of the selected TLAP competences were covered (completeness of the selected TLAP).Use words and graphics to display both benefits and tradeoffs of all ideas before picking a design.

UML Modeling Game (week 6)
C1,C2,C8,C9 In order to design pedagogical content we define learning outcomes in terms of competences and we align teaching activities with strategies (TS1-TS9) proposed by Crismond and Adams [12].In Table 3 we map each activity with the selected teaching strategies.A detailed description of each activity proposed is given below:

Scratch
We used Scratch [38] in the very first weeks with the purpose of improving students' programming experiences and motivation.Scratch can be taken as an auxiliary tool of students' learning of programming, stimulating the students' learning motivation and cultivating their ability of solving practical problems with the computational thinking approach [39].A list of exercices is given to students.The list includes exercises to develop in Scratch (like to draw a particular grid, see (Fig. 1) ), sample codes with simple mistakes to correct and examples to complete.Infographics are graphic visual representations of information, data and knowledge that includes text and images, narratives, descriptions, maps, etc.This activity aims to generate computer graphics containing answers to: "What is Java?," "Where?", "Who developed it?","When?".As a homework, each student searchs the required information and develops the infographic.In (Fig. 2) there is a sample infographic developed by one student.Students must bring to class the picture.In class, all infographics are discussed and the main topics are referred.

Wordle
Students are asked to create at home a "word cloud" from Java code using the Wordle tool [40].The class "Truck" is the first complete design example of a Java class that is presented in the course.The model consists of a truck, with attributes color and plate number.This includes instance variables and methods of access and modification.Each student creates his or her own "cloud" and in the next class all must provide an explanation of the words that appear more prominently (Fig. 3).

Modeling Clay
To become familiar with the concept of identifying objects, aliasing and message passing we use a kinesthetic learning activity modeling clay promoting comprehension of object oriented concepts as proposed in [41] .This activity focused on improving the comprehension on the difference between object and class, memory, creation of objects, garbage collector, aliasing and message passing (Fig 4).This activity takes about 20 minutes.The teacher brings to class various objects (eg disposable razor, kitchen grater, sandpaper drill, soap, bucket, cotton swabs, etc.).Each group of 4 students choose an object.On a sheet they must write: a) name of the group, b) drawing the chosen object, c) analysis of the object's characteristics: functionality, materials, costs, etc.).
Students must answer: what would you like to know more about the subject?What new questions arise?(eg Cost?, Who makes it?Is it recyclable?, etc.).Students must describe how they would answer these questions.From the responses on the characteristics and properties of specific tangible objects, they must find an analogy to desirable properties and characteristics of the software (such as usability, efficiency, reliability, etc.).The activity takes about 50 minutes.

Rubric
In class we study two tasks from a previous course: one developed properly and one with errors.They are given to each group with the rubric to assess the work.The rubric contains the following areas: organization of documentation, writing and spelling, class diagram, test data, listing, coding style and execution.Work is classified in each area as excellent, good or poor.Each group must assess and justify the chosen category.The activity takes about 30 minutes.

Puzzle Algorithm
Students should solve the problem of finding the maximum of a list of numbers.We provide students with an algorithm puzzle.Each piece of it is a line of code and there are some extra pieces (different valid solutions could be constructed).Students must select the lines, and recompose the algorithm.In (Fig 5) is presented an initial (and partially incorrect) solution proposed by one student.Students present their own solutions putting it on the blackboard, and in groups discuss and select the most appropiate.This activity takes about 45 minutes.Peer Instruction (PI) [42] engages students during class through activities that require each student to apply the core concepts being presented, and then to explain those concepts to their fellow students.A class taught with PI is divided into a series of short presentations of certain concepts, each focused on a central point and followed by a related conceptual question.We use this activity to develop concepts of "Array".The activity takes about 50 minutes.

Minute Test
Angelo and Cross [43] suggest assigning minute papers at the end of class.In one minute, students answer the following questions: (1) What is the most significant thing you learned today?, and (2) What relevant question is in your mind at the end of today's session?.The answers are discussed in groups and the most remarkable points are selected.

Video
Pair programming consists of two programmers sharing a single workstation (one screen, keyboard and mouse among the pair).The programmer at the keyboard is usually called the "driver", the other, also actively involved in the programming task but focusing more on overall direction is the "navigator"; it is expected that the programmers swap roles every few minutes or so [44].A short video of pair programming [45] is presented to discuss the concepts of the practice.Students discuss what is the true spirit of pair programming and the advantages and disadvantages of such methodology.The main purpose of this activity is to take into account the required skills and the problems of working in groups.This activity takes about 15 minutes.

UML Modeling Game
The teacher brings to class several toys in their original packages (e.g.race car, deck of cards, puzzle), see (Fig 6).
Students are asked to model the game in UML notation.Information of attributes is implicit in the packages (identification, recommended age, pieces, etc.).Different models are sketched on the blackboard and several valid options are analyzed.This activity takes about 20 minutes.

Experimentation
The research question that motivated this study was: What kind of competence-oriented activities can we design to engage students and obtain better results?.
In terms of GQM [46] (Goal, Quality, Metrics): Our study aims to generate informed design activities in order to enhance the development of several TLAP competences, providing better results in CS1 course from the educational perspective in the context of introductory programming (CS1) course at Universidad ORT Uruguay.
The independent variable chosen in the experimental design was the teaching method, which incorporates IDTLM design strategy and selected TLAP competences.The dependent variable is the course result.
An initial experiment was conducted in the second semester of 2012.One random group of 20 freshmen in the experimental group (EG) was selected to participate in the activities and 16 students were in the control group (CG).Students in the EG receive instruction though IDTLM design strategy and selected TLAP competences and students in CG received instruction through the use of traditional lecture.Common syllabus, notes, assignments, teachers experience and classroom were used in (CG) and (EG).Both groups included a high number of students who fail in previous course.A limitation of the study was the low number of freshmen.Considering only the students who take the course for first time, in the selected group, 44% (4/9) of them had completed the course and in the control group, 22% (2/9).
Based on that experience, we designed some improvements in the activities and replicated the study.In the first semester of 2013, two groups of students corresponding to CS1 courses were randomly selected to participate in the experiment.In the EG were 24 students and in the CG were 22 students.
An initial test on Java, object oriented concepts and programming was designed and applied in both groups.The scores showed that both groups were similar in terms of CS knowledge and the students were essentially novices.
As we exposed, final results of assessments establish a classification into 3 groups according to final scores in a 0-100 scale: students that must retake the course (1-69) (FL), students that must take a final exam (70-85) (AP2) and finally the category of students that approve the course (86-100) (AP1).Fig. 7 illustrates final results of assessments for EG and CG.
The dot plot shows a distribution of final result of each student.In the EG, most of the students are in the AP1 area (86-100 points) and in the CG they are distributed all over the values.In order to evaluate the proposed pedagogical design, we tested the following hypotheses: Null Hypothesis: H0: There is no difference in applying informed design activities with a competencies oriented approach in the final results of our students.
Alternative Hypothesis: H1: There is a difference in applying informed design activities with a competencies oriented approach in the final results of our students.
Different non-parametric methods can be applied in our experiment.We select Mann-Whitney U [48] (MWU) as the non-parametric context of the experiment.MWU is the non-parametric equivalent of Student's test or the t-test for two samples.Table 6 illustrates the significance level (p-value) of the Mann Whitney U Test, obtained by SPSS [49].In terms of statistical significance (measured by alpha (α)) a high level of significance (α < 0.01) was found.As the value of p-value shown in Table 7 is 0,001 (less than 0.01) we reject H0 and accept H1.In this context, there is statistical evidence to conclude that the performance of the experimental group is significantly different.
Considering the detailed final results of the students and the MWUT, we could infer that the proposed pedagogical design produce significantly better final results.
Teachers of the experimental group referred a high level of enjoyment.In class, informally talking with students they show high levels of motivation.
Related to the activities, a survey was conducted among students in the experimental group (EG).We asked them to order the activities according to their preference.Each student selected their three preferred activities.Integrating the answers, the three activities preferred by all the students were: Concept Test, Puzzle Algorithm, and Modeling Clay.A brief description of the results of student's survey preference of proposed activities is shown in Table 7.

Conclusion and Future Work
Unpacking the IDTLM allowed out teachers to generate informed design activities based on competences in order to satisfy learning goals.Our findings suggest that this pedagogical design approach seems to be more effective in engaging students in order to achieve better results.
As mentioned in the experimentation section, in the experimental group 83.3% of the students approved the course and in the control 50%.Also, teachers referred to high motivated students in the experimental group.The main internal threat to validity that could be identified is the size of the groups.
This set of didactic units and activities are the outcome of the work and didactic units are available to all CS1 teachers at Universidad ORT Uruguay.A subset of these exercises obtained one of the first prizes in the PRECITYE Program [50] (Regional Program in Engineering Entrepreneurship and Innovation).
In the future we will incorporate the Informed Design Rubric [51] proposed by Crismond to assess student learning over one or more design activities proposed.Also, we will incorporate other approaches and monitoring the performance of students in the following courses.

Table 2 :
Unpacking the Matrix

Table 3 :
Activities Vs Teaching Strategies

Table 4
illustrates final results for both (EG) and (CG).

Table 6 :
Mann Whitney U Test

Table 7 :
Activities Survey