Applying a Usability Technique in the LibreOffice Writer Project

As a result of the growth of non-developer users of OSS applications, usability has over the last ten years begun to attract the interest of the OSS community. The OSS community has some special characteristics which are an obstacle to the direct adoption of many usability techniques as specified in the HCI field. The aim of this research is to adapt and evaluate the feasibility of applying the Personas usability technique to one OSS project from the viewpoint of the development team. The applied research method was a case study of the following OSS project LibreOffice-Writer. We formalized the application procedure of the adapted usability technique. We found that either there were no procedures for adopting usability technique in OSS or they were not fully systematized. Additionally, we identified the adverse conditions that are an obstacle to their adoption in OSS and propose the special adaptations required to overcome the obstacles. To avoid some of the adverse conditions, we created web artefacts (e.g. wiki) that are very popular in the OSS field. Additionally, we found that there are obstacles to the application of the technique. Despite these obstacles, it is feasible to apply the adapted Personas technique in OSS project.


Introduction
Open source software (OSS) has spread so swiftly that it now rivals commercial software systems in terms of deployment [1]. Some OSS communities nowadays do not have processes in place to guarantee that, taking into account the features of this community as a whole, the developed software is good [2]. Shortcomings with respect to process, activity, task and technique definition in the field of OSS development has led researchers from different fields to take an interest in this field of research and try to remedy the failings [3][4] [5].
Usability is one of the key software development quality attributes [6]. In recent years, OSS has come to be an important part of computing [7][8] [9][10] [11] [12] [13] [14][15] [16] [17] [18]. However, several authors have acknowledged that the usability of OSS is poor [19] [20] [21]. In the empirical study conducted by Raza et al. [22], 60% of respondents (non-developer users) claimed that poor usability is the main obstacle to be overcome by OSS applications in order to encourage users to migrate from commercial software [21]. The main reasons for the generally poor usability of OSS developments are: OSS developers have tended to develop software for themselves [4] [23], the developer community is very much in the dark about who its users are [19] [24], OSS communities operate according to a meritocracy based on software code input, and, due to the unavailability of resources, testing and bug reporting are almost exclusively conducted by volunteers and end users [7].
On one hand, the human-computer interaction (HCI) field offers usability techniques whose key aim is to build usable software. However, they are applied as part of HCI methods and not within the OSS development process. On the other hand, the OSS development process focuses on source code and thus on feature development. The OSS development process has special characteristics (e.g., the members of the community are geographically distributed, resources are in short supply and the culture may be quite alien to interaction designers) [7][20] [22]. This prevents many of the HCI usability techniques from being adopted directly [25].
The OSS community has now started to adopt some usability techniques, mostly usability evaluation techniques [25]. The OSS community has not adopted many techniques related to requirements analysis. Some usability techniques have been adapted ad hoc for adoption in OSS development projects [25]. The adoption of usability appears to be less straightforward in the OSS development process than in commercial development due to some of the characteristics of the OSS community, like: (i) its feature-centred development, (ii) worldwide geographical distribution, (iii) shortage of resources, (iv) a culture that may be alien to interaction designers [7][20] [22]. Consequently, usability technique adoption is a demanding task because most HCI techniques are not designed for the type of environment in which OSS is developed [25]. Our research aims to determine how to adopt a usability technique (i.e., the Personas technique) in the OSS development process. To do this, we analyze and identify which obstacles have to be solved to be able to apply this technique in an OSS project (LibreOffice Writer). Our research work addressed two areas: SE and HCI. With the aim of bringing these two areas together, we use usability techniques as a bridge in order to adopt knowledge of the HCI area in the SE area, specifically in the OSS development process [25].
This research has two aims. First, we intend to adapt one usability technique the requirements engineering techniques for adoption in the OSS development process. The selected technique related to requirements engineering was Personas to conduct user analysis. For our research, we selected the technique Personas because it should improve and enrich the requirements analysis activity. Second, we set out to determine the feasibility of adopting this usability technique adapted to one real OSS project. In the following, we describe the criteria used to select the OSS project. LibreOffice Writer was picked because it is project that have organized and structured user community, which is beneficial for software usability evaluation. The research method that we used to validate the feasibility of our proposal for adopting a usability technique adapted to an OSS project is a case study [26]. This method is useful for extending the information on the phenomenon under study (adoption of adapted usability techniques) and determining whether the results are consistent, achieving sounder conclusions with a real setting (OSS projects). Consequently, we had to volunteer for an OSS project and join the respective OSS community. This paper is organized as follows. Section 2 introduces related work. Section 3 reports the research method followed to apply the usability technique in the LibreOffice Writer project. Section 4 describes the proposed solution. Section 5 discusses the results. Section 6 describes the study validity threats. Finally, Section 7 outlines the conclusions and future work.

Related work
In recent years, the worldwide OSS community has adopted just over 50% of evaluation related HCI techniques. However, only about 20% of usability techniques related to requirements engineering and design activities have been adopted [25]. Therefore, more research is required to support the adoption of techniques related to requirements engineering in OSS developments. In view of the importance of HCI and SE, it is only logical to study the user-centred software development activities in OSS projects. This is especially true of the requirements engineering stage, because the discovery of user requirements during the early development activities is useful for putting right any software defects detected later on [27].
There are papers in the literature reporting the usability evaluation of some OSS applications [14][28] [29]. Assa et al. [14] studied the usability issues facing software developers using code analyzers by evaluating one of the popular open-source static-code analysis tools. Al-Odan and Al-Daraiseh [28] conducted a thorough study, placing five of the most popular free and open source software tools side by side for comparison with respect to both user acceptance and technical specifications. Ternauciuc and Vasiu [29] tried to inventory existing methods for testing and improving usability, with a particular focus on e-learning platforms. However, usability technique definition and integration into OSS projects is a complicated process, which has not been researched at length [20][27] [30] [31] [32]. Existing papers suggest that usability techniques should be reconceptualized. However, they do not explain how the OSS community should go about adaptation. Nichols and Twidale [4] and Ternauciuc and Vasiu [29] are the only authors to put forward some general ideas for improving usability. However, the issues to be considered in order to adopt such techniques in OSS developments are unclear.
On the other hand, Castro [25] proposes a framework for integrating usability techniques into OSS developments. This framework is composed of a number of adaptations in response to the adverse conditions for adopting usability techniques in OSS development projects. In order to adopt usability techniques in OSS development projects, it is necessary to: (i) study the adverse conditions preventing the use of HCI techniques, and (ii) analyse what types of and which adaptations are necessary if these techniques are to be used in OSS projects [25]. The adverse conditions are classified into three major groups (families of adaptations). First, some usability techniques require a usability expert (most OSS projects do not have experts on their team). Second, some techniques require the participation of one or more users meeting face to face (OSS users are geographically distributed all over the world). Third, some techniques have to be applied step by step, prepared in advance or require preliminary information (OSS community work is wholly voluntary and done by members in their spare time) [25].
Although research examining usability in OSS has been published, there is no standardized procedure for determining how to adopt usability in OSS development [33][34] [35]. Likewise Scacchi states that there is no internationally accepted OSS development model defining OSS development in practice [36]. The first step in our research is to study how the OSS community uses usability techniques in its development projects. In order to discover the activities conducted in the OSS development process, we conducted a systematic mapping study (SMS) [33]. Castro's is the only published research to study usability problems and techniques occasionally adopted in OSS projects in an integrated manner and to report the current state of usability in the OSS community [25]. As a result of the literature review, we can say that only one of the research papers reports a general and systematic proposal for integrating usability techniques into the OSS development process, considering the particular characteristics, philosophy and idiosyncrasy of the OSS development process, without forfeiting the essence of usability techniques [37]. Two SMS related to usability in OSS were conducted in advance of our research. A SMS reviews the literature on a particular field of interest [37]. The first SMS was conducted by Castro [25], reviewing papers published up until 30 July 2013. The second SMS was conducted with a search range from 1 august 2013 to 30 july 2017 [38].
The literature review discovered some papers reporting and describing studies that report the adoption in OSS projects of the Personas technique [20] [27], the necessary information for applying the Personas technique was gathered from descriptions provided by the OSS community and not through face-to-face interviews of user groups. Faily and Lyle [40] presented four guidelines that software engineering tools should incorporate to support the design and evolution of personas. This technique was not applied as specified by HCI because these guidelines are based on their experiences of modifying the open-source CAIRIS requirements management tool to support design and development activities for the EU FP7 webinos project.

Research Method
The research method used to validate our research is the case study [41] according to which we gather the results and experience of applying the Personas technique adapted to LibreOffice Writer. This research method is used when the phenomenon under investigation (in this case, the adoption of an adapted usability technique) is studied within its real setting (in this case, an OSS project). OSS projects are the perfect setting for the case study reported here because OSS communities are, according to several authors [7] [47].
The main criterion for the selection of this method (case study) is that there is little room for experimentally manipulating the phenomenon under study: The adoption of the Personas technique in a real OSS project [48]. Additionally, it is a qualitative study requiring the collection of data using techniques that do not aim to measure or associate measurements with numbers such as group discussion, open interviews, group or community interaction and introspection [48]. Therefore, we decided to use web artefacts (e.g., online surveys) to collect data. The research questions are stated in Section 5 and are related mainly to how to adapt the Personas technique for adoption in the LibreOffice project.

Identified Adverse Conditions and Usability Technique Adaptations
The technique analyzed in this research (Personas) need to be adapted for adoption in the OSS development process because these OSS communities have characteristics that the HCI world does not account for like, for example, worldwide geographical distribution of their members, a code-centred world view, unavailability of resources and a culture that may be somewhat alien to interaction designers [7][20] [22]. Although usability techniques require conditions that cannot be generally met in the OSS world, they can be adapted to the idiosyncrasy of OSS projects.
The usability technique adaptation protocol entails specifying the techniques for adoption in the OSS development process. HCI authors do not specifically define the procedures for applying usability techniques in OSS projects. Besides, generally speaking, the procedures are, not defined in enough detail, that is, this problem applies not only to OSS but also to usability techniques generally. Therefore, the adaptation of the usability techniques that we propose is a four-stage process: (i) formalize all the steps to be taken to adapt the usability technique, i.e. clearly specify the details of the steps of which the technique adaptation procedure is composed, (ii) analyze each of the steps to identify the adverse conditions that are an obstacle to technique application in OSS, (iii) propose the adaptations required to overcome the above obstacles to their adoption in the OSS development process, (iv) apply the adapted technique in the selected OSS project. In the remainder of this section, we describe the adaptations that have been considered for the adoption of this technique in the LibreOffice project.

Personas Technique and Its Adaptations
The aim of the Personas technique is to output a representation of end users as guidance for application design [49] [51]. This technique is capable of gathering, analysing and synthesizing information related to users that are to interact with the software system and, therefore, help to focus software analysis and design on the features and objectives of the product end user [50] [51]. The Personas technique cannot be applied directly in the OSS development process because the OSS community has characteristics to which the HCI world is unaccustomed (e.g., a code-centered world view, unavailability of resources and a culture that may be somewhat alien to interaction developers). The Personas technique [49] is composed of seven steps. In the following, we describe the first three steps of this technique and report the adverse conditions that are an obstacle to its adoption in OSS developments.
The aim of Step 1 of the technique (identify behavioral variables) is to identify the behavioral variables of product end users (e.g., attitude towards information technologies). Cooper et al. [49] suggest that users be interviewed to gather the necessary information from which to put together the behavioral variables. However, OSS project users are geographically distributed all around the world. Thus, this characteristic of OSS projects is an adverse condition for technique application. To deal with this adverse condition, we propose that users participate in an online survey. The researchers designed the questionnaire (questions and measurement scales) used in the online survey.
Step 2 (map interviewed subjects to behavioral variables) establishes ranges on the behavioral variables identified from scalar responses to the questions designed for the survey proposed in Step 1. Cooper et al. [49] do not specify how to calculate the behavioral variable ranges for mapping. This step must be performed by a usability expert, which is an obstacle because experts very seldom participate in OSS development. As no usability expert is available, we propose substituting the usability expert by a team of junior experts supervised by a senior expert. In the Step 3 (identify significant behavioral patterns), Cooper et al. [49] do not specify how to identify behavioral patterns. This step requires technique and usability expertise. As no usability expert is available, we propose substituting the expert by a team of junior experts supervised by a senior expert. Table 1 summarizes the adverse conditions identified and the main adaptations proposed for Cooper et al.'s Personas technique [49]. We adopted some of the steps proposed by Cooper in order to facilitate the application of the Personas technique [49]. Table 2 shows, for each of the adapted Personas technique steps, the tasks to be performed for their application in an OSS project.

LibreOffice Writer Case Study Results
This section presents the results obtained from applying the Personas technique in LibreOffice Writer. LibreOffice is a free and open source office software package developed by The Document Foundation. It was created in 2010 as a fork of OpenOffice.org, another old open source project. Your environment is programmed in C++, Java, and Python computer languages. As explained in section 4.2, in the first step of the Personas technique, an online survey was prepared using Google Forms tool and distributed to the community through the two user mailing lists that can be obtained in the forum the LibreOffice project: users@global.libreoffice.org and users@es.libreoffice.org. The online survey sought to obtain information about behavioral variables related to issues similar to those defined in [49][51]: (i) Experience of using computers, (ii) frequency of use of the application, (iii) motivation to use the application, (iv) age, (v) profession or trade and (vi) purpose of using the application.
A total of 48 responses were received from users from the two mailing lists during the 15 days the survey was open. It should be noted that user participation is voluntary, meaning that they cannot offer continuous participation while this research is taken place. Thus, in the online survey, all the necessary questions were added to obtain the information that in the third step, check for redundancy and completeness, allowed to elaborate a detailed user´s personality profile and refine the Persona, for example: (i) What are your job responsibilities? and (ii) What activities do you usually do on your day?  [49] Adverse conditions Proposed adaptations • Identify behavioral variables.
• No users or physical spaces are available for face-to-face meetings.
• Users participate online via an online survey. • Map interviewed subjects to behavioral variables.
• Expertise of people familiar with the technique and usability is required.
• The usability expert is either a developer, an expert OSS project user, or a team of junior experts supervised by a senior expert.
• Identify significant behavioral patterns.
• Synthesize key characteristics and goals of personas.
• Expertise of people familiar with the technique and usability is required. • The format of the document associated with this step is not specified.
• The usability expert is either a developer, an expert OSS project user, or a team of junior experts supervised by a senior expert.
• The format of the output product is specified.
• Check for redundancy and completeness.
• Expertise of people familiar with the technique and usability is required.
• The usability expert is either a developer, an expert OSS project user, or a team of junior experts supervised by a senior expert. • The format of the output product is specified.
• Expand the description of personas attributes.
• Expertise of people familiar with the technique and usability is required.
• The format of the document associated with this step is not specified.
• Define and specify types of personas.
• Expertise of people familiar with the technique and usability is required. The survey applied in LibreOffice Writer is found on different websites, specifically, in two forums available in both Spanish 1 and English 2 . To synthesize the relevant characteristics and objectives of personas, the Weka tool was used [52]. As mentioned before, this tool contains the K-Means algorithm for data analysis and predictive modeling. The survey questions for the analysis with Weka had answers in closed format, which facilitated their handling with the application. After executing the K-Means algorithm, two user segments named Cluster 1 and Cluster 2 were obtained. Table 3 summarizes the variables and the dominant attribute in each of the user segments. For the next step, check for redundancy and completeness, the most representative users of each segment were identified, that is, those whose responses most closely matched the attributes of the segment. Finally, in the last step, the Cluster 1 segment was designated as the representative of the primary persona for having the most preponderant attributes among the surveyed users. On the other hand, the Cluster 2 segment represents the secondary persona who has some characteristics similar to the primary persona, but its main variations are related to the level of knowledge of the application and the frequency of use of the application. Finally, the primary persona foundation document is presented in Fig. 1, while the secondary persona foundation document is presented in Fig. 2

Discussion of Results
In this section, we discuss and answer the research questions stated in this research.

RQ1: How can Personas Usability Technique be Adopted in Real OSS Projects?
Usability techniques were created for another type of software development, that is, were not designed taking into account the specific characteristics of the OSS development process. On this ground, it is necessary to adapt the techniques. These adaptations are based on the adverse conditions for technique application. Some of the adverse conditions are overcome using certain web artefacts (for example, wikis, forums, etc.), which are known to the OSS community. Consequently, OSS community members will be familiar with many of the adaptations, and this is likely to encourage the application of usability techniques. Below, we describe the major technique adaptations.

ROLES AND TASKS
a. He is employed in a private sector company. b. His job responsibilities are in the area of teaching. He is a teacher. c. He regularly uses a computer at work. d. The activities he usually does on his day include staying informed on scientific topics. e. The activities he usually does with LibreOffice Writer include re-writing and supporting in the forums.

SKILLS AND KNOWLEDGE
a. His level of education is Graduate Master. b. He has been using computers for more than 4 years. c. He likes to work with computers. d. The applications he uses the most are word processors.

SKILL WITH LIBREOFFICE WRITER
a. He uses LibreOffice Writer every day of the week. b. He considers his experience with the application as an advanced user. c. The main difficulties he encounters in the application are the formula editor. d. His favorite features are styles for paragraphs, characters, etc.

CONTEXT / ENVIRONMENT
a. He place to access LibreOffice Writer is at home. b. He uses a laptop as devices to access LibreOffice Writer. c. He has a laptop as his only personal device.

Figure 2: Secondary Persona Foundation Document
There are mainly two adaptations for the Personas technique. First, users participate online using electronic mail. Second, we suggest that the usability expert be replaced by a developer, expert user or a team of junior experts supervised by a senior expert. The usability expert was replaced by a team of junior experts supervised by a senior expert. During the application of the Personas technique, we came up against two problems. First, it is difficult, due to time zone differences, to arrange meetings with users and developers in order to create and discuss documents associated with this technique. Second, a usability expert is required to analyze and interpret the results of the application of the technique. As a result of the application of the technique, we discovered that to be able to define user segments, the material for gathering the information must be distributed using social networks, due to the low OSS community participation rate (i.e., of users contacted through the OSS forum). Additionally, OSS user participation was lower than expected on two grounds. First, we expected, based on statistics reported on the official website of the application, to get a large number of users to participate. Second, it was hard to contact and engage users to participate in this research.
In the following, we describe the impact generated using the adapted usability technique with the application in LibreOffice Writer project. In this particular case, however, we created two plug-ins (the first plug-in 3 to insert a page number in the header/footer and the second plug-in 4 to apply the capital letter in a text paragraph) in order to improve some LibreOffice Writer features. Due to the small number of users, the plug-ins were not sent to the LibreOffice Writer project for feedback from developers at this preliminary stage of the research project, which is due to be conducted on a larger scale in the near future. Considering the software life cycle, the Personas technique, which are part of the user analysis activity during the requirements engineering stage, would be adapted and applied before requirements elicitation to provide details about prospective software users, as OSS user segments are not previously defined. Note that the results of the usability technique application were sent to the selected software project developers. Some responded that our results would be used later. Note, however, that these results are potentially applicable by any OSS development team. The results of the application of the Personas technique would facilitate the adoption of usability mechanisms as part of requirements engineering activities, thus helping to improve the usability of the software system under development.

RQ2: Which are the Types and Characteristics of the OSS Projects in which it is Possible Work with Users and Experts to Adopt Adapted Usability Techniques?
There is a wide variety of OSS projects classified according to the level of development, size of community and resources: (i) large projects with an organized and structured user community and with company support (for example, Sun Microsystems contributed to GNOME [53]), (ii) medium-sized projects with a sizeable community of members (from 100 to 500 users) and a growing group of developers (from 5 to 10 developers) without company support (for example, FreeMind), (iii) small projects with very few developers and without a list of representative users or company support. Depending on the project type, the adaptations of the technique may vary. For example, for large projects that have HCI teams [19], it will not be necessary to make adaptations in this respect. Then again, it is necessary to implement usability technique adaptations for small projects that do not have many resources. LibreOffice Writer is a large and longstanding community. It is large-scale OSS project, which is well organized and well structured, and they also have a large number of users. The project administrator had a list of users but had not identified which were representative. Generally, OSS developers working on large-scale projects are not acquainted with the profile of their users [24] [54]. The adaptation of the technique for LibreOffice Writer project affected both the project and the team. In the LibreOffice Writer project, not only did one of the administrators give us permission to apply the usability technique and answered our questions, he also directly participated in the design of the questionnaire used to apply the user profiles Personas technique.

Study Limitations
There are some limitations on the validity of this study, because it is eminently qualitative [55]. We identified the limitations of our results using the guidelines provided by Runeson [26]. Regarding construct validity, both, internal and external, we found no problems. Finally, regarding reliability, the case study should provide for the participation of all OSS user types, that is, we should not just recruit real users (regular OSS tool users that are registered with the community). Therefore, before applying the technique, we suggest that other options for recruiting users who are genuinely interested in participating in this type of research (for example, social networks) should be explored.

Conclusions
The aim of this research was to adapt and evaluate the feasibility of adopting HCI Personas usability technique in real OSS projects. Technique adaptations are necessary to address some adverse conditions caused by OSS development characteristics that are an obstacle to the application of techniques prescribed by the HCI area. In particular, we made adaptations to the Personas usability technique, for application in the LibreOffice Writer OSS project. The main adverse conditions that we found with the technique are: (i) the need for a usability expert to apply the Personas technique, (ii) user participation, and (iii) the need for several users to be present in person to apply the technique. To address these conditions, we propose three adaptations: (i) substitute the usability expert by a team of junior experts supervised by a senior expert (a MSc in Usability Engineering student supervised by two HCI expert researchers); (ii) remote user participation, that is, the users use web artefacts (for example, online surveys); and (iii) use biased sampling including family and friends of the observer as users, and (iv) conduct remote observations/interviews of geographically distributed OSS users using different tools for text, voice and video transfer over the Internet.
Collaboration is one of the principles underlying OSS community [27] [56]. However, we did not get much collaboration during the application of the usability technique, perhaps because real users (i.e., users registered on the SourceForge project website) were short of time, unfamiliar with the importance of usability, or had no incentive to participate. On this ground, we suggest using social networks to publicize technique application and recruit as many participants as possible.
In the following, we detail the key recommendations that researchers and developers should consider to apply usability techniques in OSS projects: (i) encourage the OSS community to start to attach importance to software development questions addressed by HCI, (ii) use other means of communication (like social networks) to promote the application of techniques and not only official OSS project forums, (iii) promote some sort of incentive to recruit as many participants as possible for these initiatives, and (iv) consider the software development status when selecting the OSS technique and project.
After analysing and applying the Personas usability technique, we found had adverse conditions, such as the number of participants, biased information provided by developers, geographical and temporal distribution, and OSS community motivation. Despite all these problems, it is feasible to apply the adapted Personas technique in OSS projects. We believe that it is necessary for this community to start to attach importance and take note of the repercussion of the software development issues dealt with by HCI. Thus, as it is necessary to adapt the HCI techniques for adoption in OSS development projects, this community should take a broader view of software development to consider its usability and not just focus on feature development.
Below we detail how our research has advanced the state of the art in this field. Terry et al. [27] propose the reconceptualization of HCI methods for adaptation to OSS but do not explain how to carry out this reconceptualization, that is, they do not define the steps required to carry out each HCI and the web artifacts to be used for adoption in OSS. However, we set out the adaptations for the adoption of the Persona technique in OSS, specifying the steps and tasks associated with each step. These adaptations are implemented in the LibreOffice Writer project. Some researchers have evaluated the usability of OSS applications without the participation of application users [57] [58]. However, our research was carried out with real users, as OSS users have different requirements and needs to developers. Additionally, the problems that we found in our evaluations were validated by experts (junior and senior) and then presented to the developers in order to improve their application usability. Castro [25] proposes a general framework for usability technique integration in OSS development projects. This technique framework is very generic and does not report the steps for applying the techniques in OSS. The research reported here is an advance insofar as it specifies the Personas technique for application in the OSS projects, specifically describing the steps for this technique, as the procedures proposed by the original authors are not explicitly defined.
As future work, we intend to conduct further case studies to adapt and apply other usability techniques for OSS projects. We will analyze other web artefacts that can be adapted to OSS communities in order to improve communication (for example, social networks) and gradually raise the awareness of OSS developers about the benefits of applying HCI usability techniques.