On the Design of Group Decision Processes for Electronic Meeting Rooms

This paper reports a set of experiments motivated by the observation that the design of group decision processes is crucial to the success of electronic meeting room usage. Decision processes can be designed with more emphasis either on exchanging meanings (discussing issues) or exchanging artefacts (gene rating and structuring topics). Our problem is that, given a particular case to be discussed in an electronic meeting room, we do not know how to design the meeting for best performance. The paper builds a framework for studying this problem based on the n otion of communication mode. The experiments already made confirm that quality of results varies when different communication modes are used and show that meetings that do not exchange meanings result in solutions with inferior quality.


Introduction
The origin of this work is a project which main goal is to set up an electronic meeting room at INDEG, a public institute owned by ISCTE (University) dedicated to provide Masters degrees in Management Sciences.The creation of the electronic meeting room pursues two fundamental purposes: (1) provide an infrastructure to teach topics related to management sciences; and (2) demonstrate the environment to companies with links to the institute.Another purpose, which is of major importance to the project team, is to execute scientific experiments with the room, particularly in what concerns the effects of software usage on decision making processes.
The room is now operational with the following infrastructure (Figure 1): seats to a maximum of eight people, eight notebook client computers, one server, one Smart Board front projection unit from Smart Technologies Inc., one video projector serving the Smart Board, and two video cameras dedicated to record meetings.Concerning software, we have installed Meeting Works for Windows from Enterprise Solutions Inc. and GroupSystems from Ventana Corp. [46].One aspect we had to consider when planning the room concerned training people to manage its usage.Although ISCTE has several experts in meeting facilitation, none of them had experience in computer supported meetings.A great effort has been spent in understanding how meetings should be designed and actually in designing meetings.
Running various meeting sessions, we observed that it was extremely difficult to design meetings that combine both communicational and informational interactions.This classification differentiates situations where participants discuss issues (communicational interactions) from situations where participants generate or organise topics (informational interactions).
Communicational interactions are not supported by both software tools installed in the room.This lack of support means that the system does not provide, for instance, audio, video or textual communication channels.Two design alternatives may be considered under these circumstances: (1) either allow users to communicate face-to-face; or (2) rely solely on informational support tools (one alternative consists in using brainstorming tools) to accomplish decision making.However, informal experiments showed us that the second alternative lead participants to abandon system usage.
In this paper, we build the concept of mediation channels as a means to classify the possible combinations of communicational and informational interactions.This variable is subsequently used to study design alternatives.
The paper is organised in the following way.We start by summarising the experiments and results reported in the literature concerning electronic meeting rooms.Then, we describe some informal experiments and observations that lead to the definition of the problem addressed by this paper.Finally, we describe the controlled experiments, their results and our conclusions.

Related Work
The development and use of electronic meeting rooms has increased rapidly.Although they are mostly located at universities and other research facilities, we are starting to find them at corporate locations.The University of Arizona GroupSystems facility [24], now called Center for the Management of Information (CMI) [38], was one of the firsts.Under the direction of Dr. Jay F. Nunamaker Jr., it now includes 3 electronic meeting rooms, with networks of 29, 24, and 14 computers.In the corporate field, the most relevant example is IBM Corporation, which has built more than 50 GroupSystems facilities at its sites [23].
Other electronic meeting rooms include the Decision Support Lab [40], at the University of Nevada, with a network of 12 computers; the Management Decision Centre, at the George Washington University [43]; the Queen's Executive Decision Centre [41], at Queen's University in Ontario, Canada, which is directed by Dr. Brent Gallupe.
In the Electronic Meeting Room (EMR), at the University of Hawaii at Manoa [39], team members who are unable to attend meetings can join the session remotely, through dial-in lines that can be accessed via modem connections.
In Europe, electronic meeting rooms are less common than in North America.Still, several ones exist, like the Concert Lab, at the German National Research Center for Information Technology (GMD) [42] in Darmstadt, Germany.This lab is equipped with 4 SunSPARC workstations, a Xerox Liveboard, several video cameras and microphones, where images of remote participants can be projected on a large screen.At GMD, there is also the Pen Lab which uses pen-based technologies.Still in Germany, we can find the CATeam room at the University of Hohenheim [44] in Stuttgart, which has a network of 12 computers and a videoconferencing system.Since March 1995, the KBS-Media Lab [45] is running at Lund University, with facilities for electronic meetings up to six persons.
In Australia, there is the Strategic Planning and Decisions (SPD) Unit at Curtin University in Perth, exploring both Group Communication Support Systems and GDSS.In South Africa, it has recently opened the Centre for Information Systems (CIS), which is allied to the University of Cape Town's Graduate School of Business.The facilities of this centre include a network of 12 PC's connected to a SUN server and the software installed includes GroupSystems for Windows.

Research Framework
Increasing research in the area of GDSS has been made during the past years.In this section, we overview the most relevant experiences in the field, their results and issues that remain open.
There are several reviews of the experimental studies made in this area, the most recent and relevant ones by Benbasat and Lim [1] and Hollingshead and McGrath [16].Both propose a set of variables for studying the impacts of GDSS use based on the McGrath [19] framework: (a) task characteristics, (b) group characteristics, (c) contextual factors, and (d) technological factors.In this overview, we have added a few more variables that have been researched recently, mostly related to technological factors.Both reviews also describe the most commonly studied dependent variables addressed by GDSS research.These can be related to: (a) performance, (b) satisfaction, and (c) group structure.Hollingshead and McGrath [16] consider one more category of variables, concerning operating conditions which govern group activities.The set of variables is presented in Figure 2 and complemented with a brief description in annex.

Overview of Results
We now summarise the reports from experimental studies related to the variables presented in Figure 2.

Group Characteristics
Group size.Group size has been exhaustively researched.Dennis et al. [6] studied the effect of group size on performance and satisfaction, using small, medium and large groups (3, 9 and 18-person groups, respectively).Their results were: (1) group performance increased with size, (2) member satisfaction and confidence with outcome also increased with size, and (3) member participation was not affected by group size.Gallupe et al. [13][12] used four different group sizes (2, 4, 6, and 12-person groups) and studied the effects on production blocking and evaluation apprehension, comparing GDSSsupported and non-GDSS supported groups.Their results show that GDSS-supported groups have less production blocking and evaluation apprehension.Group performance is better in GDSS-supported groups, but the most relevant differences were found for larger groups, not being relevant in small groups.Performance also increased with group size for GDSSsupported groups, but not for non-GDSS supported groups.Valacich et al. [35] studied the difference between homogeneous and heterogeneous groups of different sizes, and concluded that the performance increases more with group size for heterogeneous groups than homogeneous groups.
Member proximity.The results obtained in this area have not been conclusive.Valacich et al. [33] conclude that distributed groups have better performance than face-to-face groups, but Chidambaram and Jones [2] found no significant differences.
Formal hierarchy.The study of Benbasat and Lim [1] concludes that the presence of leadership reduces the advantages of GDSS use, as meetings tend to take more time and members become less satisfied.
Group history.In their review, Benbasat and Lim [1] conclude that established groups have better performance than ad hoc groups.

Task Characteristics
Task complexity.Experiments focused mostly in generating and choosing task types.The usage of GDSS would be expected to have more impact in complex tasks, but the study of Benbasat and Lim [1] concluded otherwise.Their results show that usage of GDSS in simpler tasks was more efficient.Inadequate existing systems was perceived by the authors to be the reason.

Contextual Factors
Conflict.Valacich and Schwenk [29][36] made two experiments in this area, concerning two conflict techniques known as devil's advocacy (DA) and dialectical enquiry (DE).The first experiment [29] compares the use of these two techniques in the case of individuals and groups.Their results show that DA provides overall better performance than DE, although the difference is more significant in the individual case.The second experiment [36] compares these two techniques with a third one known as expert approach (EA).They concluded that: (1) DA generates more number of alternatives than the other two, but EA generates more than DE; (2) the use of DA and EA requires more voting rounds to reach agreement than DE; and (3) satisfaction with process and outcome was equal for all techniques.Poole et al. [27] have studied conflict management in GDSS-supported and nonsupported groups, and their results show that there were differences in the level of conflict.

Technological Factors
GDSS/non-GDSS support.An experiment by Gallupe et al. [11] found that usage of GDSS resulted in (1) the improvement of decision quality, (2) increase in the number of generated alternatives, (3) less confidence, satisfaction and consensus between the group members.The results in (1) and ( 2) are consistent with later experiments by Dennis and Valacich [7] and Valacich et al. [34].A more recent experiment by Massetti [18] also concludes that usage of GDSS results in the generation of more creative ideas.
Level of GDSS support.Sambamurthy et al. [28] conducted a set experiments to compare the use of Levels 1 and 2 GDSS.They conclude that: (1) while the Level 1 system leads to the generation of more ideas, the use of the Level 2 system produces higher quality ideas; and (2) Level 2 system users solve problems with more independence (without exterior help) than Level 1 users.In their review, Benbasat and Lim [1] conclude that this variable is the one with more impact in the group interaction process.
Facilitation.Not many experiments have been conducted in this area, but one by Dickson et al. [9] showed that chauffeured groups have better performance and reach a higher level of consensus than facilitated or non-facilitated groups.
User interface.A recent experiment by Sia et al. [30] compares icon-based with text-based user interfaces.Their conclusions show that icon-based interfaces result in better performance and more equality of participation.

Roomware.
Several experiments on the configuration of roomware have been made by Streitz et al. [32].They compared the use of a network of workstations (WS), a liveboard, pencil and paper (LB), and a network of workstations and a liveboard (WS+LB).Their results show that the WS+LB condition leads to the best performance.

Operating Conditions
Anonymity.Jessup et al. [17] conducted two experiments which showed that anonymous groups had better performance than identified groups, and that distributed anonymous groups were more efficient than the rest.Connolly et al. [3] studied anonymity along with evaluation tone (supportive versus critical tone), and concluded that anonymous groups which were given a critical tone had the best performance, and identified groups which were given supportive tone were the most satisfied.Hiltz et al. [14] studied the effect of pen names, a kind of anonymity, on inhibition and individuation of group members, and their results showed that the groups which used pen names were more des-inhibited, more critical, and reached a higher degree of consensus than the rest.
Production blocking.The effect of this variable is not considered relevant in electronic meetings, since all group members usually have the opportunity to work simultaneously.However, some experiments have been made where production blocking was introduced in GDSS on purpose, to study the resulting effects.Experiments by Gallupe et al. [10] studied the effect of production blocking in electronic brainstorming groups.The results showed that purposely blocked groups had less performance, generated more redundant ideas, and thought the task was harder than non-blocked groups.
Open Issues Nunamaker et al. [24] stress that future research should focus on explaining specific characteristics and features of GDSS systems, rather than performing more broad GDSS versus non-GDSS experiments.According to this perspective, there is a need to expand knowledge over operational conditions influencing GDSS usage.

Some Observations and Definition of a Problem
The first decision processes designed for the INDEG room concerned basic training of facilitators and demonstrations to undergraduate and MBA classes.Some of the experiments included generating a new name for ISCTE, generating a new name for a bleach product, and identifying the major problems in ISCTE's facilities and possible solutions.These experiments may be characterised as oriented towards "lists building".
The above meeting designs resulted generally well.Participants with low computing skills did not influence meetings, since both GroupSystems and Meeting Works are easy to use and do not require special training.The productivity of groups changed according to participants' interest and involvement in the subject, for instance the Bleach Product session was artificial to the MBA student attendants, but generally we can say that a reasonable number of items was generated.
After recognising that the design of "lists building" processes was a straightforward task, we designed a second set of processes: Board of Directors, a risk-decision scenario concerning the launch of a new product in an industrial company; Moon Landing and Alaska Plane Crash, two classical NASA problems that require participants to prioritise a list of tools in an emergency scenario; and Tele-Centre, a multi-criteria situation concerning the identification and selection of important requirements for a teleworking centre.These processes can be characterised as oriented towards "decision making", and requiring discussion between participants.
The Alaska Plane Crash experiment resulted in a complete failure.The process was designed to allow each participant to identify the most important items from a set of 15 and provide reasons for that selection.This task was done individually and in complete silence.Afterwards, the participants discussed their rankings using the GDSS (GroupSystems' topic commenter tool).At a point, during the electronic discussion phase, one participant said loudly "this [task] is completely wrong.First, we must define a strategy and only then select the items."Then, users started to communicate face-to-face and abandoned the GDSS.This event made us realise that the process, as it was designed, blocked the opportunities for open discussion.
The Board of Directors experiment showed a similar problem.One of the participants, playing the role of industrial director, complained that he had crucial information but could not stress its importance to the group while using the GDSS.The final decision gave more importance to comments made by one participant playing the role of marketing director.
The Tele-Centre experiment was then designed with face-to-face discussions during most of the process.Only the voting phase was done in silence.Although some remarks were made by the users, stating that they would prefer more freedom to manipulate items in the GDSS, the meeting was successful.
As it became clear with the cases described above, users' interactions can be characterised as either targeted at producing, organising and structuring data, or oriented towards expressing meanings to the group.We classify the former as informational interactions and the later as communicational interactions.One major difference that we perceive between both types is that communicational interactions have the potential to receive full attention by group participants.
In this scenario, the GDSS acts as a technological mediator, providing (or not) support for informational and communicational interactions, and putting more emphasis on one or the other.As meeting designers, facing multiple alternatives, we must understand how these mediation channels affect meeting outputs.

Framework for Studying the Problem
A comparison with the variables presented in the related work associates our problem with the level of GDSS support.In particular, Levels 1 and 2 make a distinction between communicational and informational interactions.However, in our perspective, this characterisation does not cover the full spectrum of possible combinations of the two interaction types.Though, we propose the mediation channel as a new input variable that combines communicational and informational interaction support.
This new meeting input is defined as follows.The mediation channel is classified in two axis.
The information axis is divided in non-GDSS and GDSS support.The communication axis is divided in three categories: face-to-face, none (or forbidden) and computer support.
The combination of the above alternatives defines six different mediation channels (see Figure 3): • Virtual conference -The computer is a technological substitute for face-to-face discussion, using textual, audio and/or video channels.• Co-located GDSS -The GDSS support to information sharing is complemented with face-to-face discussion.• Nominal GDSS -The meeting is limited to information sharing via the GDSS.
• Virtually co-located GDSS -The GDSS supports both communicational and informational interactions.

Informational
No GDSS GDSS

Controlled Experiments
We set controlled experiments to assess the influence of different mediation channels in group meetings.This section describes the experimental setting and meetings setup.

Experimental Setting
Problem.Are there any significant differences in what concerns decision quality between processes using different mediation channels?
Variables.One single dependent variable was studied in the experiments: decision quality.

Hypotheses.
The current experiments are limited to the following hypotheses: • H1: We will observe differences between co-located and nominal GDSS mediation channels.Lack of support to communicational interactions results in lower decision quality.• H2: We will observe differences between table conference and co-located GDSS.The use of GDSS for information sharing results in the improvement of decision quality.
Sample and procedure.The chosen population was composed by university students from public and private institutes in Lisbon.The variables used to select the sample were education, age and knowledge of Windows user-interfaces.The sample was made by a nonrandom method (family and friends) and had 72 participants (12 groups of 6 persons).The groups were randomly assembled.

Meetings Setup
There were three experimental conditions: (1) table conference; (2) co-located GDSS, with face-to-face interaction; and (3) nominal GDSS, with participants face-to-face but not allowed to communicate verbally.These conditions were applied, respectively, to three, four and five groups of different participants.
For all the experimental conditions the problem presented to subjects was the same -Moon Survival Problem [1].This problem is a rank ordering problem, and "the task requires that the subjects imagine themselves crash-landed on the moon 200 miles from base.All but 15 pieces of equipment have been destroyed.The remaining items are to be ranked in order of declination in contribution to survival on the walk to safety" [37].The task was presented to subjects as an exercise in individual and group problem solving.
Decision quality was measured as the absolute difference between the rank assigned by the group to the items and the rank assigned by the NASA Crew Research Unit.This variable can range between 0, as the best solution, and 210, as the worst.

Table conference situation
The six participants went to the room and took their places (without any pre-established order).The facilitator introduced himself, told the instructions and requested the participants to fill a user profile.
Primarily, the participants had to solve the problem by themselves using paper and pencil.When finished their individual solution, they were asked to discuss the problem among themselves.The main role of the facilitator was to involve all participants in the discussion and solve conflict problems that could have been brought up during the discussion.After the group had discussed all essential points (aprox.40 min.),they were asked to solve the problem again, in silence.

Co-located GDSS situation
The modifications to the experimental setting were the following.The facilitator introduced GroupSystems and certified that there were no doubts about the software (aprox.10 min.).The problem and instructions were then presented, and participants were requested to fill a user profile using GroupSystems.
Participants were asked to solve the problem by themselves, using the GroupSystems survey tool.When finished, the global solution was presented in the SmartBoard by the facilitator.Then, the subjects were asked to discuss the global solution.After the discussion, the subjects were asked to solve the problem in silence, using the survey tool.

Nominal GDSS situation
The modification to the previous setting was that the subjects were asked to examine the problem without any face-to-face interaction.GroupSystems' electronic brainstorming tool was used to support information sharing.The system was configured to automatically circulate subjects through all pages managed by the brainstorming tool.

Results and Observations
Currently, our analysis of the results compares the quality of individual rankings for each experimental condition.The results are summarised in Figures 5 and 7, where the horizontal and vertical axis display respectively the quality of initial and final rankings.The figures also display linear regressions of individual rankings.We start by comparing results from co-located and nominal situations (Figure 5).According to the set-up, the only difference between both meetings is that one is designed to allow participants to discuss face-to-face while the other requires users to share information using the computing system.These results show that quality diminished when participants were forced to use the system.Applying the T statistic to analyse if differences are significant (Figure 6), for a confidence level of 95%, we obtain that the null hypothesis is rejected.Therefore, hypothesis H1 is validated.Our observations, and also participants' comments, indicate that lack of communication is responsible for the bad performance of nominal GDSS groups: participants spent most time writing their arguments, neglecting attention to others' arguments.It is interesting to note that the system was configured to automatically circulate pages with comments between users, which was expected to reduce the attention problem.
These results raise the problem that, besides attention, other mechanisms used in face-to-face interactions are absent in nominal situations.Possible mechanisms include emphasising the importance of some comments or perceiving others' acceptance or rejection of one comment.
This has implications to software design and requires further experiments to evaluate which software mechanisms are necessary to preserve quality of results in nominal GDSS.
Figure 7 allows to compare the co-located and table conference situations.The results do not show any significant difference, which denies hypothesis H2.We emphasise that these results were obtained in the context of a decision process characterised by moderate complexity, rational decisions and no need for creative solutions.
These results are somewhat unexpected, given that one would expect at least two positive contributions from computer support: (1) it allows users to easily check and modify their rankings during the initial and final phases; and (2) displaying group rankings allows users to more easily perceive agreements and disagreements.To further investigate the above issues we selected the two items considered most important by the specialists -oxygen and water -and analysed their rankings.Figure 8 presents the results.
We also attempted to understand how participants changed their final rankings, either towards consensus (average of initial rankings) or towards the optimal solution (oxygen is number one and water is number two).As shown in Figure 8, participants seem to prefer the optimal solution (1.5 versus 4 for co-located participants, and 2.18 versus 4.56 for table conference participants).
Note, finally, that co-located participants start closer to the optimal solution, when compared with table conference participants, and hold that advantage.
From the above results, we conclude that there is very limited evidence that the co-located situation is better than the table conference, and that differences must be attributed to initial rankings.Since participants progress towards the optimal solution, the type of information provided by the computing system, i.e. displaying averages, does not fit well with users' intentions.

Conclusions
This paper departed from our observation that the design of group decision processes for electronic meeting rooms is a difficult task due to, in the one hand, multiple design alternatives and, in the other hand, incomplete understanding of implications carried by different designs to group decisions.
In our perspective, the definition of mediation channels contributes to clarify and build a framework for the multiple alternatives faced by facilitators when designing decision processes.Mediation channel is an input variable which results from the combination of different communicational (face-to-face, none and computer) and informational (no GDSS and GDSS) interactions.
The experiments described in this paper assess three different mediation channels: table conference, co-located GDSS and nominal GDSS.Results show that table conference and colocated GDSS provide better quality group decisions than nominal GDSS.The results also indicate that there is no significant differences in quality between table conference and colocated GDSS channels.
Some observations can be made concerning the above results.First, nominal GDSS groups require the development of specific computational mechanisms capable to reproduce functionality specific to communicational interactions, such as, for instance, attention.Second, in order to optimise GDSS results, it seems necessary to scrutinise adequacy to users' needs, a task that requires detailing precisely the properties of GDSS tools.Some of these properties are known and have already been studied, notably anonymity, group memory, parallelism [24], production blocking, or access to medium [10].The list seems however to be incomplete.
The experimental results were obtained in the context of a decision process characterised by moderate complexity and rational decisions.To understand if results apply to complex negotiation or strongly conflicting processes remains open.
Other mediation channel alternatives, namely nominal conference, virtual conference and virtually co-located GDSS must be assessed in future experiments.Furthermore, each mediation channel can be fine-grain characterised, using multiple degrees of either communicational or informational support.For instance, co-located GDSS can range from simple list building, organisation, and evaluation, up to more complex action planning.A complete understanding of the design of group decision processes for electronic meeting rooms requires results from such fine-grained experiments.
The electronic meeting room at INDEG is currently running and supporting teaching courses.
Lessons have been learned and resulted in meeting designs that accommodate and try to take most profit from face-to-face discussions in co-located GDSS meetings.Still, strategies devised to increase software usage during sessions are needed.

Figure
Figure 7 -Quality results

Figure 8 -
Figure 8 -Rankings of the two most important items 1

•
Table conference -No technological support is provided.This channel allows to frame experiments cited previously concerning GDSS versus non-GDSS supported groups.• Nominal conference -Nominal means silent and independent discussion, as for instance in the Nominal Group Technique [31], where participants give feedback but do not engage in argumentation or conflicting situations.