assessing decision making quality in face-to-face teams versus virtual...

This article was downloaded by: [UOV University of Oviedo]On: 17 October 2014, At: 04:19Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Decision SystemsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/tjds20

Assessing Decision Making Quality in Face-to-FaceTeams versus Virtual Teams in a Virtual WorldJeannie Pridmore a & Gloria Phillips-Wren aa Sellinger School of Business and Management , Loyola University Maryland , 4501 N.Charles Street, Baltimore , MD , 21210 , USAPublished online: 18 Apr 2012.

To cite this article: Jeannie Pridmore & Gloria Phillips-Wren (2011) Assessing Decision Making Quality in Face-to-FaceTeams versus Virtual Teams in a Virtual World, Journal of Decision Systems, 20:3, 283-308, DOI: 10.3166/jds.20.283-308

To link to this article: http://dx.doi.org/10.3166/jds.20.283-308

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/tjds20

http://www.tandfonline.com/action/showCitFormats?doi=10.3166/jds.20.283-308

http://dx.doi.org/10.3166/jds.20.283-308

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

JDS – 20/2011. Decision Making in Web 2.0, pages 283 to 308

Assessing Decision Making Qualityin Face-to-Face Teams versus VirtualTeams in a Virtual World

Jeannie Pridmore — Gloria Phillips-Wren

Sellinger School of Business and ManagementLoyola University Maryland, 4501 N. Charles StreetBaltimore, MD 21210, [email protected]@loyola.edu

ABSTRACT. Virtual teams use communication technologies to interact and accomplish workwithout physical presence. Prior research has shown that virtual teams can encounterdifficulties in making decision. However, never ICTs have enabled computer-based simulatedenvironments that look real and in which avatars can interact with each other. In comparisonwith research on decision making in face-to-face teams, literature on virtual teams in virtualworlds is just emerging. Our objective is to investigate how virtual world technology impactsteam decision making, and to compare with face-to-face teams. We collected data on teamsoperating in both environments and used an analytical hierarchy process model to comparetheir decision quality. Virtual teams in the virtual world took longer to reach a decision, buttheir decision accuracy was better than face-to-face teams. The results were surprising andsuggest that virtual world technology can be an effective team operating environment.RÉSUMÉ. Les équipes virtuelles utilisent les NTIC pour travailler sans contacts directs. Alorsque des recherches ont montré que ces équipes rencontraient certains problèmes dans laprise de décision, les NCIT du 21e siècle offrent de nouvelles possibilités. Par comparaisonavec la recherche sur la prise de décision en mode direct, la recherche sur les équipesvirtuelles en est toutefois à son tout début. Notre objectif est de rechercher l’impact desméthodes de travail virtuelles sur la prise de décision, en comparant avec les méthodesdirectes. Nous utilisons des données sur les comportements d’acteurs dans des groupes desdeux types et suivons un processus de hiérarchie analytique pour faire la comparaison. Leséquipes virtuelles sont plus lentes, mais la précision de leur prise de décision semblesupérieure. Ces résultats prouvent que les environnements virtuels peuvent faciliter le travailde groupe.KEYWORDS: Decision Making, Virtual Worlds, Virtual Teams, Decision Support, Web 2.0.MOTS-CLÉS : prise de décision, environnements virtuels, équipes virtuelles, aide à la décision,web 2.0.

DOI:10.3166/JDS.20.283-308 © 2011 Lavoisier, Paris

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4

284 JDS – 20/2011. Decision Making in Web 2.0

1. Introduction

Over the past several years, it has become the norm for innovative organizationsto create geographically dispersed teams often referred to as virtual teams that worktogether through Internet Communication Technologies (ICTs) enabled with Web2.0. According to a 2008 Gartner Group report, virtual work in the form of virtualteams will account for 60% of global work assignments (Bell, 2005). This trendsuggests that business professionals will most likely be part of a virtual team at somepoint in their career. In addition, the effective use of virtual teams has beenrecognized as one of the “five imperatives” emerging from information technologythat will force organizations to change structures and human behavior processes(Herman, 2001).

ICTs typically used for virtual teams, such as video and web conferencing, aimto provide a face-to-face experience; however, these technologies still do not allowfor an experience that is equivalent to the presence achieved in face-to-facecommunication (Davis et al., 2009). There has been a call for research to understandhow virtual team processes such as decision making are impacted by new ICTs suchas virtual world (VW) technology given their unique technical capabilities (avatars,shared spaces, and activities) to create a real sense of presence (Davis et al., 2009;Mennecke et al., 2011). In this article, we investigate the decision making processand decision making quality of teams working virtually through a VW as comparedto teams working face-to-face.

VW technology is defined as an immersive three-dimensional environment, inwhich people interact as avatars (i.e. virtual people) with each other using themetaphor of the real world (Bainbridge, 2007). Uses of VW technology have shownthat immersive worlds can support a variety of human activities and interactions thatimpart a wealth of new opportunities and challenges for enriching how we learn,work, and play (Boulos et al., 2007; Prasolova-Forland et al., 2006). VW technologyhas the potential to provide virtual teams with new ways of overcoming andmanaging geographic distance as well as other barriers to work together, but theircapabilities have yet to be examined in depth (Davis et al., 2009).

There are many VW in existence today: Open Sim, Active Worlds, and SecondLife, to name a few. Currently, Second Life has the largest population withapproximately 18 million residents. To better understand the complex socialinteractions that can occur in a virtual team operating in a VW and to increase ourknowledge of how team decision quality is impacted through the use of VWtechnology, we investigated the use of Second Life in a structured virtual teamexercise.

This paper addresses the following research questions:1) Does virtual world technology impact a team’s decision making process

compared to teams who meet face-to-face?

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4

Decision Making in Face-to-Face vs Virtual Teams 285

2) Can virtual teams using virtual world technology achieve equivalent decisionquality compared to teams who meet face-to-face?

VW technology provides an opportunity to advance knowledge on virtual teamsand technology capabilities needed to support them. Studies of VW technology andvirtual teams provide important knowledge for both researchers and practitioners onthe phenomena related to communication and information sharing (Majchrzak andMalhotra, 2003; Pinsonneault and Caya, 2005; Powell et al., 2004) as well as thechallenge of overcoming limits to location and dispersion (Cousins and Robey,2005; Jin and Robey, 2008; Sotto, 1997). The paper is organized as follows. First,we discuss virtual teams and VW technology. Next, we discuss a multi-criteriamodel of decision quality that considers both the process of, and outcomes from,decision making by comparing decision making and decision making quality inteams operating face-to-face versus virtual teams operating in a VW. In the nextsection the methods and experiment are explained, and the results are presented andanalyzed. Finally, the research findings are discussed and summarized, and ourcontributions to literature are addressed along with research implications andlimitations.

1.1. Virtual teams

Teams provide a method for combining various skills, talents and perspectives toachieve business goals (Siebdrat et al., 2009). The importance and use of teams inorganizations has been well established (Paul et al., 2004) to take on a variety ofissues such as product customization, new product development, and strategicdirection. Organizations believe that teams operating effectively can lead to acompetitive advantage (Cooke et al., 2001).

As the workplace has globalized and technology has delivered the ability tocommunicate across geographic boundaries, virtual teams have become afundamental business proposition. It is estimated that 61% of employees in largeorganizations have worked on a virtual team, and in the United States alone, 8.4million employees are estimated to work in virtual teams (Furst et al, 2004). Thesenumbers can only be expected to increase with globalization, joint ventures, andpartnerships (Kanawattanachai and Yoo, 2002).

Teams working face-to-face are characterized with having a high level ofpresence and visibility, but even face-to-face team decision making can be flawed(McNamara et al., 2008). Virtual teams rely on technology to carry out allcommunication and decision making processes (Davis et al., 2009, Dube and Pare,2004; Zigurs, 2003). Face-to-face communication is limited if it occurs at all.Therefore, virtual teams can be more complex than traditional face-to-face teams,because they cut across time, distance, and even organizations.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Virtual team research hails from a wide range of fields including organizationalbehavior, human resources, communication, psychology, information systems,production operations, education, project management, medicine, strategy, and smallgroup research (Schweitzer and Duxbury, 2010). Despite the popularity of virtualteams, research in the field is still “maturing” (Saunders and Ahuja, 2006, p. 663).Previous research has found that virtual teams perceive it harder to communicatenonverbally than face-to-face teams (Cramton, 2002), underperform in their abilityto comprehend and exchange information (DeMeyer, 1991; Galegher and Kraut,1994; Bordia, 1997) than face-to-face teams, and are more task focused and lesssocially focused and have weaker relational links to teammates than face-to-faceteams (Chidambaram and Bostrom, 1993; Walther, 1995; Burke and Chidambaram,1996; McDonough et al., 2001). One study found that face-to-face teams had higherlevels of team cohesiveness (Warkentin, 1997), but other studies indicate that whilevirtual teams begin with lower unity, over time virtual team members may be able toexchange enough social information to develop cohesion (Chidambaram andBostrom, 1993; Chidambaram , 1996).

A recent meta-analysis found teams operating virtually were found to have moreequal participations and influence, generate more unique ideas, and have lessmember dominance than face-to-face teams (Rains, 2005). However, virtual teamstend to struggle with intra-team processes (Cramton, 2001; Driskell et al., 2003;Thompson and Coovert, 2003), and recent organizational studies have shown thatdistributed work teams are assigned some of the most important tasks in innovative,knowledge-intensive organizations including making decisions and solvingproblems (Mazneviskia and Chuboda, 2000; Leinonen et al., 2005).

In summary, previous research suggests that virtual teams may have difficultyinteracting as effectively as face-to-face teams. These difficulties may be overcomeif the ICT being used could create a media-rich environment that mimics a realisticface-to-face environment. VW technologies offer virtual teams an entirely new wayof working together by providing an environment that has the potential for richerand more engaging collaboration than has been possible with traditionaltechnologies, leading to more efficient processes and better outcomes.

1.2. Virtual World Technology

VWs, also known as metaverses, are three-dimensional communities that mimicthe real world without its physical limitations. They reinvent the ideas of “beingtogether” and awareness for distributed teams. Within virtual worlds, globallydispersed individuals can create avatars and work together in virtual teams as if theywere operating in a face-to-face situation. VW avatars provide team members with alevel of presence and realism that that is not possible with other ICTs. Avatarsrepresent the embodiment of the user in a VW which allows for a greater sense ofcontrol within the immersive environment, allowing users to more readily engage in

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


events as they were occurring in real life (Gazzard, 2009). The vivid and interactivefeatures of virtual worlds allow users to experience circumstances that mayinfluence their awareness and subsequent decision making quality.

Davis et al. (2009) put forth a model for researching VW with five interactingcomponents (1) the VW itself, (2) people/avatars, (3) VW technology capabilities,(4) behaviors, and (5) outcomes (Figure 1). The circular relationships within the VWand with the outcomes illustrate the on-going social interactions that affect and areintended to represent interlay among these constructs in a non-unidirectionalrelationship (Davis et al., 2009).

Figure 1. Conceptual Model for virtual world adopted from Davis et al. (2009)

The VW is an environment that is capable of supporting effective teaminteraction (Schroeder et al., 2006). People are the team members who arerepresented in the VW by their avatar appearance and behavior. An avatar is a user-created digital representation that symbolizes the team member’s presence in theVW (Bailenson et al., 2005). People use avatars to represent themselves in the VWand to interact with their team members. Previous research suggests that theappearance and interaction of avatars in a realistic VW environment can affectindividual’s sense of presence (Biocca et al., 2003; Blascovich, 2002; Lombard andDitton, 1997).

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


We provide definitions needed for the paper. VW technology includes a set ofcapabilities for communicating, rendering life-like images, interactions with otheravatars and the environment, and team process such as decision making (Davis etal., 2009). Behavior is manifested through the interaction and communications ofavatars (Davis et al., 2009). Prior research has shown that virtual behaviors canaffect team collaboration, performance, and outcomes (Peters and Manz, 2007;Zigurs, 2003). Coordination is defined as the means through which the teammembers and technology work together to carry out activities in order to accomplishthe stated goal (Khazanchi and Zigurs, 2005). Trust is defined as having confidencein other’s motives and willingness to act on the basis of words, actions, or decisionsof others (McAllister, 1995). Role clarity refers to individuals having anunderstanding of the roles within the team. Shared understanding is the mutualknowledge, beliefs, and assumptions that team members develop during ongoingcommunication (Khazanchi and Zigurs, 2005) and a common understanding of thestrategic direction of the team (Liedtka, 1996).

Outcomes can include both task and team-related outcomes. Hackman (1983)suggested that team effectiveness is a function of multiple tasks, the capability of theteam members to work together, and individual growth. Time-Interaction-Performance theory takes a multi-dimensional approach by emphasizing taskperformance, interactions among team members, and team member support(McGrath, 1991).

Figure 2. Inside the sellinger business building in Second Life

For the purpose of this study, we focused on how the VW technologycapabilities in Second Life impact virtual team decision quality based on decisionmaking time and decision accuracy. To provide a sense of place, Loyola UniversityMaryland’s Second Life Campus used in this study is shown in Figures 2 and 3.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Since VW technology provides the illusion that people are in the same space byremoving physical boundaries and separation among avatars, and allows for bothverbal and non-verbal rendering through the technology and controlled by theavatar, the use of VW technology offers a feeling of presence that naturally allowsfor more complex interactions as well as increased interactivity and achieving agood level of decision quality. A key theme of presence is the idea that realisticinteractions with responsive representation in contextually accurate settings couldenhance the quality of interaction (Blascovich et al., 2002). Previous researchsuggests that the appearance of avatars in the virtual world coupled with theirinteractions, can affect a person’s sense of presence and decision making (Biocca etal., 2003; Blascovich, 2002; Lombard and Ditton, 1997). This increase in presenceof the user is due to their avatar living in the virtual world. Therefore, we put forththe following hypotheses.

H1: Virtual teams operating in a virtual world and face-to-face teams will nothave a significant difference in the process of decision making.

H2: Virtual teams operating in a virtual world and face-to-face teams will nothave a significant difference in the decision accuracy achieved.

H3: Virtual teams operating in a virtual world and face-to-face teams will nothave a significant difference in decision quality achieved.

Figure 3. Outside on the Loyola University’s grounds in Second Life

2. Assessing decision quality

The process of decision making involves the procedures and steps that a decisionmaker utilizes in making a decision. Table 1 displays four well-known decision

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


making models. Groups and teams use specific processes such as collaborationwithin each of the phases in order to arrive at a decision.

Table 1. Comparison of decision making process models

Simon(1977)

Boyd(1950)

Turban and Aronson(1998)

Forgionne(1999)

Intelligence

Observe– Observe unfoldingcircumstances; Gatherexternal information

Intelligence– Organizationalobjectives; Datacollection; Problemidentification, ownership,classification

Identifyingobjectives;Recognizing theproblem

Design

Orient– Perceiveopportunities andthreats; Focusthinking on particulardirection

Design–Formulate model; Setcriteria for choice; Searchfor alternatives; Predictand measure outcomes

Gathering data;Generatingalternatives;Establishing criteria;Evaluatingalternatives

Choice

Decide– Make a decision;Choose; Evaluate;Perform sensitivityanalysis

Choice– Solution to problem;Sensitivity analysis;Selection of alternatives;Plan for implementation

Making a choice

Imple-mentation

Act– Act on the decision

Implementation– Implement the decision

Implementing thechoice

LearningSynthesizing

All of the models describe similar phases in the decision making process. Thedecision making phases generally proceed sequentially, with iteration and feedbackloops as more is known about the problem. Sometimes the phases are so tightlylinked that they are difficult to separate. Simon’s (1977) model was based onmanagerial decision making, and it is probably the best known model.Implementation was added later as a fourth phase. Boyd’s (1950) model wasdeveloped for real-time events that change quickly, and Turban and Aronson’s(1998) description is close to that of Simon. Forgionne (1999) separated outLearning as a distinct phase, although some researchers argue that learning isembedded in Simon’s model.

All of the models in Table 1 describe a process culminating in a decision. Thus,we have included both the process of, and outcome from, decision making in our

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


evaluation of decision making. A review of the literature shows that researchersinclude both process and outcome metrics to evaluate decision making and decisionsupport systems (Phillips-Wren et al., 2009). Although outcome is perhaps easier toquantify, process metrics could be, for example, faster decision making or betterdata during the intelligence phase. Outcomes from decision making depend on thespecific decision problem, and examples of outcome metrics are accuracy or reducedcost. A generalized multiple-criteria model to determine decision quality is shown inFigure 4 and incorporates Simon’s phases (intelligence, design, choice,implementation and learning) as well as outcomes from decision making.

Figure 4. Generalized multiple-criteria model for decision quality (based onPhillips-Wren et al., 2009)

The model shown in Figure 4 can be evaluated with the Analytic HierarchyProcess (AHP). AHP is a multiple-criteria method (Harker, 1988) that canincorporate both qualitative and quantitative criteria into a single metric (Saaty,1977, 1994). AHP provides a logical and scientific basis for multiple-criteriadecision making and has been widely applied to both individual and group decisionmaking scenarios since the early 1980s (Wind and Saaty, 1980; Saaty and Vargas,1994).

In order to evaluate the model using the AHP, pair wise comparisons are madebetween alternatives on each element at the lowest level of the hierarchy. The AHPmodel then computes the decision quality by combining the criteria in the modelsuch that the sum of the relative rankings of the alternatives at each step in the AHPis one. In the research in this paper, we will compare virtual teams in a virtual world(VT) to face-to-face (FF) teams. The relative comparison of VT to FF will be aninput for each element at the lowest level of the hierarchy. The AHP will thencompute the ratio VT/FF for Process and for Outcome. At the top of the hierarchy,the AHP will provide the ratio VT/FF. We posit that there is no difference inDecision Quality between VT operating in a virtual world and FF teams. We willpresent quantitative metrics from the empirical experiment that will provide theinput for the initial comparison.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


3. Research methodology

The focus of our study is the decision making process and team decisionoutcomes for teams working face-to-face and in Second Life. This design led to theidentification of 55 subjects from an undergraduate information technology businesscourse. To begin this study, one class session (75 minutes) was devoted to SecondLife lab time. During this time, the subjects were introduced to Second Life andcreated their avatars. Once everyone had successfully created their avatars, part ofclass was conducted in Second Life. This time was spent visiting businesses,government entities, and other universities in Second Life. The goal was tointroduce the subjects to the real world uses of Second Life and to help them gainexperience using the interface, gestures, and verbal communication capabilities.

Next, the first of two Second Life assignments were assigned. This first requiredthe subjects to photograph themselves while visiting several different islands inSecond Life. During the next class period, we reflected on their experiences,discussed technology difficulties, and stressed the importance of using voice andgestures while operating in Second Life. The second assignment required subjects topractice using virtual voice and gesture commands. The deliverable includedphotographs of them communicating with others while using gestures. The focus ofthis assignment was to give the subjects time to acclimate themselves tocommunicating through their avatars and to stress the importance of verbal and non-verbal communication. We concluded this assignment by discussing the experiencesin class and addressing technology difficulties that the students encountered.

The team decision making exercises picked for the virtual team decision makingstudy included two different team survival exercises (a survival at sea scenario and asurvival in space scenario) with a known optimal solution. The subjects were given asurvival scenario and were instructed to rank in order of importance 15 items with 1being the most important item to their survival and 15 being the least importance totheir survival. Teams were randomly formed and operated in a face-to-faceenvironment and in a virtual world environment (headsets or microphones wererequired) in Loyola University Maryland’s Second Life campus.

Different teams were formed to work together in a face-to-face environment andin a virtual world environment. To avoid issues with team familiarity, differentteams were randomly formed for the face-to-face exercise and for the virtual worldexercise. To help ensure validity and to reduce the possibility of order effects, theorder of the team exercises was randomly assigned, i.e. half of the teams performedthe first team exercise in Second Life and then the second team exercise face-to-face. The other half of the teams performed the first team exercise face-to-face andthen the second team exercise in Second Life. This research design gave teamperformance results in Second Life and in real life. An outline of the execution forthe experiment is located in Table 1. A total of 55 subjects were put into 18 teams of3 or 4 team members. The research design is fairly consistent with much of the

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


research reported by Fjermstad and Hiltz (1999) regarding number of groups,subjects per group, and total number of subjects.

Table 2. Execution of experiment

Class 1– Introduction to Second life– Create Avatars– Conduct part of class in Second Life

Assignment 1

– Conduct the Second Life tutorials on Welcome Island– Visit several different Second Life Island– Submit pictures from the tutorial and from the islands visited– Submit a brief write up describing your experience and thoughts on

using Second Life

Class 2

– Discuss Second Life experiences and perceptions– Identify and discuss technology problems– Stress the importance and requirement of verbal (voice) and non-

verbal (gestures) while communication in Second Life

Assignment 2

– Practice communicating in Second Life– Submit photos of your avatar communicating verbally and nonverbally in Second Life with other avatars– Submit a brief write up discussing your experiences and perceptions

Class 3 – Discuss their experiences and perceptions– Identify and discuss technology problems

Assignment 4 – Assign the face-to-face and the virtual world team experiments

The data collection instrument (see Appendix) for this study consisted of teamstracking their decision making process times as well as a structured survey andunstructured open ended questions based on previous studies to obtain informationfrom the students about their experience of working in a virtual team in Second Lifeand in a face-to-face environment. The survey questions were based on a 5 pointLikert scale with 1 very easy/ not challenging and 5 being very hard/verychallenging. In addition, decision accuracy was calculated on a team basis to gain afuller picture of their experiences including the team’s performance. Decisionaccuracy was calculated by comparing the item ranked by the individual and theteam to the actual ranking of the item. Therefore, the higher the calculated number,the less accurate the decision was.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


4. Results

Teams recorded the time it took for various decision-making tasks and submittedtheir final decisions. In addition, they responded to qualitative questions designed tocapture their opinions of decision making in the two environments. The meanamount of time for the decision-making tasks for virtual teams and face-to-faceteams is shown in Table 3.

Table 3. Mean times for virtual teams and face-to-face teams

Mean time (min.) to: Virtual team Face-to-face team

Understand the problem 1.76 1.24

Generate alternatives and makechoice 12.97 10.92

Decision making process total 14.87 12.57

Since the rankings of the various items in each scenario have a known solution,the difference between the correct answer as judged by location in the 1-15 list ofitems and the team decision can be calculated. The mean differences are shown inTable 4.

Table 4. Mean difference (-) from correct decision for virtual teams and face-to-face teams

Virtual Team Face-to-Face Team

47.5 57.9

Hypotheses H1, H2 and H3 can be tested using the model in Figure 2 and theresults in Tables 2 and 3. In our decision problem, the Intelligence phase componentof Process is measured by the time to complete this stage. Subjects discussedalternatives and determined final ranking of items with an iterative approach as theyprogressed from Item 1 to Item 2, etc. Thus, we combined the Design and Choicephases in Figure 3. We did not measure Implementation or Learning. The Outcomecomponent of Decision Quality in Figure 3 is measured by the accuracy of the finaldecision as determined by the mean difference between the team decision and thecorrect decision as shown in Table 3.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


5. Analysis of results

Our hypotheses H1, H2 and H3 are that virtual teams operating in a virtual worldand face-to-face teams will not have a significant difference in the process ofdecision making, the outcome from decision making, and decision quality. Wetested the hypotheses two ways: (1) H1 and H2 were tested with a two sample t-testwith a level of significance of 0.05, and (2) H3 was tested with a multi-criteriamodel implemented with the analytic hierarchy process since we considered bothoutcome and process to be components of decision quality. We used group-levelaggregate scores for the variables in our statistical analyses. The use of groups as theunit of analysis is consistent with prior research (Gallupe and DeSanctis, 1988;Valacich et al., 1992).

H1 proposes that the process of decision making would not be significantlydifferent in the two environments. The metric used for comparison was the meantime the teams spent making their decisions, that is, does it take longer to make adecision in one environment compared to the other. The t-test results are shown inTable 4 and reveal that there is no significant difference in the amount of time tomake a decision for virtual teams in a virtual world and face-to-face teams at the0.05 level of significance. The results support H1.

Table 5. t-test results for mean difference in time to make a decision (process ofdecision making) for virtual teams in a virtual world and face-to-face teams

Mean St. Dev.

Face -to-face teams 12.57 5.9

Virtual world teams 14.87 6.72

95% CI (-5.21, 0.61)

p-value = 0.119

H2 proposes that the outcome of decision making would not be significantlydifferent in the two environments. The metric used to compare the two groups wasthe mean difference in the ranking of items in the decision problem. The t-testresults are shown in Table 6 and reveal that there is no significant difference indecision accuracy for virtual teams in a virtual world and face-to-face teams at the0.05 level of significance. The results support H2.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Table 6. t-test results for mean difference from correct decision (outcome ofdecision making) for virtual teams in a virtual world and face-to-face teams

Mean St. Dev.

Face -to-face teams 57.9 15.9

Virtual world teams 47.46 9.71

95% CI (-5.21, 5.86)

p-value = 0.907

Figure 5. Comparison of FF to VT by overall decision quality

H3 proposes that the overall decision quality would not be significantly differentin the two environments. We used the multi-criteria model shown in Figure 4 andevaluated it using the analytic hierarchy process with equal weightings at all levels.At the bottom of the hierarchy the two alternatives (face-to-face and virtual teams)are compared on each criterion such that the sum is equal to one. The time torecognize the problem and the time to generate alternatives/make a choice providedthe relative comparisons under Process, with shorter time taken as indicative of abetter process. The mean difference from the correct decision was used to comparethe alternatives under Outcome Value. The AHP then computes higher levelcomparisons, with the Decision Quality computed as 0.5009/0.4991 for face-to-faceversus virtual teams where we have equally weighted each criterion. Although theAHP does not provide a statistical comparison, the values are close, with a slight

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


advantage to face-to-face teams. Interestingly, we can see that virtual teams havebetter decision accuracy or “Outcome”, while face-to-face teams show better“Process” of decision making. The results suggest support for H3, i.e. the there is nodifference in decision quality by face-to-face teams compared to teams operating ina virtual world.

6. Discussion

In this research, we compared the decision quality of teams operating in face-to-face dialogue with those communicating through a VW, Second Life. We found thatthere was no significant difference in the process of, and outcome from, decisionmaking. The decision quality was considered to show no difference for the twoenvironments. These results show promise for virtual team decision making. Theresults indicate that the additional presence achieved through virtual worldtechnology may help overcome some of the limitations that have been previouslyreported in the literature with the use of virtual teams.

Table 6. Strengths of decision making for virtual teams in a virtual world asreported by participants

Strengths

We were able to agree on the rankings in a timely matter.

In typical team projects, we have a leader and 2 minions. However in the virtual world,we were all equal.

Co-operation between our team members was a big strength, because we all listened toeach other and agreed together.

Being able to work from home.

It was efficient and saved time.

Being able to share our ideas easily and efficiently.

It was easy to roll out of bed and go to work.

We were able to share ideas and collaborate.

It felt like we were all standing together.

It made the decision process easier.

Using the voice part really increase my feeling of being together.

We all felt a significant level of presence that contributed us being able to reach consensuseasily.

We had great verbal communication

It was easy to express our ideas and come to a consensus.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Qualitative information was obtained from the experiment by solicitingcomments about the experience of decision making in a virtual team in Second Life.A summary of the strengths listed are located in Table 6, and a summary of theweaknesses are located in Table 7.

Table 7. Weaknesses of decision making for virtual teams in a virtual world asreported by participants

Weaknesses

As much as I was expecting weaknesses of working as a team to complete this exercisein the virtual world, there weren’t any. Everyone was comfortable in Second Life. I amcomfortable enough with the online world, that I didn’t see a huge difference between thetwo assignments.

We experienced some problems with the sound.

A little confusing communicating at first.

It took longer than when worked face-to-face.

I don’t think there were any weaknesses.

Using the sound was difficult at times due to feedback.

It was annoying when we started working together in Second Life.

We experienced some technical difficulties at first.

Getting used to using Second Life as a team was a bit confusing at first.

There were no clear weaknesses to our team.

It was a little awkward because we could not actually see each other.

It was a little hard to read people.

It was a little difficult to organize the discussion.

Our results provided some interesting findings. First even though there are morestrengths listed than weaknesses, several of the participants reported difficultiesgetting started as a team in Second Life. Participants reported technical problemssuch as sound and discomfort of working as a team in Second Life even withtraining in the virtual world. Post-experiment comments were consistent with thet-test results of how hard it was to discuss the problem with their team (shown inTable 8), and how hard was it to discuss the most important item with their team(shown in Table 9). Virtual teams had significantly more difficulty than face-to-faceteams on both items.

The results obtained from the t-tests comparing the participant’s perceptionsbetween face-to-face team work and virtual team work showed that that there was asignificant difference perceived in the difficulty level during the beginning stages ofthe team decision making. However, the t-test comparisons between the face-to-face

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


teams and virtual teams show no significant difference in the level of difficultywhen making their team decisions. Table 10 and Table 11 display our results.

Table 8. t-test of difficulty in discussing the decision problem

How hard was it to discuss the problem with your team?

Mean St. Dev.

Face-to-face 1.84 1.19

Virtual World 2.76 0.971 95% CI (-1.43, -0.42)

p-value = 0.000

Table 9. t-test of difficulty in determining the most important item in the decisionproblem

How hard was it to discuss the most important item?

Mean St. Dev.


Virtual World 2.74 0.98

95% CI (-1.40, -0.34)

p-value = 0.002

Table 10. t-test of difficulty in discussing the rankings of the items in the decisionproblem

How hard was it to discuss the rankings with your team?

Mean St. Dev.


Virtual World 2.79 0.905 95% CI (-0.79, 0.18)

p-value = 0.215

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Table 11. t-test of difficulty in agreeing on the rankings of the items in the decisionproblem

How hard was it to agree on the rankings with your team?

Mean St. Dev.


Virtual World 2.4 0.89

95% CI (-0.71, 0.13)

p-value = 0.178

Coupling the qualitative comments and the t-test results of difficulty providesinsight into how the participants perceived team decision making in Second Life.Although individuals perceived difficulty in initial communication, they overcamethese issues and worked together productively. This observation is an indicator ofthe importance of training employees in using virtual technology before initiatingvirtual teams in a virtual world.

Some of the participants reported that it took more time to make a decision in theVW than when meeting face-to-face, and the results suggest that this is true,although they are not significantly different. Again, these comments emphasize theimportance of training and education with virtual worlds before companies utilizethe technology for virtual teams.

6.1. Implication for practitioners

Our results indicate that virtual teams can operate in a VW and produce decisionquality equivalent to face-to-face teams. However, to successfully implement avirtual team using VW technology, practitioners should consider the time thatneeded to get the team comfortable with the environment. Our results indicate thatvirtual teams need somewhat more time in the beginning to learn to work togetherdue, in our case, to technical and social issues. Although these issues were overcomequickly, this extra time should be taken into consideration. In addition, in order forthe team project to be successful, it is necessary to clearly set out the expectationsand require participants to script the steps that are needed to complete the project.Since studies have indicated that the productivity of teams is threatened by a lack ofcommon ground and lack of awareness among team members (Beers et al., 2005;Makitalo et al., 2005; Dillenbourg, 2002; Barab et al., 2004), scripting steps tofollow before beginning the project is expected to enhance a virtual team’sperformance in a virtual world environment. For example, when group membersknow more about their working processes, they can plan their team work and

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


individual work more effectively. Rummel et al. (2003), in an investigation of howcollaboration scripts affect medical and psychology students, showed that thosedyads of students who had followed process script during the shared workingprocesses produced significantly better results than the dyads in a non-scriptedcondition. Given the newness of VW environments, scripting the processes thatvirtual team members should follow is expected to improve their efficiency and theiroutcomes. In addition, knowledge and learning in one virtual environment isexpected to transfer to other virtual teaming arrangements.

6.2. Limitations and future research

Limitations do exist in our study. First, even though our sample size of 55 islarge enough to draw statistical conclusions, future studies should be designed toincorporate larger sample sizes. Secondly, all of our participants are from the sameorganization and some of them may know each other so that trust and other socialinteractions could have been established before the experiment began. Results maydiffer with teams of people who have not met or are from different cultures. Sincemany virtual teams are comprised of individuals who have never met face-to-face,future research should include virtual team work with subjects who have never metface-to-face. Our study is also limited in that the team decision was relatively simpleand performed in a synchronous mode. Many virtual team projects operate in bothsynchronous and asynchronous modes. Future research should investigate virtualteams operating in a VW that are complicated enough to utilize both synchronousand asynchronous communication modes. In addition, future research shouldinvestigate other possible outcome measures such as user satisfaction. We did notdirectly test for satisfaction; however, qualitative comments from participantssuggest that this could have impacted our results.

7. Summary and contribution

Virtual worlds are interesting new technologies that permit teams tocommunicate and meet across physical boundaries more realistically than moretraditional methods such as phone or video conferencing. The virtual environmentcan be made to emulate a real location, and participants can create avatars that looklike themselves and use their own voice. As virtual teams become more common inbusiness settings, practitioners can utilize these results to increase the effectivenessand efficiency of these teams.

This study has contributed to the literature by assessing the impact of a virtualworld on team decision making compared to face-to-face teams. The study hasprovided quantitative and qualitative data that suggest that virtual teams operating ina virtual world can achieve the same level of process, outcome, and decision qualityas teams operating face-to-face. Our results provide support for the view that

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


organizations which properly train and educate their employees could utilize virtualteams without negatively impacting the decision making process or the quality of theoutcome reached.

Acknowledgements

The authors would like to thank Salvatore Lenzo and Michael Jaworski inTechnology Services at Loyola University Maryland for their work in developingthe Second Life campus used in this study. We also express appreciation toProfessor Daniel Power helping us learn the capabilities available in Second Life.

8. References

Anderson F., Shane H., “The Impact of Netcentricity on Virtual Teams: The NewPerformance Challenge”, Team Performance Management, Vol. 8, No. 1-2, 2002, p. 5-7.

Bailenson J.N., Swinth K., Hoyt C., Persky S. et al., “The independent and interactive effectsof embodied-agent appearance and behavior on self-report, cognitive, and behavioralmarkers of copresence in immersive virtual environments”, Presence: Teleoperators &Virtual Environments, Vol. 14, No. 4, 2005, p. 379-393.

Bainbridge W.S., “The scientific research potential of virtual worlds”, Science Magazine,Vol. 317, 2007, p. 472-476.

Barab S., MaKinster S., Scheckler R., “Designing System Dualities: Characterizing an OnlineProfessional Development Community”, Designing for Virtual Communities in theService Learning, Barab S., Kling R., and Gray J. (eds.), New York, CambridgeUniversity Press, 2004, p. 307-359.

Barron B., “When Smart Groups Fail”, The Journal of The Learning Sciences, Vol. 12, No. 3,2003, p. 307-359.

Beers P., Boshuizen H., Kirschner P., Gijselaers W., “Computer Support for KnolwedgeConstruction in Collaborative Learning Environments”, Computers in Human Behavior,Vol. 21, No. 4, 2005, p. 623-643.

Bell M.A., Virtual Hybrid Workgroups Are Critical to Successful Offshore Sourcing,Stamford, CT, Gartner, 2005.

Biocca F., Harris C., Burgoon J.K., “Toward a more robust theory and measure of socialpresence: Review and suggested criteria”, Presence, Vol. 12, No. 5, 2003, p. 456-480.

Blascovich J., “A theoretical model of social influence for increasing the utility ofcollaborative virtual environments”, 4th International Conference on CollaborativeVirtual Environments, Bonn, Germany, 2002, p. 5-20.

Blascovich J., Loomis J., Beall C., Swinth K.R., “Immersive virtual environment technologyas a methodological tool for social psychology”, Psychological Inquiry, Vol. 13, No. 2,2002, p. 103-124.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Bordia P., “Face-to-face versus Computermediated Communication: A Synthesis of theExperimental Literature”, The Journal of Business Communication, Vol. 34, No. 1, 1997,p. 99-120.

Boulos M.N., Hetherington L., Wheeler S., “Second Life: An overview of the potential of 3-Dvirtual worlds in medical and health education”, Health Information and LibrariesJournal, Vol. 24, No. 4, 2007, p. 233-245.

Boyd J., “A Discourse on Winning and Losing”, unpublished collection of lecture slides. Afew copies are available via Interlibrary Loan from such sources as the Marine CorpsUniversity Library, the depository for Colonel Boyd’s papers, 1950.

Brennan S., “The Grounding Problem in Conversation with and through Computers”, Socialand Cognitive Approaches to Interpersonal Communication, Fussel S.R. and Kreuz R.J.(eds.), Mahwah, NJ, Erlbaum, 1998, p. 201-225.

Bromme R., “Beyond One’s Own Perspective”, Practicing Interdisciplinary, Weingart P. andStehr N. (eds.), Toronto, Canada, University of Toronto Press, 2000, p. 115-133.

Burke K., Chidambaram L., “Do Mediated Contexts Differ in Information Richness? AComparison of Collocated and Dispersed Meetings”, Proceedings of the Twenty-NinthAnnual Hawaii International Conference on System Sciences, Hawaii, 1996, p. 92-101.

Chase N., “Learning to Lead a Virtual Team”, Quality, Vol. 38, 1999, p. 76-78.

Chidambaram L., “Relational Development in Computer-supported Groups”, MIS Quarterly,Vol. 20, No. 2, 1996, p. 143-163.

Chidambaram L., Bostrom R., “Evolution of Group Performance Over Time: A RepeatedMeasures Study of GDSS Effects”, Journal of Organizational Computing, Vol. 3, No. 4,1993, p. 443-469.

Cooke N.J., Kiekel P.A., Helm E., “Measuring team knowledge during skill acquisition of acomplex task”, International Journal of Cognitive Ergonomics: Special Section onKnowledge Acquisition, Vol. 5, 2001, p. 297-315.

Cousins K.C., Robey D., “Human agency in a wireless world: Patterns of technology use in nomadiccomputing environments”, Information and Organization, Vol. 15, 2005, p. 1151-1180.

Cramton C.D., “Finding common ground in dispersed collaboration”, OrganizationalDynamics, Vol. 30, 2002, p. 356-367.

Davis A., Murphy J., Owens D., Khazanchi D., Zigurs I., “Avatars, People, and VirtualWorlds: Foundations for Research in Metaverses”, Journal of the Association forInformation Systems, Vol. 10, No. 2, 2009, p. 91-117.

DeMeyer A., “Tech Talk: How Managers are Stimulating Global R&D Communication”,Sloan Management Review, Vol. 32, 1991, p. 49-59.

Dillenbourg P., “Over-Scripting CSCL: The Risks of Blending Collaborative Learning withInstructional Design”, Three Worlds of CSCL Can We Support CSCL, P.A. Kirschner(ed.), Heerlen, Open University Nederland, 2002, p. 61-91.

Driskell J.E., Radtke P.H., Salas E., “Virtual teams: Effects of technological mediation onteam performance”, Group Dynamics, Vol. 7. 2003, p. 297-323.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Dube L., Pare L., “The Multi-faceted Nature of Virtual Teams”, Virtual teams: Projects,protocols, and processes, D.J. Pauleen (Ed.), Idea Group Publishing, 2004.

Fjermestad J. and Hiltz S., “An assessment of group support systems experimental research:methodology and results”, Journal of Management Information Systems, Vol. 15, No. 3,1999, p. 7-149.

Flor N., Hutchins E., “Analyzing Distributed Cognition in Software Teams: A Case Study ofTeam Programming during Adaptive Software Maintenance”, Reading in Groupware andComputer Supported Cooperative Work, R. Baecker (Ed.), San Mateo, CA, Morgan-Kaufman, 1991.

Forgionne G., “An AHP model of DSS Effectiveness”, European Journal of InformationSystems, 1999, p. 95-106.

Furst S.A., Reeves M., Rosen B., Blackburn R.S., “Managing the life cycle of virtual teams”,Academy of Management Executive, Vol. 182, No. 2, 2004, p. 6-20.

Galegher J., Kraut R.E., “Computermediated Communication for Intellectual Teamwork: AnExperiment in Group Writing”, Information Systems Research, Vol. 5, No. 2, 1994,p. 110-138.

Gallupe R.B., DeSanctis G., “Computer-based support for group problem-finding: AnExperimental Investigation”, MIS Quarterly, Vol. 12, No. 2, 1988, p. 277-296.

Gazzard A., “The Avatar and the Player: Understanding the Relationship Beyond the Screen”,Conference in Games and Virtual Worlds for Serious Applications (VS-Games), IEEEPiscataway, NJ Coventry, UK, 2009, p. 190-236.

Gross T., Prinz W., “Awareness in Context: A Light-Weight Approach”, Proceedings of theEight European Conference on Computer Supported Cooperative Work, ECSCW’03,Kuutii K., Karsten E., Fizpatrick G., Dourish P., and Schmidt K. (Eds.), 14-18 September2003, Helsinki, Finland, 2003, p. 295-314.

Hackman J.D., A Normative Model of Work Team Effectiveness, Office of Naval Research, 1983.

Hacker P., “The Art and Science of Decision Making: The Analytic Hierarchy Process”,Working Paper 88-06-03, Decision Science Department, The Wharton School, Universityof Pennsylvania, Philadelphia, PA, 1988.

Hardin C., and Higgins E., “Shared Reality: How Social Verification Makes the SubjectiveObjective”, Handbook of Motivation and Cognition: Foundation of Social Behaviour,R.M. Sorrentino and E.T. Higgins (Eds.), New York, Guilford, 1996, p. 28-84.

Herman S., “Counterpoints: Notes on OD for the 21st century – part II”, OrganizationalDevelopment Journal, Vol. 19, No. 1, 2001, p. 115-118.

Hewitt J., Scardamalia, “Design Principles for Distributed Knowledge Building Processes”,Educational Psychology Review, Vol. 10, No. 1, 1998, p. 75-96.

Higgins E., “Social Cognition: Learning about What Matters in the Social World”, EuropeanJournal of Social Psychology, Vol. 30, 2000, p. 3-39.

Jin L., Robey D., “Bridging social and technical interfaces in organizations: Aninterpretive analysis of time-space distanciation”, Information and Organization,Vol. 18, No. 3, 2008, p. 177-204.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Kanawattanachai P., Yoo Y., “Dynamic nature of trust in virtual teams”, Journal of StrategicInformation Systems, Vol. 11, 2002, p. 187-213.

Kerr N., Bruun S., “Dispensability of Member Effort and Group Motivation Losses: FreeRider Effects”, Journal of Personality and Social Psychology, Vol. 44, 1983, p. 78-94.

Kezsbom D., “Creating Teamwork in Virtual Teams”, Cost Manufacturing, Vol. 42, No. 10,2001, p. 33-37.

Khazanchi D. and Zigurs I., Patterns of effective management of virtual projects: Anexploratory study, Newton Square, PA, Project Management Institute, 2005.

Leinonen P., Jarvela S., Hakkinen P., “Conceptualizing the Awareness of Collaboration: AQualitative Study of a Global Virtual Team”, Computer Supported Cooperative Work,Vol. 14, 2005, p. 301-322.

Leinonen P., Jarvela S., Lipponen L., “Individual Students’ Interpretations of theirContribution to the Computer-mediated Discussions”, Journal of Interactive Learning,Vol. 14, No. 1. 2003, p. 99-122.

Liedtka J.M., “Collaborating across lines of business for competitive advantage”, Academy ofManagement Executive, Vol. 10, No. 2. 1996, p. 20-34.

Lombard M., Ditton T., “At the heart of it all: The concept of presence”, Journal ofComputer-Mediated Communication, Vol. 3, No. 2, 1997.

Majchrzak A., Malhotra A., Deploying Far Flung Teams: A Guidebook for Managers,Chicago, IL, Society for Information Management, 2003.

Makitalo K., Weinberger A., Hakkinen P., Jarvela S., Fischer F., “Epistemic CooperationScripts in Online Learning Environments: Fostering Learning by Reducing Uncertainty inDiscourse?”, Computers in Human Behavior, Vol. 21, No. 4, 2005, p. 603-622.

Maznevski M., Chudoba K., “Bridging Space Over Time: Global Virtual-team Dynamics andEffectiveness”, Organization Science, Vol. 11, No. 2, 2000, p. 473-492.

McAllister D.J., “Affect- and cognition-based trust as foundations for interpersonal cooperation inorganizations”, Academy of Management Journal, Vol. 38, No. 1, 1995, p. 24-59.

McDonough E., Kahn K., Barczak G., “An Investigation of the Use of Global, Virtual, andCollocated New Product Development Teams”, The Journal of Product InnovationManagement, Vol. 18, No. 2, 2001, p. 110-120.

McGrath J.E., “Time, interaction, and performance (TIP): A theory of groups”, Small GroupResearch, Vol. 22, No. 2, 1991, p. 147-174.

McNamara K., Dennis A.R., Carte T.A., “It’s the Thought that Counts: The mediating effectsof information processing in virtual team decision making”, Information SystemsManagement, Vol. 25, 2008, p. 20-32.

Mehrabian A., Silent messages, Belmont, CA, Wadsworth, 1971.

Mennecke B.E., Triplett J.L., Hassall L., Jordan-Conde Z., “Embodied social presencetheory”, Proceedings of the 43th Hawaiian International Conference on System Sciences,Koloa, HI, ACM Press, 2010, p. 1-10.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Paul S., Samarah I., Seetharaman P., Mykytyn P., “An Empirical Investigation ofCollaborative Conflict Management Style in Group Support System-Based Global VirtualTeams”, Journal of Management Information Systems, Vol. 21, No. 3, 2004, p. 185-222.

Peters L.M., Manz C.C., “Identifying antecedents of virtual team collaboration”, TeamPerformance Management, Vol. 13, No. 3/4, 2007, p. 117-129.

Phillips-Wren G., Mora M., Forgionne G., Gupta J., “An integrative evaluation framework forintelligent decision support systems”, European Journal of Operational Research, Vol.195, No. 3, June 16, 2009, p. 642-652.

Pinsonneault A., Caya O., “Virtual Teams: What We Know, What We Don’t Know”,International Journal of e-Collaboration, Vol. 3, No. 1, 2005, p. 1-16.

Prasolova-Forland E., Sourin A., Sourina O., “Place Metaphors in Educational Cyberworlds:A Virtual Campus Case Study”, Proceedings 2005 International Conference onCyberworlds, CW’05, 2005, p. 221-228.

Prince C., Stout R., Salas R., Team situational awareness preliminary lessons learned:Identification of skills, behaviours, design guidelines, and training strategies frominterviews and research, Technical Report, FAA and Naval Air Warfare Center, TrainingSystems Division, 1994.

Powell A., Piccoli G., Ives B., “Virtual teams: team control structure, work processes, and teameffectiveness”, Information Technology and People, Vol. 17, No. 4, 2004, p. 359-379.

Rains S., “Leveling the Organizational Playing Field--Virtually: A Meta-Analysis ofExperimental Research Assessing the Impact of Group Support System Use on MemberInfluence Behaviours”, Communication Research, Vol. 32, No. 2, 2005, p. 193-234.

Rockett L., Valor J., Miller P., Naude P., “Technology and Virtual Teams: Using GloballyDistributed Groups in MBA Learning”, Campus-Wide Information Systems, Vol. 15,No. 5, 1998, p. 174-182.

Robey D., Schwaig K.S., Jin L., “Intertwining material and virtual work”, Information andOrganization, Vol. 13, 2003, p. 111-129.

Rummel N., Spada F., Casper F., Ophoff J., Schornstein K., “Instructional Support forComputer-Mediated Collaboration”, “Designing for Change in Networked LearningEnvironments”, B. Wasson, S. Ludvigsen and U. Hoppe (eds.), Proceedings of theInternational Conference on Computer Support for Collaborative Learning 2003,Dordecht, Kluwer Academic Publishers, 2003, p. 199-208.

Saaty T.L., “A Scaling Method for priorities in Hierarchical Structures”, Journal ofMathematical Psychology, 1977, p. 234-281.

Saaty T., Vargas L., Decision Making in Economic, Political, Social and TechnologicalEnvironments with the Analytic Hierarchy Process, RWS Publications, Pittsburgh, PA, 1994.

Saunders C.S., Ahuja M.K., “Are all distributed teams the same? Differentiating betweentemporary and ongoing distributed teams”, Small Group Research, Vol. 37, 2006, p. 662-700.

Schmidt K., “The Problem with “Awareness”, Computer Supported Cooperative Work:Journal of Collaborative Computing, Vol. 11, No. 3-4, 2002, p. 285-298.

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


Schroeder R., Heldal I., Tromp J., “The usability of collaborative virtual environments andmethods for the analysis of interaction”, Presence, Vol. 16, No. 5. 2006, p. 655-667.

Schweitzer L., Duxbury L., “Conceptualizing and measuring the virtuality of teams”,Information Systems Journal, Vol. 20, No. 3, 2010, p. 267-295.

Siebdrat F., Hoegl M., Ernst H., “How to Manage Virtual Teams”, MIT Sloan ManagementReview, Vol. 50, No. 4, 2009, p. 62-69.

Simon H., The new science of management decision, 3rd ed. Englewood Cliffs, NJ, Prentice-Hall, 1977.

Sotto R., “The Virtual organization”, Accounting, Management, and InformationTechnologies, Vol. 7, No. 1, 1997, p. 37-51.

Thompson L.F., Coovert M.D., “Teamwork online: The effects of computer conferencing onperceived confusion, satisfaction, and post discussion accuracy”, Group Dynamics, Vol.7, 2003, p. 135-151.

Turban E., Aronson J., Decision Support Systems and Intelligent Systems, Upper SaddleRiver, NY, A. Simon and Schuster Company, 1998.

Valacich J.S, Dennis A.R., Nunamaker J.F. Jr., “Group Size and Anonymity Effects on Computer-Mediated Idea Generation”, Small Group Research, Vol. 23, No. 1, 1992, p. 49-73.

Walther J., “Relational Aspects of Computermediated Communication: ExperimentalObservations over Time”, Organization Science, Vol. 6, No. 2, 1995, p. 186-203.

Warkentin M.E., Sayeed L., Hightower R., “Virtual teams versus face-to-face teams: Anexploratory study of a Web-based conference system”, Decision Sciences, Vol. 28, No. 4,1997, p. 975-995.

Wilson S., “Forming Virtual Teams”, Quality Progress, Vol. 36, No. 6, 2003, p. 36-41.

Wind Y., Saaty T.L., “Marketing Applications of the Analytic Hierarchy Process”,Management Sciences, Vol. 26, No. 7, 1980, p. 641-658.

Zigurs I., “Leadership in Virtual Teams: Oxymoron or Opportunity?”, OrganizationalDynamics, Vol. 31, No. 4, 2003, p. 339.

Appendix. Data Collection questionnaire

How was the team exercise performed?a. Face-to-faceb. Virtually

What time did you start discussing the problem? Start Time:__________________

What time did you start discussing which items are most important to solve the problem?Start Time:__________________

What time did your team start discussing the #1 ranked item? Start Time:__________________

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4


What time did your team start discussing the #2 ranked item?Start Time:__________________



What time did your team start discussing the #5 ranked item?Start Time:__________________

What time did your team start discussing the items ranked 6-10?Start Time:__________________

What time did your team start discussing the items ranked 11-15?Start Time:__________________

How did your team decided on the rankings?c. Voted. One person decidede. Consensusf. Majority

How hard was it to discuss the problem with your team?1 2 3 4 5Very Easy Easy Very Hard

How hard was it to discuss which items were the most important with your team?1 2 3 4 5Very Easy Easy Very Hard

How hard was it to discuss the item rankings with your team?1 2 3 4 5Very Easy Easy Very Hard

How hard was it for your team to agree on the ranking order? 1 2 3 4 5Very Easy Easy Very Hard

Describe your experience of working as a team to perform a team exercise.

What were the strengths of working as a team for completing this exercise?

What were the weaknesses of working as a team to complete this exercise?

Dow

nloa

ded

by [

UO

V U

nive

rsity

of

Ovi

edo]

at 0

4:19

17

Oct

ober

201

4

assessing decision making quality in face-to-face teams versus virtual...

Documents