Research Design 1

Research Design, Practice, Logistics and Ethics

Edgardo Donovan

RES 603 – Dr. Alan B. Flaschner

Module 1 – Case Analysis

Monday, October 18, 2010

Research Design 2

Research Design, Practice, Logistics and Ethics

Eveland and Bikson utilize an experimental research design in conducting their study

titles “Work Group Structures and Computer Support: A Field Experiment” as they attempted to

scientifically measure the propensity that people provided with technological support would

gravitate towards the use of the latter in a field environment measured over a long period of time.

It is frequently suggested that work groups that have computer technology to support

activities such as text editing, data manipulation, and communication develop systematically

different structures and working processes from groups that rely on more conventional

technologies such as memos, phone calls, and meetings (Eveland 354).

This field experiment created two task forces, each composed equally of recently retired

employees and employees still at work but eligible to retire. They were given the identical tasks

of preparing reports for their company on retirement planning issues, but they were randomly

assigned to different technology conditions. Interviews were conducted four times during the

year-long project; in addition, electronic mail activity was logged in the on-line group (Eveland

354).

Figure 1. (Research Design)

Research Design 3

Eveland’s research can be defined as utilizing a quantitative, field, and experimental

design. We can classify designs into a simple threefold classification by asking some key

questions. First, does the design use random assignment to groups? If random assignment is

used, we call the design a randomized experiment or true experiment. If random assignment is

not used, then we have to ask a second question: Does the design use either multiple groups or

multiple waves of measurement? If the answer is yes, we would label it a quasi-experimental

design (Trochim 2). The main strength of the research design comes from random assignment to

experimental conditions plus control over other potentially interfering variables such as type of

task, type of technology, prior experience with electronic mail, preexisting group structures, and

the like (Eveland 358).

Figure 2. Questions about the Real World (McGrath 181)

The research process can be viewed as a series of interlocking choices, in which we try

simultaneously to maximize several conflicting desiderata. There is not one true method that will

Research Design 4

guarantee success (McGrath 179). A researcher begins by noticing a real world phenomenon and

attempts to create new knowledge by inquiring about its dynamics. The research problem is then

incorporated into a specific design which then in turn gives birth to an operational plan. That

plan may take on different approaches. In most cases the end-result from the execution of that

plan involves acquiring data that proves correlations or associations among a well defined set of

variables that support a new way of understanding. Although there are many different ways to

sequence research, typically the series of choices is locally directional: plan must come before

execution; data collection must come before data analysis (McGrath 180).

Figure 3. The Research Process (Gabaney)

Research Design 5

Eveland probably chose the experimental design based on field gathered data probably

because cross-sectional or retrospective research designs do not allow this hypothesis to be tested

with much power (Eveland 354). A qualitative research design based on the conventional

subjective wisdom of the literary tradition related to the topic at hand would undoubtedly have

the potential of being very interesting for the field. However, it would have been very difficult to

extrapolate universal truths related to how employees utilize technology when given an

opportunity to do so. Another advantage of this type of design was that it was conducted in the

field and not in a lab setting. Further research perhaps in different geographical areas as well as

organizational cultures would be required to compound the validity of Eveland’s results.

Regardless, Eveland credibly demonstrates that groups that are given the resources to utilize new

technology over time gain an edge over similar groups not as well resourced while expending

greater time and effort in learning new ways of doing things. This knowledge is very applicable

in the private sector. Technological savvy acquired after technology tools are resourced

extensively over time will give a company an edge over its competition. However, this comes at

not only a financial cost but by also focusing employee effort away from productive endeavors

towards learning and mastering new ways of doing things. This in the short-term does not benefit

an organization. This phenomenon would have been more difficult to prove in a lab where

mediating variables and control mechanisms would be required in an attempt to increase the

potential for validity.

Any research project has limitations related to time, money, and connections which forces

its creators to become adept at the art of making trade-offs during the design process. These

severe constraints must be taken into consideration early if one wishes to avoid failure in the

early to mid stages of a project. Dr. Eveland was fortunate enough to participate in a research

Research Design 6

project commissioned by RAND which typically is very experienced in resourcing large projects

for the government and a variety of private sector organizations. Even though capital, time,

personnel, and access to data constraints were probably less of a problem with RAND’s backing

they were potential showstoppers nonetheless. It was necessary to find an organization that

would be willing to make the financial and time sacrifices necessary to commit its people to

undergo a long and intensive research participation project. Dr. Eveland had to ensure that the

subjects agreed to take part and to continue their participation in randomly assigned groups.

They were to be selected from a common “community”; that is, they should come from a

common culture, share some concerns, and have some reason to think they might want to work

with one another (Eveland 358).

Dr. Eveland needed to select a sample group that conveyed enough validity but also

generalizability to the project. The randomly assigned sample size selected was small. Resource

constraints limited them to only two task forces of about 40 members each. All the members

were older men whose careers led to midlevel management or professional positions; we do not

know how the inclusion of younger employees, women members, or representatives of the top or

bottom of the organizational hierarchy might have affected the results. Third, participation in the

task force was voluntary; results might not be the same for collaborative activities that are part of

regular job assignments (Eveland 358).

It is necessary to strike the right balance when attempting to incentivize a group to

participate in a study for conflicts of interest have a strong potential for rendering the results of

otherwise well designed research to be invalid. Typically, it is important for researchers to be

honest about their research goals so as to not mislead and disgruntle participants later on who

will respond with a lack of seriousness. Sometimes financial incentives may succeed in swaying

Research Design 7

people to participate but will create unnecessary conflicts of interest. The longer a research

project is the more fragile it becomes for it is more difficult to ensure that participants participate

wholeheartedly over a long period of time.

Eveland and Bikson utilize an experimental research design in conducting their study

titles “Work Group Structures and Computer Support: A Field Experiment” as they attempted to

scientifically measure the propensity that people provided with technological support would

gravitate towards the use of the latter in a field environment measured over a long period of time.

Research Design 8

Bibliography

Eveland, JD & Bikson, TK (1988). Work group structures and computer support: a field

experiment. ACM (Association for Computing Machinery). 6 (4). 354-379.

Gabaney, Steve (2007). Flow chart describing the research process. Indiana State

University.

McGrath, Joseph E & Brinberg, David (1983). External validity and the research

process: a comment on the calder/lynch dialogue. The Journal of Consumer Research.

10(1). 115-124.

McGrath, Joseph E. (1981) Dilemmatics: The study of research choices and dilemmas.

American Behavioral Scientist. 25(2). 179-211.

Research Design. PDF of powerpoint presentation. University of Western Ontario.

Available at

http://www.ssc.uwo.ca/psychology/undergraduate/psych266a/lectureslides/Psych%20266

%20Research%20Methods%20x3-%20web%20version.pdf

Schlichter, J. & Brüggemann-Klein (1997). A CSCW (Computer Supported Cooperative

Work). Technische Universität München, Germany.

Trochim, William (2007). Introduction to research design. Cornell University.

Sabherwal, R., Jeyaraj, A., & Chowa, C. (2006). Information system success: individual

and organizational determinants. Management Science, 52(12), 1849.