qualresearch100512-100512120502-phpapp02

7/27/2019 qualresearch100512-100512120502-phpapp02

1/50

Department of Informatics Engineering

UNIVERSITY OF COIMBRA

A. Dias de Figueiredo, 2010 PhD Program in Information Sciences & Technologies - Research Methods

A CONCISE INTRODUCTION TO

QUALITATIVE RESEARCH METHODSIN INFORMATION SCIENCES & TECHNOLOGIES


2/50

University of Coimbra, Ph.D. in Information Sciences & Technologies Research Methods 2009/2010

A. Dias de Figueiredo, 2010 Slide 2

1. QUANTITATIVE vsQUALITATIVE METHODS

2. A TYPICAL QUALITATIVE RESEARCH APPROACH

3. QUALITATIVE RESEARCH METHODS

4. DATA COLLECTION TECHNIQUES

5. DATA ANALYSIS TECHNIQUES: CODING

6. TRIANGULATION AND MIXED METHODS RESEARCH

7. RELIABILITY AND VALIDITY


3/50











4/50



Research in Information Sciences and Technologies is traditionally

supported by two radically distinct categories of methods:

1. QUANTITATIVE VSQUALITATIVE METHODS

Quantitative Methods

Qualitative Methods

Although they are sometimes said to be incompatible

(namely by members of the quantitative camp),

they should be seen as complementary to each other.


5/50



The articulation between the two visions quantitative and

qualitative is actually leading to a third major research paradigm:

Mixed Methods



6/50



Quantitative methods are traditionally used in the natural sciences to study

natural phenomena with mathematical rigor, namely using statistical analysis.

A limited amount of variables is always assumed, and these

variables are seen as independent from external factors, measurable,

and holding mathematical relationships between each other.

Quantitative research generally emphasizes planning,

hypotheses, large random samples, and objective measures.

It assumes the existence of a distinction between

researcher and subjects, and aims at generalizing, i. e.

at producing laws applicable to much broader realities.



7/50



Qualitative methods are being used increasingly in the social and human

sciences for the study of highly complex and contingent phenomena, where

the numbers of variables is too high to be handled by quantitative methods.

The growing attraction of Information Sciences and Technologies

toward qualitative methods results, to a large extent, from the increasingly

complex, social and human, nature of the phenomena they deal with.

In the past, the social and human sciences, for fear of

looking less respectable, tended to resort mainly to the

quantitative methods of the natural sciences.

Today, they are putting increasing emphasis on qualitative

research and, as a consequence, obtaining much richer results.



8/50



The Information Systemsdiscipline is considered, today, a

socio-technical discipline, founded on the need to

reconcile technological solutions with the social and human

dimensions of business and the organizational reality.

The main international Information Systems journals, such as

MISQuarterly, clearly show this tendency, and so does

The Communications of the ACM, one of the most popular

Information Science and Technologies journals.

The same happens with most journals and

conferences devoted to Information Systems.



9/50



The field ofHuman-Computer Interaction (HCI)is also receiving

similar influences from the social and human dimensions of technology use.

A few years ago, user interfaces tended to be developed from rather

mechanistic perceptual and cognitive models. Today, social

and organizational models of interaction have become essential.

Most usability studies are conducted today on the basis of

socio-technical approaches inspired by those used in the social sciences.

In the meantime, we witness a shift of interest from usability seen

as just effectiveness, flexibility, and satisfaction toward a

concern with the human experience of living with technology.



10/50



A similar evolution is occurring in Software Engineering.

In fact, it started almost thirty years ago:

New software development approaches, such asAgile Programming

and Extreme Programming, have in common the characteristic of putting

social and human issues at the center of the development process.

Concepts such as social norms, values, beliefs, symbolisms, representations,

patterns of behavior, which were looked with suspicion in the past,

are considered today as essential to the success of technological projects.

Personnel attributes and human relations activities provide by far the

largest source of opportunity for improving software productivity(Boehm, 1981)



11/50



Certainty - cause

and effect linkages

can be determined

PLOTTING THE QUANTITATIVE vs QUALITATIVE CAMPS IN

STACEYS CERTAINTY & AGREEMENT REFERENCIAL

Certainty Uncertainty

Agreement

D

isagreement

Uncertainty -cause and effect

linkages cannot

be determined

and situations

are unique.

Agreement the level

of agreement between

all the parts involved

varies along the axis.



12/50











13/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation

Berg, B. L. (2007)



14/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation

Research begins with rough Ideas that turn slowly into research questions.

Berg, B. L. (2007)



15/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation

Research begins with rough Ideas that turn slowly into research questions. The Literature Review gradually builds up knowledge about the research ideas.

Berg, B. L. (2007)



16/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation


The Design is the plan to be followed in order to carry out the research.It must include strategies for the selection of the samples.

Berg, B. L. (2007)



17/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation


The Design is the plan to be followed in order to carry out the research.It must include strategies for the selection of the samples.

The Data Collection and Organization phase is devoted to gathering the data forthe research and organizing it, so that it can be properly analyzed. These are difficult

tasks, since the volume of data collected in qualitative research can be enormous.

Berg, B. L. (2007)



18/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation

The Analysisincludes three concurrent flows of action: Data Reduction, which focuses, simplifies, and

transforms raw data into more manageable forms;

Data Display, which presents the data as organized and compressed assembliesof information that permit conclusions to be analytically drawn; and

Conclusions & Verification, where the researchers review and finalize all theirconclusions and make sure that they satisfy the requirements of validity.

Berg, B. L. (2007)



19/50



IdeasLiterature

ReviewDesign

Data

CollectionAnalysis

Dissemi-

nation

Dissemination takes the form of very well writtenand detailed documents, so that other researchers

can evaluate the analysis and conclusions obtained

and decide if they trust the results and want

to use them to feed their own research.

Berg, B. L. (2007)



20/50











21/50



From a large variety of qualitative research methods used in the

social and human sciences, five stand out as more relevant

in Information Sciences and Technologies research:

Case Studies

Ethnographic Research

Grounded Theory

Action-Research


Design-Based Research


22/50



Case studies

Case studies are the most common kind of qualitative method

used in Information Sciences and Technologies research.

They let us study a phenomenon in its real context, specially when

the boundaries between phenomenon and context are not clear.

Case Studies


Grounded Theory

Action-Research

Typically, the researcher studies a case or variety of cases of real-world

organizations where information sciences and technologies are being

used and concludes about its impacts on the organizational context.

Case studies can also be used for quantitative research, in which case

they tend to follow a positivist approach. One of the best known books

on case studies (Yin, 1994) corresponds to this option. Other authors,

on the contrary, take constructivist and interpretivist approaches.

Design-based Research



23/50



Ethnographic research

Ethnographic research is inspired by the practices of cultural

and social anthropology, where the researcher integrates for a

period of time the community where the study is taking place.

This practice is common, not just in Information Sciences and Technologies

research, but also in Human-Computer Interaction, in projects that try tounderstand the behavior of the users, so that better interfaces can be developed.

The approach is very common when developing and assessing information

systems. E.g., to understand how the 4200 workers of a company react to the

setting up of an CRM solution so as to improve that solution and makesure that future solutions do not suffer from similar problems.

Case Studies


Grounded Theory

Action-Research



It is also being used to understand the behavior of Software

Engineering teams (namely large, complex, distributed, and multi-

national teams) and improve their performance.


24/50



Grounded Theory

Grounded Theory is a research approach proposed the sociologists

Barney GlaserandAnselm Strauss, who claim that research should

depart from the ground. The researcher categorizes empirically

collected data in order to build a general theory that fits the data.

In essence, it is based on the generation of theory from data.

This approach radically defies the traditional positivist

approaches, which claim that the researcher must depart from a

theory, establish hypotheses that conform to the theory, and than

get to the field to confirm the hypotheses in light of the theory.

The models developed by using Grounded Theoryare

quite distinctive from the traditional ones in that

they reveal a strong foundation on the concrete.

Case Studies


Grounded Theory

Action-Research




25/50



Action-research

Action-Research is today one of the more promising qualitative research

approaches in Information Sciences and Technologies research.

In essence, it consists of repeatedly going through the cycle:

Planning => Action => Reflection

Action-research corresponds to what John Dewey

called the Principle of Intelligent Action.

Case Studies


Grounded Theory

Action-Research



We start by making a plan of our action in a crude first approximation,

we act following that plan, and we then reflect on the results obtained.

From this reflection, we correct our previous plan, act in agreement

with the new plan, and reflect on the results we have now obtained.

The cycles go on, repeatedly, until we are happy with the results.


26/50




Case Studies


Grounded Theory

Action-Research



Design-based Research is a research method where

knowledge is built in successive approximations while

designing, building, and evaluating an artifact.

The successive improvements introduced in the artifact as it

is put to test represent opportunities for consolidating the

knowledge that emerges from its design and application.

The artifact may be almost anything: a piece of equipment, a

software application, the solution to a social or technical

problem, a theoretical framework, or even a whole theory.

In essence, it consists of repeatedly going through the cycle:

Awareness of Problem => Suggestion =>

Development => Evaluation => Conclusion


27/50











28/50



Four data collection techniques are more relevant

in Information Sciences and Technologies research:

Document Analysis

Interviews

Participant Observation

Surveys



29/50



Document Analysis

Document Analysiscovers a broad range of techniques

devoted to the analysis and interpretation of the

documents used as primary data sources.

The term document is understood very broadly, including

not just texts, but also sound, photos, videos, and anymaterials that carry relevant messages.

Document Analysis

Interviews


Surveys

Typical varieties of document analysis include:

Conversational Analysis Discourse Analysis Narrative Analysis

Objective Hermeneutics



30/50



Interviews

Interviewsare used to collect data from the

subjects interactively, usually one-on-one.

Document Analysis

Interviews


Surveys

They are particularly useful to clarify the meaning of a phenomenon to

the subjects and to obtain personal accounts about the development

of a process in which the subjects are engaged.

They are time consuming, but they are more flexible and adaptable

than surveys. They can present various levels of structure:

structured interviews semi-structured interviews

unstructured interviewsWhen many people areinterviewed simultaneouslyand participants are free tointeract with each other, we

talk about focus groups.

They are also useful to support the exploratory work that precedes aquantitative study and to clarify and enrich the results of quantitative studies.



31/50




In participant observation the researchers observe first-

hand the activities of the subjects under study and

collect data from this observation.

The technique works well when the activities observed

are frequent, the groups observed are small, and

there is little risk of disturbing the subjects.

Document Analysis

Interviews


Surveys

There are two basic forms of participant observation:

overt covert

It is a time consuming technique, but it is generally

more accurate than post-hoc self-reporting.



32/50



Surveys

In surveys the subjects complete a survey form

without the intervention of the researcher.

Surveys are particularly useful when the subject population is

large (or distributed geographically), majority opinions are

sought, and the subjects are motivated to respond.

Document Analysis

Interviews


Surveys

Special care must be taken to minimize the length of the

survey, manage the choice of words and terminology, look after

the balance of its structure, fully plan the strategies for its

subsequent analysis, and test-pilot it thoroughly.

Surveys can include closed and open questions, but the

number of open questions should be reduced to a minimum.



33/50











34/50



Codes are tags that categorize the data collected

during a study to assign meanings to them.

Coding makes it easier to search the data, make comparisons

and identify patterns that require further investigation.

It can also be used to extract quantitative data from qualitative data.

A large variety of coding schemes exist, as well as

many software tools like NVivo and Atlas TI used to assistin coding and in helping to organize the resulting patterns.

Codes can be based on: themes, topics, ideas, concepts, terms,

phrases, or keywords found in the data, but they can also correspond

to passages of audio or video recordings and to parts of images.



35/50











36/50



research

ideas

research

findings

Research Process



37/50



research

ideas

research

findings

With triangulation, the

same issue is studied in

various perspectives

that complement and

verify each other.


TRIANGULATION


38/50



Multiple Theories

research

ideas

research

findingsMultiple Methods

Multiple Data

Multiple Researchers

TRIANGULATION4 main kinds of triangulation



39/50



research

ideas

research

findings


Multiple Theories

Multiple Methods

Multiple Data

Multiple Researchers

The new research paradigmof Mixed Methods has

emerged from this kind oftriangulation

TRIANGULATION


40/50



Mixed Methods Research (also called, by some authors,

Multiple Method Research orMultimethod Research)

is becoming the third major research approach,

along with quantitative research and qualitative research.

Janice Morse (2003) provides the following distinctions:

Multimethod design - the use of two or more research methods,

each conducted rigorously and complete in itself, in one project.

The results are then triangulated to form a complete whole.

Mixed methods design - the incorporation of various qualitative

and quantitative strategies within a single project, that

may have either a qualitative or quantitative theoretical drive.



41/50



Multimethod designs, according to Morse (2003)

For multimethod designs Morse (2003) defines three principles:

Principle 1: identify the theoretical drive

(inductive ordeductive) of the project.

Principle 2: develop overt awareness of the dominance (QUAN

orQUAL, and simultaneous orsequential) of each project.

Principle 3: observe methodological integrity.



42/50











43/50



The inclusive trend, which argues that the credibility of qualitativeresearch can only be widely accepted if the language of mainstream

(quantitative) research is maintained, although operationalized

to meet the new conditions and circumstances.

Two major trends in the debate about rigor

and validity in qualitative research:

The exclusive trend, for which the qualitative paradigm is soradically different from the quantitative paradigm that a new

language must be used to express its rigor and validity.

(Guba & Lincoln, 1981; Guba & Lincoln, 1982; Guba & Lincoln, 1989 )

(Yin, 1994; Morse, Barret, Mayan, Olson, & Spiers, 2002, Creswell, 2009)



44/50



THE EXCLUSIVE TREND

The proponents of the exclusive trend claim that the terms

validity and reliability from qualitative research do not

make sense in qualitative research, so they should be replaced:

Quantitative Research Qualitative Research

internal validity credibility

external validity transferability

reliability dependability

objectivity confirmability



45/50



Credibilityis achieved when the results are seen

as believable by the participants in the research.

The participants decide about credibility.

Transferability exists when the results

can be applied to other contexts.


Techniques for establishing credibility include: prolonged

engagement, persistent observation, triangulation, peer debriefing,

negative case analysis, referential adequacy, and member checking.

The researcher should describe in detail the context and

underlying assumptions of the research (thick description), so that

transferability is possible, but the person who transfers the resultsto a different context is responsible for the transfer.


46/50



Confirmabilitydemonstrates that the inquiry is free of bias, values and

prejudice, i.e. that the data interpretations and outcomes are rooted in

contexts and persons apart from the researcher and are

not mere products of the researchers imagination.


The researcher must document the procedures, so that others can check

and recheck the data throughout the study. Techniques that may be used

to strengthen confirmability include: prolonged engagement, persistentobservation, peer debriefing, negative case analysis; and triangulation.

Dependability emphasizes the stability of the data over time.

The researcher must be able to account for the permanently changing context in which

the research takes place, describing any changes that occur and how these changes

affect the research. This requires what is sometimes called progressive subjectivity.

After the study, a data audit or external audit should be

conducted by a researcher not involved in the research process,

to examine both the process and product of the research study.


47/50



THE INCLUSIVE TREND

Many proposals exist within the inclusive trend.

The one summarized here is proposed by Creswell (2009):


Qualitative validitydoes not have the meaning used in quantitative

research: it just means that that the researcher checks for the

accuracy of the findings by employing adequate procedures.

Qualitative reliabilityindicates that the researchers approach is

consistent across different researchers and different projects.

Qualitative generalization is a term used in a limited way in qualitative

research, since the intent is not to generalize finds, but rather to

explore particular phenomena in the context where they occur.


48/50



Qualitative Validity

Creswell (2009) proposes the use of eight primary strategies:


TriangulationMember checking

Rich, thick descriptionAvoidance of researcher bias

Negative case analysisProlonged engagement

Peer debriefingExternal auditing

Two strategies to avoid researcher

bias are reflexivity (researcher self-

awareness and self-reflection) andresearcher journaling (detailed and

timely documentation of the

researcher thoughts).


49/50



Qualitative reliability

Yin (2003) suggests that qualitative researchers should:


document the procedures of their case studiesdocument as many steps of the procedures as possible

set up a detailed case study protocol and database

Gibbs (2007) suggests several reliability procedures:

Checking transcripts for mistakesCheck the persistence of the meaning of the codes

Coordinate communication among codersCross-check codes developed by different researchers


50/50



Berg, B. L. (2007). Qualitative Research Methods for the Social Sciences (6th Ed.). Pearson Education.Boehm, B.W. (1981). Software Engineering Economics. Englewood Cliffs, N.J.: Prentice-Hall.Creswell, J. W. (2007). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (3rd Ed.), Sage.Gibbs, G. R. (2007). Analyzing Qualitative Data . In U. Flick(Ed.). The Sage Qualitative Research Kit. Sage.Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries, Educational Communication

and Technology Journal, 29 (2), 75-91.

Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation: Improving the usefulness of evaluation results throughresponsive and naturalistic approaches. San Francisco, CA: Jossey-Bass.Guba, E. G., & Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry. Educational

Communication and Technology Journal, 30 (4), 233-252.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.Morse, J. M., Barret, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and

validity in qualitative research. International Journal of Qualitative Methods, 1(2), 1-19.Morse, J. M. (2003). Principles of Mixed Methods and Multimethod Research Design. In Tashakkori, A, & Teddlie, C.

(Eds.). Handbook of Mixed Methods in Social and Behavioral Research . Sage. 189-208.

Yin, R. K. (1994). Case Study Research: Design and Methods (2nd Ed.). SageYin, R. K. (2003). Case Study Research: Design and Methods (3rd Ed.). Sage

REFERENCES:

qualresearch100512-100512120502-phpapp02

Documents