Doing Educational ResearchHatesh Kumar, S. Faiza Hassan, Samreen RiazMSAL, NED UET

Doing Educational Research

A Guide for First Time Researchers Edited By: Clive Opie

Book Review Presented By Mr. Hatesh Kumar Ms. Samreen Riaz Ms. Faiza Hassan

What is Educational

Research

Methodology, Procedures And Ethical Concerns

Writing Research

Reliability, Validity

And Credibility

Research Approaches

Research Procedure

sPresenting

Data

Using NUD.IST

Using Atlas.

ti

Chapters covered…

What is Educational

Research

Methodology,

Procedures And Ethical Concerns

Writing Research

What is Educational Research?Hatesh Kumar

Chapter :1What is Educational Research?

Clive Opie

What is Educational Research?Can I do educational Research?

Research is…. Seeking through methodological processes to add one’s body of

knowledge and, hopefully, to that of others, by the discovery of non-trivial facts and insights. (Howard & Sharpes, 1983)

A search or investigation directed to the discovery of some fact by careful consideration or study of a subject; a course of critical or scientific inquiry. (OED, 2001)

Educational Research… ‘The collection and analysis of information on the world of education so

as to understand and explain it better’, with a significant practicing teachers in that it should be

‘viewed as critical, reflexive, and professionally oriented activity… regarded as crucial ingredient in the teacher’s professional role….generating self knowledge and personal development in such a way that practice can be improved.(Hitchcock and Hughes, 1989)

Myths surrounding Educational ResearchResearch requires The collection of quite large amount of data Results which can be generalized A hypothesis The undertaking of experiments Objectivity rather than subjectivity The use of statistics That something is proved Specific expertise, as it is difficult

Can I do educational Research?

PracticalityHard workTime consumingProblematicDifficultRequires expertise

Educational Research

Educational Research is ‘doable’, albeit at different levels of depth and sophistication, by all interested in making a ‘systematic, critical and self-critical enquiry which aims to contribute to the advancement of knowledge’ (Bassey, 1990) of the world around them.

Methodology, Procedures and Ethical Concerns

Chapter :2Methodology, Procedures and Ethical Concerns

Pat SikesWorking DefinitionsSelecting Methodologies and ProceduresResearcher PositionalityEthical issues and questions

Methodologies

Methodology refers to the theory of getting knowledge, to the considerations of the best ways, methods, or procedures, by which data that will provide the evidence basis for the construction of knowledge about whatever it is that is being researched, is obtained.

Methodology is concerned with the description and analysis of research methods rather than with the actual, practical use of those methods.

Methodological work is philosophical , thinking, work. Methodology is the overall approach to a particular research project, to the

overarching strategy that is adopted.

Procedures/ MethodsSpecific research techniques that are used to collect and analyze data.

Methodology A Case Study Action Research

project

Procedures/ Methods Interviews,

questionnaire, observation and documentary analysis

Tests, questionnaires, and interviews ,

To collect information in order to evaluate the interventions that was its focus.

Selecting Methodologies and Procedures

Researchers have to be able to justify and argue a methodological case for their reasons for choosing a particular approach and specific procedures.

Methodology and procedures determine the nature of the findings of the research.

Decisions, about which methodology and procedures will be used, influenced by

What can actually be done; What is practical and feasible; by situational factors of various kinds; and by personal predilection and interests.

Think about the implications for research design

The physical, social, political and historical contexts in which their research project will be located.

The reasons why they are doing the research. How they conceptualize the situation they are

researching. The sorts of questions that they are seeking

answers for. The type of information that their questions will

produce. The scale of the research and the financial,

personnel and other resources available to conduct it.

Continue…The nature of the research population and the ability of informants to provide particular types of response.

Ethical and moral issues relevant at various stages of the research process.

If applicable, what are the requirements and expectations of any organization or body that is commissioning and/or funding the research?

When, and over what timescale the research will be done.

Researcher Positionality

The most significant factor that influences choice and use of methodology and procedures is ‘where the researcher is coming from’ in terms of their philosophical position and their fundamental assumptions concerning:

Social reality: their ontological assumptions; The nature of knowledge: their epistemological

assumptions; Human nature and agency: specifically their

assumptions about the way in which human beings relate to and interact with their environment.

Assumptions

Assumptions are colored by values and beliefs that are based in political allegiance, religious faith, and experiences that are consequent upon social class, ethnicity, gender, sexuality, historical and geographical location and so on.

Paradigm: A basic set of beliefs that guides action.

Education Research: Two main paradigmsQualitative and Quantitative

Research Paradigms

Hard, real, capable of being transmitted in tangible form

Do you view the nature of knowledge as

Softer, subjective, based on experience and insight of an essentially personal nature

OR

Empiricist’s view of knowledgei.e. No knowledge exists that which is objectively, immediately observable.

You are classed as a positivist, objectivist.

Rationalist’s view of knowledgei.e. knowledge is perceived as created in the mind of the individual. You are

classed as an anti-positivist, interpretevist, subjectivist.

You are likely to employ quantitative procedures such as surveys &

undertake large studies searching for generalisable results.

You are likely to employ qualitative procedures, which focus on

individuals or small groups, more concerned with understanding

personal constructs and relatability.

Ontological assumptions concerning the nature of social reality Ontological assumptions focus on whether a person sees social reality- or

aspects of the social world- as external, independent, given and objectively real, or, instead, as socially constructed, subjectively experienced and the result of human thought as expressed through language.

How they view the social world has the implications for the sorts of methodologies and procedures they are likely to consider to be valid means of collecting ‘valid’ data, that can be used to make a ‘valid’ interpretation, thus creating ‘valid’ knowledge.

If a social constructivist position is taken it will be necessary to collect subjective accounts and perceptions that explain how the world is experienced and constructed by the people who live in it.

Epistemological assumptions concerning the bases for knowledge Epistemology is the theory of knowledge.

Griffiths suggests that, epistemology, and particularly the relationship between methodology and procedures and knowledge and truth, is a contentious and controversial area for researchers and consumers of research.

Emphasis on accounts given by informants- either verbally in interviews or written in the questionnaire.

Issue of how words can actually reflect reality and experience is complex and problematic.

Researchers state their position explicitly …tentative and cautious in presenting conclusion.

Assumptions concerning human nature and agency

Concerned with the ways in which human beings are seen to act within the world.

Actions ….innate instinctual forces or external conditions and forces. Whatever you view inevitably applies to you and research population. It is a complex area that highlights the issues of social power and

agency as well as raising questions about natural behaviors. If people are believed to behave in a predetermined or reactive way,

then observation and experiment will be appropriate techniques. If they are felt to make decisions about what to do then procedure will

seek explanation and understanding from their perspective will be needed.

Ethical issues and questions Research Design Access Procedure of Data Collection Research relationships Interpretation and analysis Writing up Data dissemination Avoiding harm/Doing wrong

Research DesignWhat exactly do you want to know and why do you want to know it? Can you justify your interest?

If you are intending to do anything that is in any way ‘experimental’ what are the implication for the people who will be involved? If you are using a ‘control group’ will people assigned to it miss out any thing that you suspect will be beneficial? Can it be justified?

Research DesignInsofar as you are able, have you thought about potential unintended or unexpected consequences either to the people directly involved in the research or as a result what you might find out?

If you are intended to do covert research of some kind, can you justify it?

How do you regard the people you are going to be ‘researching’?

Access

How are you going to access your research population? If you choose to do your

research with people who don’t posses much social power (e.g. children, captive

populations, your own students) can you justify why? And are you exploiting their

weakness?

Procedure of Data CollectionAre you asking people things you wouldn’t want to be asked?

Are you asking people to do things you wouldn’t want to be asked to do?

Research relationships

You have a basic moral responsibility towards the people you are working with. Are you sure that you are doing as you would be done by?

Could you be accused by ‘rape research’? Are you manipulating people and relationships in order to get

‘good’ data? Are you sensitive to the implications of any differences in terms

of social power between researcher and ‘researched’.

Interpretation and analysis

Do you acknowledge any theoretical frameworks or value systems that may influence your interpretations and analysis?

Writing up

Do you ‘own’ your research in your writing up.

Do you make the research process appear to be neat and unproblematic?

Are informants sufficiently protected in written accounts?

Data disseminationAre my informants sufficiently protected when it comes to data dissemination?

Avoiding harm/Doing wrong

The aim is not to harm anyone or do any moral wrong. This isn’t simple because one can never know what the unintended outcomes will be?

Do the ends ultimately justify the means?

Writing Research

Chapter : 3Writing Research

David HyattExpected requirements for student academic writing

A consideration of academic writing conventions

Audience in academic writing FeedbackStructuring your research-based assignmentSome advice I’ve been given (or wish I’d been given!)

Conclusion

Expected requirements for student academic writing

Using a range of sourcingCriticalityEvidenceMake your own pointPresentationPlagiarism

If you must write a prose or poemsThe words you choose should be your ownDon’t plagiarize or take ‘on loan’There’s always someone, somewhereWith a big nose, who knowsWho’ll trip you up and laugh when you fall.

1980s song (Morrissey and Marr, 1986)

A consideration of academic writing conventionsSpecific points to consider……when to us ‘I’ , ‘We’, ‘You’ etc

Avoid Direct ClaimsUse ‘Tentative’, ‘Distancing’ attitudinal modality

Use of nominalization as opposed to verbalization

Other problems Avoid overgeneralizations Avoid unsubstantiated claims Be careful with rhetorical questions Be specific Use subject specific lexis Avoid dramatic/loaded language Don’t over claim/under claim Avoid sexist or gender stereotyping language

Audience in academic writing

The duality of the audienceThe message is being directed at two discrete audiences.

For experts related to the fieldFor lay-person

Feedback………Tutor’s comment

Phatic

Developmental

Structural

Content

Methodological

Comment , encouragement

Alternatives, future, reflective, informational

Discourse, sentence, stylistic level Positive and negative evaluation,

non-evaluative summary

Approach, methods, process and administration

Structuring your research-based assignment

Abstract Introduction Literature Review Methodology Results and Analysis Conclusion Limitation Implications and Recommendation

Some advice I’ve been given (or wish I’d been given!)Time managementSpace managementReferencing as you goProofreadingThe value of critical friend

Conclusion

Good academic writing is clear, concise, critical, credible, evidenced, well structured and well presented.

Remember to enjoy your writing.Goethe claimed that ‘The most original authors are not so because they advance what is new, but because they put what they have to say as if it had never been said before.’

Samreen RiazChapters covered…

Reliability, Validity

And Credibilit

y

Research Approache

sResearch

Procedures

Reliability, Validity And CredibilityJon Scaife

The systematic, rigorous investigation of a situation or problem in order to generate new knowledge or validate existing knowledge.

WHAT IS RESEARCH?

McMillan and Schumacher (1989) define research as “a systematic process for collecting and analyzing information (data) for some purpose.”

Kerlinger defines scientific research as “systematic, controlled, empirical and critical investigation of natural phenomena guided by theory and hypotheses about the presumed relations among such phenomena.”

It has been argued that the results of educational research will lead to the improvement of educational practice; therefore, professional practitioners should maintain a continued interest in research.

The results of educational research are reported in a way that requires a knowledgeable person to read and implement them.

Although educational research is complex and demanding, the broad spectrum of research activities ranges from the simple, single operations to complex combinations of qualitative and quantitative procedures.

Educational research is systematic and within a broad framework follows the steps of the scientific method. However, across different types of studies, there is extensive flexibility in how the steps are implemented.

To make research systematic, researchers use the approach of scientific inquiry and scientific method.

Scientific Inquiry: search for knowledge through recognized methods of data collection, analysis and interpretation.

Scientific Method: research process is considered to consist of a series of sequential steps.

Scientific Method:

1) Identifying a problem

The nature of the problem is to be defined; related knowledge is identified and a framework to conduct the research is established. In addition, necessary assumptions and conditions are also identified.

2) Review information

The researcher reviews how others approached a similar problem; i.e. Literature review.

3) Data collection:

Collecting data requires a proper organization and control to validate the data to make decisions upon them

4) Data analysis:

Data analysis must be done in a manner appropriate to the problem.

5) Drawing conclusions

Following data analysis, researchers draw conclusions and make generalizations based on the data they had collected.

The Validity of Educational Research:Researches must be based on facts; i.e. capable to be justified. There are two concepts: internal validity and external validity.Internal Validity: it is the extent to which the results of a research can

be interpreted accurately and with confidence. External Validity: the extent to which research results can be

generalized. Internal validity is a prerequisite for external validity because if the

results cannot be interpreted accurately with confidence, researchers cannot generalize them.

Generalizability does not mean that the study must be generalized to many various situations and populations; external validity, rather, depends upon the conditions and purposes of the specific research study.

It is impossible to reach perfect internal and external validity; researchers must work to reach a balance so that results can be interpreted with confidence and still have some useful generalizability.

The Reliability of Educational Research:

Reliability refers to the consistency of the research and the extent to which studies can be replicated.

1. Internal reliability refers to the extent that data collection, analysis, and interpretation are consistent under the same conditions.

- If internal reliability is lacking, the data becomes a function of who collected them rather than what actually happened.

2. External reliability deals with the issue of whether or not independent researchers can replicate studies in the same or similar settings with consistent results.

Reliability is a necessary characteristic for validity; a study cannot be valid unless it is reliable. If it is unreliable, results cannot be interpreted with confidence and cannot be generalized.

Reliability + validity = credibility of research

Research Approaches

Correlational research refers to the systematic investigation or statistical study of relationships among two or more variables, without necessarily determining cause and effect.

Descriptive research refers to research that provides an accurate portrayal of characteristics of a particular individual, situation, or group. Descriptive research, also known as statistical research. These studies are a means of discovering new meaning, describing what exists, determining the frequency with which something occurs, and categorizing information.

Ethnographic research refer to the investigation of a culture through an in-depth study of the members of the culture; it involves the systematic collection, description, and analysis of data for development of theories of cultural behaviour.

• It studies people, ethnic groups and other ethnic formations, their ethno genesis, composition, resettlement, social welfare characteristics, as well as their material and spiritual culture.

Experimental research is an objective, systematic, controlled investigation for the purpose of predicting and controlling phenomena and examining probability and causality among selected variables.

Exploratory research is a type of research conducted for a problem that has not been clearly defined. Exploratory research helps determine the best research design, data collection method and selection of subjects

Grounded theory research is a research approach designed to discover what problems exist in a given social environment and how the persons involved handle them; it involves formulation, testing, and reformulation of propositions until a theory is developed.

Historical research is research involving analysis of events that occurred in the remote or recent past

Phenomenological research an inductive, descriptive research approach developed from phenomenological philosophy; its aim is to describe an experience as it is actually lived by the person

TYPES

QUANTITATIVE RESEARCH DESIGN

QUALITATIVE RESEARCH DESIGN

Krathwohl defined them as (1993), “Qualitative research: research that describes phenomena in words instead of numbers or measures… Quantitative research: research that describes phenomena in numbers and measures instead of words (P. 740)”

In terms of conducting research, the difference between them is not a dichotomy but a qualitative-quantitative continuum.

General Methodology: Qualitative and Quantitative Research

Qualitative research is research dealing with phenomena that are difficult or impossible to quantify mathematically, such as beliefs, meanings, attributes, and symbols

Qualitative researchers aim to gather an in-depth understanding of human behaviour and the reasons that govern such behaviour. The qualitative method investigates the why and how of decision making, not just what, where, when.

Quantitative research refers to the systematic empirical investigation of any phenomena via statistical, mathematical or computational techniques. The objective of quantitative research is to develop and employ mathematical models, theories and/or hypotheses pertaining to phenomena

Quantitative research is generally made using scientific methods, which can include:

• The generation of models, theories and hypotheses

• The development of instruments and methods for measurement

• Experimental control and manipulation of variables

• Collection of empirical data

• Modelling and analysis of data

• Evaluation of results

From a practical point of view, qualitative and quantitative procedures are often mixed; however, their methodologies can be placed on the continuum– not on dichotomy–as they tend towards the qualitative or quantitative.

Classification of Educational Research

Two systems are described: one based on the goal of the research, and another on the way in which the research is conducted.

Applied Research- Purpose: solve an immediate,

practical problem; - It may contribute to the general

knowledge of some field as it produces a solution for a specific problem. (supplemental purpose).

Basic Research- Purpose: adding to the existing

body of knowledge in a discipline;- Although not ruled out, basic

research does not necessarily provide results of immediate practical use (supplemental purpose).

Basic and applied research are important; they should not be differentiated by hierarchy of value judgments; instead, the purpose of research is the criterion.

An example of Basic Research:- An experiment on learning in a laboratory setting. The purpose of the experiment

is to contribute to the knowledge about how learning takes place.

An example of Applied Research: - A curriculum committee is surveying elementary school teachers about materials

of reading programs. The results of the survey would provide the necessary information for deciding which program to adopt.

Basic research and Applied research are differentiated by their purpose. The primary purpose of basic research is the extension of knowledge; the purpose of applied research is the solution of an immediate, practical problem.

Action ResearchAction research is one type of applied research. It is conducted by a professional educator to aid in making decisions in local schools.

Since it is local, there is concern upon generalizing its results to other educational settings. Teachers are curious about their own practices rather than generalizing the outcomes.

Action research is less rigorous in terms of methodology and design than other educational research.

Action research + research literature = viable approach to making educational decisions at local level.

Research Procedures

Research Procedures

The research design is the master plan specifying the methods & procedures for collecting & analyzing the needed information in a research study.

Research design is a plan of how & where data are to be collected & analyzed.

Research design can be defined as a blue print to conduct a research study, which involves the description of research approach, study setting, sampling size, sampling technique, tool & methods of data collection & analysis to answer specific research questions or for testing research hypothesis

ELEMENTS OF RESEARCH DESIGN

ELEMENTS

APPROACH

POPULATION, SAMPLE & SAMPLING TECHNICMETHOD OF ANALYSIS

TOOL & METHODSTIME & METHOD OF DATA COLLECTION

Qualitative

Quantitative

Both

With /without conceptual framework

1. Nature of the research problem.

2. Purpose of the study.

3. Researcher’s knowledge & experience.

4. Researcher’s interest & motivation.

5.Research ethics & principles.

6. Subjects & participants.

7. Resources.

8. Time.

9. Possible control on extraneous variables.

10.Users of the study findings.

Chapters covered…

Presenting Data

Using NUD.IST

Using Atlas.ti

Presenting DataClive Opie

Presenting Data

Collecting data?=how to analyze data

Data

Quantitative

Descriptivestatistics

Inferentialstatistics

Parametric testsNon-Parametric

tests

Qualitative

Themes

Issues

Quantitative

Nominal

Mode frequencies

Statistic analysis

Ordinal

Spearman rank Order

Kruskal-Wallis test

Interval Ratio

ANOVA

T-test

Nominal DataPuts data into categories

Ethnic GroupsSample of Species in a selected ground

EarthwormsAnts

Sample of Speech communities in a given areaPunjabiPotohariSiraiki

Ordinal Data Indicates Order of Numbers is Meaningful

Scores Example: Scores in a sample population

Low score vs high scores Likert Scale Position of Schools on City-League

No arithmetic significance Intervals between sets of scores Difference between 60 & 70 Vs 45 and 55

Interval Ratio DataRefers to numbers which have regular intervals

Intervals can be interpretedCan be extrapolated

Generalized to other schools in the localityExample:

# of Minority Community students in schools of a selected locality

Descriptive StatisticsDescribes group

Age ranges in a class

Trap of Superfluous Data

Age Range Group Total

Cumulative Percentage

19-21 178 78.422-24 42 96.925-27 7 100

Inferential StatisticsTests

Non-Parametric TestsParametric Test

Non-Parametric Data Data

Nominal Ordinal Data

Can not be ExtrapolatedSphinx Survey, ExcelKruskal Wallis Test-find if 3 more groups belong to same population

Spearman’s Rank Order-find significant relationship between two sets of Ordinal Data

Parametric Data Data

Interval-RatioCan be ExtrapolatedRequires careful attention to characteristicsANOVA-mean of more than two samplesPearson’s product Moment correlation-strength of relationship between two interval scales variables

t-test: -testing the level of significance between the mean of two samples

Presenting Data Table Bar Charts Graphs

Guidelines Colors Simplicity Labels Total Necessity

19 20 21 22 23 24 25 26 270

10

20

30

40

50

60

70

80

Age profile

Age profile

19-21 22-24 25-270

50

100

150

200

Age profile

Age profile

QUALITATIVE Data Data

Open Questions InterviewsDescriptionsObservation Notes

ChallengesVolumeCreating meaning (themes and issues)Analysis is not straight forward or quickOpen to subjectivity of researcher

QUALITATIVE Data Validity & reliability

Table or charts can be addedPositive aspects

Connects to social realityAnswer more reliably why findings are the way they areStumble upon new themes

Using NUD.ISTAnn-Marie Bathmaker

Using NUD.ISTNon-Numerical, Unstructured Data Indexing, Searching

and Theorizing

Software Package for analyzing Qualitative DataTo study interviews and Data Analysis

In this Chapter…

Practicalities of Using NUD.IST

Highlights of Pros & Cons (Criticism)

Recommendations for new Users

Analyzing Data in Qualitative Study

Loads of text for analysis

Context matters

Tone is important & holds meaning

Difficult to get a grip on and make connections

Difficult to Predict what will be important

ResearchersFeel the need

To organize data

Build connections

Cluster parts of texts together

Save time

Starting with NUD.IST

Comes with an Introductory tutorial

Lasts 5 hours

Printed Manual

NUD.IST

Works as a code and retrieve system

Import Text Doc

Code Parts of text

Retrieve Coded Doc

Analyze the text

Other features

Nodes can be further connected to get sub-code

Memos can be attached to the nodes

Coding & links can be modified

Preparing Text for coding

Name files and maintain an index

Change names before importing

Decide units for codes: Sentence or Paragraphs

Criticism

Might distance the researcher from actual dataCategories emerge from data

Links are more important

CriticismUnnecessary or forced linking:

It is important to avoid the misapprehension that the coding and computing lend a scientific

gloss to qualitative research. (Coffey et al, 1996) Manual analysis may lead to more unclear stringing

which is difficult to get back to. Computer encourages users to be clear about what

they are doing. (Fielding and Lee, 1998)

CriticismFacilitates the development of theory in highly organized and systematic waySystematic IndexingResearchers develop their own interpretations which they have to make explicit and justify

CriticismTends to distance you from actual data to concentrate on conceptual notions and, ponder and visualize these concepts

CriticismIncrease reliability, validity and generalizabilityEase of access and, back and forth movement

RecommendationsNot to be influenced by what the software can do

Be clear at all times why an action has been taken

The world of human experience should be studied from the perspective of culturally and historically situated individual.

Using Atlas.tiMichael Pomerantz

Using Atlas.tiSoftware; used to study, analyze and compare data for Qualitative analysis

Gives good qualitative descriptionsPreferred by researchers working with grounded theory approach

(engage in interview/real life dialogue)

Preparing Text for coding

Name files and maintain an index

Change names before importing

Decide units for codes: Sentence or Paragraphs

Atlas.ti

Works as a code and retrieve system

Import Text Doc

Code Parts of text

Retrieve Coded Doc

Analyze the text

Other features

Nodes can be further connected to get sub-code

Memos can be attached to the nodes

Coding & links can be modified

Additional features

Has left & right side panes for codes & texts

One word codes

Can auto code a word across texts

Users ViewsAllows to build networks of Codes (tree & Higher hierarchy)

Allows time and space to go back and start all over again

Allows categorization of codes; (important, superordinate, Sub ordinate)

Allows later division and merger of codes

Thank you