Observationas aResearch Instrument

An Overview• What is the purpose of your investigation / study /

research?

– Rationale, key issues (focus), reference to the work of others (keeping track!!).

• Will data be prestructured or evolve? Is the approach to be tightly or loosely structured?

• Do you need a hypothesis?

Not essential and will depend on type of study, approach etc. What is more important is ‘whether you have thought about what is, and what is not, worth investigating and how the investigation will be conducted’

An Overview continued . . .• Who is the intended audience?

• What outcomes are intended – short term/long term?

• Which methodological approach will you take?– Will this draw on/be supported by the work of others?

• Which research instruments?– Will this draw on/be supported by the work of others?

• How will you support such things as: rigour, the need to be systematic, the need to operate ethically, reliability, validity, etc?

• How will you select a sample, gain access to the associated location, obtain consent, ensure confidentiality/anonymity, involve your participants actively etc?

An Overview continued

McNiff and Whitehead (2005 p.1) provide the following list of issues for consideration:

– What is my concern?

– Why am I concerned?

– What kind of evidence can I produce to show why I am concerned?

– What can I do about it?

– What will I do about it?

– What kind of evidence will I produce to show how the situation is unfolding?

– How do I make sure that any conclusions I come to are reasonably fair and accurate?

– How do I modify my practice in the light of my evaluations?

Some key Issues:Once you are sure that observation as a research method (instrument) is fit for purpose (a sound research question(s) – methods fit), consider:

• Strengths:– Observation gives direct access to social interactions.

– “include the opportunity to record information as it occurs in a setting” (Cresswell, p.221)

– Observation can give permanent and systematic records of social interaction (by way of field notes, numerical data that may be nominal, continuous or ordinal, video, audio etc.)

– Observation can enrich and supplement data gathered by other techniques/approaches (in a mixed approach it may, for example supplement results drawn from the analysis of questionnaire responses).

– Observation techniques are extremely varied (producing qualitative and or quantitative data, can be used in different contexts/locations, to answer different types of research question).

• Weaknesses:– It demands a great deal of time.– It is susceptible to bias.– Limits you to situations where you can gain access

Validity & Reliability:‘Validity is about whether or not the researcher has recorded, or measured, what was intended. Three techniques in particular will enhance the validity’:

– Use more than one source of evidence and or more than one data collection technique. (methodological triangulation).

– Get those who provide you with data to comment on your interpretations.(respondent validation).

– Be aware of your own impact … monitor this carefully and make this explicit in your submission (reflexivity).

Reliability is the idea that another evaluator, or the same evaluator on another occasion, should come up with the same results. There are twodimensions of reliability:

– Inter-judge reliability – which is about whether two evaluators studying the same event come up with the same conclusions;

– Intra-judge reliability – which is about the consistency of the same evaluator’s findings about the same things on different occasions.

Validity and reliability can be supported by:

– a very detailed account of your approach (rationale, procedures, techniques, management of data analysis and the drawing of conclusions etc – sometimes referred to as an audit trail) that is open to scrutiny;

– This includes a very rigorous/systematic approach to data collection and analysis.

Based on Craft, A. (2000 pp. 109-110) Continuing Professional Development A Practical Guide for Teachers and Schools (2nd Edition), London, RoutledgeFalmer

How?• Who/What do you need to observe? Individuals, groups,

on the basis of gender, ability, age etc?

• Setting the Scene (Access, Ethics - including consent, Building a Rapport etc.)

• When, where and how will you observe?

– Participant Observation (e.g. linked to Action Research).

– Partial Participant Observation (nearby, but not taking an active role)

– Non Participation Observation. (e.g. Experimental work).

– Third Party Observation (e.g. a colleague observing you).

• How will data be collected? (How often, over what length of time etc?)

– Tape (provides detail that can be analysed during repeated listening – a useful back up to in-situ note taking etc. Quality of audio, doesn’t record everything (non-verbal language, impact on participants, time consuming etc. Transcript offers greatest level of depth)

– Video (excellent visual and audio record, can be viewed repeatedly and will provide good support to in situ note taking, but what about camera position, affect on participants etc? Transcript offers greatest level of depth.)

– Note Taking – Descriptive / Narrative (Anecdotal) (Immediacy, economic use of time, but what might get missed, is recoding superficial, how might this approach impact on participants?)

Descriptive: Some pre-set categories but more flexibly applied, allowing the researcher to ‘consider the context of the behaviours, their sequences, their meanings etc.

Narrative: No structure or restrictions, based on general guiding questions –evolving focus allowing for the collection of very rich data. Categories emerge from data analysis.

Sample:Here you need to consider the extent to which your‘sample’ – a relatively small number of individuals drawnfrom the population – is representative of that samepopulation. This links to the notion of ‘generalisation’.

A general rule of thumb is that a sample smaller than 30 isunlikely to allow you to make useful comparisons,particularly of a statistical nature.

Please note that the selection noted below is by no means definitive.

• Random - This approach gives each eligible element/unit an equal chance of being selected in the sample; random procedures are employed to select a sample e.g. names out of a hat, every tenth name on a list etc.

• Stratified - This takes account of different sub-groups – men/women, 0-5s/6-11s/12-18s, different ethnic or religious groupings etc.

• Cluster - This involves successive sampling of units (or clusters); the units sampled progress from larger ones to smaller ones (e.g. health authority/health board, trust, senior managers).

• Convenience – Sometimes called ‘opportunistic sampling’ a strategy that uses the most easily accessible participants for a study, selected quite arbitrarily. The selection is unstructured.

• Purposive - Purposive/purposeful sampling: a strategy in which the researcher selects participants (or has them selected) who are considered to effectively represent a particular population. For example, a selection of able pupils, or a mixed ability group. This is sometimes also called judgmental sampling.

However your sample is selected, you must provide a justification for your approach as partof a methodology section

Research question:

What might non-verbal language tell us

about power relations between adult

interviewers and young children?

Data from a video recording

Event Sampling:Frequency

Event Sampling: Duration

AWJ: Arguing without justifying a personal position

RVN: responding very negatively

UBL: using bad language

Participant Categories

AWJ RVN UBL Row Totals

Student A 0 (secs) 10 0 10

Student B 14 8 4 26

Student C 21 0 21 42

Student D 59 12 37 108

Column Totals 94 30 62 186

Time/Interval Sampling:

• Simpson and Tuson (2003) indicate that the timeinterval adopted by professional researchers liesbetween 5 seconds and 5 minutes, with pre-datacollection trials used to determine the mostappropriate time interval.

• Select a set of indicators/categories associated with a particular area of focus (in the example shown on the next slide - Non-talk pupil activity) and, during regular time intervals (every two minutes in this example), occurrences of these indicators are noted.

Time/Interval Sampling (cont’d):

• Recording is based on a simple yes (tick) to indicate that a category has been observed at some time during the interval. No more than one tick is recorded, irrespective of how many times the behaviour may be seen.

• This can be useful in providing an indication of how forms of behaviour change over time. For example one of the categories might occur frequently at the start of a lesson, not be seen whilst children are working on their task, but feature again during the plenary session.

• However, this form of recording would not provide the same data as a tally sheet linked to the frequency of particular occurrences/events.

References

• Bell, J. (1999) Doing Your Research Project – A Guide for first time researchers in education and social-sciences (3rd Edition), Buckingham, Open university Press.

• Craft, A. (2000 pp. 109-110) Continuing Professional Development A Practical Guide for Teachers and Schools (2nd Edition), London, RoutledgeFalmer

• Cresswell, J. (2008) Educational Research. (3rd Edition) New Jersey: Pearson International

• McNiff, J. and Whitehead, J. (2005) Action Research for Teachers: A Practical Guide, London, David Fulton Publishers

• Simpson, M and Tuson, J. (2003 pp. 16-20) Using Observation in Small-Scale Research : A Beginners Guide (Revised Edition), Glasgow, SCRE

• The Open University (2001 p.207) Masters Programme in Education: Research Methods in Education Handbook, Milton Keynes, The Open University