FACULTY OF ARTS & SOCIAL SCIENCES

Widening ParticipationEvaluation considerations

Department of Gender and Cultural StudiesTess Lea QEII Fellow

Working with young or marginalised people 1

What’s the best approach?

Comes down to:

› What do we want to do?

› How much time have we got?

› Who have we got?

› What resources do we have?

› Who is our target audience?

› Best imaginable outcome for results?

Some techniques

› Interviews

› Observations

› Focus Groups

› Small questionnaires with crisp, manageable questions

› Review of documents, records, media

› Limited archival research

› Performance based methods

- Interactive role playing

› Follow up (cohort) surveys to map changing decisions/perceptions

› Participant led data gathering

Pros and cons

› Focus group

- Economical but risk getting dominant ‘group think’

› Observations

- less intrusive but takes longer saturation to get to patterns

› Interview format/interactive performance based

- flexible and rich but time intensive

› Survey

- larger potential population

- useful for longitudinal or cohort analyses

- not ideal for understanding detail or social complexity

- Also dealing with questionnaire-fatigued target groups

5

Sampling: size versus depth

› If want larger sample, must tame the monster: ask mainly yes/no questions; have only one open-ended question

› If want detail, limit the sample

› Most reliable method for young people is face to face interviews and participatory research

› In any case, design data analysis before you administer instrument of choice

- Clarify what data you actually want and ask for it, not what you think should be asked

- Involve target group in the revisions

- Anticipate and design the data analysis tools (the database and coding systems) before going live

7

› Research shows that poorly worded questionnaires bore young people, or provokes conventional answers – needs to be attractive and easily interpreted

› Also important to avoid having it be another piece of school work

› True consent not possible in power structures

› Thus even more essential instruments are well designed

- Conduct a small trial: test the questions – do people respond as you had hoped? Ask young people how they found it. Gauge reactions and modify

- Limit time required to 15 minutes or less

- Be aware of biased terms – e.g. ‘assistance to the poor’ (positive) versus ‘welfare’ (negative)

- Keep it uncluttered – less is more

- Think about the order – make it a logical progression, not choppy

What are the right questions?

› What is the age group? Gender? Social background?

› What data story do you need to be able to tell and with what ambition (For further funding? Publications?)

- Can’t over-state need for clarity about purpose

› Vignettes of transformational stories are excellent

- Write with journalistic flair

› Combine these with other impact data

- The more ‘factual’ and repeatable over time, the better

8

Youth oriented evaluations that yield convincing data

› Given reality of need to maximise responses in shortest time frame and at least cost, how to get the holy grail of youth-centred evaluations that yield convincing data?

› Many interventions presume a deficit (in esteem, in awareness, in comfort) and rush to fill that, without any primary research

› Foregrounding the students and making the evaluations as student-centred as the interventions vital especially given academic persona as another teacher/adult

› Be creative

9

Brainstorming time 2

Widening Participation

Are we making an impact? How known?

What is meant by/needed for sustainability?

Is it worth expanding/scaling?

What do we not want to be doing more of?

Brainstorming time

› Group discussion of project status in relation to the timelines and matching methods to questions to desired outcomes

› Contact details: tess.lea@sydney.edu.au; 9351 6777