Making Social Work Count Lecture 3

An ESRC Curriculum Innovation and Researcher Development Initiative

Who is being studied?

Different samples and their implications for research

and practice in social work

Why are samples important?

• Perhaps the most important question to ask of any study is – who is being studied?– Who are researchers trying to study? (sample type)– Who did they actually study? (response rate)

• Understanding and interpreting findings depends on these questions

• More important for social work than most subjects– Much of psychology is based on US students (we know

more about this group than any other in history)– Social work tends to be “hard to research” groups

Why are samples important?

• Not just a research question – shapes our understanding of policy issues and responses

• Also allows us to critique stereotypes and is therefore important for anti-discriminatory and reflective practice

Why are samples important?

• Policy questions where samples matter:– Do children do badly in care?– Does parental alcohol misuse harm children?

• Simple stereotypes overheard:– “Children’s services should not give families s.17

payments as they will just come back for more.”– “Australian social workers are better than British

ones.”– “People are at risk of violence from mentally ill

people.”

Learning outcomes

Understand what a sample is

Understand the difference between

different types of sample

Appreciate the ways in which

different samples can influence

research findings

Understand the importance of

response rates for influencing

samples

Apply issues of sample bias in

research to anti-discriminatory

practice issues and stereotyping in

social work practice and policy

Defining the sample

• How does the study define its sample?• How would this affect the sample it is focussed

on?• For instance, interested in parental substance

misuse: what is difference between using alcohol, misusing it or being dependent?

• “Misuse” is alcohol related harm to self or others – how can this be identified and defined?

Defining a sample: example of people who misuse alcohol

All People

Use AlcoholMisuse Alcohol

Dependent

Defining a sample: example of people who misuse alcohol

All UK People

63 million

Dependent

200,000

ONS, 1998

Defining a sample: may be even more complicated than that

All People

Dependent

Winston Churchill

“When I was younger I made it a rule never to take strong drink before lunch. It is now my rule never to do so before breakfast.”Dependent drinker through 1930s and 1940s – but managed to win WWII … and a Nobel prize for literatureWhether he was in a sample would depend on how you defined misuse / dependency

Whole population sample

All the people to be studied

Whole population sample

• This is rarely possible in research – and almost unheard of in social work research

• Why not?– In practice most “samples” are too large – Often getting hold of people is not possible– Sometimes defining the sample is very difficult,

and different definitions lead to different samples and different findings

You can’t study all the sample

All Misuse Alcohol

Sample

Example of children of drug users

First, how do you define it? What drugs? Used how often? Etc• Different definitions lead to very different samples – not just size, but

also nature of sample

Second, the size of the sample is too large for a 100% sample• The Hidden Harm (2003) report extrapolated from treatment findings

to estimate 200-250,000 children with a parent who has a drug problem (about half living at home) – you can’t include them all in a study

Third, how do you get hold of them?• Like most social work populations – the issue is complex, surrounded by

stigma and this makes it hard to identify children• In practice in social work research often those known to services

Sampling and soup

• Is your sample “representative” of the soup?

• Is it properly mixed? • Did you taste enough? • Is it burnt at the

bottom? • Did you get a crouton?

Sampling

Probability Sampling

• Each member of the population has a known non-zero probability of being selected• Highly representative if all subjects participate - the ideal

Non Probability Sampling

• Members are selected from the population in a non-random manner• Less likely to be representative of the rest of the population

Probability Sampling

The ideal sampling for drawing conclusions about a whole population is usually probability sampling, because it increases the likelihood of obtaining samples that are representative of the population– Random - Each individual in the population of interest has an

equal likelihood of selection. Not possible without complete list of population members and potentially uneconomical to achieve

– Stratified – A mini-reproduction of the population. Before sampling, the population is divided into characteristics of importance for the research. For example, by gender, social class, education level, religion, etc. Then the population is randomly sampled within each category or stratum

Probability Sampling and Soup

• Random – you take enough of the soup from different parts of the pot to be pretty sure you have got a good taste

• Stratified – you make sure you had some liquid, noodles … and a couple of croutons

Non probability sampling

Not truly representative of the population but the validity can be increased by trying to approximate random selection, and by eliminating as many sources of bias as possible– Purposive - with a purpose in mind. A non-representative

subset of some larger population. Decisions concerning the individuals to be included in the sample are taken based upon a variety of criteria which may include specialist knowledge of the research issue, or capacity and willingness to participate in the research e.g. age range, ethnicity etc.

– Convenience - select first available subject who meets criteria. A matter of taking what you can get - an accidental sample e.g. Use of students (available and convenient!)

Probability Sampling and Soup

• Purposive – you were particularly interested in croutons – so mainly tasted them

• Convenience – you did not have a ladle so just use a teaspoon…

Response rates

• Whatever your sample – the response rate is the percentage of those you tried to get data for who you actually got data for

• Why is this important?

• Why particularly important for social work?

Response rates

• The people who did not take part are very likely to be different from those who did take part – More serious problems? – Less strong opinions?– Other issues

• Particularly big issue for social work – as our client groups tend to have more difficulties and are often effectively “silenced” e.g. learning difficulties, victim of violence, little or no English

Response rates

• Not getting hold of certain groups may distort the findings…

• Only including those who loved (or hated) a service

• Excluding those in crisis or with serious problems – and therefore underestimating an issue or overestimating impact of service

Policy questions

Do children do badly in care?• Government Green Paper Care Matters relied

largely on comparison of outcomes for children who left care (16-18) with the general population

• Concluded care failing• Is this a valid sample to conclude this for?

DfES (2006)

Policy questions

Is care bad for children?

Do children do badly in care?

Does ‘care’ work?

The government DfES) argued that:• Children “in care” or who have left care do much worse than

general population on range of measures – Poor educational results (43% get a GCSE compared to 95%) (SEU,

2003; DFES, 2006); – Poor health outcomes (DH, 2002); – are four times more likely to be unemployed and 60 times more

likely to be sent to prison (UK Joint Working Party on Foster Care, 1999).

• We should therefore reduce the number of children in care...

• What are the problems with this conclusion?

Some problems

1. What does “care” mean? Do these figures include all children who have been in care? If not, how does this influence them?

2. Is comparison with the “general population” reasonable?

Who are children in care?

• 60,300 in March 2006 (DfES, 2006)• Very disparate group– From babies given up for adoption… – to 15-year olds whose parents can not cope with their

difficult behaviour– Wide range of placements and other services

• A lot of movement within the group– Small minority remain “in care” for more than two years– Most return home or move to permanent alternatives

• Impossible to generalise about this disparate population– Care therefore not “good” or “bad”

Care and leaving care

• Most children in care at 16 entered care as teenagers

• They therefore spent most of childhood at home

• Others enter for a short time – maybe a year or two from birth. They are not in these comparisons

M4, care and samples

• Congestion around the M25 does not mean the whole motorway is “failing”

• Most of the traffic at that point is relatively local traffic (eg Heathrow)

• Large stretches are not congested• Research point of view need to

either:• Focus study e.g. the last 20

miles/ children in care over 16• OR sample across the whole

motorway

Policy questions

• Does parental alcohol misuse harm children?– What type of “misuse”? What level of

“harm”?– What sample?

• Velleman and Orford (1999) found that with a sample of young adults whose parents misused the impact was less than might be thought – Sample recruited by advertising for

volunteers• Forrester and Harwin (2008) found many

social work cases involved PSM and the children affected by alcohol misuse did poorly– Sample of families allocated a social

worker

Stereotypes and samples

• “Children’s services should not give families s.17 payments as they will just come back for more.”

• “Australian social workers are better than British ones.”

• “People are at risk of violence from mentally ill people.”

Stereotypes and samples

• “Children’s services should not give families s.17 payments as they will just come back for more.”– Forrester (2006) found most families did not come back,

indeed was issue LEAST likely to lead to re-referral – but a small number came back repeatedly

• “Australian social workers are better than British ones.”– Based on knowledge of small number of Australian

workers in the UK – but workers who travel around the world likely to be very different from those who stay at home

Stereotypes and samples

“People are at risk of violence from mentally ill people.”

– Based on small number of “high profile” killings

– When there are violent killings by people who are not mentally ill the press do not talk about the dangers of “sane” people…

– Statistically people with mental illness are six times more likely to be victims of violence (BBC, 2012)

– They are slightly more likely to be violent – but this is solely because they are more likely to misuse drugs or alcohol (BMJ, 2010)

Stereotyping

• A complex psycho-social phenomenon

• One element is inappropriate generalisation from small samples…

• An understanding of research methods contributes to a well-educated scepticism about stereotypes

Samples

• Our knowledge is almost always partial• One key limitation is what sample it is based

on• Understanding different types of samples

helps us critically engage with research, critique policy and think about our own and others’ stereotyping

Learning outcomes

Are you able to:• Define what a sample is • Identify different types of sample• Appreciate the ways in which

different samples can influence research findings

• Understand the importance of response rates for influencing samples

• Apply issues of sample bias in research to anti-discriminatory practice issues and stereo-typing in social work practice and policy

References

• Advisory Council on the Misuse of Drugs (2003) Hidden harm: responding to the needs of children of problem drug users, Report of an inquiry by the Advisory Council on the Misuse of Drugs, Home Office; London

• BBC (2012) Mentally ill 'at high risk of being victim of violence, http://www.bbc.co.uk/news/health-17182626

• BMJ (2010) Violent crime among mentally ill people is due more to substance misuse than inherent factors341:c4909

• Department for Education and Skills (2006) Care Matters: Transforming the Lives of Children and Young People in Care, HMSO; Norwich

• Forrester, D. (2006) Patterns of Re-referral to Social Services: A Study of 400 Closed Cases, Child and Family Social Work , 12 (3), pp 286-299

• Forrester, D., Cocker, C., Goodman, K., Binnie, C. and Jensch, G. (2009) What is the impact of public care on children’s welfare? A review of research findings and their policy implications, Journal of Social Policy, 38, 3, pp439–456

• Forrester, D. and Harwin, J. (2008) Outcomes for children whose parents misuse drugs or alcohol: a 2-year follow-up study, British Journal of Social Work, 38, December 2008, pp 1518 - 1535

• Office of National Statistics (1998), How many people drink?, Summary Report accessed on NHS Avon website: http://www.avon.nhs.uk/alcohol/the_facts.htm

• Velleman, R. and Orford, J. (1999) Risk and Resilience. Adults who were the children of problem drinkers, OPA: Amsterdam