Key issues for longitudinal research design:

Lessons from the Growing Up in Scotland study

Paul Bradshaw

Definitions of longitudinal research

• In the broadest sense longitudinal research involves the follow up of any set of entities in which changes can be observed over time. • Individuals• Households• Institutions such as hospitals or schools• Nations

• Most commonly focussed on individuals • No recognised definition of what period or what

number of follow-ups constitutes ‘longitudinal’ • Aim is usually to determine causes and processes

which lead to change and/or to particular outcomes

The UK experience

• The UK has a long, and highly regarded, history of (particularly quantitative) longitudinal social enquiry

• A number of ongoing internationally renowned longitudinal studies providing information on the life histories of people as they move from birth to old age

• Current QNLR being undertaken in Britain includes:• Birth cohort studies• Age cohort studies• Family/Household panel studies• Area studies • Census-based studies

The challenges of longitudinal research

“The success of a longitudinal study depends on stable leadership from a committed principal investigator and a team of highly skilled researchers”

(Bynner et al, 2006)

SampleData collection

Duration (& relevance)

EthicsCost

Data analysis

The Sample

Two main issues:• Sample design

• Non-response and attrition

Sample design considerations

• What is the population of interest?• All vulnerable children?• Sub-groups of interest? E.g. with particular characteristics –

age, family circumstances, area where they live, interventions received

• What do you want to be able to say about them?• Use sample to generalise to population?• Compare outcomes and experiences between children in

different groups?

• How will you find/recruit them?• Is there a sampling frame with the info you need?• Will you need the help of an agency/organisation to recruit?

Sampling precision (1)• When using a sample, the data produces ‘estimates’ of what the

real value may be in the population. The precision of these estimates is affected by:• Sample size• Sample clustering

• Measuring sample precision:• Use ‘confidence intervals’ and ‘standard errors’

• Confidence interval (CI)• Typically 95% meaning 95 times out of 100 this interval will capture the

true population value that we are trying to estimate• Expressed as e.g. 64% (+/- 6%) suggesting the true value is

somewhere between 58% and 70% • Differences will usually have to be of the magnitude of the CI for it to be

statistically significant – e.g. in example above, an increase or decrease of around 6% would be necessary.

• Smaller sample sizes produce larger CIs requiring greater change for statistically significant differences to be detected

Sampling precision (2)Sample numbers required in each group to demonstrate

significant differences (with a power of 0.8)

Size of difference between groups (percentage points)

Lower of the two percentages:

5 7 10 15

10 687 374 200 101

25 1252 653 330 153

50 1566 797 389 170

75 1095 541 251 101

90 436 195 75 -

GUS response and attrition rates

No. cases issued

No. cases achieved

Response rate

As % of sw1 achieved

Birth cohort

Sweep 1 6583 5217 79% 100%

Sweep 2 5217 4512 86% 86%

Sweep 3 4665 4193 90% 80%

Sweep 4 4394 3994 91% 77%

Child cohort

Sweep 1 3605 2859 79% 100%

Sweep 2 2859 2500 87% 87%

Sweep 3 2599 2332 90% 82%

Sweep 4 2460 2200 89% 77%

GUS: Sweep 5 response by age of mother

70

75

80

85

90

95

% a

chie

ved

Under 25 25-29 30-34 35-39 40+

Mother's age at child's birth

Non-contact and refusal rates

No. cases issued

No. non- contact

% non-contact

No. refused

% refused

Birth cohort

Sweep 2 5217 162 3% 316 6%

Sweep 3 4665 102 2% 268 6%

Sweep 4 4394 93 2% 203 5%

Child cohort

Sweep 2 2859 100 3% 153 5%

Sweep 3 2599 51 2% 142 5%

Sweep 4 2460 51 2% 136 6%

Sample maintenance strategies

• Keeping in touch• Tracing• Keeping respondents informed• Valuing and appreciating respondents (including, but

not restricted to, use of incentives)• Boosts of key sub-groups of interest and on-going

sample refreshment• Making reasonable demands

Sample maintenance: keeping in touch• Potentially willing respondents can be lost by virtue of

moving home. • Important to establish effective procedures for

obtaining updated contact information.• Techniques include:

• Collecting as much contact information as possible at first contact including telephone/mobile numbers and e-mail

• Information about ‘stable contacts’ – someone who knows the respondent and would know where they had moved to

• Regular mailings with return address – undelivered items act as address checks ahead of fieldwork

• Updates from administrative and service databases

Sample maintenance: keeping respondents informed• In any research project, it is considered important to

provide good information to respondents about the purpose and nature of research

• Respondents must understand the longitudinal nature of the study so that they also recognise why they are repeatedly visited and why they are irreplaceable

• Many studies adopt a ‘branding’ approach – a name and logo that clearly identifies the study and which is easily recognisable by respondents

• Websites can be extremely useful in providing more detailed information

Sample maintenance: valuing and appreciating respondents• If respondents are going to continue to take part in a

longitudinal study they need to feel that their time and effort is valued and is worthwhile.

• This can be demonstrated through:• The interviewer• Thank you letters sent after each interview• Providing evidence that survey findings have been used• Illustrating the impact they have had on government policy

• Use of incentives• Non-financial rewards – gifts such as pens, fridge

magnets, calendars

Being part of NCDSThere’s one thing I’m guaranteed, all of my entire life,

is one birthday card.I’ve always liked being part of it, I’ve always enjoyed being part of it because it’s different and I’m quite

proud of it really.I think I could almost put this in as a landmark in my

life.I haven’t kidded on about anything in my life, warts

and all, I’ve been honest and I’ve said it all. ….. I think, you know, it does make you feel special in

certain ways, like really…..I haven’t ever seen anything negative in it actually, not anything negative at all…..I’ve always felt comfortable

and I always know that if I don’t want to answer a question I don’t have to………… I think it’s just been

fascinating

Recognising the importance of NCDS

I can remember becoming part of the NCDS, I didn’t understand it.... I think I’ve just grown to

understand it more as I’ve got older and the importance of it and how it’s helping everybody

really, that’s what I think.Well, I think to a certain extent you’ve got to say it’s mainly for helping others, you know. Like you say it’s no benefit to me to do it, but then again it’s no skin off my nose not to do it, so. It’s one of those things

like, you know….I don’t see a reason [not to]……I think it’s interesting that they’re actually following all these people, right through their life and ……they find out comparisons. Aye, I think it’s really…. that’s why I take the bother to, aye, come. I want to be part

of it because I’ve been in it all my life.

Data collection: what to ask and when• Decisions on content of data collection require good

planning – important to ask the right questions at the right time• Take a ‘longitudinal’ view:

– What ‘stage’ are your subjects at? – What stage will they reach? – What situations, characteristics or contexts might you want to

compare between those two stages?– What might be significant at stage 1 when you are comparing

outcomes at later stages?

• Decisions on content will usually be theory or hypothesis-based

• What if you miss something?

Data collection: intervals between fieldwork• How often do you follow-up your sample? This is

dependent upon:• Respondent burden• Developmental stages, processes or transitions that you are

interested in• Budget

• There is no recognised nor legitimate pattern – it’s largely dependent upon the objectives and focus of the research

Study

Child’s age in years

0 1 2 3 4 5 6 7 8 10 11 16

NCDS (1958)

BCS (1970)

MCS (2000)

GUS BC (2005)

GUS CC (2005)

ALSPAC (1991)

Follow-up intervals of selected UK child cohort studies

Sw1 Sw2 Sw3 Sw4 Sw5 Sw6(2005/6) (2006/7) (2007/8) (2008/9) (2009/10)

BC only

(2010/11)

BC only

Main carer

Main carer Main carer Main carer Main carer Main carer

Partner

Child height & weight

Child height & weight

Child height & weight

Cognitive assessmts

Cognitive assessmts

Health

records

Health records

Health

records

Health records

Health records

Health records

School records

School records

GUS: Sources of data

Data analysis

• Benefits of longitudinal data• Measuring change over time:

– Flows into and out of ‘states’ (e.g. poverty, unemployment, being looked after)

– The effects of change, or of state durations, on outcomes

– Impact of interventions

– ‘Individual’ development

• Temporal ordering of events• Improved control for omitted explanatory variables • Improved control for the effects of previous states• Exploring the effects of ageing and cohort membership

Data analysis

• Drawbacks of longitudinal data• Data management

– Complex structure/relations

– Complex variable/samples

– Resultant file and variable management requires training and skills of good practice

• Software issues – Complexity of methods

– Some methods only available via specific software packages

Longitudinal models

• Two main modelling approaches in social science research:• Event history analysis, time to an event

(Also known as: duration analysis; survival analysis; failure time; duration economics; hazard modelling)

• Panel data analysis– Regression models suitable for repeated observations

– Time generally conceptualised as being discrete

– Extension of standard regression models

– Closely related to multilevel modelling

– (Simple methods can also be used)

Ethical issues (1)

• Confidentiality, data security and data access• Safeguarding confidentiality through

– Restricting the data that are released

– Controlling the arrangements under which potentially disclosive data is released

• Data access/release– Providing data for researchers to analyse on their desktop with

the level of sensitive detail restricted

– Providing more sensitive data under a special licence

– Remote access where raw data is never released and analyses are run in house on behalf of researchers who receive edited outputs

– Safe settings where researchers are required to visit a protected site where access to the data is carefully managed.

Ethical issues (2)

• Informed consent• What is ‘informed consent’?

– Respondents must understand that the project is longitudinal in nature and what that means for them

– They are free to withdraw at any time from the project as a whole or any aspect of it

• One-off or repeated consent?– No universal practice, but

– Some form of repeated consent would normally be involved

Summary

• Longitudinal research in any methodological discipline presents a set of core challenges

• The key issue is to consider, in detail, how the fundamental temporal nature of the project affects the basic aspects of its design including:• The sample• Data collection• Data analysis• Ethics

• Requires a more complex design, but a good design will produce tremendously valuable data