share pretest 2010 feedback & results
DESCRIPTION
SHARE Pretest 2010 Feedback & Results. Schedule. Schedule. What we've done since Paris. Further testing, revision and one (major) change of the instrument Sample Management System Translation IT meeting for agencies (Apr 23) Concepts, ex ante v ex post harmonization - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/1.jpg)
1
SHARE Pretest 2010
Feedback & Results
![Page 2: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/2.jpg)
2
ScheduleDate 1. Generic
QuestionnaireDesign,Programming, Testing
2. SMS anddata transfer
3. Country-SpecificQuestionnaireDesign,TranslationProgramming, Testing
4. Agencies,fieldworkprep,contracts
5. Fieldwork 7.Conferences,Meetings
8. Deadlines
20 – 21J ul 2010
Post-pretestmeeting inOdense
01 – 15Aug 2010
Area Teams,Country Teamsrevise genericand country-specificinstrument onpaper (->version 5)
MEA,CentERdatarevise SMS (->Version 5)
Area Teams,Country Teamsrevise genericand country-specificinstrument onpaper
15 – 31Aug 2010
CentERdataprogramsGeneric Version5
CentERdataprograms SMSVersion 5
01 – 15Sep 2010
MEA, AreaTeams,Country Teamstest GenericVersion 5
MEA, AreaTeams,Country Teamstest SMSVersion 5 (->Version 6)
MEA finishespreloaddatabase formain survey
16 – 30Sep 2010
CentERdataprograms CAPIFinal GenericVersion 6
CentERdataprograms SMSFinal GenericVersion 6
Surveyagencies testpreloaddatabase formain survey
Generic Version6 frozen forMain SurveyTraining
![Page 3: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/3.jpg)
3
Schedule01 – 07Oct 2010
Country teamstranslate finalchanges toCAPI & SMS inLMU
08 – 15Oct 2010
CentERdatacompilestranslatedinstruments
16 – 22Oct 2010
Country-teamstest,implement finalchanges inLMU;CentERdatacompiles finaltranslatedinstruments
22 Oct2010
FinalTranslatedCAPI and SMSVersion 6
27 – 28Oct 2010
TTT inGermany
02 Nov2010 - 30J un 2011
Fourth wave ofSHARE
-- Everybodyworks hard
![Page 4: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/4.jpg)
4
What we've done since Paris
• Further testing, revision and one (major) change of the instrument– Sample Management System– Translation
• IT meeting for agencies (Apr 23)– Concepts, ex ante v ex post harmonization
• TTT for new and old countries (May 25-26)– Organized and conducted by MEA– Mock interviews of new modules/questions– SMS– Preload– Gaining respondent cooperation– Proxy etc.
![Page 5: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/5.jpg)
5
Pretest goals
• 100 realized household interviews– at least 50 refresher households
• Test logistics in all phases of the fieldwork– includes proxy interviews if necessary– includes exit interviews where necessary– included interviews in institutions / nursing homes
where necessary– have drop-off for each respondent
![Page 6: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/6.jpg)
6
Pretest goals
• Testing the CAPI instrument– Do the structural changes to the CAPI work?– In particular new/much changed sections SN and CH– Is the translation/adaptation correct?– Where are we in terms of length?
• Testing the functionality:– SMS Client CAPI instrument– SMS Client SMS Sample Distributor– SMS SD CentERdata
• Get ready for the main survey!
![Page 7: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/7.jpg)
7
Qualitative Feedback
• All feedback received until July 18 (Sunday)
• 7 agencies, 6 countries– DK, ES, EE, FR, BN, IT– also received by now: CH, BF, NL, HU
• Going to forward to area teams– please deal with detailed comments
![Page 8: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/8.jpg)
8
Qualitative Feedback
• Gaining R cooperation– getting harder and harder to convince people to
participate– length of interview does not help– reluctance to answer personal questions to people
living in their community (IT)
![Page 9: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/9.jpg)
9
Qualitative Feedback
• Particular problems to gain cooperation– older R (afraid, easily tired)– working R (too busy)– R living in appartment buildings (difficulty getting to
door)– earlier refusals (i.e. who did refuse SHARELIFE) hard
to convince to be interviewed again– couples (two interviews are just too much in one
evening session)– older Rs often first asked children whether they
should participate --> special info aimed at children?
![Page 10: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/10.jpg)
10
Qualitative Feedback
• General complaints about length, repetitiveness and detail– tiredness– lack of interest– anger
![Page 11: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/11.jpg)
11
Qualitative Feedback
• Specific length problems identified– Introductions are too long, too much blablabla (sic!) -
time robbers (DK, FR)– Smoother way to do "all proxy interview" needed– Mental health module too long– second person refused because of length
• Brochures– helpful but should be more neutral (i.e. not linking low
education and obesity)– third wave results needed
![Page 12: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/12.jpg)
12
Qualitative Feedback
• Preload errors– year of last interview
• What was lacking– questions about personal interests and activities
• Unfolding bracket entry points were unrealistic (FR)
• A comprehensive QxQ should be available before the data collection
![Page 13: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/13.jpg)
13
Qualitative Feedback
• SN:– too early in the interview, resp do not feel comfortable– reluctance to give names/details of friends and relatives without
their permission (interview should be about the R)– people seem to forget people they live with– better shift after CF because it makes it easier to separate
couple Rs– hard to separate couples right at beginning of the interview when
insisted both should be present
• CH:– becomes confusing if too many children with similar
names/nicknames (ES)
![Page 14: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/14.jpg)
14
Qualitative Feedback
• DO– respondents, especially old did not like filling in (ES) – between 10 and 30 minutes (DK)– favorable reaction to have health care questions
(costs) on paper rather than CAPI– people got annoyed, questions seemed to be already
administered– people did not understand why they had to fill in DO
after such a long interview– illiterate respondents
![Page 15: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/15.jpg)
15
Qualitative Feedback
• Measurements:– are generally liked because they bring variation to the
interview (BN)– 3 test are too many (IT)– why can't interviewers choose when to perform test
(like in SHARELIFE)– The preferred section amongst several respondents,
they were very cooperative and interested during the physical tests
![Page 16: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/16.jpg)
16
Qualitative Feedback
• Financial questions:– too difficult, too detailed, too nosy!!! (sic!)– questions are too long; easier to understand when
you read them than when you hear them– respondents do not understand what is the purpose of
questions
![Page 17: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/17.jpg)
17
Data on which numbers are based
• All data received until July 20 (yesterday)
• 12 countries
– no data yet from DE, GR, HU, PT, LU(?)
• 844 household interviews
• Target was 100 hh i'views per country(reached in only 4 countries so far)
![Page 18: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/18.jpg)
18
Number of interviews by country
0
20
40
60
80
100
120
140
AT BE CH CZ DK EE ES FR IT NL SE SI
Single Couple
![Page 19: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/19.jpg)
19
Length
• Excluding all interviews shorter than 20 minutes and longer than 5 hours, the overall average and median length are:– 80 and 76 minutes for single interview, 83 and 79 if
weighted– 130 and 125 minutes for couple interview, 130 and
125 if weighted
• Note: this is pure interview time including interviewer observations but excluding general setup and coverscreen
![Page 20: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/20.jpg)
20
Length
• The averages do not tell us much about the variance– 35% of the total variance is between countries– as always, there are "long" and "short" countries
![Page 21: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/21.jpg)
21
Length by country, single hh
050
100
150
200
min
utes
CH NL AT FR EE BE SE DK ES CZ IT SIexcludes outside values
![Page 22: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/22.jpg)
22
Length by country, couple hh
5010
015
020
025
030
0m
inut
es
CH DK SE AT NL BE FR EE ES CZ SI ITexcludes outside values
![Page 23: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/23.jpg)
23
Length (Pretest v Main)
• Comparison of wave 2 timings between pretest and main survey– Single HH took 1.3 minutes longer in main
than pretest– Couple HH took 4.5 minutes longer in main
than pretest• Changes between main and pretest?
– Added Chair Stand and Peak Flow!– Added CASP and CES-D to AC module– Various minor cuts and additions
![Page 24: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/24.jpg)
24
Length (Pretest v Main)
• Accounting for CS, PF, AC additions– Single HH: 5.9 minutes shorter– Couple HH: 8.3 minutes shorter
• Yet again: huge between-country variance
![Page 25: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/25.jpg)
25
Pretest v Main(single HH - wave 2)
-40
-20
02
04
0M
ain
- P
rete
st
bf at de dk pl it bn es ch nl se grCountry
![Page 26: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/26.jpg)
26
Length by area
• Single / couple hh– Health: 34.8 / 65.6 min– Econ: 23.6 / 42.6 min– Social: 19.8 / 32.0 min
• Note: allocation of modules to areas does not allow for overlaps– e.g. CF, DN, HO
![Page 27: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/27.jpg)
27
Length by module0
510
1520
min
utes
CF EP PH EX HC HO AC MH SN CH BR AS WS GS DN CO PF CS SP FT IV HHexcludes outside values
![Page 28: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/28.jpg)
28
If to cut, where to cut?
• Expecting a too long interview, we have discussed in May with area teams a contigency plan– two round of discussion– designed to take on average 10 minutes off the CAPI
questionnaire
• achieved reasonable degree of agreement on what to cut
• scheduled some of the dropped item for next wave / in-between data collection
![Page 29: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/29.jpg)
29
If to cut, where to cut?
• Health– Drop Walking Speed/Chair stand– Skip HC including end of life care – with the exception of
• HC002 (seen a doctor in last 12 months), • HC003 (seen a GP in last 12 months), • HC004 (seen a specialist in last 12 months), • HC005 (which specialist), HC012 (been in Hospital in the last 12 months), • HC013 (How often in Hospital in last 12 months), • HC014 (total nights in hospital in last 6 months), • HC029 (been in nursing home last 12 months), and the
– Health insurance questions in drop-off?
![Page 30: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/30.jpg)
30
If to cut, where to cut?
• Social– Drop year of birth from SN loop– FT: drop FT004, FT006, FT011, FT013, FT015 etc.
large gifts or inheritances (amount and reason for transfer)
– HO071, HO072, HO073: Quality of neighborhood– SP: return to the single solicitation of help
given/received outside the household (i.e. do not distinguish between personal care and practical help)
![Page 31: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/31.jpg)
31
If to cut, where to cut?
• Econ– EP021, EP022, EP057, EP058: Responsibility for
supervising employees; how many– EP059: Opportunities to work after retirement age– EP065: Retirement been a relief or concern– HO035: Years living in community– HO038, HO039, HO040: Apart from vacations or brief visits,
do you regularly spend part of the year in another residence? Where is this residence located? In Which country
– HO011, HO012: ask only at baseline or if moved house since last wave...
![Page 32: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/32.jpg)
32
If to cut, where to cut?
• Econ– Drop all asset income question (AS005, AS009,
AS015, AS058) – replace by summary question– AS061 reason no bank account only at
baseline– Keep expectations module only for baseline– Drop financial literacy questions from EX
module– Drop detailed ISCO questions
![Page 33: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/33.jpg)
33
If to cut, where to cut?
• Scheduled for next wave / in-between wave data collection:– End of life decisions– Quality of living environment– Prevention– Financial literacy– Walking speed / Chair Stand / Other
measurement?
![Page 34: SHARE Pretest 2010 Feedback & Results](https://reader036.vdocuments.site/reader036/viewer/2022062409/56815025550346895dbe0e2f/html5/thumbnails/34.jpg)
34
Questions?
?