iccs marker training hamburg july 2008 final note on marking reliability marking report from windem...
TRANSCRIPT
![Page 1: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/1.jpg)
ICCSMarker Training
HamburgJuly 2008
Final note on marking• Reliability marking report from WinDEM will
include record of scores for double-marked items, AND marker IDs for first and second marking BUT
• This will only work if the same marker marks all (2 or 3) item in a booklet (i.e. the ID is linked to the booklets)
• Marking by booklet is no problem• Marking by item requires you to make sure
that the markers score the same booklets (they will need to return to booklets to score most recently trained items)
• We can help make sense of the output if you need it.
![Page 2: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/2.jpg)
Coding of parental occupations
ICCS Marker Training
Hamburg, July 2008
![Page 3: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/3.jpg)
ICCSMarker Training
HamburgJuly 2008
Content of presentation
• Open-ended questions in student
questionnaire
• Overview of coding requirements
• Coding tools
• Coding procedures
• Rules for coding
![Page 4: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/4.jpg)
ICCSMarker Training
HamburgJuly 2008
Questions on parental occupation
• The ICCS student questionnaire includes two pairs of open-ended questions– Mother’s occupation– Father’s occupation
• Open-ended responses need to be coded• For the analysis, the classified occupations
will be recoded into status scores (International Socio-economic Index)
![Page 5: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/5.jpg)
ICCSMarker Training
HamburgJuly 2008
Open-ended questions in StQ
![Page 6: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/6.jpg)
ICCSMarker Training
HamburgJuly 2008
Response codes using ISCO classification
• Framework recommended by International Labour Organisation (ILO)
• Currently being updated to ISCO-08
• ICCS will have to use ISCO-88
• Some modifications of ISCO-88 classification!
![Page 7: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/7.jpg)
ICCSMarker Training
HamburgJuly 2008
Coding tools
• ISCO manual (ISCO-88) published by ILO in 1990– http://
www.ilo.org/public/english/bureau/stat/publ/index.htm
– Also available in French or Spanish (http://www.ilo.org/public/english/bureau/stat/isco/isco88/major.htm )
• Countries may also use available national occupational classifications– BUT codes have to be converted into
ISCO-88 afterwards
![Page 8: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/8.jpg)
ICCSMarker Training
HamburgJuly 2008
Valid reasons for using national classifications
• No translation of ISCO manual into country’s language available
• Centre has coders who are familiar and experienced with national classification
• Using national classification makes data more useful for national analysis
![Page 9: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/9.jpg)
ICCSMarker Training
HamburgJuly 2008
ISCO-88 classification
• Structure– Major groups (first digit)– Sub-major groups (second digit)– Minor groups (third digit)– Unit groups (fourth digit)
• Mapping to occupational status– International Socio-economic Index (SEI)– Recoding of unit groups to SEI scores
![Page 10: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/10.jpg)
ICCSMarker Training
HamburgJuly 2008
Modifications of ISCO-88 - 1
• General Managers’ classified as ‘[Small Enterprise] General Managers’ and ‘Department Managers’ as ‘[Large Enterprise] Department Managers’
• Several categories of armed forces personnel (including similar civilian occupations)
• Distinction between ‘academic teachers’ (2321) and ‘vocational teachers’ (2322)
![Page 11: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/11.jpg)
ICCSMarker Training
HamburgJuly 2008
Modifications of ISCO-88 - 2
• ‘Traditional Chiefs and Heads of Village’ (1130) was modified to cover all local (as opposed to national) officials.
• Distinguished ‘Oilers and Greasers’ from other mechanics and fitters
• Other codes for very broad categories:(1240) ‘Office managers’, (7510) ‘Non-farm manual foremen and supervisors’, (7520) ‘Skilled workers/artisans’, (7530) ‘Apprentices’, (8400) ‘Semi-skilled workers’
![Page 12: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/12.jpg)
ICCSMarker Training
HamburgJuly 2008
Modifications of ISCO-88 - 3
• Additional codes:– 9501 for home duties– 9502 for student– 9503 for social beneficiary (e.g.
unemployed, retired, etc.)– 9504 for “I don’t know” and similar
responses– 9505 for vague responses
![Page 13: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/13.jpg)
ICCSMarker Training
HamburgJuly 2008
General coding instructions
• Only ISCO-88 codes should be submitted
• Questionnaire responses to be entered into WinDEM– New for main survey!– replaces EXCEL template used in the
field trial
• Demonstration how to enter data in WinDEM will follow...
![Page 14: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/14.jpg)
ICCSMarker Training
HamburgJuly 2008
Coding procedures
• Enter data into WinDEM spreadsheet
• Sorting function for occupations by job title
• Generally, coding in sorted spreadsheets is more efficient and reliable than direct coding of hard-copies
![Page 15: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/15.jpg)
ICCSMarker Training
HamburgJuly 2008
Rules for coding - 1
• Coders should not interpret according to own interpretations but follow instructions
• Coders should only use information provided by the two open-ended responses
![Page 16: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/16.jpg)
ICCSMarker Training
HamburgJuly 2008
Rules for coding - 2
• Begin by looking at job title (Q6a/Q8a)– if job title is sufficient for coding select
appropriate ISCO code– if job title is not sufficient, use job
description to find appropriate code
• If more than one job title is provided, select first job for coding unless job description indicates otherwise
![Page 17: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/17.jpg)
ICCSMarker Training
HamburgJuly 2008
Rules for coding - 3
• Every pair of responses should be coded to the most detailed level possible (4 digits) – If not possible, closest aggregated level should
be selected (e.g. “teacher”)– Aggregated codes should be revised by
supervisor of coding procedures
• National centres should always apply the rules described in ISCO Manual (ILO, 1990)– “Approaches to some specific issues” – “Notes to some particular occupations”
![Page 18: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/18.jpg)
ICCSMarker Training
HamburgJuly 2008
National coding instructions - 1
• National coding index should be supplemented with – National ICCS Coding Instructions
• When using national classifications instead of ISCO-88– add specific coding instructions for
modified occupational categories– add any other necessary coding
instructions for ICCS coding
![Page 19: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/19.jpg)
ICCSMarker Training
HamburgJuly 2008
National coding instructions - 2
• Try to obtain national coding instructions from – relevant office for statistics in your country– (or) International Labour Organisation
(ILO)
• If no external version can be obtained, instructions will have to elaborated based on ICCS guidelines– Field trial data should then be used as a
basis for coding index for the main study
![Page 20: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/20.jpg)
ICCSMarker Training
HamburgJuly 2008
Coder training and queries
• Training material– National ICCS coding instructions– Should be obtained from relevant statistical body
in country and adapted for ICCS purposes
• In case of coding problems– First try to resolve at national centre between
coder and supervisor– If it cannot be resolved try to use other national
expert advice– As a last resort, contact ISC ([email protected])
![Page 21: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/21.jpg)
ICCSMarker Training
HamburgJuly 2008
National expertise
• Try to engage expert in national labour statistics
• This person should not be directly involved in coding but give advice to supervisor in especially difficult cases
• The ISC will try to help but national experts will have greater knowledge regarding country-specific coding problems
![Page 22: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/22.jpg)
ICCSMarker Training
HamburgJuly 2008
Coders and coding - 1
• ISCO coders should be:A.Literate and cooperative.
B.Clerically accurate and careful.
C.Willing and able to follow detailed instructions conscientiously.
D.Not likely to alter procedures in order to reduce the amount of work.
E.Persistent and willing to work steadily for long periods.
![Page 23: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/23.jpg)
ICCSMarker Training
HamburgJuly 2008
Coders and coding - 2
• Screening test may used during recruitment to check some of the criteria recommended by the ILO
• It is recommended to have two coders with one supervisor
• Ensure support from specialist with experience in this area (if available)
• Training should mainly consist of practical instruction under close supervision
![Page 24: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/24.jpg)
ICCSMarker Training
HamburgJuly 2008
Double-coding
• When training coders national centres may want to use double-coding of sub-set of responses
• Can be used at the beginning to identify problems and
• At the end for a final measure of agreement.
![Page 25: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/25.jpg)
ICCSMarker Training
HamburgJuly 2008
Submission of occupational data
• Ensure that all students have valid ISCO-88 codes
![Page 26: ICCS Marker Training Hamburg July 2008 Final note on marking Reliability marking report from WinDEM will include record of scores for double-marked items,](https://reader036.vdocuments.site/reader036/viewer/2022062409/5697c00f1a28abf838cca4c7/html5/thumbnails/26.jpg)
ICCSMarker Training
HamburgJuly 2008
Questions or comments?