Measuring Disability in a Measuring Disability in a Survey or Census Context: Survey or Census Context:

Parallel Work Advancing the Parallel Work Advancing the FieldField

Barbara M. Altman, Ph.D.Barbara M. Altman, Ph.D.

Disability Statistics ConsultantDisability Statistics Consultant

Active Active MeasurementMeasurement Efforts Efforts

Washington Group – Measurement on an Washington Group – Measurement on an international levelinternational level

ISDS – Federal efforts to prepare for improved ISDS – Federal efforts to prepare for improved data collectiondata collection

ACS – Development of improved questions ACS – Development of improved questions for Census Surveyfor Census Survey

Measurement Issues Measurement Issues Issue 1 - Addressing the problems of the wide Issue 1 - Addressing the problems of the wide

ranging variety of questions and wording.ranging variety of questions and wording. Issue 2 – Understanding the nature of the conceptual Issue 2 – Understanding the nature of the conceptual

model serving as a basis for definition and model serving as a basis for definition and measurement. For example, a theoretical or measurement. For example, a theoretical or legislative model.legislative model.

Issue 3 – How does the Issue 3 – How does the purposepurpose of data collection of data collection dictate the way the conceptual element from the dictate the way the conceptual element from the model is operationally defined? model is operationally defined?

Issue 4 - Is the measurement of a ‘true’ population Issue 4 - Is the measurement of a ‘true’ population feasible or desirable? Is there “one” true number?feasible or desirable? Is there “one” true number?

The Washington Group on The Washington Group on Disability StatisticsDisability Statistics

In June of 2001, the UN International Seminar In June of 2001, the UN International Seminar on the Measurement of Disability on the Measurement of Disability recommended the development of principles recommended the development of principles and standard forms for global indicators of and standard forms for global indicators of disability to be used in censuses disability to be used in censuses

There was a broad consensus on the need for There was a broad consensus on the need for population based measures of disability for population based measures of disability for individual country use and for international individual country use and for international comparisonscomparisons

The Washington Group was set up to: The Washington Group was set up to:

Foster international cooperation in the area of Foster international cooperation in the area of health and disability statisticshealth and disability statistics

Develop disability measures suitable for censuses Develop disability measures suitable for censuses and surveys that will provide basic information on and surveys that will provide basic information on disabilitydisability

Untangle the web of confusing and conflicting Untangle the web of confusing and conflicting disability estimatesdisability estimates

Washington Group: ObjectivesWashington Group: Objectives

Develop a small set/s of general disability Develop a small set/s of general disability measures for use in Censuses measures for use in Censuses

Recommend extended set/s of items to Recommend extended set/s of items to measure disability as components of measure disability as components of population surveys / supplementspopulation surveys / supplements

Address methodological issues associated with Address methodological issues associated with disability measurementdisability measurement

Current International and National Current International and National SituationSituation

Currently national censuses in developing Currently national censuses in developing countries use one of three types of questions countries use one of three types of questions that provide widely differing estimates of that provide widely differing estimates of national prevalence of disability:national prevalence of disability:

The three types of questions include:The three types of questions include: Generic question about the presence of a conditionGeneric question about the presence of a condition Generic questions about the presence in the household of a Generic questions about the presence in the household of a

person with a disability followed by a list of impairmentsperson with a disability followed by a list of impairments Checklist of impairmentsChecklist of impairments

Current problems nationallyCurrent problems nationally Question domains and wording varied across surveys and Question domains and wording varied across surveys and

census –ability to crosswalk surveys limitedcensus –ability to crosswalk surveys limited Many questions in use predate improved theoretical Many questions in use predate improved theoretical

approachesapproaches

Moving from CMoving from Conceptoncept to D to Definitionefinition to to MMeasurementeasurement: : The Conceptual ModelThe Conceptual Model A conceptual model of the key components A conceptual model of the key components

and their relationship is a basic tool for and their relationship is a basic tool for developing or selecting questions.developing or selecting questions.

WG selected the ICF as the conceptual model:WG selected the ICF as the conceptual model: Common point of reference Common point of reference Provides a common vocabulary, classifies Provides a common vocabulary, classifies

domains within conceptual areasdomains within conceptual areas Does not provide an operational definition or a Does not provide an operational definition or a

way to measure the conceptsway to measure the concepts

Moving from Moving from ConceptConcept to to DefinitionDefinition to to MeasurementMeasurement: : The Definitional ParadoxThe Definitional Paradox

There is no single There is no single operationaloperational definition of disability definition of disability Different operational definitions lead to different estimatesDifferent operational definitions lead to different estimates The question you are trying to answer (The question you are trying to answer (the purposethe purpose) will ) will

determine which determine which operationaloperational definition to use definition to use Need to Need to understand the choicesunderstand the choices that are being made when a that are being made when a

purpose and an purpose and an operationaloperational definition are chosen definition are chosen Need to Need to understand the choicesunderstand the choices that are being made when that are being made when

time, expenses and respondent burden time, expenses and respondent burden limitlimit number of number of questionsquestions

Purpose of Data CollectionPurpose of Data Collection

3 major classes of purposes at aggregate level3 major classes of purposes at aggregate level Service ProvisionService Provision Monitoring functioning in the populationMonitoring functioning in the population Assess equalization of opportunitiesAssess equalization of opportunities

2 criteria for selection of a purpose2 criteria for selection of a purpose Relevance—particularly for policy makers Relevance—particularly for policy makers

and program officialsand program officials FeasibilityFeasibility

Purpose: Service provisionPurpose: Service provision

Seeks to identify those with specific needs, Seeks to identify those with specific needs, usually the most serious problemsusually the most serious problems

Requires detailed information about the Requires detailed information about the person and the environmentperson and the environment

Influenced by the organization and Influenced by the organization and structure of service organizations within a structure of service organizations within a particular cultureparticular culture

Purpose: Monitoring functioning in the Purpose: Monitoring functioning in the populationpopulation

• Seeks to identify Seeks to identify all those with all those with activity or activity or participation participation limitationlimitation

• Response Response comparability comparability problematic since problematic since participation is participation is culturally and culturally and environmentally environmentally determined determined 0

5

10

15

20

25

Pro

port

ion (

%)

2005 2010 2015 2020 2025

Population reporting work limitation

Purpose: Equalization of opportunitiesPurpose: Equalization of opportunities

• Seeks to identify Seeks to identify all those at all those at greater riskgreater risk than than the general the general population for population for limitations in limitations in activity or activity or participationparticipation

• Disability as a Disability as a demographicdemographic 0

30

60

90

Pro

port

ion (

%)

Nondisabled Disabled

% Employed

Issue 4 –What Population?Issue 4 –What Population?

Actually, if we identify disability as an outcome of Actually, if we identify disability as an outcome of the person’s interaction with his/her environment we the person’s interaction with his/her environment we are seeking to identify a person with a are seeking to identify a person with a Greater Risk Greater Risk ofof Disability based on health related functional Disability based on health related functional limitationslimitations

The Population would be defined as: The Population would be defined as: persons with persons with functional limitations who are at risk for the loss functional limitations who are at risk for the loss or restriction of opportunities to take part in the or restriction of opportunities to take part in the normal life of the community on an equal level normal life of the community on an equal level with others due to physical, social and other with others due to physical, social and other environmental barriers.*environmental barriers.*

*Adapted from Scott Brown’s presentation at the ISDS Meeting

Locating Risk in the ICF ModelLocating Risk in the ICF Model

Health Condition

Body Functions & Structure Participation

Environmental Factors

Personal Factors

Source: ICF, WHO, 2001

?

THE ‘At Risk’ POPULATION: In THE ‘At Risk’ POPULATION: In Pure ICF TermsPure ICF Terms

ICF ComponentsICF Components

Body FunctionsBody Functions&&

StructuresStructures

Activities Activities & &

ParticipationParticipation

Environmental Environmental FactorsFactors

BarriersBarriers

FacilitatorsFacilitators

Functions Functions

Structures Structures

CapacityCapacity

PerformancePerformance

Moving from CMoving from Conceptoncept to D to Definitionefinition to to MMeasurementeasurement: : Measurement of Measurement of equalization of opportunitiesequalization of opportunities

Locate the definition of disability at the most Locate the definition of disability at the most basic level of activity/participation in core basic level of activity/participation in core domainsdomains

This level is associated with the ability or This level is associated with the ability or inability to carry out basic bodily operations inability to carry out basic bodily operations at the level of the whole person (i.e. walking, at the level of the whole person (i.e. walking, climbing stairs, lifting packages, seeing a climbing stairs, lifting packages, seeing a friend across the room)friend across the room)

Criteria for inclusion of domainsCriteria for inclusion of domains

Suitability for self-reportSuitability for self-report ParsimonyParsimony Validity across various methodological modesValidity across various methodological modes WG Perspective – Cross cultural comparability WG Perspective – Cross cultural comparability Possible domains that fulfill these criteria:Possible domains that fulfill these criteria:

WalkingWalking SeeingSeeing HearingHearing CognitionCognition

Benefits of this approachBenefits of this approach

Development of a demographic means of Development of a demographic means of understanding disability (can compare understanding disability (can compare persons with and without disability)persons with and without disability)

Connection between disability and Connection between disability and participation can be made during data participation can be made during data analysisanalysis

Effectiveness of programs / policies to Effectiveness of programs / policies to promote full participation can be monitoredpromote full participation can be monitored

Intended use of dataIntended use of data

Compare levels of participation in employment, Compare levels of participation in employment, education, or family life for those with disability education, or family life for those with disability versus those without disability to see if persons versus those without disability to see if persons with disability have achieved social inclusionwith disability have achieved social inclusion

Monitor prevalence trends for persons with Monitor prevalence trends for persons with limitations in specific basic activity domains limitations in specific basic activity domains

Measurement Measurement Answers*Answers* Issue 1 - Addressing the problems of the wide ranging variety Issue 1 - Addressing the problems of the wide ranging variety

of questions and wording.of questions and wording. For international comparability we have found it necessary to locate For international comparability we have found it necessary to locate

our population of interest within the basic levels of functioning of the our population of interest within the basic levels of functioning of the whole person. If we want comparability within the US across surveys it whole person. If we want comparability within the US across surveys it is essential to have a similar basic standardized operational measure.is essential to have a similar basic standardized operational measure.

Issue 2 – Nature of the conceptual model serving as a basis for Issue 2 – Nature of the conceptual model serving as a basis for definition and measurement. For example, a theoretical or definition and measurement. For example, a theoretical or legislative definition.legislative definition. Commonly accepted theoretical approaches need to underlie the Commonly accepted theoretical approaches need to underlie the

conceptualization of the components of disability that are then conceptualization of the components of disability that are then operationalized. Legislative definitions need to be treated as “purpose” operationalized. Legislative definitions need to be treated as “purpose” of data collection since the varieties of legislation and legislative of data collection since the varieties of legislation and legislative definitions are directed at specific policy purposes. definitions are directed at specific policy purposes.

*Personal Opinion

Measurement Measurement Answers*Answers* Issue 3 - Purpose of data collection effort may dictate the Issue 3 - Purpose of data collection effort may dictate the

conceptual component from the model as well as the specific conceptual component from the model as well as the specific operational definition of disability.operational definition of disability. Major disconnect occurs between data collection process, use and Major disconnect occurs between data collection process, use and

interpretation. While we can push for purpose related interpretation. While we can push for purpose related operationalization, how the measures are used in analysis requires operationalization, how the measures are used in analysis requires careful planning and interpretation and understanding of the limitations careful planning and interpretation and understanding of the limitations of the measures.of the measures.

Issue 4 - Is the measurement of a ‘true’ population feasible or Issue 4 - Is the measurement of a ‘true’ population feasible or desirable? Is there “one” true number?desirable? Is there “one” true number? We can’t represent the ‘true’ population but we can represent a We can’t represent the ‘true’ population but we can represent a

consistent population by using a standardized subset of questions consistent population by using a standardized subset of questions across surveys (or across countries). That is both feasible and desirable across surveys (or across countries). That is both feasible and desirable and does not preclude inclusion of other measures for specific and does not preclude inclusion of other measures for specific purposes.purposes.

*Personal Opinion