data quality standards at the u.s. census bureau
DESCRIPTION
Data Quality Standards at the U.S. Census Bureau. Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233 European Conference on Quality in Official Statistics July 2008. Agenda. Background Development Process Framework and Structure - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/1.jpg)
1
Data Quality Standardsat the
U.S. Census Bureau
Pamela D. McGovern and John M. BusheryU.S. Census BureauQuality Program Staff
Washington, DC 20233
European Conference on Quality in Official StatisticsJuly 2008
![Page 2: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/2.jpg)
2
Agenda
• Background• Development Process• Framework and Structure• What Worked Well• Next Steps• Challenges
![Page 3: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/3.jpg)
3
Background
![Page 4: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/4.jpg)
4
Office of Management and Budget (OMB)
• Provides direction on Government-wide budget and management issues
• Issued Information Quality Guidelines (2002)• Issued Standards and Guidelines for
Statistical Surveys (2006)
![Page 5: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/5.jpg)
5
U.S. Census Bureau
Composed of separate program directorates:• Economic• Demographic• Decennial
![Page 6: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/6.jpg)
6
Quality Program
• Initiated in 1998• Purpose
– Advance and maintain the Census Bureau interests in quality of its products and processes
• Key players– Methodology and Standards (M&S) Council– Quality Program Staff
Reference: Landman, et al., (Q2001)
![Page 7: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/7.jpg)
7
Methodology and Standards (M&S) Council
• Steering Committee for the Quality Program• Chiefs of methodology & research divisions
and senior statisticians and methodologists• Ensure sound statistical and methodological
practices
![Page 8: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/8.jpg)
8
Quality Program Staff (QPS)
• Support staff for the M&S Council• Facilitate development and implementation of
standards and guidelines• Evaluate implementation of standards
![Page 9: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/9.jpg)
9
Development Process
![Page 10: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/10.jpg)
10
Original Approach
• Informal process– Team formed based on need– No formal deadlines
• Detailed requirements• Development periods lengthy
– Took from 1 to 4 years
![Page 11: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/11.jpg)
11
New Approach
Structured approach that uses:• Dedicated Staff• Contractor Assistance• Directorate Reviewers• Focus Groups
![Page 12: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/12.jpg)
12
Dedicated Staff (QPS)
Two Mathematical Statisticians and one Survey Statistician
Advantages:• Development times decrease• Cohesion of content improve• Consistency of format improve
![Page 13: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/13.jpg)
13
Contractor Assistance
Advantages:• Provides expertise in conducting focus
groups• Brings outside perspective
Disadvantage: • Lacks familiarity with Census Bureau issues
![Page 14: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/14.jpg)
14
Directorate Reviewers
Focus group participants and program area managers
Advantages:• Allows QPS to obtain directorate input early in
process• Promotes buy-in
Disadvantage:• Risk exists that can lose buy-in
![Page 15: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/15.jpg)
15
Focus GroupsSeven to ten mid-level managers; Two hours in
lengthAdvantages:• Forum for discussing important issues across
directorates• May increase buy-in• Participants more attentive during the review
Disadvantage:• Scheduling difficulties can slow down process
![Page 16: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/16.jpg)
16
Process Steps• QPS sends draft standard to reviewers• QPS compiles comments and produces
discussion issues for focus group• Contractor conducts focus group• QPS revises standard and conducts walk-
throughs with the M&S Council• QPS prepares “final draft” standard and sends to
reviewers along with responses• QPS and quality assurance staff review the
requirements to determine if they are auditable• QPS submits set for final review and approval
![Page 17: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/17.jpg)
17
Framework and Structureof the Standards
![Page 18: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/18.jpg)
18
Framework
• Lists standards and their purpose– Based on the OMB standards – Based on our existing Census Bureau standards
(See our Information Quality website http://www.census.gov/quality/)
• Consists of two types of standards– Process Standards (18)– Supporting Standards (5)
![Page 19: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/19.jpg)
19
Process Standards
Reflect the phases of the life cycle of a data product:
• Planning and Design• Obtaining Data• Processing Data• Analyzing Data• Disseminating Products
![Page 20: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/20.jpg)
20
Supporting Standards
Apply to all phases of the life cycle:• Protecting Confidentiality• Managing Data and Documents• Ensuring Quality• Applying for a Waiver
![Page 21: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/21.jpg)
21
Structure of the Standards
High-level requirements in list form:• State “what” is required; avoid the “how”• Include examples to provide guidance• Provide links to reference documents and to
related standards
![Page 22: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/22.jpg)
22
What Worked Well
![Page 23: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/23.jpg)
23
Accelerated Development
Reduced development times due to:• Early input• Straightforward requirements• Individual walkthroughs with the M&S Council
members
![Page 24: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/24.jpg)
24
Program Area Input
• Interviews with program area experts to get assistance in drafting requirements
• Focus group approach– Structure the discussion based on comments– Overlap participants on related standards– Record the session
![Page 25: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/25.jpg)
25
Focus Group Evaluation
Scale: 1= strongly disagree to 5= strongly agree
Characteristic Rating
The goal of the focus group was clear. 4.2
The participant’s point of view was heard. 4.5
The goal of the focus group was accomplished. 4.0
The length of the focus groups was sufficient. 4.2
The participant’s satisfaction with the outcome. 4.0
![Page 26: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/26.jpg)
26
Next Steps
![Page 27: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/27.jpg)
27
Next Steps
• Review the full set of standards for completeness, consistency, and cohesion
• Submit the standards to the directorates for final review and approval
• Expect completion by September 2009
![Page 28: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/28.jpg)
28
Next Steps
• Implementation of the standards– QPS assists the program directorates– Training
• Evaluations of the implementation– Identify aspects of the standards that need
improvement
![Page 29: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/29.jpg)
29
Challenges
![Page 30: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/30.jpg)
30
Challenges
• Risk of excessive burden– How compliance will affect costs and schedules
• High-level standards– Avoid being too vague– Program areas interpret standards differently
• Verification of compliance– Audit program (Bushery et al., Q2006)
![Page 31: Data Quality Standards at the U.S. Census Bureau](https://reader035.vdocuments.site/reader035/viewer/2022081513/56813aee550346895da35d63/html5/thumbnails/31.jpg)
31
Thank you
Contact Information:Pamela D. McGovern4600 Silver Hill RoadU.S. Census Bureau
Washington, DC [email protected]