the implementation of tools to support the data quality of the survey frame mario ménard november...
TRANSCRIPT
The implementation of tools to support the data quality of
the survey frame
Mario Ménard
November 2008
23-04-19Statistics Canada • Statistique Canada2
Outline
Data sources to populate and maintain the survey frame
Quality Assurance Survey Interceptor Module Survey Frame Assessment Tool
23-04-19Statistics Canada • Statistique Canada3
Data Sources used to maintain the frame
Central SurveyFrame
BusinessRegister
CanadianRevenueAgency(CRA)
NAICSCoding
Survey Feedback
Profilersand FrameAnalysts
• 99% of businesses have only one operating entity
• Legal name, business address, main activity description, geographical location, size measures etc.
• 400 000 new businesses each year• 45 % are coded automatically based on the description provided at the time of registration• 55 % requires manual intervention
• Identification of inactive businesses
• # of misclassified businesses
• Data collection centers transmit survey feedback to the Register
• Interception of legal name, business status and NAICS changes
• Review the validity of the survey feedback
• Review the updates performed on business structures and its
characteristics
• Updates business structure and its characteristics
23-04-19Statistics Canada • Statistique Canada4
Quality Assurance Survey
QAS measures the number of inactive businesses and the quality of NAICS coding for units on the BR
Monthly sample of 600 businesses• Allows small constant surveys conducted by a few individuals• Reduces QAS interviewer re-training and ensures survey
consistency from month to month• Provides timely, relevant measurements• Gives an indication of the inactive units and coding quality over
time Units eligible for selection exclude very large and complex
businesses and recently surveyed units
23-04-19Statistics Canada • Statistique Canada5
Quality Assurance Survey Resultsfrom June 2007 - July 2008
Sectors
% Death Rate
% NAICS 2 Digit Error
Rate
% Error Rate by Revenue
% Volatility Rate
All sectors 18.5 13.2 1.3 2.5
11 Agriculture, Forestry, Fishing & Hunting 14.8 8.2 0.5 5.2
21 Mining, Oil and Gas Extraction 19 23 3.0 6.5
22 Utilities 14.5 7.6 0.0 -
23 Construction 17.4 7.3 0.9 0.1
31-33 Manufacturing 16 20.2 0.3 2.2
41 Wholesale Trade 21.8 24.2 0.3 5.7
44-45 Retail Trade 16.4 8.3 0.1 2.1
48-49 Transportation & Warehousing 21.6 12.8 4.7 5
51 Information & Cultural Industries 22.7 13. 6 1.5 3.1
52 Finances & Insurance 19.2 22.7 2.8 2.2
53 Real Estate, Rental & Leasing 24.9 16.6 1.9 2.2
54 Professional, Scientific and Technical Services 18.4 10.5 1.0 0.1
55 Management of Companies and Enterprises 23 33.5 8.5 3.8
56 Administrative & Support 17.3 29.7 8.9 5.5
61 Educational Services 16 16.9 1.4 1.6
62 Health Care & Social Assistance 16.1 3.5 6.2 -
71 Arts, Entertainment & Recreation 20.2 12 14.2 2.6
72 Accommodation & Food Services 17.9 3.5 0.0 0.1
81 Other Services except Public Administration 18.8 9.9 4.8 1.7
23-04-19Statistics Canada • Statistique Canada6
Quality Assurance Survey Resultsfrom June 2007 - July 2008
Sectors
% Death Rate
% NAICS 2 Digit Error
Rate
% Error Rate by Revenue
% Volatility Rate
All sectors 18.5 13.2 1.3 2.5
11 Agriculture, Forestry, Fishing & Hunting 14.8 8.2 0.5 5.2
21 Mining, Oil and Gas Extraction 19 23 3.0 6.5
22 Utilities 14.5 7.6 0.0 -
23 Construction 17.4 7.3 0.9 0.1
31-33 Manufacturing 16 20.2 0.3 2.2
41 Wholesale Trade 21.8 24.2 0.3 5.7
44-45 Retail Trade 16.4 8.3 0.1 2.1
48-49 Transportation & Warehousing 21.6 12.8 4.7 5
51 Information & Cultural Industries 22.7 13. 6 1.5 3.1
52 Finances & Insurance 19.2 22.7 2.8 2.2
53 Real Estate, Rental & Leasing 24.9 16.6 1.9 2.2
54 Professional, Scientific and Technical Services 18.4 10.5 1.0 0.1
55 Management of Companies and Enterprises 23 33.5 8.5 3.8
56 Administrative & Support 17.3 29.7 8.9 5.5
61 Educational Services 16 16.9 1.4 1.6
62 Health Care & Social Assistance 16.1 3.5 6.2 -
71 Arts, Entertainment & Recreation 20.2 12 14.2 2.6
72 Accommodation & Food Services 17.9 3.5 0.0 0.1
81 Other Services except Public Administration 18.8 9.9 4.8 1.7
Sectors
% Death Rate
% NAICS 2 Digit Error
Rate
% Error Rate by Revenue
% Volatility Rate
All sectors 18.5 13.2 1.3 2.5
23-04-19Statistics Canada • Statistique Canada7
Interceptor Module
Intercepts requests for updates received from the data collection centres
The following events are intercepted:• Classification changes• Legal name changes• Business Status changes
Survey program managers have the choice to decide if they want to use this module
After 35 days in the interceptor module, requests for updates are processed automatically to avoid the accumulation of events
23-04-19Statistics Canada • Statistique Canada8
Number and status of update requests received in the Interceptor Module
From January 2008 to September 2008
# of update request
s
# of update
requests cancelled
% of update
requests cancelled
# of update request
s accepte
d
% of accepte
d request
s
Total # of update requests
13 500 4 900 36 % 8 600 64 %
Services Surveys 7 797 1 941 25 % 5 856 75 %
Distributive Trade Surveys
1 571 899 57 % 672
43 %
Manufacturing Surveys
1 533 919 60 % 614 40 %
Others 2 599 1 141 44 % 1 458 56 %
23-04-19Statistics Canada • Statistique Canada9
Survey Frame Assessment Tool
Indicates changes on the survey frame from one period to the other Easily accesses the details of the changes Analysis performed just before the release of the monthly Generic
Survey Universe File Identifies the largest contributors to the change from one period to
the another Measures the quality of work being performed by the people making
updates to the frame Provides constructive feedback and a good indication of additional
training that may be required
23-04-19Statistics Canada • Statistique Canada10
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by type of updates between the June 1st and July 1st G-SUF, 2008
Type of updates
# of additional
establishments
Total revenue
(000,000)
New establishments:
1) From profiling activities 349 $ 9 400
2) From assigning a NAICS code 11 800 $ 5 699
3) From obtaining a size measure from administrative source
11 329 $ 6 480
NAICS Changes to existing establishment:
4) From profiling activities 1 100 $ 5 600
5) From the Quality Assurance Survey 125 $ 10
6) From a new description received from CRA
270 $ 76
Total 24 973 $ 27 265
23-04-19Statistics Canada • Statistique Canada11
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by type of updates between the June 1st and July 1st G-SUF, 2008
Type of updates
# of additional
establishments
Total revenue
(000,000)
New establishments:
1) From profiling activities 349 $ 9 400
2) From assigning a NAICS code 11 800 $ 5 699
3) From obtaining a size measure from administrative source
11 329 $ 6 480
NAICS Changes to existing establishment:
4) From profiling activities 1 100 $ 5 600
5) From the Quality Assurance Survey 125 $ 10
6) From a new description received from CRA
270 $ 76
Total 24 973 $ 27 265
23-04-19Statistics Canada • Statistique Canada12
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by type of updates between the June 1st and July 1st G-SUF, 2008
Type of updates
# of additional
establishments
Total revenue
(000,000)
New establishments:
1) From profiling activities 349 $ 9 400
2) From assigning a NAICS code 11 800 $ 5 699
3) From obtaining a size measure from administrative source
11 329 $ 6 480
NAICS Changes to existing establishment:
4) From profiling activities 1 100 $ 5 600
5) From the Quality Assurance Survey 125 $ 10
6) From a new description received from CRA
270 $ 76
Total 24 973 $ 27 265
23-04-19Statistics Canada • Statistique Canada13
Survey Frame Assessment Tool
Operating entity
NAICS
Before
NAICS
After
Revenue
Size
Changes User ID Division Update
Source
S76921157 444190 1 679 790 761 Birth – New NAICS
stlogae BRD On-line - Profiling
S77093355 413190 413190 1 512 330 738 Birth - New Establishment
Stat flags
menamar BRD On-line - Profiling
S75628386 412110 419120 1 277 462 141 NAICS Change corejoa IOFD On-line - Profiling
S77021577 336310 336310 689 831 265 Birth – Size
measure
External BRD Activation - CRA
S66586043 336310 545 678 900 Birth – NAICS lemaroc BRD On –line - NAICS
Example of the detailed regarding each update that contributed to the changes in the GSUF from June 1st and July 1st G-SUF, 2008
23-04-19Statistics Canada • Statistique Canada14
Questions
Thank you