sharing best practices for the redesign of three business surveys

1

Sharing best practices for the Sharing best practices for the redesign of three business redesign of three business

surveyssurveys

Charles Tardif, Business Survey Methods Division,Statistics Canada

presented at the ICES-III, session 75

June 21st, 2007

2

Alternative titleAlternative title

Can we design a monthly Unified Enterprise Survey?

3

Outline

What are monthly business surveys? – a simplified view

Common objectives of monthly business surveys

Statistics Canada (STC) monthly business surveys:

Monthly Survey of Manufacturers (MSM)

Monthly Survey of the Food Services and Drinking Places (MFS)

Monthly Wholesale and Retail Trade Survey (MWRTS)

4

Outline (cont’d)

Commonalities between the surveys

Comparing survey elements

Summary – A monthly unified enterprise survey?

Conclusion

5

What are monthly business surveys? – a simplified view

Production of monthly estimates mainly on financial information (sales, revenues, expenses, etc):

By geographic level

By industrial level, classified by the North American Industrial Classification System (NAICS)

With a targetted quality, expressed in terms of Coefficients of Variation (CV).

6

Common objectives of monthly business surveys

Measure trends and levels for key financial variablesEstablish best possible survey design to meet the surveys’ objectivesMeet the 6 elements of the STC Quality Framework:

Relevance, accuracy, timeliness, accessibility, interpretability, coherence

Maximise use of administrative data, mainly tax data, to reduce response burden

7

STC monthly business surveys

MSMManufacturers

MFSRestaurants

MWRTS Retail

MWRTS Wholesale

Variables of interest

Shipments, inventories and orders

Sales,

Number of locations

Sales,

Number of locations

Sales, Inventories

Geo level Provinces, territories

Provinces, territories



Industrial level (NAICS)

311 to 339, at the 3rd to 6th digit

722, at the 4th digit

44 and 45, for 19 trade groups (TG)

41, for 15 trade groups (TG)

8

STC monthly business surveys (cont’d)

MSMManufacturers

MFSRestaurants

MWRTS Retail

MWRTS Wholesale

Population size(establishments or clusters of est.)

100,000 90,000 180,000 100,000

Collection units 11,000 2,100 12,000 8,000

Number of domains

More than 1,000

52 247 possible domains (19 TGs)

195 possible domains (15 TGs)

Yearly revenues (in billions)

550 36 350 450

9

Commonalities between the Commonalities between the surveyssurveys

All surveys:All surveys:share common objectivesshare common objectivesundertook or completed a redesign or a undertook or completed a redesign or a restratification in the last 2 yearsrestratification in the last 2 yearsare facing pressure to make a more extensive are facing pressure to make a more extensive use of Tax datause of Tax datahave skewed populations have skewed populations have an annual counterpart, all integrated in have an annual counterpart, all integrated in the Unified Enterprise Survey (UES)the Unified Enterprise Survey (UES)are managed by different subject matter are managed by different subject matter divisions, although centralized methodological divisions, although centralized methodological support.support.

10

Commonalities between the Commonalities between the surveyssurveys

One monthly unified surveyOne monthly unified survey??We’ll see what has been done so far to We’ll see what has been done so far to harmonize the different surveys.harmonize the different surveys.

11

Comparing survey elements Comparing survey elements

MSM Manufacturers

MFS Restaurants

MWRTS Wholesale, Retail

Frame

Before BR, no exclusions

Business Register (BR), with the exclusions of the non-employing establishments

Now BR, no exclusions, using a Survey specific Universe Frame

Sampling

Unit

Before Establishment level

Company level

Now Establishment level or cluster of establishments

12

Comparing survey elementsComparing survey elements

MSM Manufacturers

MFS Restaurants

MWRTS Wholesale,

Retail

Stratification variables

Before Province/territory, NAICS and a size measure: Gross Business Income (GBI) from the BR

Now Province/territory, NAICS and a size measure: annualized monthly data, data from annual

survey, tax data, GBI

Take-none stratum

Before Bottom 2% by province

No take-none stratum

5% by geo and TGs

Now Bottom 10% (MSM, MFS) or 5% (MWRTS) by province and stratification NAICS, subject to a

cap.

13


Stratification – Now:Stratification – Now: Differences between surveys, but all:Differences between surveys, but all:

are using the Lavallée-Hidiroglou algorithm are using the Lavallée-Hidiroglou algorithm (efficient with skewed population) to stratify (efficient with skewed population) to stratify their population with the following their population with the following characteristics:characteristics:

1 Take-all / must-take stratum by prov/NAICS1 Take-all / must-take stratum by prov/NAICS1 or a few take-some strata by prov/NAICS1 or a few take-some strata by prov/NAICSMinimum sample size per stratumMinimum sample size per stratumCapped design weightCapped design weightOversampling for out-of-scopes, deaths and non-Oversampling for out-of-scopes, deaths and non-responseresponse

14


Use of GST - Now:Use of GST - Now:All surveys are making use of GST data, All surveys are making use of GST data, although in a different way:although in a different way:

MSM and MWRTS: MSM and MWRTS: Micro approach, with selected units replaced by Micro approach, with selected units replaced by GST data through modeling. Units have a design GST data through modeling. Units have a design weight > 1.weight > 1.

Use of GST data varies by NAICS for MSMUse of GST data varies by NAICS for MSM

Standard stratification design explained in the Standard stratification design explained in the previous slide is usedprevious slide is used

15


GST micro approach, selected units

Take-somes 1

Take-somes 2

Take-nones

Take-alls

S1 S2

16


Use of GST - Now:Use of GST - Now:All surveys are making use of GST data, All surveys are making use of GST data, although in a different way:although in a different way:

MFS: MFS: Micro approach, where a sample of units known as Micro approach, where a sample of units known as simples are selected to model the value of all the simples are selected to model the value of all the other simple units. All units have a weight of 1.other simple units. All units have a weight of 1.

Independent stratification design for these simple Independent stratification design for these simple units.units.

Applied for selected combinations of provinces and Applied for selected combinations of provinces and NAICS.NAICS.

17


GST micro approach, all units modelledTake-alls

Take-nones

Simples

Com

plex

uni

ts

18


MSM Manufacturers

MFS Restaurants


Sample selection

STC Generalized Sampling (GSAM) system for sample selection

Random sampling within each stratum

Random sampling within each stratum and for the GST modelling

Systematic sampling within each stratum

Same sample, sampling births every month

19


MSM Manufacturers

MFS Restaurants

MWRTS Wholesale,

Retail

Edit and imputation

BeforeMSM in-house E&I program

MFS in-house E&I program

MWRTS in-house E&I program

Now

BANFF, a STC generalized system for E&I

MWRTS in-house E&I program

All are doing outlier detection, historical edits, trend and mean imputation

20


MSM Manufacturers

MFS Restaurants


Estimation STC Generalized Estimation System (GES) used to produce the estimates.

Variance -Sampling variance computed using GES-Computing variability from other sources (imputation, use of admin data) being considered.

Publication -Estimates published in The Daily, a STC publication.-Estimates available in CANSIM.

21

Summary - A monthly unified Summary - A monthly unified enterprise survey? enterprise survey?

AdvantagesAdvantagesTaking advantage of the best practices of Taking advantage of the best practices of each survey to integrate all monthly each survey to integrate all monthly surveyssurveys

Easier to ensure coherence between the Easier to ensure coherence between the monthly surveys – and their annual monthly surveys – and their annual counterpart, for comparison purposescounterpart, for comparison purposes

Annual surveys are already integratedAnnual surveys are already integrated

22


Advantages (cont’d)Advantages (cont’d)Would facilitate the implementation of Would facilitate the implementation of changes to the surveys (no need for a changes to the surveys (no need for a guinea pig):guinea pig):

For example, introduction of the GST was For example, introduction of the GST was done at different times for the 3 surveysdone at different times for the 3 surveys

Implementation of better measures of Implementation of better measures of variability could be done more efficiently:variability could be done more efficiently:

Integrating more components of variabilityIntegrating more components of variability

Variance of the differences between the estimatesVariance of the differences between the estimates

23


Issues - MethodologicalIssues - MethodologicalAs seen, there are conceptual differences As seen, there are conceptual differences between the surveys:between the surveys:

Different data elements collectedDifferent data elements collected

Level at which the information is collected is Level at which the information is collected is differentdifferent

Use of administrative data is differentUse of administrative data is different

Although these differences could be Although these differences could be factored in the sample designfactored in the sample design

24


Issues - OperationalIssues - OperationalCurrently managed by three different Currently managed by three different subject matter divisions, but centralized subject matter divisions, but centralized methodological supportmethodological support

Impact on the field workImpact on the field work

25

ConclusionConclusion

Past:Past:Used to be important differences between the Used to be important differences between the monthly surveysmonthly surveys

Present:Present:Important steps have been made to share best Important steps have been made to share best practices between the monthly surveys to practices between the monthly surveys to harmonize themharmonize them

Future:Future:Should we go one step further and integrate Should we go one step further and integrate them in a Monthly Unified Enterprise Survey?them in a Monthly Unified Enterprise Survey?

26

For more information, please contact

Pour plus d’information, veuillez svp contacter

[email protected]

sharing best practices for the redesign of three business surveys

Documents

possible survey design

different surveys

surveysall surveys

surveys objectivesmeet

use of gst data

sample of units

simple units

different way