trying out new survey techniques – lessons learned 2013 northwest hazardous materials management...
TRANSCRIPT
![Page 1: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/1.jpg)
Trying Out New Survey Techniques – Lessons Learned
Trying Out New Survey Techniques – Lessons Learned
2013 Northwest Hazardous Materials Management Conference
Liz Tennant, Strategic Advisor, Office of the Program Director
June 25, 2013
2013 Northwest Hazardous Materials Management Conference
Liz Tennant, Strategic Advisor, Office of the Program Director
June 25, 2013
![Page 2: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/2.jpg)
The Local Hazardous Waste Management Program in King
County
The Local Hazardous Waste Management Program in King
County
A regional partnership to protect and enhance public health and environmental quality by reducing the threat posed by the production, use, storage and disposal of hazardous materials.
![Page 3: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/3.jpg)
Program PartnersProgram Partners
• 37 Cities and 2 tribes in King County
• King County Water & Land Resources
• Public Health--Seattle & King County
• Seattle Public Utilities
• King County Solid Waste
• 37 Cities and 2 tribes in King County
• King County Water & Land Resources
• Public Health--Seattle & King County
• Seattle Public Utilities
• King County Solid Waste
![Page 4: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/4.jpg)
King County, WashingtonKing County, Washington
1.92 million residents
799,000 households
62,000 businesses
2,134 square miles
14th most populous
county in the U.S.
1.92 million residents
799,000 households
62,000 businesses
2,134 square miles
14th most populous
county in the U.S.
![Page 5: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/5.jpg)
![Page 6: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/6.jpg)
2012 Survey Goals2012 Survey Goals
Updated snapshot of residents’ attitudes, awareness and behavior about household hazardous materials and waste.
Update on who is using our collection services, and how satisfied they are with the services.
Better understanding of why people are not using the facilities and how to better reach and motivate them to dispose of hazardous wastes properly.
Updated snapshot of residents’ attitudes, awareness and behavior about household hazardous materials and waste.
Update on who is using our collection services, and how satisfied they are with the services.
Better understanding of why people are not using the facilities and how to better reach and motivate them to dispose of hazardous wastes properly.
![Page 7: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/7.jpg)
ConsiderationsConsiderations
Last surveys conducted 5 years ago.
Need for statistically valid samples.
Need to align with increasingly diverse demographics.
Particular concern with input from South King County.
Last surveys conducted 5 years ago.
Need for statistically valid samples.
Need to align with increasingly diverse demographics.
Particular concern with input from South King County.
![Page 8: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/8.jpg)
ApproachApproach
Three complementary surveys:
1. General survey of residents throughout King County
2. Focus groups of special populations in South King County
3. Survey of collection customers
Three complementary surveys:
1. General survey of residents throughout King County
2. Focus groups of special populations in South King County
3. Survey of collection customers
![Page 9: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/9.jpg)
Decision to Try New MethodsDecision to Try New Methods
General residential survey using address based sampling instead of random-dialed phone calls
Oversampled the non-Caucasian population in South King County
Supplemented with focus groups
General residential survey using address based sampling instead of random-dialed phone calls
Oversampled the non-Caucasian population in South King County
Supplemented with focus groups
![Page 10: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/10.jpg)
What is Address Based Sampling?What is Address Based Sampling?
Obtain a random sample of addresses in the area you are surveying.
Mail a letter or postcard to the selected addresses.
Provide information about how to respond.
Analyze the responses you receive.
Potentially can adjust the sample.
Obtain a random sample of addresses in the area you are surveying.
Mail a letter or postcard to the selected addresses.
Provide information about how to respond.
Analyze the responses you receive.
Potentially can adjust the sample.
![Page 11: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/11.jpg)
Final Survey Design:Final Survey Design:
Worked with consultant to design an approach that would meet survey goals. Survey much shorter than 2007.
Letter and paper survey mailed to 10,000 randomly selected households.
Multiple ways to respond: mail back paper survey (postage provided) On-line survey in English or Spanish Toll-free phone number in English or
Spanish
Worked with consultant to design an approach that would meet survey goals. Survey much shorter than 2007.
Letter and paper survey mailed to 10,000 randomly selected households.
Multiple ways to respond: mail back paper survey (postage provided) On-line survey in English or Spanish Toll-free phone number in English or
Spanish
![Page 12: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/12.jpg)
Final Survey Design:Final Survey Design:
Oversample of 300 non-Caucasian households in South King County. Telephone calls to targeted phone numbers
(cell and land line)
Monitored responses to ensure demographic alignment.
Built in contingency for random dialed phone calls if needed to get a demographically aligned sample.
Oversample of 300 non-Caucasian households in South King County. Telephone calls to targeted phone numbers
(cell and land line)
Monitored responses to ensure demographic alignment.
Built in contingency for random dialed phone calls if needed to get a demographically aligned sample.
![Page 13: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/13.jpg)
2012 General Survey Responses2012 General Survey Responses
Survey Responses General Survey
South County Over-sample
Total
# of Paper Surveys 837 837
# of On-Line Surveys 176 176
# of Call-ins 3 3
# of Targeted Phone calls
0 300 300
Total 1,016 300 1,316
![Page 14: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/14.jpg)
High Level Findings (n=1,270)High Level Findings (n=1,270)
Generally people are using the collection sites in their region.
Most people say they don’t have hazardous waste.
Slightly under half say they know about sites; about one third say that they have used them.
Strong support for importance of government providing these services from both collection site users and non-users.
Generally people are using the collection sites in their region.
Most people say they don’t have hazardous waste.
Slightly under half say they know about sites; about one third say that they have used them.
Strong support for importance of government providing these services from both collection site users and non-users.
![Page 15: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/15.jpg)
South County Focus GroupsSouth County Focus Groups Selected 4 non‐English speaking
communities based on demographic data: Spanish‐speaking Vietnamese Filipino
(Tagalog‐speaking) Korean Focus Group
Korean
Conducted 5 focus groups (two Hispanic) recruited by and facilitated by a multi‐cultural, multi‐lingual team.
Selected 4 non‐English speaking communities based on demographic data: Spanish‐speaking Vietnamese Filipino
(Tagalog‐speaking) Korean Focus Group
Korean
Conducted 5 focus groups (two Hispanic) recruited by and facilitated by a multi‐cultural, multi‐lingual team.
![Page 16: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/16.jpg)
South County Focus GroupsSouth County Focus Groups Focus group leaders worked to recruit a
diverse mixture of people in each 15 person focus group.
Conducted 43 individual interviews in the African American community.
General survey questions were modified for focus groups and interviews.
Materials were translated into the relevant language by each focus group facilitator.
Focus group leaders worked to recruit a diverse mixture of people in each 15 person focus group.
Conducted 43 individual interviews in the African American community.
General survey questions were modified for focus groups and interviews.
Materials were translated into the relevant language by each focus group facilitator.
![Page 17: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/17.jpg)
Focus Group FindingsFocus Group Findings Results can’t be generalized due to small
sample size. Communities differ in their approach and
level of concern. Common perception that small amounts of
hazardous waste improperly disposed don’t make a difference.
Widespread confusion about what is hazardous.
Confusion about solid waste disposal and hazardous waste disposal.
Results can’t be generalized due to small sample size.
Communities differ in their approach and level of concern.
Common perception that small amounts of hazardous waste improperly disposed don’t make a difference.
Widespread confusion about what is hazardous.
Confusion about solid waste disposal and hazardous waste disposal.
![Page 18: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/18.jpg)
Focus Group FindingsFocus Group Findings
The Spanish community members did not know about the services that we provide, despite our two year Hispanic outreach campaign.
The elderly were unaware of the home collection services we provide.
Generally that services are inconvenient and should be expanded.
The results help illuminate some of the responses in the other surveys.
The Spanish community members did not know about the services that we provide, despite our two year Hispanic outreach campaign.
The elderly were unaware of the home collection services we provide.
Generally that services are inconvenient and should be expanded.
The results help illuminate some of the responses in the other surveys.
![Page 19: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/19.jpg)
Lessons LearnedLessons Learned
Project was too complex with too many moving parts.
Project management schedule was too ambitious.
Both consultants sought revised budgets to cover core activities.
There were problems with communications, oversight and work quality with both consultants.
Schedule regular face to face meetings with consultant at key checkpoints.
Make sure the consultant understands what level of statistical analysis you are looking for and other expectations.
Make sure consultant includes adequate time and budget for writing the report.
Project was too complex with too many moving parts.
Project management schedule was too ambitious.
Both consultants sought revised budgets to cover core activities.
There were problems with communications, oversight and work quality with both consultants.
Schedule regular face to face meetings with consultant at key checkpoints.
Make sure the consultant understands what level of statistical analysis you are looking for and other expectations.
Make sure consultant includes adequate time and budget for writing the report.
![Page 20: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/20.jpg)
Lessons Learned - Next TimeLessons Learned - Next Time Don’t schedule the General Survey and the
Focus Groups at the same time.
Develop a realistic project schedule.
Make sure that the consultant includes adequate time and budget to cover: Needed meetings Analyzing the results Writing the report
Schedule regular face to face meetings with the consultant at key checkpoints.
Don’t schedule the General Survey and the Focus Groups at the same time.
Develop a realistic project schedule.
Make sure that the consultant includes adequate time and budget to cover: Needed meetings Analyzing the results Writing the report
Schedule regular face to face meetings with the consultant at key checkpoints.
![Page 21: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/21.jpg)
Lessons Learned - Next Time Lessons Learned - Next Time
Make sure that the consultant understands: The type and level of analysis that you
expect. They need to translate technical presentation
of statistics into commonly understood terms. They are expected to meet standard report
requirements. An outline of the report must be approved
prior to drafting it. The first draft should not have grammar or
spelling errors and should be close to final.
Make sure that the consultant understands: The type and level of analysis that you
expect. They need to translate technical presentation
of statistics into commonly understood terms. They are expected to meet standard report
requirements. An outline of the report must be approved
prior to drafting it. The first draft should not have grammar or
spelling errors and should be close to final.
![Page 22: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/22.jpg)
Lessons Learned – Next timeLessons Learned – Next time
Develop focus group questions before or after the general survey questions, not at the same time.
Work closely with the consultant to ensure that concepts are accurately translated and are asked consistently.
Interview consultants to be sure that consultant team members have adequate English proficiency and an understanding of the content that we are trying to collect.
Develop focus group questions before or after the general survey questions, not at the same time.
Work closely with the consultant to ensure that concepts are accurately translated and are asked consistently.
Interview consultants to be sure that consultant team members have adequate English proficiency and an understanding of the content that we are trying to collect.
![Page 23: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/23.jpg)
2012 Surveys and Key Findings:2012 Surveys and Key Findings:
http://lhwmp.wordpress.com/category/publications/
![Page 24: Trying Out New Survey Techniques – Lessons Learned 2013 Northwest Hazardous Materials Management Conference Liz Tennant, Strategic Advisor, Office of the](https://reader036.vdocuments.site/reader036/viewer/2022062407/56649d9c5503460f94a85bad/html5/thumbnails/24.jpg)
Questions?Questions?
For more information:
206-284-7974