e-survey workshop : guidelines i and practice 曾淑芬 元智大學資訊社會學研究所
TRANSCRIPT
E-Survey WorkshopE-Survey Workshop : :
Guidelines I and PracticeGuidelines I and Practice
曾淑芬元智大學資訊社會學研究所
What is Electronic Survey?Email SurveyWeb-based Survey
The rapid development of surveys on the WWW is leading some to argue that soon internet surveys will replace traditional methods of survey data collection. Major advantages have been mentioned: cost savings, speed, limited geographical constraint, ease of contacting respondents, more efficient data processing, ability for customization. Others are urging caution or even voicing skepticism about the future role Web surveys will play.
Why E-Survey?
Promises and Challenges
Promises Strong reach / Penetration Fast response speed Low cost Response flexibility Control of anonymity Minimized data-entered error Minimized interviewer bias
Promises and Challenges
Challenges Generalizability and Response Rate:Generalizability and Response Rate:
response rate to internet based surveys have been declining over the past 10 years
Accessibility:Accessibility:no national or global online directory of email addresses exist
Design Concerns:Design Concerns:use design survey response features judiciously to maximize data quality and minimize error
Sources of Errors in Web-based Survey
Coverage ErrorThe result of all units in a
defined population not having a known nonzero probability of being included in the sample drawn to represent the population. A mismatch between the target population and the frame population.
Sources of Errors in Web-based SurveyCoverage Error
Universal coverage of the Web remains quite limited. Coverage error represents the biggest threat to the representativeness of sample surveys conducted via the internet.
Some populations do not exhibit large coverage problems: employees of certain organizations, members of professional organizations, certain types of businesses, students at many universities and college, and groups with high levels of education
Sampling errorThe result of surveying a sample of the population rather than the entire population. Not all members of the frame population are measured.
Sources of Errors in Web-based Survey
Sampling error
Sources of Errors in Web-based Survey
While coverage error refers to people missing from the frame, sampling error arises during the process of selecting a sample from the frame population, necessitating a means of identifying people on the frame.
A misguided assumption behind many Web surveys that large samples necessarily mean more valid responses.
Sources of Errors in Web-based Survey
Measurement error The result of inaccurate responses that stem from poor question wording, poor interviewing survey mode effects and/or some aspect of the respondent’s behavior.
Sources of Errors in Web-based Survey
Measurement error
Measurement error is the deviation of the answers of respondents from their true values on the measure.
The appearance of a survey can vary from respondent to respondent because of different browser settings, user preferences, variations in hardware, and so on.
There is much work to be done to determine optimal designs for different groups of respondents and types of surveys.
Sources of Errors in Web-based Survey
Non-response error The result of non-response from people in the sample, who, if they had responded, would have provided different answers to the survey questions than those who did respond to the survey.
Sources of Errors in Web-based Survey
Non-response error
Non-response error arises through the fact that not all people included in the sample are willing or able to complete the survey.
Non-response error is a function of both the rates of non-response and of the differences between respondents and non-respondents on the variables of interest.
There is at present little experimental literature on what works and what does not, in terms of increasing response rates to web surveys.
Types of Web survey (Couper, 2001)
Probability-based Web-based surveysIntercept surveysList-based samples of high coverage populationsMixed-mode designs with choice of completion methodPre-recruited panels of internet usersProbability samples of full population
Non-Probability-based Web-based surveyPolls as entertainmentUnrestricted self-selected surveysVolunteer opt-in panels
Seven Response Types for Web-based Surveys (Bosnjak and
Tuten, 2001)
1. Unit non-responders2. Complete responders3. Answering drop-outs4. Lurkers5. Lurking drop-outs6. Item non-responders7. Item non-responding dropouts
Source: Bosnjak and Tuten,2000 (http://ascusc.org/jcmc/vol6/issue3/boznjak.html)
Seven Response Types for Web-based Surveys (Bosnjak and
Tuten, 2001)
Design of Web-bases Surveys (Dillman and Bowker, 2001)
1. Introduce the web questionnaire with a welcome screen (Non-response error)
2. Provide a PIN number (Sampling, Coverage)
3. First question should be interesting to most respondents, easily answered, and fully visible on the first screen of the questionnaire. (Non-response)
4. Present each question in a conventional format similar to that normally used on paper self-administered questionnaires. (Measurement, Non-response)
Design of Web-bases Surveys (Dillman and Bowker, 2001)
(Cont.)5. Restrain the use of color so that
figure/ground consistency and read-ability are maintained, navigational flow is unimpeded, and measurement properties of questions are maintained. (Measurement)
6. Avoid differences in the visual appearance of questions that result from different screen configurations, operating systems, browsers, partial screen displays and wrap-around text. (Coverage, Measurement, Non-response)
7. Provide specific instructions on how to take each necessary computer action for responding to the questionnaire and other necessary instructions at the point where they are needed. (Non-response)
8. Use drop-down boxes sparingly, consider the mode implication, and identify each with a “click here” instruction. (Measurement)
9. Do not require respondents to provide an answer to each question before being allowed to answer any subsequent ones. (Non-response)
Design of Web-bases Surveys (Dillman and Bowker, 2001)
(Cont.)
10. Provide skip directions in a way that encourages marking of answers and being able to click to the next applicable question. (Measurement)
11. Construct web questionnaires so they scroll from question to question unless order effects are a major concern, and/or telephone and web survey results are being combined. (Coverage, Measurement, Non-response)
12. When the number of answer choices exceeds the number that can be displayed in a single column on one screen, consider double-banking with an appropriate grouping device to link them together. (Measurement)
Design of Web-bases Surveys (Dillman and Bowker, 2001)
(Cont.)
13. Use graphical symbols or words that convey a sense of where the respondent is in the completion process, but avoid ones that require significant increases in computer memory. (Coverage, Non-response)
14. Exercise restraint in the use of question structures that have known measurement problems on paper questionnaires, e.g. check-all-that-apply and open-ended questions. (Measurement, Non-response)
Design of Web-bases Surveys
(Dillman and Bowker, 2001) (Cont.)
Other Web Survey Design Suggestion: Zanutto (2001)
1. Use a cover letter with the questionnaire2. Make the survey simple, and have it take
no longer than 20 minutes3. Give an estimated time that it will take to
complete the survey4. Be sure the first question is interesting,
easy to answer, and related to the topic of the survey
5. Be concerned about privacy issues for the respondents and the data that is collected.
6. Allow an alternate mode of completion if people are concerned about privacy, i.e. print and mail in the survey.
E-Survey WorkshopE-Survey Workshop : :
Part III Part III 實作教學實作教學
元智大學資訊社會學研究所黃心怡
開始你的 Web-survey
分析結果確認研究主題
設計問卷
問卷程式設計 問卷前測 發送 E-mail 邀請
資料收集
架設你的網路問卷平台自行設計 套裝軟體
優點 可客製化所需的問卷形式
容易使用,不需太多前置作業許多套裝軟體的功能也包括了資料整理與分析
缺點 自行設計所花的時間較多需有簡單程式基礎
所需的金錢花費較高
執行 利用 HTML 或是 ASP 等程式語言,設計一問卷網頁,放置於自己的網路伺服器上,儲存受訪者的回覆。
中文網站如 : http://my3q.com 或是 spss 等軟體都提供現成的程式,研究者只要輸入題目與簡單設定即可上線使用。
The Advantage of Web-Survey
Cost savingEase of contacting
respondentsMore efficient data
processingAbility for customization
How to designSTEP1:IntroductionSTEP1:Introduction
(1) 選擇問卷形式傳統問卷格式 題組形式每題一頁
(2) 設定版面大小較常使用的是 800*600 ,建議在網頁中加註讓受訪者瀏覽時較好觀看
STEP2:LayoutSTEP2:Layout
Frame( 表單 ) Radio( 按鈕類 )Checkbox( 勾選類 )Drop-Down( 下拉式 )Text( 文字欄 )Hyperlink( 超連結 )
STEP3:Webpage ComponentSTEP3:Webpage Component
文字檔簡單 HTML 語法 , 藉由網頁製作軟體如 : FrontP
age 、 Dreamwaver 等,受訪者在完成線上問卷後,可直接回傳結果至研究者的網路伺服器,以文字檔儲存
資料庫當調查規模龐大且題目眾多時,資料庫有著良好的儲存與管理的功能;相對的,與資料庫連結的網路問卷也需要較複雜的程式技巧與工具,其與法如 ASP 、 PHP 、 JAVA 。
STEP4:Save ResultSTEP4:Save Result
網路問卷降低了過去人工 coding 的時間與金錢
線上調查的電子檔容易讓統計軟體讀取 ( 通常 .txt, .csv 為最普遍的可讀取格式 )
STEP5:Analyze ResultSTEP5:Analyze Result
STEP BY STEP: STEP BY STEP: Form (Form ( 表單表單 ))
STEP BY STEP : FormSTEP BY STEP : Form
STEP BY STEP : Pixels(STEP BY STEP : Pixels( 瀏覽像素瀏覽像素 ))
STEP BY STEP : PixelsSTEP BY STEP : Pixels
STEP BY STEP : Table(STEP BY STEP : Table( 表格表格 ))
STEP BY STEP : TableSTEP BY STEP : Table
STEP BY STEP : TableSTEP BY STEP : Table
STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button
STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button
STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button
名稱 = Q1
是 = 1
否 = 0
不選擇預設值
STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button
STEP BY STEP : Check BoxSTEP BY STEP : Check Box
STEP BY STEP : Check BoxSTEP BY STEP : Check Box
名稱 = Q2_1
有勾選 = 1
不選擇預設值
STEP BY STEP : Check BoxSTEP BY STEP : Check Box
STEP BY STEP : DropdownSTEP BY STEP : Dropdown
STEP BY STEP : DropdownSTEP BY STEP : Dropdown
STEP BY STEP : DropdownSTEP BY STEP : Dropdown
名稱 = Q3
新增選項 = 台北市
勾選指定值 =01
起始狀態 = 未選取
STEP BY STEP : DropdownSTEP BY STEP : Dropdown
STEP BY STEP : DropdownSTEP BY STEP : Dropdown
驗證功能 :
受訪者送出問卷後,檢查其填寫狀態,避免漏答
STEP BY STEP : Text BoxSTEP BY STEP : Text Box
STEP BY STEP : Text BoxSTEP BY STEP : Text Box
名稱 = Q4
字元寬度 =50
密碼欄位 = 否
STEP BY STEP : Text BoxSTEP BY STEP : Text Box
驗證功能 :
文字格式 = 文字
STEP BY STEP : STEP BY STEP : 跳答跳答
STEP BY STEP : STEP BY STEP : 跳答跳答
STEP BY STEP : STEP BY STEP : 跳答跳答
STEP BY STEP : STEP BY STEP : 跳答跳答
STEP BY STEP : STEP BY STEP : 跳答跳答
STEP BY STEP : SubmitSTEP BY STEP : Submit
STEP BY STEP : SubmitSTEP BY STEP : Submit
STEP BY STEP : SavingSTEP BY STEP : Saving
STEP BY STEP : SavingSTEP BY STEP : Saving
檔案格式 :
文字資料庫使用 tab鍵來區隔
STEP BY STEP : SavingSTEP BY STEP : Saving
確認頁的 URL:
Thank_you.html
STEP BY STEP : SavingSTEP BY STEP : Saving
表單儲存 :
1. 檢查是否每一題皆為獨立題號
2. 可選擇日期格式與受訪者的其他資訊
實作時的提醒
1.選擇調查方法時仍需因時地而制宜,免得不符研究初衷
2.在應用資訊科技進行調查之時,最重要的是能夠充分運用電子化後的便利與快速。因此,千萬要讓你的網路調查設計的比傳統問卷調查更易於整理、分析
3.跳答與漏答的項目需事先規劃,並寫在程式中
4.盡量遵循著 Dillman的 web-survey設計準則,將有助於回覆率與可能的失誤
Reference
Bosnjak, M. M. and Tuten, T. L., 2001. “Classifying Response Behaviors in Web-Based Surveys” Journal of Comupter-Mediated Communication, Vol6, No.3, at http://www.ascusc.org/jcmc/vol6/issue3/boznjak.html.
Couper, M. P., 2000. “Web Surveys a Review of Issues and Approaches” Public Opinion Quarterly, Vol64, No.4, pp.464-481.
Dillman, D. A., 2000. Mail and Internet Surveys: The Tailored Design Methods. Second edition. New York: Wiley.
Dillman, D. A., 2002. “Navigating the Rapids of Change: Some Observations on Survey Methodology in the early 21st century” presidential address to the American Association for Public Opinion Research.
Dillman, D. A. and Bowker, D. K., 2001. “The Web Questionnaire Challenge to Survey Methodologists” at http://survey.sesrc.wsu.edu/dillman/zuma_paper_dillman_bowker.pdf.
Reference
Dillman, D. A., Tortora, R. D. and Bowker, D., 1998. “Principle for Constructing Web Surveys” Pullman, Washington. SESRC Technical Report 98-50, at http://survey.sesrc.wsu.edu/dillman/papers/websurveyppr.pdf.
Smith, T. W., 2001. “Are Representative Internet Surveys Possible?” proceedings of Statistics Canada Symposium.
Sheehan, K. B., 2002. “Online Research Methodology: Reflections and speculations” Journal of Interactive Advertising, Vol3, No.1.