using the internet to collect primary research data postgraduate research methods day conference,...
TRANSCRIPT
![Page 1: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/1.jpg)
Using the Internet to Collect Primary Research Data
Postgraduate Research Methods Day Conference,
The Open University,
Camden Regional Centre
Claire Hewson,
Department of Psychology,
The Open University
![Page 2: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/2.jpg)
A (very) Brief History of Internet-mediated Research (IMR)
Pioneers started implementing online data collection methods around the mid-1990s Bordia (1996); Dillman (1991); Gaiser (1997); Hewson (1994).
Reports (e.g. validation studies) have since supported the quality of data that can be obtained in IMR Corley & Scheepers (2002); Hewson & Charlton (2005).
A new era of IMR (Web 2.0) The web as organic, collaborative, interactive, ever-changing (e.g. Wikipedia); the rise of social media (e.g. Facebook, Twitter, YouTube).
![Page 3: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/3.jpg)
IMR Data Collection Methods
Obtrusive Approaches Surveys, experiments, interviews, participant and / or disclosed observation approaches.
Unobtrusive Approaches Observation using traces and archives, e.g. discussion group archives, browsing behaviours, social networking links (SNA); lurking?
'Big Data' Data sets which become so large as to be difficult to manage; presently a 'hot topic'.
![Page 4: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/4.jpg)
The Visualisation of Complex Networks
Nike+ involves the placement of a sensor underneath the footbed of your Nike running shoe in order to collect data about where you've run, how long it took and where you can improve over time.
http://www.visualcomplexity.com/vc/
Visualisation of a year's worth of runs uploaded to the Nike+ website. It plays back runs throughout three cities: New York, London and Tokyo. The runs showed tens of thousands of peoples' runs animating the city and bringing it to life. The software visualizes and follows individual runs, as well as showing the collective energy of all the runners, defining the city by the constantly changing paths of the people running in it.
![Page 5: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/5.jpg)
Web-based Surveys and Experiments
There is now a large range of software for helping implement these, particularly survey packages (which can also support simple experimental designs), e.g. 'Surveymonkey', 'Qualtrics'.
Dedicated packages to support experimental web-based studies include: WEXTOR (Reips, wextor.org/wextor/en) and WEBEXP (Edinburgh webexp.info).
![Page 6: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/6.jpg)
Web-based Surveys and Experiments
New design issues and principles emerge in this novel context. Useful resources and guidelines include:
Exploring Online Research Methods: www.restore.ac.uk/orm
WebSurveyMethodology: websm.org
Couper (2008). Designing Effective Web Surveys. CUP: New York.
Reips & Krantz (2010). Conducting True Experiments on the Web. In Gosling & Johnson (eds) Advanced Methods for Conducting Online Behavioral Research. APA.
Fielding, Lee & Blank (2008). The Sage Handbook of Online Research Methods. Sage: London.
![Page 7: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/7.jpg)
Recruiting Participants in IMR
Sample Bias Issues?, e.g. Bordia (1997).
True Volunteers, place adverts in public spaces, e.g. newsgroups, webpages, IMR study clearing houses, Facebook.
List-based Sampling Frame, e.g. email lists, online panel.
Probability Sampling Methods, e.g. random selection from known list, online probability panel.
General Principle: methods which allow some estimation of the sampling frame are preferable where measures of response rate, non-response bias, etc. are important to the research design.
![Page 8: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/8.jpg)
Online Sampling Resources
● Online Study Clearing Houses, e.g. Online Psychology
Research: http://www.onlinepsychresearch.co.uk
● Newsgroups, SNSs, e.g. Facebook: facebook.com, Newsgroups (e.g. http://uk.groups.yahoo.com; https://www.groups.google.com), Mailing Lists (see the searchable Catalist database: http://www.lsoft.com/catalist.html)
● Online Panels, e.g. Knowledge Networks: http://www.knowledgenetworks.com/knpanel
Online workforces, e.g. Mechanical Turk: www.mturk.com
OU Participant Panel, Jim will talk more about this
![Page 9: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/9.jpg)
OU ‘Virtual Participant Panel’
URL: http://www.open.ac.uk/socialsciences/psychology/cogpsypanel/
People can participate in studies ad hoc and/or… … can sign up to be contacted for future studies. Sign-ups give certain information (e.g. sex, age,
location, handedness) which can be used to filter if particular characteristics are needed in a sample.
Currently has 1,245 signed-up members. Has a built-in ‘Questionnaire Manager’ to create
surveys.
![Page 10: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/10.jpg)
OU ‘Virtual Participant Panel’
Instruction text
Question text
Response text
![Page 11: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/11.jpg)
OU ‘Virtual Participant Panel’
Stimulus image (‘question’)
Response images (‘answers’)
Response buttons
![Page 12: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/12.jpg)
SurveyMonkey Example
● Computer Attitudes Surveyhttps://www.surveymonkey.com/s/computer_attitudes_study
● Qualtrics – site license for all OU social science faculty students / staff. Visit 'openss.qualtrics.com' and sign up with a valid OU (open.ac.uk) email address.
Martin discussed using Qualtrics in a radiology expertise study this morning, and Jim will demonstrate an online experiment implementation using Qualtrics.
Importing studies from SurveyMonkey into Qualtrics?
![Page 13: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/13.jpg)
Qualtrics
![Page 14: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/14.jpg)
Qualtrics
Stimulus image
Response images
![Page 15: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/15.jpg)
Ethics in Online Research
Some ethics issues require special consideration in an IMR context, the complexities of which are not always obvious to researchers or participants.
e.g. Data security, traceability (confidentiality, anonymity).
![Page 16: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/16.jpg)
Key feature of IMR: involves acquisition of data from or about individuals in absence of face-to-face co-presence
COUPLED WITH greater scope for carrying out quite complex interactive procedures
![Page 17: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/17.jpg)
The New BPS Guidelines (November 2013)
Document aims to outline some of the key ethics principles for researchers and ethics committees to keep in mind when designing and / or evaluating an IMR study
Emphasis on how to apply existing ethics principles in an IMR context
“Thinking is not optional”
![Page 18: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/18.jpg)
The New BPS Guidelines (November 2013)
● Centred around the four main principles of the BPS 'Code of Human Research Ethics' (2011).
1. Respect for the autonomy and dignity of persons
2. Scientific value
3. Social responsibility
4. Maximising benefits and minimising harm
![Page 19: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/19.jpg)
Main Ethics Issues to Consider in IMR
Public-private domain distinction Confidentiality* Copyright Consent; withdrawal; debrief*● Levels of control*● Disruption of social structures
![Page 20: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/20.jpg)
Valid Consent
Verifying characteristics (e.g. age, sex)
Engagement with consent process (check boxes?)
Adequate information – not all risks online are obvious, e.g. potential data leakage (balance between information and overload)
Risks will be higher in some contexts than others – key principle of proportionality of ethics procedures to levels of risk of harm re-emerges
![Page 21: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/21.jpg)
Withdrawal and Debrief
Robust withdrawal procedures may be more difficult to implement online (e.g. volunteer web survey)
Detecting withdrawal may be harder (use of partial data?)
Ensuring debrief difficult (e.g. exit by closing browser)
Enhanced risk in cases of deception (no debrief)?
Principle of proportionality of ethics procedures to levels of risk (of harm) re-emerges, e.g. effects of withdrawal without debrief (such as in very sensitive research)
![Page 22: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/22.jpg)
Confidentiality, Anonymity
● Enhanced traceability of data, dissemination implications, use of quotes.
Other data leakage risks, e.g. email.
Traces on personal computing equipment.
Sensitivity of data is a factor associated with levels of risk of harm.
Researchers should carefully assess risks, and perhaps inform participants of these risks.
![Page 23: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/23.jpg)
Data Protection
UK Data Protection Act (1998).
Issue of storing data on non-UK/EU servers?
Popular software packages (e.g. SurveyMonkey, Qualtrics) are US-based.
'Safe Harbour Scheme'.
![Page 24: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/24.jpg)
Office of National Statistics SurveyMonkey, Safe Harbour Scheme
![Page 25: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/25.jpg)
Levels of control
Over procedures, who takes part, participant behaviours...
Implications for scientific value (validity)
Implications for levels of harm (monitoring participant reactions, presenting debrief info., etc.)
![Page 26: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/26.jpg)
Maximising Benefits, Minimising Harm
These considerations permeate the other issues
Careful assessment needed; trade-offs will be inevitable (e.g. anonymity versus traceability/verifiability; disclosure versus social disruption/harm)
“Thinking is not optional”
![Page 28: Using the Internet to Collect Primary Research Data Postgraduate Research Methods Day Conference, The Open University, Camden Regional Centre Claire Hewson,](https://reader035.vdocuments.site/reader035/viewer/2022062716/56649dd35503460f94aca069/html5/thumbnails/28.jpg)
Some Examples
● http://www.socialpsychology.org/expts.htm http://www.onlinepsychresearch.co.uk http://psych.hanover.edu/Research/exponnet.html http://www.wexlist.net