Using the web to collect data for studies in

Psychology

Prof. Ben C Jones

Advantages of web-based testing

Advantages: Low cost

Collecting data in the lab can be very expensive, in terms of time, money and other resources

By contrast, once set up, testing online is very cheap

You have to employ fewer people to collect the same volume of data

You can collect a large volume of data very quickly

Advantages: Recruitment

Some groups of participants are hard to get to come in to the lab because they are rarely on campus

By contrast, it is often easy to get these types of people to participate in an online test

People can take part in the test in their own time (e.g. at times that might not be convenient for the

experimenters) and in the setting most convenient for them (e.g. their own home)

Advantages: Large samples

A challenge for lab-based studies is collecting the large samples required for some types of research

By contrast, collecting very large samples for an online test is easier (and cheaper and faster)

It is possible to leave a study running online for months, years and, potentially, decades in order to

collect very large samples

To tie up a lab’s resources over the same period of time would be a big gamble

Advantages: Diverse samplesA problem for some areas of psychology is that we now know a great deal about how undergraduate psychology students behave, but little about how other groups might

behave

The web allows a wider range of subjects to be included in studies (that may be more representative of the general

population than you are!)

For example, the web can be used to easily and cheaply test people from different countries and/or people of different

ages, occupations, educational backgrounds, socioeconomic statuses etc.

As web access improves, this strength of web-based research will become even more important

Advantages: Pilot testing

Web-based tests are well suited to pilot tests that you can then run in the lab

This allows lab tests to target successful projects and ensure lab resources are allocated wisely

Additionally, web tests are very useful for quickly and easily piloting, e.g., new sets of stimuli, questionnaires etc.

Advantages: Public engagement

The interactive nature of web tests allows people to learn about your research and lends itself to interactive exhibits

We have adapted our web tests for interactive public exhibitions as part of French Science Week, The Royal Society of London’s Summer Science Exhibition, The

London Science Museum and The Natural History Museum

The nature of web tests allows you to reach beyond an academic audience. My lab’s website was included in the NY Times’ list of the Top 5 Psychology Websites and has

had ~3 million visitors in the last 30 months alone

Disadvantages of web-based testing

Disadvantages: Control

Some research areas require very tight control over, e.g., viewing distances, lighting etc.

This is often not possible in web tests

Consequently, web testing is better-suited to some research areas than others

Disadvantages: Reliability There are problems (potentially) with ‘frivolous’ responses,

duplicate participation etc.

These can introduce additional noise in the data sets or even bias the results

These problems can often be addressed by ‘catch trials’ and deleting data from duplicate IP addresses

Many areas have now published papers demonstrating that online data and lab data show the same effects

For some types of data (personal information etc.) there is some evidence that the greater feeling of anonymity in online tests makes responses more (not less) reliable

Disadvantages: Sampling Bias

Although web access is very good in many countries and improving all the time, some groups (e.g., very low SES)

may be underrepresented

It used to be said that online samples were biased towards teenage, single males and against women and the elderly

This is no longer the case, however (e.g., ‘silver surfers’)

Despite possible sampling biases, it is unlikely that samples are as biased as samples of UG Psychology students, for

example

Disadvantages: Expertise

Setting up web tests is not straightforward and requires great investment in, e.g., equipment (server) and expertise

(programming)

However, it is very likely that affordable ‘off the shelf’ interfaces for web testing will become available very soon,

much as affordable ‘off the shelf’ interfaces for lab testing did in the 1980s

Ethics andweb-based testing

Ethics

In online tests, the ethical considerations are the same as those that apply to lab-based studies

For example, subjects must give informed consent, should be fully debriefed, can withdraw at any time and any

deception should be revealed and benign

Conclusions

Although not without its problems, web tests have a lot of advantages, particularly when used in conjunction with lab-

based studies