advantages and disadvantages of research on internet
TRANSCRIPT
-
8/8/2019 Advantages and Disadvantages of Research on Internet
1/29
JCMC Home
Submit
Issues
Author Index
Editors
About JCMC
Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey
research, online questionnaire authoring software packages, and web survey services.Journal of Computer-
Mediated Communication, 10(3), article 11. http://jcmc.indiana.edu/vol10/issue3/wright.html
Researching Internet-Based Populations:Advantages and Disadvantages of Online Survey Research,
Online Questionnaire Authoring Software Packages, and WebSurvey Services
Kevin B. WrightDepartment of Communication
University of Oklahoma
Abstract
This article examines some advantages and disadvantages of conducting online survey research. It explores current features,
issues, pricing, and limitations associated with products and services, such as online questionnaire features and services to
facilitate the online survey process, such as those offered by web survey businesses. The review shows that current online
survey products and services can vary considerably in terms of available features, consumer costs, and limitations. It is
concluded that online survey researchers should conduct a careful assessment of their research goals, research timeline, and
financial situation before choosing a specific product or service.
Introduction
The past decade has seen a tremendous increase in internet use and computer-mediated communication (Fox,
Rainie, Larsen, Horrigan, Lenhart, Spooner, & Carter, 2001; Horrigan, 2001; Nie & Erbring, 2000; Nie,
http://jcmc.indiana.edu/submit.htmlhttp://jcmc.indiana.edu/issues.htmlhttp://jcmc.indiana.edu/authors.htmlhttp://jcmc.indiana.edu/editors.htmlhttp://jcmc.indiana.edu/aboutus.htmlhttp://jcmc.indiana.edu/vol10/issue3/wright.htmlmailto:[email protected]://jcmc.indiana.edu/vol10/issue3/http://jcmc.indiana.edu/vol10/issue3/http://jcmc.indiana.edu/vol10/issue3/http://jcmc.indiana.edu/vol10/issue3/http://jcmc.indiana.edu/submit.htmlhttp://jcmc.indiana.edu/issues.htmlhttp://jcmc.indiana.edu/authors.htmlhttp://jcmc.indiana.edu/editors.htmlhttp://jcmc.indiana.edu/aboutus.htmlhttp://jcmc.indiana.edu/vol10/issue3/wright.htmlmailto:[email protected] -
8/8/2019 Advantages and Disadvantages of Research on Internet
2/29
-
8/8/2019 Advantages and Disadvantages of Research on Internet
3/29
more than 10 online surveys.
Advantages of Online Survey Research
Researchers in a variety of disciplines may find the Internet a fruitful area for conducting survey research. As
the cost of computer hardware and software continues to decrease, and the popularity of the Internet
increases, more segments of society are using the Internet for communication and information (Fox et al.,
2001; Nie et al., 2002). Thousands of groups and organizations have moved online, many of them
aggressively promoting their presence through the use of search engines, email lists, and banner
advertisements. These organizations not only offer information to consumers, they also present opportunities
for researchers to access a variety of populations who are affiliated with these groups.
Communication researchers may find the Internet an especially rich domain for conducting survey research.
Virtual communities have flourished online, and hundreds of thousands of people regularly participate in
discussions about almost every conceivable issue and interest (Horrigan, 2001; Wellman, 1997; Wellman &
Haythornthwaite, 2002). Areas as diverse as interpersonal (Parks & Floyd, 1996; Tidwell & Walther, 2002;
Wright, 2004), group (Hollingshead, McGrath, & O'Connor, 1993; Hobman, Bordia, Irmer, & Chang, 2002),
organizational (Ahuja & Carley, 1998), health (Rice & Katz, 2001; Wright, 2000a), and mass communication
(Flaherty et al.,1998; Flanagin & Metzger, 2001) have been studied using online surveys.
Access to Unique Populations
One advantage of online survey research is that it takes advantage of the ability of the Internet to provide
access to groups and individuals who would be difficult, if not impossible, to reach through other channels
(Garton, Haythornthwaite, & Wellman, 1999; Wellman, 1997). In many cases, communities and groups exist
only in cyberspace. For example, it would be difficult to find a large, concentrated group of people conducting
face-to-face discussions of topics such as cyber-stalking, online stock trading, and the pros and cons of virtual
dating. While people certainly discuss such issues among friends, family members, and co-workers, few meet
-
8/8/2019 Advantages and Disadvantages of Research on Internet
4/29
face-to-face in large groups to discuss them. One advantage of virtual communities as sites for research is that
they offer a mechanism through which a researcher can gain access to people who share specificinterests,
attitudes, beliefs, and values regarding an issue, problem, or activity. For example, researchers can find a
concentrated number of older individuals who use computers on the Internet-based community SeniorNet
(Furlong, 1989; Wright, 2000a, 2000c). In contrast, with traditional survey research methods it may be more
difficult to reach a large number of demographically-similar older people who are interested in computers.
Another example is the case of individuals with diseases or conditions, such as HIV, eating disorders, and
physical disabilities. Individuals with these conditions and diseases are often difficult to reach because they are
stigmatized offline. Health communication researchers have been able to go online to study these populations,
including examining how features of the computer medium help people cope with the social stigma of their
condition (Braithwaite, Waldron, & Finn, 1999; Wright, 2000b). More generally, the Internet enables
communication among people who may be hesitant to meet face-to-face. For example, individuals with
unpopular political views may hesitate to express themselves openly, and groups of individuals such as Arab-
Americans may feel uncomfortable talking about anti-Arab sentiment in public places (Muhtaseb, 2004). These
individuals and groups often can be reached on the Internet in larger numbers than would be possible using
face-to-face research methods.
Time
A second advantage is that Internet-based survey research may save time for researchers. As already noted,
online surveys allow a researcher to reach thousands of people with common characteristics in a short amount
of time, despite possibly being separated by great geographic distances (Bachmann & Elfrink, 1996; Garton et
al., 2003; Taylor, 2000; Yun & Trumbo, 2000). A researcher interested in surveying hard-to-reach populations
can quickly gain access to large numbers of such individuals by posting invitations to participate to
newsgroups, chat rooms, and message board communities. In the face-to-face research environment, it would
take considerably longer-if it were possible at all-to find an equivalent number of people with specific
attributes, interests, and attitudes in one location.
-
8/8/2019 Advantages and Disadvantages of Research on Internet
5/29
Online surveys may also save time by allowing researchers to collect data while they work on other tasks
(Llieva, Baron, & Healey, 2002). Once an invitation to participate in a survey is posted to the website of a
community of interest, emailed to people through a listserv service, or distributed through an online survey
research service, researchers may collect data while working on other projects (Andrews et al., 2003).
Responses to online surveys can be transmitted to the researcher immediately via email, or posted to an HTML
document or database file. This allows researchers to conduct preliminary analyses on collected data while
waiting for the desired number of responses to accumulate (Llieva et al., 2002). First generation online survey
researchers often used email-based surveys, which involved creating online survey forms using word
processing software, and later used products such as Macromedia's Dreamweaver. Researchers had to "cut
and paste" responses from the email responses into statistical software programs such as SAS and SPSS. More
recently, online survey creation software packages provide a variety of templates to create and implement
online surveys more easily, as well as to export data to statistical software packages. Moreover, a number of
online survey services provide survey design assistance, generate samples, and analyze and interpret data.
Some of the newer software packages and web-based services are detailed below.
Cost
Online survey researchers can also save money by moving to an electronic medium from a paper format
(Bachmann & Elfrink, 1996; Couper, 2000; Llieva et al., 2002; Yun & Trumbo, 2000). Paper surveys tend to be
costly, even when using a relatively small sample, and the costs of a traditional large-scale survey using
mailed questionnaires can be enormous. The use of online surveys circumvents this problem by eliminating the
need for paper and other costs, such as those incurred through postage, printing, and data entry (Llieva et al.,
2002; Watt, 1999; Witmer et al., 1999). Similarly, conducting online interviews, either by email, or in a
synchronous "chat" format, offers cost savings advantages. Costs for recording equipment, travel, and the
telephone can be eliminated. In addition, transcription costs can be avoided since online responses are
automatically documented. Newer online survey creation software and web survey services costs can vary
from very little to thousands of dollars depending upon the types of features and services selected; however,
-
8/8/2019 Advantages and Disadvantages of Research on Internet
6/29
this is relatively inexpensive compared to the cost of traditional paper-and-pencil surveys.
Disadvantages Associated with Online Survey Research
As discussed above, online surveys offer many advantages over traditional surveys. However, there are also
disadvantages that should be considered by researchers contemplating using online survey methodology.
Although many of the problems discussed in this section are also inherent in traditional survey research, some
are unique to the computer medium.
Sampling Issues
When conducting online research, investigators can encounter problems as regards sampling (Andrews et al.,
2003; Howard, Rainie, & Jones, 2001). For example, relatively little may be known about the characteristics of
people in online communities, aside from some basic demographic variables, and even this information may be
questionable (Dillman, 2000; Stanton, 1998). A number of recent web survey services provide access to
certain populations by offering access to email lists generated from other online surveys conducted through
the web survey service. Some offer access to specialized populations based on data from previous surveys.
However, if the data were self-reported, there is no guarantee that participants from previous surveys
provided accurate demographic or characteristics information.
Generating Samples from Virtual Groups and Organizations
Some virtual groups and organizations provide membership email lists that can help researchers establish a
sampling frame. However, not all members of virtual groups and organizations allow their email addresses to
be listed, and some may not allow administrators to provide their email addresses to researchers. This makes
accurately sizing an online population difficult.
Once an email list is obtained, it is possible to email an online survey invitation and link to every member on
the list. Theoretically, this can give researchers a sampling frame. However, problems such as multiple email
-
8/8/2019 Advantages and Disadvantages of Research on Internet
7/29
addresses for the same person, multiple responses from participants, and invalid/inactive email addresses
make random sampling online a problematic method in many circumstances (Andrews et al., 2003; Couper,
2000). One solution is for researchers to require participants to contact them to obtain a unique code number
(and a place to include this code number on the online questionnaire) prior to completing a survey. However,
requiring this extra step may significantly reduce the response rate. Another solution that some newer web
survey programs offer is response tracking. Participants are required to submit their email address in order to
complete the survey. Once they have completed the survey, the survey program remembers the participant's
email address and does not allow anyone using that email address access to the survey. This feature helps to
reduce multiple responses, although someone could potentially complete the survey a second time using a
secondary email address (Konstan, Rosser, Ross, Stanton, & Edwards, 2005).
Generating a Sample from an Online Community
Establishing a sampling frame when researching an online community presents a number of challenges. Unlike
membership-based organizations, many online communities, such as community bulletin boards and chat
rooms, do not typically provide participant email addresses. Membership is based on common interests, not
fees, and little information is required when registering to use these communities, if registration is required at
all. Some researchers attempt to establish a sampling frame by counting the number of participants in an
online community, or the published number of members, over a given period of time. In either case, the ebb
and flow of communication in online communities can make it difficult to establish an accurate sampling frame.
For example, participation in online communities may be sporadic depending on the nature of the group and
the individuals involved in discussions. Some people are "regulars," who may make daily contributions to
discussions, while others only participate intermittently. Furthermore, "lurkers," or individuals who read posts
but do not send messages, may complete an online survey even though they are not visible to the rest of the
community. The presence of lurkers in online communities appears to be highly variable (Preece, Nonnecke, &
Andrews, 2004). Studies have found that in some online communities lurkers represent a high percentage
(between 45% and 99%) of community members, while other studies have found few lurkers (Preece et al.,
2004). Because lurkers do not make their presence known to the group, this makes it difficult to obtain an
-
8/8/2019 Advantages and Disadvantages of Research on Internet
8/29
accurate sampling frame or an accurate estimate of the population characteristics.
As internet communities become more stable, some community administrators are beginning to compile
statistics on their community's participants. Many communities require a person to register with the
community in order to participate in discussions, and some communities are willing to provide researchers with
statistics about community membership (at least in aggregate form). Registration typically involves asking for
the individual's name, basic demographic information such as age and gender, and email address. Other
community administrators might ask participants for information about interests, income level, education, etc.
Some communities are willing to share participant information with researchers as a validation technique by
comparing the survey sample characteristics with those of the online community in general. Yet, because
individuals easily can lie about any information they report to community administrators, there is no guarantee
of accuracy.
When possible, using both online and traditional paper surveys helps to assess whether individuals responding
to the online version are responding in systematically different ways from those who completed the paper
version. For example, Query and Wright (2003) used a combination of online and paper surveys to study older
adults who were caregivers for loved ones with Alzheimer's disease. The researchers attempted to assess
whether the online responses were skewed in any way by comparing the responses from both subsamples.
While no significant differences between the two subsamples were found in this particular study, real
differences in responses between Internet users and non-Internet users might exist in other populations. This
may make it difficult to assess whether the observed differences are due to factors such as participant
deception or actual differences due to characteristics associated with computer and non-computer users.
Other Sampling Concerns
Although some studies of online survey methods have found that response rates in email surveys are equal to
or better than those for traditional mailed surveys (Mehta & Sivadas, 1995; Stanton, 1998; Thompson,
Surface, Martin, Sanders, 2003), these findings may be questionable because non-response rate tracking is
-
8/8/2019 Advantages and Disadvantages of Research on Internet
9/29
difficult to ascertain in most large online communities (Andrews et al., 2003). One relatively inexpensive
technique used by market researchers to increase response rates is to offer some type of financial incentive,
e.g., a lottery. Individuals who participate in the survey are given a chance to win a prize or gift certificate,
and the winner is selected randomly from the pool of respondents. However, this technique is not without
problems. Internet users frequently encounter bogus lotteries and other "get rich quick" schemes online, so a
lottery approach to increasing response rates could potentially undermine the credibility of the survey. In
addition, offering a financial incentive may increase multiple responses to the survey as participants try to
"stack the deck" to increase their chances of winning (Konstan, et al., 2005). Straight incentives such as a
coupon redeemable for real merchandise, i.e., books, may be more effective and more credible.
Self-selection bias is another major limitation of online survey research (Stanton, 1998; Thompson et al.,
2003; Wittmer et al., 1999). In any given Internet community, there are undoubtedly some individuals who
are more likely than others to complete an online survey. Many Internet communities pay for community
operations with advertising. This can desensitize participants to worthwhile survey requests posted on the
website. In short, there is a tendency of some individuals to respond to an invitation to participate in an online
survey, while others ignore it, leading to a systematic bias.
These sampling issues inhibit researchers' ability to make generalizations about study findings. This, in turn,
limits their ability to estimate population parameters, which presents the greatest threat to conducting
probability research. For researchers interested only in conducting nonprobability research, these issues are
somewhat less of a concern. Researchers who use nonprobability samples assume that they will not be able to
estimate population parameters.
Many of the problems discussed here are not unique to online survey research. Mailed surveys suffer from the
same basic limitations. While a researcher may have a person's mailing address, he or she does not know for
certain whether the recipient of the mailed survey is the person who actually completes and returns it
(Schmidt, 1997). Moreover, respondents to mailed surveys can misrepresent their age, gender, level of
-
8/8/2019 Advantages and Disadvantages of Research on Internet
10/29
education, and a host of other variables as easily as a person can in an online survey. Even when the precise
characteristics of a sample are known by the researcher, people can still respond in socially desirable ways or
misrepresent their identity or their true feelings about the content of the survey.
The best defense against deception that researchers may have is replication. Only by conducting multiple
online surveys with the same or similar types of Internet communities can researchers gain a reliable picture
of the characteristics of online survey participants.
Access Issues
Some researchers access potential participants by posting invitations to participate in a survey on community
bulletin boards, discussion groups, and chat rooms. However, members of online communities often find this
behavior rude or offensive (Hudson & Bruckman, 2004), or consider this type of posting to be "spam"
(Andrews et al., 2003). A community moderator may delete the unwanted post, or the researcher may be
inundated with emails from irate members of the community. Researchers using email invitations to participate
in a survey may face similar rejection. An unwanted email advertisement is often considered an invasion of
privacy. The invitation for the survey may be deleted, or the researcher may receive email from participants
complaining about it.
Some participants in Internet communities actually welcome studies by researchers, especially when members
are interested in how their community is perceived by others. With some diplomatic dialogue initiated by the
researcher, it is often possible to work with web community administrators and participants when proposing a
study idea (Reid, 1996). This is a more ethnographic approach. Although accessing some online communities
can be extremely challenging, seeking permission from the community and taking time to explain the purpose
of the study might help a researcher to gain access. Nonetheless, it may take a long time before receiving a
response to a request, and community sponsors may reject the researcher's request despite his or her
attempts to convey the possible benefits of the study for the community (Andrews et al., 2003). Researchers
might apologize in advance for the potentially unwanted posting, with an explanation of the importance of
-
8/8/2019 Advantages and Disadvantages of Research on Internet
11/29
conducting the research and possible benefits to members.
Researchers can foster "good will" between themselves and community participants by offering to provide
information about the results of their study to the community. One way to do this is to create a study report,
highlighting the most interesting results to the online community audience, post it on a web page, and have
community administrators post a link to the page on the community web site. Study results should be
presented so that audience members can understand them. For example, the author of this article created a
summary of research findings for the SeniorNet community after completing a study of social support among
participants (Wright, 2000a). SeniorNet administrators created a special link to this web page so that the
participants in the study (as well as other SeniorNet members) could learn about the results and their possible
implications.
It is important for researchers to include contact information, information about the study, and something
about their credentials when creating an invitation to participate in a survey. In addition to being a
requirement of most institutional research review boards in universities in the United States, this helps to
enhance the credibility of the survey and it can create opportunities for email interaction between the
researcher and participants. This is valuable, especially when participants have questions. However, as
Andrews et al. (2003) point out, providing researcher contact information has its downside. Researchers can
sometimes become the targets of abusive individuals who resent the invasion of privacy when they encounter
an online survey. Hate email or worse types of abuse can occur if some individuals on the Internet find online
surveys offensive. A man once called the author's home phone number repeatedly and left threatening
messages on his voice mail after obtaining the number from his department secretary (the department number
appeared on the informed consent for the online survey). While such incidents tend to be rare, the possibility
of irate responses is something to consider.
The above does not necessarily constitute an exhaustive review of the advantages and disadvantages of
conducting online surveys, although it represents experiences encountered by many researchers, and points to
-
8/8/2019 Advantages and Disadvantages of Research on Internet
12/29
factors that should be taken into consideration in deciding to use and designing an online survey. The next
section surveys current web survey software packages and online survey-related services available to
researchers who may be interested in conducting online survey research.
Current Web Survey Software Packages and Online Survey-Related Services
As noted above, today's researchers have help with online survey work. There are currently dozens of online
survey software packages and web survey services available to researchers willing to pay for them. Table 1
lists 20 of the more prominent packages and services, along with their web addresses.
Active Websurvey www.activewebsoftwares.com/activewebsurvey/
Apian Software http://www.apian.net/
CreateSurvey www.createsurvey.com
EZSurvey www.raosoft.com
FormSite www.formsite.com
HostedSurvey www.hostedsurvey.com
InfoPoll www.infopoll.net/
InstantSurvey www.netreflector.com
KeySurvey www.keysurvey.com
Perseus www.perseus.com
PollPro www.pollpro.com
Quask www.quask.com
http://www.activewebsoftwares.com/activewebsurvey/http://www.apian.net/http://www.createsurvey.com/http://www.raosoft.com/http://www.formsite.com/http://www.hostedsurvey.com/http://www.infopoll.net/http://www.netreflector.com/http://www.keysurvey.com/http://www.perseus.com/http://www.pollpro.com/http://www.quask.com/http://www.activewebsoftwares.com/activewebsurvey/http://www.apian.net/http://www.createsurvey.com/http://www.raosoft.com/http://www.formsite.com/http://www.hostedsurvey.com/http://www.infopoll.net/http://www.netreflector.com/http://www.keysurvey.com/http://www.perseus.com/http://www.pollpro.com/http://www.quask.com/ -
8/8/2019 Advantages and Disadvantages of Research on Internet
13/29
Ridgecrest www.ridgecrestsurveys.com
SumQuest www.sumquest.com/
SuperSurvey www.supersurvey.com
SurveyCrafter www.surveycrafter.com
SurveyMonkey www.surveymonkey.com
SurveySite www.surveysite.com
WebSurveyor www.websurveyor.com
Zoomerang www.zoomerang.com
Table 1. Twenty web survey companies
The author examined each of the websites for these 20 online survey product and service companies in order
to assess current features, pricing, and limitations, as well as to identify current trends in the online survey
product and services market. Table 2 presents a comparison of features, pricing issues, and limitations of the
20 online product and service companies.
CompanyName/Product
Features Pricing Service Limitations
ActiveWebsurvey
Unlimited surveys; softwareautomatically generatesHTML codes for surveyforms
Information unavailable onwebsite
Customer required to purchasesoftware; limited to 9 questionformats
Apian Software Full service web design andhosting available
$1195 up to $5995depending on number ofsoftware users; customercharged for technicalsupport
Customer required to purchasesoftware
CreateSurvey Standard features;
educational discount
$99 a month for unlimited
surveys and responses;free email support
Survey housed on company
server for a set amount oftime
http://www.ridgecrestsurveys.com/http://www.sumquest.com/http://www.supersurvey.com/http://www.surveycrafter.com/http://www.surveymonkey.com/http://www.surveysite.com/http://www.websurveyor.com/http://www.zoomerang.com/http://www.ridgecrestsurveys.com/http://www.sumquest.com/http://www.supersurvey.com/http://www.surveycrafter.com/http://www.surveymonkey.com/http://www.surveysite.com/http://www.websurveyor.com/http://www.zoomerang.com/ -
8/8/2019 Advantages and Disadvantages of Research on Internet
14/29
EZSurvey Unlimited surveys; mobilesurvey technology available;educational discount
$399 for basic software;additional software isextra; telephone training is$150 an hour
Customer required to purchasesoftware
FormSite Weekly survey traffic report;
multiple language support
$9.95 up to $99.95 per
month depending ondesired number of response
Survey housed on company
server for only a set amount oftime; limited number ofresponse per month
HostedSurvey Standard features;educational discount
Charge is per number ofresponses; first 250response are free, then
around $20 every 50responses.
Survey housed on companyserver for only a set amount oftime
InfoPoll Standard features; Software
can be downloaded for free
Information unavailable on
website; limited customersupport; training availablefor a fee
Software can be downloaded
free, but works best onInfoPoll server; customersappear to be charged for usingInfoPoll server
InstantSurvey Standard features; supportsmultimedia
Information unavailable onwebsite; free 30 day trial
Survey housed on companyserver for only a set amount oftime
KeySurvey Online focus group feature;unlimited surveys
$670 per year for a basicsubscription; free 30 daytrial
Survey housed on companyserver for only a set amount oftime; limited to 2000responses
Perseus Educational discount; mobilesurvey technology available
Information unavailable onwebsite; free 30 day trial
Survey housed on companyserver for only a set amount oftime
PollPro Standard features; unlimitedsurveys
$249 for single user;access to PollPro server isan additional fee
Customer required to purchasesoftware
Quask Supports multimedia $199 for basic software;access to Quask server foran additional fee
Customer required to purchasesoftware; more advancedfeatures only come with higherpriced software
Ridgecrest Standard features;educational discount
$54.95 for 30 days Survey housed on companyserver for only a set amount oftime; limited to 1000responses for basic package
-
8/8/2019 Advantages and Disadvantages of Research on Internet
15/29
SumQuest Standard features; userguidebook for creatingquestionnaire available
$495 to purchase software;free unlimited telephonesupport
Customer required to purchasesoftware
SuperSurvey Standard features $149 per week for basicpackage.
Survey housed on companyserver for only a set amount of
time; 2000 response per weeklimit
SurveyCrafter Standard features;educational discount
$495 for basic softwarepackage; free andunlimited technical support
Customer required to purchasesoftware
SurveyMonkey Standard features; unlimitedsurveys
$20 a month for a basicsubscription; free emailsupport
Survey housed on companyserver for a set amount oftime; limited to 1000 initialresponses
SurveySite Company helps with all
aspects of survey design,data collection and analysis;online focus group feature
Information unavailable on
website
Company staff rather than
customer create and conductsurvey
WebSurveyor Standard features; unlimitedsurveys
$1,495 per year forsoftware license
Customer required to purchasesoftware
Zoomerang Standard features;educational discount
$599 for software Customer required to purchasesoftware
Table 2. Comparison of online survey software and services
This is not, of course, an exhaustive list of online survey software and service businesses. However, it
represents a good cross-section of the types of online survey products and services currently available to
researchers. The following sections consider some of the current features of online survey products and
services, pricing issues, limitations, and the implications of using these products and services for online survey
research.
Current Features
Survey Creation Software vs. Expanded Services
The businesses listed in Tables 1 and 2 offer researchers two basic options for creating and conducting online
-
8/8/2019 Advantages and Disadvantages of Research on Internet
16/29
survey research. One option is the online survey software packages, which are computer programs that
researchers use to create and conduct online surveys on their own computer and server. The companies that
offer such packages also provide options for customer support, server space for the online survey (in some
cases), and several data tracking and analysis options. Other companies offer a wider range of services for
conducting online surveys, including research design, online questionnaire development, sampling and data
collection services, and data analysis and interpretation services. The major features and problems with each
option are discussed below.
Purchasing Software Option
Some companies (see Table 2) require customers to purchase online survey creation software. Owning the
software enables researchers to create multiple online surveys of any length as opposed to being charged per
survey, per time period (e.g., by the month), by number of responses, by survey length, or by some
combination of these options. Many of these companies also offer customer support, including help with
design, data collection, participant tracking, and data analysis. One disadvantage of owning the software is
that customers have to pay to upgrade software. Given rapid advances in software development, this software
may be outdated in a relatively short period of time. Customers who have purchased software receive
discounted upgrades, however. An example of this option is EZ Survey, which allows researchers to run the
software on their own computer and a server of their choice. This may be an attractive choice for researchers
who have access to free server space on their university or research organization server.
Online Questionnaire Features
The businesses listed in Tables 1 and 2 offer a wide array of options for creating online surveys, including
many different templates to help first-time web survey researchers. Each of the online survey products
reviewed offered some type of online form to collect data from participants. A "form" is an interactive type of
web page that allows Internet users to send information across computer networks. After completing an online
survey, participants click on a "submit" button on the webpage. This transmits the survey responses to the
researcher. Online survey questions are the same types as on a traditional paper/pencil questionnaire, only the
-
8/8/2019 Advantages and Disadvantages of Research on Internet
17/29
participants submit the information over the Internet rather than return questionnaires in person or by mail.
Common Gateway Interface (CGI) scripting, a type of computer language that is run on the Web server where
an online survey is housed, tells the server how to process information that is submitted.
Most Internet users are quite familiar with Web forms since search engines, including Yahoo! and Google, are
sophisticated forms. Writing scripts for processing forms can be done manually, but this type of work can be
cumbersome for a busy researcher, especially one who is not technologically proficient. All of the reviewed
companies offering online survey products provide a variety of useful questionnaire options, and a user-
friendly process to develop online questionnaires.
The businesses listed in Tables 1 and 2 typically offered a range of question types, although the number of
options varied from business to business. Basic question options usually include Likert-type scales, semantic
differential scales, checklists, textboxes (for qualitative responses), drop-down menus (for nominal or
categorical items), and filter questions (to tailor surveys to individual characteristics of survey respondents). In
addition, the majority of the reviewed products offer randomized answer choices for participants, so as to vary
the order of question responses and thus reduce question order bias.
Some products support multiple language versions of an online survey and versions for visually impaired
respondents. Additionally, some products offer more complex question-type options, such as multiple response
matrices and the ability to use multimedia, i.e., asking participants to respond to a video or audio clip. A
multimedia video or audio clip can be used to jog the memories of respondents or as a reference point for
participant responses. For example, researchers who want to measure participant perceptions of a political
candidate's positions on foreign policy could include a video clip from a recent speech. Multimedia can also be
useful when targeting low literacy populations, since video and audio messages guide participants through an
online survey. However, including multimedia can increase download times and may be frustrating to
participants who must download media players or other types of programs in order to participate in the survey
(Andrews et al., 2003). Taking the use of multimedia a bit further, the technology exists to easily construct a
-
8/8/2019 Advantages and Disadvantages of Research on Internet
18/29
-
8/8/2019 Advantages and Disadvantages of Research on Internet
19/29
offer other types of features to aid with the survey research process. For example, EZ Survey offers a free
sample size calculator, and businesses such as SurveyMonkey offer pop-up advertising to aid in recruiting
participants. Some companies, such as InstantSurvey, unsubscribe respondents from an email list after they
have completed a survey, which may help to reduce multiple responses from the same participant.
Several of the companies offer researchers even more sophisticated options for conducting survey research.
Perseus can conduct mobile surveys, using wireless handheld devices like Palm pilots. Data are sent through
wireless technology to a server (similar to other online survey forms) where the information is posted to a
database file. Mobile Internet surveys offer a number of advantages to researchers. Using a wireless device (as
opposed to a laptop computer), researchers can bring a survey to otherwise inaccessible populations in the
non-virtual world, such as patients in a busy healthcare setting, individuals in rural settings, or socioeconomic
groups that do not have access to computers or the Internet. This allows researchers to conduct face-to-face
interviews with participants while using the wireless device to store and transmit responses to the survey to a
database. In addition, some companies, such as Perseus, have the capability to merge computer technology
with traditional survey methods. They offer telephone survey capabilities where participants use a touch tone
phone to enter responses.
Other companies, such as KeySurvey and SurveySite, provide the ability to conduct online focus groups. The
Internet allows researchers to include participants from multiple geographic locations in the same focus group.
Participants view the same video, audio, and/or text in real time from remote locations. Researchers can
interact with participants via chat room applications or webcam and audio teleconferencing technologies. Real-
time computer applications are important in focus groups because researchers want participants to interact
with the focus group facilitator and with each other at the same time. The responses of one participant can
trigger ideas and responses among other participants, leading to richer results. These qualitative focus groups
are often used as a precursor to developing a quantitative survey to reach broader numbers of individuals.
Pricing
-
8/8/2019 Advantages and Disadvantages of Research on Internet
20/29
Costs of survey products and their services vary. In general, the more features and services needed from a
web survey company, the more it will cost. However, it is a "buyer beware" situation. Basic features can be
purchased for a relatively small amount of money. For instance, SurveyMonkey provides authoring tools,
server space, and simple automated survey analysis for about $20 a month ($240 per year); however, there
are limitations, such as the limitation of 1000 responses per month. SurveyMonkey charges an additional 5
cents per survey response over the 1000 response limit. Moreover, paying more does not necessarily mean
more services. Other businesses, such as KeySurvey, charge substantially more ($670 per year for a basic
subscription) for products and services similar to those offered by SurveyMonkey. Other companies charge
researchers by the survey. Companies that charge less typically do not recruit participants for customers and
do not provide consultation throughout all stages of the research process. However, for many web survey
researchers, these basic, less expensive approaches may be sufficient, especially for those experienced in
conceptualizing survey projects, data analysis, and interpretation of results. In general, if sample generation or
help with analyzing data is not needed, then businesses that include these services in the price should be
avoided, or else these services should be negotiated out of the price. Pricing for the businesses reviewed here
varied considerably even though they offered similar products, features, and services. For example,
SuperSurvey offers products, features, and services similar to SurveyMonkey for $500 to $2000 per business
quarter (depending upon number of users and number of responses desired), as opposed to only $20 a month.
As previously stated, while most companies offer free technical support, researchers are generally charged
extra for extended training and consultation. In some cases, consultation can be expensive. For example,
Perseus charges $2000 per day for personalized training, but also offers discounts for group consultation and
training. Moreover, many of the reviewed business websites offer educational discounts for academics,
including discounts on software, as well as consultations and other support services. For example, Zoomerang
offers educators one year access to their online web survey creation services, server space for surveys, and
customer support for around $350 (about $100 less than the regular price for service). Other business, such as
Perseus and SurveyCrafter, advertise educational discounts on a wider variety of services. Researchers should
-
8/8/2019 Advantages and Disadvantages of Research on Internet
21/29
inquire about these special discounts since they may help to reduce the overall cost of purchasing web survey
software or services.
Limitations
As noted above, there may be limitations associated with using web survey products and services. Some
specific limitations include issues of time, space, and number of responses allowed for a given price. For
example, companies such as SurveyMonkey and SuperSurvey will host an online survey for a set amount of
time. If a researcher wants to keep a survey on the company's server for an extended period of time (such as
more than a year), this costs extra. In addition, some companies often charge more for longer surveys and for
a number of respondents exceeding a certain amount (generally over 1000). Purchased software, in contrast,
generally does not have space or response number restrictions.
There are also generally limitations to the amount of free customer support a researcher can obtain. Customer
support may be available for minor technical problems and customer questions, but customers are generally
charged extra for extended consultations and training. Typically, minor questions can be answered for free via
telephone, email, or chat applications, but a researcher may be charged for extensive training, such as
learning advanced web page creation techniques or data analysis instruction. Researchers who use a
company's email lists to generate a sample are limited by the quality of this type of sampling frame. In cases
where a company uses the same lists again and again for different clients, the individuals who receive the
advertisements about a survey on these lists may become weary of being targeted by multiple surveys, and
this could negatively impact response rates.
Implications of Using Web Survey Products and Services
Current web survey products and services have greatly facilitated the process of creating and conducting
online surveys. Researchers can save considerable time by utilizing the products and services that are offered
by many of the businesses highlighted in this article, compared to the time that it would take most researchers
-
8/8/2019 Advantages and Disadvantages of Research on Internet
22/29
to create an online survey themselves using a web authoring program, thanks to a variety of attractive
features offered. The cost of these products and services varies depending on the types of features and
services a researcher desires. As with purchasing any product or service, researchers should assess their
research needs, budget, and research time frame, and comparison shop when deciding on which business to
use.
As we have seen, however, these products and services are not without limitations. While attractive, features
of the surveys themselves (such as multimedia) and the services (e.g., using company email lists to generate
samples) offered by web survey business can affect the quality of data in a variety of ways. Furthermore,
using these products and services does not necessarily circumvent the disadvantages of online surveys,
including issues related to sampling frames, response rates, participant deception, and access to populations.
In short, researchers should view current web survey products and services as another research tool that-like
the online survey itself-has its own unique advantages and disadvantages.
Acknowledgments
The author would like to thank the anonymous reviewers for their helpful and insightful suggestions for
improving this manuscript.
References
Ahuja, M. K., & Carley, K. M. (1998). Network structure in virtual organizations.Journal of Computer-Mediated
Communication, 3 (4). Retrieved April 18, 2005 from http://jcmc.indiana.edu/vol3/issue4/ahuja.html
Andrews, D., Nonnecke, B., & Preece, J. (2003). Electronic survey methodology: A case study in reaching
hard-to-involve Internet users. International Journal of Human-Computer Interaction, 16 (2), 185-210.
Bachmann, D., & Elfrink, J. (1996). Tracking the progress of e-mail versus snail-mail. MarketingResearch, 8
(2), 31-35.
http://jcmc.indiana.edu/vol3/issue4/ahuja.htmlhttp://jcmc.indiana.edu/vol3/issue4/ahuja.html -
8/8/2019 Advantages and Disadvantages of Research on Internet
23/29
Braithwaite, D. O., Waldron, V. R., & Finn, J. (1999). Communication of social support in computer-mediated
groups for people with disabilities. Health Communication, 11 (2), 123-151.
Couper, M. P. (2000). Web-based surveys: A review of issues and approaches. Public Opinion Quarterly, 64
(4), 464-494.
Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons.
Flaherty, L. M., Pearce, K. J., & Rubin, R. R. (1998). Internet and face-to-face communication: Not functional
alternatives. Communication Quarterly, 46 (3), 250-268.
Flanagin, A., & Metzger, M. (2001). Internet use in contemporary media environment. Human Communication
Research, 27(1), 153-181.
Fox, S., Rainie, L., Larsen, E., Horrigan, J., Lenhart, A., Spooner, T., & Carter, C. (2001). Wired Seniors. The
Pew Internet and American Life Project. Retrieved May 4th 2005 from:
http://www.pewinternet.org/pdfs/PIP_Wired_Seniors_Report.pdf
Furlong, M. S. (1989). An electronic community for older adults: The SeniorNet network.Journal of
Communication, 39 (3), 145-153.
Garton, L., Haythornthwaite, C., & Wellman, B. (1999). Studying on-line social networks. In S. Jones (Ed.),
Doing Internet Research: Critical Issues and Methods for Examining the Net(pp. 75-105). Thousand Oaks, CA:
Sage.
Hamilton, J. C. (1999). The ethics of conducting social science research on the Internet. The Chronicle of
Higher Education, 46 (15), B6-7.
Hobman, E. V., Bordia, P., Irmer, B., & Chang, A. (2002). The expression of conflict in computer-mediated and
http://www.pewinternet.org/pdfs/PIP_Wired_Seniors_Report.pdfhttp://www.pewinternet.org/pdfs/PIP_Wired_Seniors_Report.pdf -
8/8/2019 Advantages and Disadvantages of Research on Internet
24/29
face-to-face groups. Small Group Research, 33 (4), 439-465.
Hollingshead, A. B., McGrath, J. E., & O'Connor, K. M. (1993). Group task performance and communication
technology: A longitudinal study of computer-mediated versus face-to-face work groups. Small Group
Research, 24 (3), 307-333.
Horrigan, J. B. (2001). Online communities: Networks that nurture long-distance relationships and local ties.
Pew Internet and American Life Project. Retrieved October 1, 2004 from
http://www.pewInternet.org/reports/toc.asp?Report=47
Howard, P., Rainie, L., & Jones, S. (2001). Days and nights on the Internet: The impact of a diffusing
technology.American Behavioral Scientist, 45(3), 383-404.
Hudson, J. M., & Bruckman, A. (2004). "Go away:"? Participant objections to being studied and the ethics of
chatroom research. The Information Society, 20 (2), 127-139.
Konstan, J. A., Rosser, B. R. S., Ross, M. W., Stanton, J., & Edwards, W. M. (2005). The story of subject
naught: A cautionary but optimistic tale of Internet survey research.Journal of Computer-Mediated
Communication, 10 (2), article 11. Retrieved April 25, 2005 from
http://jcmc.indiana.edu/vol10/issue2/konstan.html
Lee, J. Y. (1996). Charting the codes of cyberspace: A rhetoric of electronic mail. In L. Strate, R. Jacobson, &
S. B. Gibson (Eds.), Communication and Cyberspace: Social Interaction in an Electronic Environment(pp. 275-
296). Cresskill, NJ: Hampton Press.
Llieva, J., Baron, S., & Healey, N. M. (2002). Online surveys in marketing research: Pros and cons.
International Journal of Market Research, 44 (3), 361-367.
Matheson, K. (1991). Social cues in computer-mediated negotiation: Gender makes a difference. Computers in
http://www.pewinternet.org/reports/toc.asp?Report=47http://jcmc.indiana.edu/vol10/issue2/konstan.htmlhttp://www.pewinternet.org/reports/toc.asp?Report=47http://jcmc.indiana.edu/vol10/issue2/konstan.html -
8/8/2019 Advantages and Disadvantages of Research on Internet
25/29
Human Behavior, 7(3), 137-145.
Mehta, R., & Suvadas, E. (1995). Comparing response rates and response content in mail versus electronic
mail surveys.Journal of the Market Research Society, 37(4), 429-439.
Mutahseb, A. (2004). The Internet as an Alternative Source of Information and Alternative Forum of
Expression for Arab Americans. Unpublished Ph.D. dissertation. University of Memphis.
Nie, N. H., & Erbring, L. (2000). Internet and Society: A Preliminary Report. Palo Alto, CA: Stanford Institute
for the Quantitative Study of Society Press.
Nie, N., Hillygus, S. & Erbring, L. (2002). Internet use, interpersonal relations and sociability: Findings from a
detailed time diary study. In B. Wellman (Ed.), The Internet in Everyday Life (pp. 215-243). London: Blackwell
Publishers.
Nonnecke, B., Preece, J., Andrews, D., & Voutour, R. (2004, August). Online Lurkers Tell Why. Paper
presented at the 2004 Americas Conference on Information Systems, New York City, NY.
Parks, M. R., & Floyd, K. (1996). Making friends in cyberspace.Journal of Communication, 46 (1), 80-97.
Preece, J. (1999). Empathetic communities: Balancing emotional and factual communication. Interacting with
Computers: The Interdisciplinary Journal of Human-Computer Interaction, 12 (1), 63-77.
Preece, J. J., & Ghozati, K. (2001). Experiencing empathy on-line. In R. E. Rice & J. E. Katz (Eds.), The
Internet and Health Communication: Experiences and Expectations (pp. 237-260). Thousand Oaks, CA: Sage.
Preece, J., Nonnecke, B., & Andrews, D. (2004). The top five reasons for lurking: Improving community
experiences for everyone. Computers in Human Behavior, 20 (2), 201-223.
Query, J. L., Jr., & Wright, K. B. (2003). Assessing communication competence in an on-line study: Toward
-
8/8/2019 Advantages and Disadvantages of Research on Internet
26/29
informing subsequent interventions among older adults with cancer, their lay caregivers, and peers. Health
Communication, 15(2), 205-219.
Reid, E. (1996). Informed consent in the study of on-line communities: A reflection on the effects of computer
mediated research. The Information Society, 12, 169-174.
Rice, R. E., & Katz, J. E. (2001). The Internet and Health Communication: Experiences and Expectations.
Thousand Oaks, CA: Sage.
Schmidt, W. C. (1997). World-Wide Web survey research: Benefits, potential problems, and solutions.
Behavior Research Methods, Instruments and Computers, 29 (2), 274-279.
Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51
(3), 709-725.
Taylor, H. (2000). Does Internet research work? Comparing electronic survey results with telephone survey.
International Journal of Market Research, 42 (1), 51-63.
Thompson, L. F., Surface, E. A., Martin, D. L., & Sanders, M. G. (2003). From paper to pixels: Moving
personnel surveys to the Web. Personnel Psychology, 56 (1), 197-227.
Tidwell, L. C., & Walther, J. B. (2002). Computer-mediated communication effects on disclosure, impressions
and interpersonal evaluations: Getting to know one another a bit at a time. Human Communication Research,
28 (3), 317-348.
Walther, J. B. (1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal
interaction. Communication Research, 23 (1), 3-43.
Walther, J. B., & Boyd, S. (2002). Attraction to computer-mediated social support. In C. A. Lin & D. Atkin
(Eds.), Communication Technology and Society: Audience Adoption and Uses (pp. 153-188). Cresskill, NJ:
-
8/8/2019 Advantages and Disadvantages of Research on Internet
27/29
Hampton Press.
Walther, J. B., & Burgoon, J. K. (1992). Relational communication in computer-mediated interaction. Human
Communication Research, 19 (1), 50-88.
Watt, J. H. (1999). Internet systems for evaluation research. In G. Gay & T. L. Bennington (Eds.), Information
Technologies in Evaluation: Social, Moral, Epistemological, and Practical Implications (pp. 23-44). San
Francisco: Jossey-Bass.
Wellman, B. (1997). An electronic group is virtually a social network. In S. Kiesler (Ed.), Culture of the
Internet(pp. 179-205). Mahwah, NJ: Lawrence Erlbaum.
Wellman, B., & Haythornthwaite, C. (Eds.) (2002). The Internet in Everyday Life. Oxford, UK: Blackwell.
Whittaker, S., & Sidner, C. (1997). Email overload: Exploring personal information management of email. In S.
Kiesler (Ed.), Culture of the Internet(pp. 277-295). Mahwah, NJ: Lawrence Erlbaum Associates.
Witmer, D. F., Colman, R. W., & Katzman, S. L. (1999). From paper-and-pencil to screen-and-keyboard:
Toward a methodology for survey research on the Internet. In S. Jones (Ed.), Doing Internet Research: Critical
Issues and Methods for Examining the Net(pp. 145-161). Thousand Oaks, CA: Sage.
Wood, A. F., & Smith, M. J. (2001). On-line Communication: Linking Technology, Identity, and Culture.
Mahwah, NJ: Lawrence Erlbaum Associates.
Wright, K. B. (1999). Computer-mediated support groups: An examination of relationships among social
support, perceived stress, and coping strategies. Communication Quarterly, 47(4), 402-414.
Wright, K. B. (2000a). Computer-mediated social support, older adults, and coping.Journal of Communication,
50 (3), 100-118.
-
8/8/2019 Advantages and Disadvantages of Research on Internet
28/29
Wright, K. B. (2000b). Perceptions of on-line support providers: An examination of perceived homophily,
source credibility, communication and social support within on-line support groups. Communication Quarterly,
48 (1), 44-59.
Wright, K. B. (2000c). The communication of social support within an on-line community for older adults: A
qualitative analysis of the SeniorNet community. Qualitative Research Reports in Communication, 1 (2), 33-43.
Wright, K. B. (2000d). Social support satisfaction, on-line communication apprehension, and perceived life
stress within computer-mediated support groups. Communication Research Reports, 17(2), 139-147.
Wright, K. B. (2002a). Social support within an on-line cancer community: An assessment of emotional
support, perceptions of advantages and disadvantages, and motives for using the community.Journal of
Applied Communication Research, 30 (3), 195-209.
Wright, K. B. (2002b). Motives for communication within on-line support groups and antecedents for
interpersonal use. Communication Research Reports, 19 (1), 89-98.
Wright, K. B. (2004). On-line relational maintenance strategies and perceptions of partners within exclusively
Internet-based and primarily Internet-based relationships. Communication Studies, 55(2), 418-432.
Yun, G. W., & Trumbo, C. W. (2000). Comparative response to a survey executed by post, email, and web
form.Journal of Computer-Mediated Communication, 6 (1). Retrieved April 18, 2005 from
http://jcmc.indiana.edu/vol6/issue1/yun.html
About the Author
Kevin B. Wright (Ph.D. University of Oklahoma 1999) is an assistant professor in the Department of
Communication at the University of Oklahoma. The majority of his research has focused on computer-
mediated support groups and health outcomes for people with health concerns, interpersonal communication
issues within the context of computer-mediated interaction, and online survey methodology. More information
http://jcmc.indiana.edu/vol6/issue1/yun.htmlmailto:[email protected]://jcmc.indiana.edu/vol6/issue1/yun.htmlmailto:[email protected] -
8/8/2019 Advantages and Disadvantages of Research on Internet
29/29
is available at: http://www.ou.edu/cas/deptcomm/facpages/wright.html
Address: Department of Communication, 610 Elm Avenue, Room 101, University of Oklahoma, Norman, OK
73019 USA
2005 Journal of Computer-Mediated Communication
http://www.ou.edu/cas/deptcomm/facpages/wright.html%20http://www.ou.edu/cas/deptcomm/facpages/wright.html%20