1/30

Characterizing Audience for Informational Website Design:

A Case Study

Jennifer Turns, Ph.D.

Assistant Professor, Technical Communication

Faculty AffiliateProgram for Educational Transformation through Technology (PETTT)

Center for Engineering Learning and Teaching (CELT)

Acknowledgements: This work has been supported by the Program for Educational Transformation through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Tracey Wagner, Aaron Louie, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

2/30

Overarching UCD Questions

• How do design teams incorporate information about users into their design process?

• When characterizing users/audience? – Which dimensions are most significant?

– Which methods under which circumstances?

– How will the insights inform design?

– What are realistic expectations?

3/30

Informational Website Design

Some of my projects in this domain…

• Arthritis Source: Medical information on the web

• Teaching challenges of engineering faculty: Information for educators

• Also– Legal information online– Information about architecture building methods

4/30

Today’s Goal

• Describe Arthritis Source, informational website

• Describe our approach to audience analysis

• Present sample of results from our audience analysis, and their impacts on the design

• Discuss moving beyond the case study

Use a case study of audience analysis for website design to reflect on decisions in audience analysis

5/30

Arthritis Source

• Developed in 1995 by Dr. Frederick Matsen

• Focus on arthritis• Authorized information• User-centered information

• Research test bed

6/30

Arthritis Source Content

• Articles as basic metaphor– Content is organized into articles – Examples: Rheumatoid Arthritis,

Rotator Cuff Surgery, Pregnancy

• Templates underlie articles:– The template for each article is the set of

questions answered in each article– All content is based loosely on five

templates: conditions, surgery, treatment, living with, medication

– Templates are informed by research on user questions

• Content is dynamically generated

7/30

Arthritis Source Authoring

• Authors – Volunteer– Are subject matter experts (e.g., doctors)– Are not trained as technical communicators– Create content for entire template or specific question– Can create content online or in Word

• Authoring is actually distributed– Team creates templates based on user information– “Authors” create content– Administrator edits

8/30

Multiple Access Paths

• Multiple Access Paths– Navigation comes from

article templates– Spotlighted content– Question-based search

9/30

Embedded, Ongoing Evaluation

• Online survey– “Tell us about yourself”

• Quick polls– “How useful is this article”

• Online quizzes– “What do you know…”

• Online research studies– “Exploring use over time”

10/30

Arthritis Source Community

Community includes• Users

• Technical Communication Professionals

• Domain Experts (i.e., doctors)

• Learning Scientists

• Developers

• Administrator

Share responsibility for• Determining scope

• Creating content

• Evaluation quality

11/30

Arthritis Source Timeline

‘00 ‘01 ‘02

1995

Creation

Dynamic Content

Question-based

Navigation

Embedded

evaluations

Audien

ce Ana

lysis

Design

ing be

gins

Survey

Analys

is 1,

n~20

0Int

erview

Analys

is 1,

n=20

Survey

analy

sis 2,

n~40

0

Template-based

content

Design

Audience Analysis

Search Engine

Optimization

12/30

Audience Analysis

• Goals– Inform our own design and evaluation

– Contribute to broader discussion

• Decisions– Dimensions?

• Inform design, Speak to team, Theoretical traditions?

– What methods?• Breadth/Depth, Acknowledge distributed nature of users

13/30

Multidisciplinary Influences

• Dimensions– Roles

– Goals

– Knowledge

– Circumstances of Use

– Culture

– Ergonomics

• Theoretical Perspectives– Technical Communication

– Reader Response Theory

– Cognitive Science

– Constuctivism

– Distributed Intelligence

– Situated cognition

– Socio-Cultural Theory

– Human Factors

14/30

Defining Dimensions

• Role – Dominant persona of users (job, affiliation)

• Goals – Reason for the interaction

• Knowledge – The extent and nature of prior relevant knowledge

• Circumstances of Use – Setting, resources, strategy, timing

• Culture – Group level beliefs, language, preferences

• Ergonomics – Relevant perceptual & motor abilities, skills

15/30

Method – Online Survey• Questions: Adaptive, ~25 questions

• Participants: – Duration: 9/1/2000 – 7/2/2001 (10 months)– 472 respondents / 710 starts

• Analyses1 –

– Descriptive Statistics – Content Analysis – Qualitative Coding – Statistical Analysis

1Acknowledgments: Tracey Wagner, Kristina Liu, Alice Tanada, Kristen Schuyler

16/30

Method - Phone Interview

About Visit

• Could you tell me about your visit or visits to the Arthritis Source?

• Could you tell me what you were trying to do when you visited the Arthritis Source?

• Did you benefit from your visit or visits to the Arthritis Source?

• What kind of information do you think other arthritis patients should know?

About Knowledge of Condition

• Could you tell me what you think arthritis is in general?

• Could you tell me how RA/OA affects the body?

• Do you know what contributes to getting RA/OA?

• Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis?

• What is most difficult to understand about RA/OA?

17/30

Phone Interview

• Participants – 20 users (10 OA, 10 RA)

• Analyses1

– Conceptions/misconceptions– Overarching Goals – Specific Information Needs

1Acknowledgments: Tracey Wagner, Kristina Liu

18/30

Mapping Data to Categories

Data and Sources Role Goals Knowledge

Circum-stances of

Use CultureErgo-

nomicsOnline Survey

Visitor Type XXX XX X XAge X X X XXHome Community X XXGeographical Area X XXType of Arthritis X XXLevel of Education XXTime since diagnosis X XName of ConditionWhy visiting XX XXX XCame in from XXUse of site in past XX XXSources of information X XX

Phone InterviewKnowledge of Condition XXXGoals XXXSpecific Information Needs XXX

19/30

Results - Overview

• What we learned… – 19% international (culture)– 26% rural (circumstances of use)– 21% over 60 (ergonomic, through vision implications)

• Highlight specific examples where results had identifiable impact on design– Role– Goals– Knowledge– Circumstances of Use

20/30

Roles

• Users with many roles • “Person with arthritis” is too simplistic…– Person with pain

– Person with condition that they do not consider arthritis

– Person who is exploring whether they have arthritis

Relation10%

Medical Professional

5%

Researcher2%

Student1%

Other20%

Person with Arthritis

62%n=462/472

21/30

Role – Design Implications

• Added new types of information– Differential diagnosis

• Identified writing guidelines…– Avoid statements that assume reader has the

particular condition… (avoid -- “your condition” or “you need to…”)

22/30

Knowledge – Misconceptions?

• From interview data, we identified several possible misconceptions:– Low bone density is associated with Osteoarthritis

– Not drinking enough milk increases the risk of Osteoarthritis

– Bone spurs cause arthritic pain

– Osteoarthritis causes bone erosion.

– OA is caused by the "wear and tear" of the joints due to overuse and aging.

– There is little you can do

23/30

Knowledge – Design Implications

• Added material to confront misconceptions

• Added “common myths” question to generic templates

• Developed Osteoarthritis knowledge quiz– 2513 Respondents for question 1– 470 Respondents for entire 7-question quiz

24/30

Goals – Social Support?Description Percentage

(n=458)1. No indication for social support 75%

2. Implicit desire for social support (mostly in the form of direct arthritis questions raised and requests for other links/sites)

20%

3. Seeking doctor’s (or hospitals) referrals 2%

4. General desire to talk to people 3%

5. Request for live interaction on the site 0%

6. Seeking local, face-to-face support 0%

25/30

Goals – Design Implications

• On providing access to social support– Reduced priority for this within our system– Added content to help users find to support

• On learning from a surprise– 20% of statements were direct questions– Indicates need to help users start with own questions– Question-based search permits this directly– Templates are based on user questions

26/30

Circumstances of Use - Virtual

• Coming from… • Implications

– Improve placement of site on search engines

– Orient users coming from search engines - accelerated implementation of the navigation based on the article templates.

Bookmark7%

Website22%

Referred by6%

Search Engine47%

Other18%

n=372/472

27/30

Ongoing Audience Research

• Ongoing analysis of user questions (goals)– From emails– From question-based search– Collected earlier…

• Studying use over time through user online journaling (goals, circumstances of use, knowledge and learning)

28/30

Where we’ve been

• Starting Point:When characterizing users/audience…

• Which dimensions are most significant?

• Which methods under which circumstances?

• How will the insights inform design?

• What are realistic expectations?

• Contribution here: Case Study…

29/30

Moving beyond case studyFraming audience analysis decisions

• Effort: Manager perspective– Resources to design, collect, analyze, interpret, decide

• Insight: Researcher perspective– Rigorousness, representativeness, triangulation?

• Impact: Designer perspective– Clarity of links to design, Persuasiveness of the data

30/30

Designing Audience Analysis

• Effort: Manager perspective– Resources to design, collect, analyze, interpret, decide

• Insight: Researcher perspective– Rigorousness, representativeness, triangulation?

• Impact: Designer perspective– Clarity of links to design, Persuasiveness of the data

Observation: Opportunities lie here…

31/30

Web-based Health Information

• Site quality– Owner credentials, update dates (Hoffman, 2000)

• Quality of information– Comprehensiveness (e.g., Chen, 2000)– Accuracy (e.g., Chen, 2000)– Providing references (e.g., Hellawell, 2000)

• Findability of information– Time required (e.g., Gotwald, 2000)– Getting to real questions (e.g., Lechner, 1996)

• Need for evaluation methods– (e.g., Wu, 2000, Delamsthe, 2000, Charatan, 1999)

32/30

When users are learners?

• Characteristics of learners– Growth (Soloway, 1994)– Diversity (Soloway, 1994)– Motivation (Soloway, 1994)– Prior Understandings (NRC, 1999)

• Thinking about implications– What to know and why– Knowing users over time– Variations (how many to know)– Time required to truly “know”

Users

Learners