Avatars Versus Agents: A Meta-Analysis Quantifying the Effect of Agency on Social Influence

Jesse Fox, Sun Joo (Grace) Ahn, Joris H. Janssen, Leo Yeykelis, Kathryn Y. Segovia & Jeremy N. Bailenson

Presented by: Pantea Habibi

Outline

◉ Overview ◉ Research Design◉ Meta-Analysis◉ Results ◉ Implications◉ Critique & Discussion

2

Overview 1

3

Overview

◉ Virtual representations of people in computer-mediated interactions ○ Avatars ○ Agents○ Hybrids

◉ Ability to behave in a human-like manner◉ unique capabilities to influence users◉ modify users’ attitudes and behaviors

4

Overview

◉ Social influence ■ social norms ■ conformity ■ compliance ■ Identification

5

◉ Social presence◉ processes

○ Identification○ social categorization

◉ perceive the representation

Overview

◉ Social influence ■ social norms ■ conformity ■ compliance ■ Identification

6

◉ social presence◉ processes

○ Identification○ social categorization.

◉ perceive the representation

I have a feeling that there is another dimension to this in that whether it matters would depend on the specificity of the task. For some tasks, it may matter and some others it might not. For some types of tasks, the exact identity might matter and for others, a less specific variation like gender might matter.[Harish]

Research Design2

7

Research purpose

◉ Virtual representations○ education, health, marketing, and other persuasive

contexts○ important to identify the differences

◉ Determine whether perceived agency of a virtual representation affected social influence

8

Variables of Interest

9

◉ Perceived Control◉ To enrich the analyses and application of the findings, relevant

moderators included:○ Level of Immersion

■ Immersive/Desktop○ Type of Measure

■ Objective(behavioral)/Subjective(self-reported) DV○ Task Type

■ Competitive/Cooperative/Neutral○ Actual Control○ Time

Variables of Interest

10

◉ Perceived Control◉ To enrich the analyses and application of the findings, relevant

moderators included:○ Level of Immersion

■ Immersive/Desktop○ Type of Measure

■ Objective(behavioral)/Subjective(self-reported) DV○ Task Type

■ Competitive/Cooperative/Neutral○ Actual Control○ Time

I like how they could dissociate the actual content of the papers and just talk about high level goals of research they wanted to achieve. However, would the context / content affect the meta-analysis? (Comparatively the second paper, explained thoroughly each papers context and how it blended or did not with the other).[Aditi]

Research Question

◉ Will agency effects on social influence vary between objective variables and subjective variables?

◉ Will actual control moderate the effect of perceived control on social influence?

11

Hypothesis

◉ Avatars will yield greater influence than agents.◉ Studies conducted in immersive virtual environments will

demonstrate smaller differences in the effects of agency than studies conducted in desktop environments.

◉ Competitive tasks will show the greatest differences in agency, followed by cooperative tasks, and neutral tasks will show the smallest differences in agency.

12

Meta-Analysis 3

13

Study Selection

14

◉ Bibliographic indices relating○ Virtual reality, communication, psychology, video gaming

◉ Databases: ScienceDirect, Google Scholar, etc.◉ Search terms: Virtual representation, avatars, etc.◉ Once a paper identified:

○ References / reverse search◉ Advertised on relevant listservs and contacted researchers◉ Measured social influence◉ 119 studies

Study Selection

15

◉ Bibliographic indices relating○ Virtual reality, communication, psychology, video gaming

◉ Databases: ScienceDirect, Google Scholar, etc.◉ Search terms: Virtual representation, avatars, etc.◉ Once a paper identified:

○ References / reverse search◉ Advertised on relevant listservs and contacted researchers◉ Measured social influence◉ 119 studies

I like the idea of contacting other researchers to get relevant work. I think that is also a good way to find more relevant papers.[Nina]

Study Criteria

16

◉ DVs - quantitative measures of social influence ○ presence or affect ratings, physiological measures, or

interpersonal distance◉ Have a visual representation◉ Explicitly manipulate agency◉ 36 studies

Final Round of Selection

17

◉ Minimally required statistics:○ means, standard deviations, and sample sizes per

condition○ t values○ F values with degrees of freedom○ r values○ Cohen’s d values

◉ 32 studies

Final Round of Selection

18

◉ Minimally required statistics:○ means, standard deviations, and sample sizes per

condition○ t values○ F values with degrees of freedom○ r values○ Cohen’s d values

◉ 32 studies

I also like how they were able to get sample sizes and needed statistics by contacting authors. But, for the one study that they couldn't get enough information, why didn't they just exclude the study instead of assuming values?[Nina]

I agree with Nina for that one study that assumed the sample sizes but I am not very critical of that assumption as there is a clear selection criterion for the groups (8 and 5 grades), I might be wrong but I arrived at the reasoning based on papers where the American core syllabus for mid and high schoolers was being designed and had a similar assumption.[Sourabh]

“The only case for which we were not able to obtain information was the Eastin and Griffiths (2006) studies, for which we assumed equal sample sizes across conditions.”

Data

19

◉ Average sample size within each study: 71◉ Desktop: 21, immersive VE: 11◉ Tasks:

○ Gaming: 16■ 7 Human, 9 Computer

○ Non-gaming: 16■ 15 Computers, 1 Human

◉ Type of tasks:○ Cooperative: 4○ Competitive: 11○ Cooperative and Competitive: 1

Effect Size Calculations

20

◉ Datasheet○ Studies

■ DVs● Measures (objective/subjective)

○ Statistics + number of participants◉ For each DV, calculated effect sizes

○ r value■ t value

● means, standard deviations, and the number of cases in each condition

Effect Size Calculations

21

◉ Missing r value: ○ Coded two values for r:

■ coded r as zero● avoids inflated results due to publication bias of

significant effects■ maximum non-significant effect size

● based on the sample size◉ Sign of r signifies the direction of the effect

○ Positive: avatar condition yielded stronger responses on the DV than the agent condition

Effect Size Calculations

22

◉ Surveyed studies○ reported more than one DV of social influence

■ conducted an analysis with DV as the unit of analysis

◉ conducted analyses using study as the unit of analysis

Data Sheet Sample

23

Statistical Analyses

24

◉ Calculated r values - weighted by sample size○ weighted r values were transformed to Fisher’s Z values○ After aggregation, the averaged Fisher’s Z values were

transformed back to r values◉ Grand mean and variance of r was calculated

○ first source of variance around the mean r ■ sampling error variance ??■ still unexplained variance left?

● moderator variables◉ Assessed the n-r correlation

Statistical Analyses

25

◉ Calculated r values - weighted by sample size○ weighted r values were transformed to Fisher’s Z values○ After aggregation, the averaged Fisher’s Z values were

transformed back to r values◉ Grand mean and variance of r was calculated

○ first source of variance around the mean r■ sampling error variance■ still unexplained variance left

● moderator variables◉ Assessed the n-r correlation

I did not clearly understand how they calculated the effect sizes they have brief discussion about filling in missing values and the n-r calculations, I feel that the authors not talking about what the dependent variables were in concrete details disrupted this understanding.[Aditi]

Results 4

26

Results

27

◉ Overall effect of agency in which avatars were more influential than agents

◉ Computed Fisher’s Z values from the r values of each DV and submitted all sets of Fisher’s Z values to a one-sample t test

◉ All DVs

Results

28

◉ Average effect size per study

Results

29

◉ Possible factors that influenced the effect sizes◉ Subtracted the sampling error variance from the total variance

○ Resulting variance was greater than zero○ Likely to be moderator variables to explain part of the

variance in the data

Results

30

◉ Effect of year of publication on the effect of agency◉ Regressed year on the four sets of r values

Results

31

◉ Effects of Immersion and DV type

◉ Submitting the Fisher’s Z values of each of the 4 sets to ANOVA

◉ Between-DV factors

Results

32

◉ means and standard errors◉ Pairwise comparisons with Bonferroni corrections

Results

33

◉ Effect of task type on agency◉ Submitted the Fisher’s Z values of all four sets to an ANOVA

with task type as a between-DV factor

Results

34

◉ Pairwise comparisons using Bonferroni corrections◉ r values were higher in both gaming tasks compared to the

non-gaming tasks

Results

35

◉ Actual control○ made any difference on the effect of the perceived control

◉ Fisher’s Z values of all four sets to an ANOVA○ between-DV

Results

36

◉ Investigated the n-r correlation as a measure of publication bias

◉ Significant negative correlations between n and r○ some studies were not uncovered in the analysis

Implications5

37

Implications

◉ Theoretical◉ HCI Research and Design

○ persuade an audience via a virtual representation to convince them they are interacting with a real person rather than an algorithm

○ Adopt multiple methods of measures

38

Implications

◉ Theoretical◉ HCI Research and Design

○ persuade an audience via a virtual representation to convince them they are interacting with a real person rather than an algorithm

○ Adopt multiple methods of measures

39

This might be a good conclusion, might have important implications when designers think about the system.However, the question of how exactly one is supposed to achieve it is as old at the field of computer science itself. Is the goal of the designer here to make sure the software passes the Turing test? Perhaps that was not the idea that the authors were going for, but how else could we put that in the context of avatars vs agents?[Harish]

Critique & Discussion6

40

Critique◉ Strengths

○ agency is an important factor to consider when examining social influence in mediated interactions

○ Multiple Variables

◉ Weaknesses○ Bias of reporting and publishing only significant results○ Relies on the original researchers’ execution and reporting

■ did not report system features○ Few studies - experimental studies - small sample sizes

◉ Possible Improvement○ Role of behavioral realism○ Investigate how an optimal hybrid can be constructed for different contexts○ Variable categorizing 41

Discussion

◉ The paper is easy to read.[Nina]

◉ I think this is a good example of a meta-analysis. It supports what we learned in

class and I had a few of the "aha" moments while reading it.[Nina]

◉ I liked how the paper cited various forms of virtual representation. The

Blascovich study that suggests that user makes a decision without regarding if

the avatar is human or computer controlled was an interesting mention.

Research questions are well defined based on previous studies.[Sourabh]

◉ A very easy to understand example for meta-analysis, selection criteria are well

defined. [Sourabh]

42

Any questions ?

Thanks!

43