Ambiguous Problem Complexity, Group Synergy and Performance: An Experiment

Elliot Bendoly

Svenja Sommer

Stylianos (Stelios) Kavadias

• An ongoing debate over the benefits of brainstorming (Stroebe and Diehl 1994; Paulus et al. 1996)

• Unclear group benefits in problem contexts with non-obvious links between solution details (‘decision’) and solution performance (apprehension decreased, etc. – Gallupe et al. 1991)

• Non-monotonic role of problem complexity (group can in fact lead to more or less productive results (Kavadias and Sommer 2009)

“Nominal” solution or “best of set”

Group collaborative

solution

> (In LOW complexity)

<(In HIGH complexity)

Motivation

Context: an adaptation of Ederer and Manso’s (2012) incentives experiment

Continuous Decisions:Price, Lemon content, Sugar content

Discrete Decisions:Color (2), Location (3)

In this problem, subjects are asked to decide on a set of parameters associated with a simple retail context:

a Lemonade Stand

Search Experiment

• All participants exposed to 2 solution development settings: Group vs. Individual (Nominal)• first 15 min in one, next 15 min in the alternate setting

• 2 levels of control of parameters in first 15 min: Generalists vs. Specialists

• 2 Initial solution development settings for Generalists: Group vs. Individual (Nominal)

• 3 Levels of complexity: Low, Medium, High

• Pay for performance scheme used for recruitment and compensation

Experimental Design

Low

Co

mpl

exity

All p

rofit

func

tions

sho

wn

for (

Pink

Le

mon

ade

at 2

5c /gl

ass)

Business District School Stadium

Med

ium

Co

mpl

exity

Business District School Stadium

Hig

h Co

mpl

exity

Business District School Stadium

Interface – Decision Variable controls, Feedback and Retrospective archive

Study 1: Generalists setting

Subjects population: MBA students

Total number of subjects: 308 ::: 122 Groups

Study 2: Specialists setting (only nominal first)

Subjects population: University students (diverse, but control variables like age, gender, years in college, background not significant)

Total number of subjects: 168 ::: 56 3-person groupsLow = 16, Medium=20, High =20

The Study

Initial Task Exposure (1st 15 min) Re-exposure to Task (2nd 15 min)

Blocking or Freeriding

Study 1: Performance of Nominal vs. Collaborative Groups

• Complexity matters! • Nominal” groups settings generally seem to benefit in more complex task settings• “Nominal group technique” (first individual, then in groups) performs poorly in

very complex task settings

Study 2: Performance of Generalist vs. Specialist Groups

• Collaborative groups of Specialists perform significantly better than collaborative groups of Generalists. - - “Nominal group technique” benefits from this difference.

40%

50%

60%

70%

80%

90%

100%

SpecialistGeneralist

1st 15 Minutes (Nominal)

Ta

sk P

erfo

rma

nce

(%

of

Ma

x)

40%

50%

60%

70%

80%

90%

100%

Specialist

Generalist

2nd 15 Minutes (Collaborative)

Task

Per

form

ance

(% o

f Max

)

Low Medium High Complexity Complexity Complexity

What causes these differences?

• Search Process– Number of “ideas”

– Coverage of space

– Step size

• Engagement– Affective Award

– Learning

• Groups effects– Production blocking

– Evaluation Apprehension

– Freeriding

First Period Individual Generalists Individual Specialists Collaborative GeneralistsIndependent Indiv. Performance Indiv. Performance Group PerformanceVariables Beta SE Beta SE Beta SEConstant 0.666 (0.187) *** 0.720 (0.125) *** 1.396 (0.372) ***High Complexity -0.202 (0.481) *** -0.142 (0.042) *** -0.558 (0.048) ***Medium Complexity -0.069 (0.494) -0.113 (0.042) *** -0.492 (0.039) ***NumSolutions -0.005 (0.002) ** -0.001 (0.001) -0.001 (0.002)Soln. Coverage 0.568 (0.166) *** 0.242 (0.191) 0.018 (0.166)Avg. Increment -0.411 (0.139) *** -0.175 (0.085) ** 0.123 (0.121)Affective Reward 0.021 (0.023) 0.003 (0.022) -0.045 (0.034)Learning 0.032 (0.020) 0.005 (0.021) 0.009 (0.020)Lack of Blocking -0.051 (0.043)Evaluation App. -0.045 (0.035)Freeriding 0.017 (0.032)R2 0.283 0.093 0.893Adj R2 0.248 0.053 0.867N 150 168 62

Impact Factors in First Period Search

Not Number of Solutions, but space coverage matters!

Need to search intelligently...

First Period Individual Generalists Individual Specialists Collaborative GeneralistsIndependent Indiv. Performance Indiv. Performance Group PerformanceVariables Beta SE Beta SE Beta SEConstant 0.666 (0.187) *** 0.720 (0.125) *** 1.396 (0.372) ***High Complexity -0.202 (0.481) *** -0.142 (0.042) *** -0.558 (0.048) ***Medium Complexity -0.069 (0.494) -0.113 (0.042) *** -0.492 (0.039) ***NumSolutions -0.005 (0.002) ** -0.001 (0.001) -0.001 (0.002)Soln. Coverage 0.568 (0.166) *** 0.242 (0.191) 0.018 (0.166)Avg. Increment -0.411 (0.139) *** -0.175 (0.085) ** 0.123 (0.121)Affective Reward 0.021 (0.023) 0.003 (0.022) -0.045 (0.034)Learning 0.032 (0.020) 0.005 (0.021) 0.009 (0.020)Lack of Blocking -0.051 (0.043)Evaluation App. -0.045 (0.035)Freeriding 0.017 (0.032)R2 0.283 0.093 0.893Adj R2 0.248 0.053 0.867N 150 168 62

Impact Factors in First Period Search

Nominal Group Technique

Impact of individual phase on group performance?

Combined individual solution coverage

Production blocking

Group solution coverage

Nominal Group Performance

Group Performance

Generalists Specialists

+ **

+ +

+**

+

+*** +

+** +*

+*** ***

+***

+

Some Evidence from Questionnaires

Low Collaborative Performance“…We couldn’t agree on whether to raise or lower the lemon content. One of the group members said they could get better results if it was lowered. We went back and forth and spent most time on it, but maybe should have thought more about the other issues. Still the debating probably helped use avoid bad solutions.”

High Collaborative Performance“The mouse was mine, so probably did more than the others. They both wanted to go in two different directions, and not what I thought was best (who knows). I kind of tuned out early on and drove. I kept saying I’d “test that after this” but since we had momentum we usually didn’t go back…”

There seem to be two paths to performance : • Leadership that ends bickering by blocking and promoting search • No blocking – seems more likely when combined individual coverage is

contained (shared mental models?)

Nominal Group Technique

Impact of individual phase on group performance?

Combined individual solution coverage

Production blocking

Group solution coverage

Nominal Group Performance

Group Performance

Generalists Specialists

+ **

+ +

+**

+

+*** +

+** +*

+*** ***

+***

+

No bickering, since difference in expertise of members is recognized by everyone.

What about collaboration before individual search?

Impact of group phase on individual performance?

Group solution coverage

Individual Solution Coverage

Individual Performance

Production Blocking

Collaborative Group

Performance

+*

+**

**

+

+

Some more evidence from Questionnaire

High Nominal Performance, and Low Blocking“When we were in the group we all took turns posting ideas since we were all new to it. Good dynamic overall. I think we made some progress figuring out where the best solutions were. I basically picked up where we left off when the group split up and when I was working on my own…”

Low Nominal Performance, and High Blocking

“I didn’t have much of a chance to impact things when working in the group, so I stopped thinking about the problem after a while. When I was on my own it was like my first time on the problem I guess. I honestly don’t know if I missed something the group discovered earlier, it was hard to connect back.”

Some results so far...

• Nominal groups (best of individuals) benefit relatively from increasing complexity

• Solution space covered (intelligent search) more important than number of solutions

• “Nominal Group Technique” (first individuals, than in groups) performs poorly under high complexity – unless expertise difference of members is recognized by everyone (credentials) .

• Production blocking might be beneficial in case of nominal group technique: Leadership ends bickering by blocking and promoting search

• Not blocking also helps, but more likely when combined individual coverage is contained (shared mental models = less bickering...)

Questions? Suggestions? Feedback?

Results From a Model

Theorized Dynamics in Detail:

Kavadias and Sommer (2009) propose a normative model designed to characterize distinctions between collaborative and “nominal” group activity under various conditions of task complexity. • Brainstorming activity: multi-agent searches on problems where groups cannot

describe the performance function in advance.

• Sufficient initial consideration of these problems, drive meaningful mental models that link decisions to performance; as a result progress to good solutions can be made.

• HOWEVER, increases in the complexity (performance interactions) make it increasingly more difficult for this to happen.

• Group dynamic effects like production blocking and evaluation apprehension, make group performance to suffer particularly more so than “nominal” groups.