new media consortium,social computing conference nov.19, 2004 - 1 two examples of credibility in...

New Media Consortium,Social Computing Conference Nov.19, 2004 - 1

Two Examples of Credibility in Collaborative Decision-Making:

Constructing a Media EventAnnotating A Formally Approved Document

Ursula Wolz, Michael Massimi, Eric Tarn, Peter BabinskiThe College of New Jersey

Susan Weinman Thomas Paine Elementary School, Cherry Hill, NJ


Introduction

• Social computing requires consideration for the credibility of group members.

– Who is allowed to make changes to a collaboratively developed artifact

– How are individuals allowed to influence the collective decision-making of a group


Overview of Hopewell

• How to update and comment upon a fixed artifact– A school district curriculum document– Developed by committee

• Credibility involves who is allowed access to – Make changes– Comment– Comments of others– Who recommends changes?– How can notes be kept private?


Overview of

• Group dynamics in sharing a fixed resource in an ad hoc network

• Example is sharing music through good speakers

• In natural settings decision-making is influenced by individual credibility.


The Hopewell Challenge

• Hopewell Valley Regional School District has a formal curriculum approved by the Board of Education

• How to allow teachers to comment and take private notes on the curriculum without changing it?


Educational Objectives• Meaningful dialog: enhance collaboration and communication among

teachers• Efficiency: tools to assist curriculum development and refinement• Analysis: resources for multi-modal data collection and analysis• Support initiatives: facilitate a culture where teachers gain

administrative support for ‘grass roots’ change.• Explicitly address balance between

– Individual privacy – Collaborative needs– Institutional security

• Initiating improvement comes from many sources

HVRSD Mission: A coherent, sequential curriculum that reflects essential knowledge, inspires lifelong learning in a rapidly changing world


BEFORE THE DATABASE


Social Constraints

• Teachers in four schools (at least 10 miles apart) come together at most twice a year

• Different committee tackle different problems:(subject, mapping, special education)

• Data is lost: – Teachers make informal comments– May take paper notes– Administration only has anecdotal evidence of need for change

• Collaboration requires establishing trust– Private notes– Informal question raising– Formal proposal for change


THE VISIONA database that supports:• “Messy” data

• Simple methods for querying (Wizard of Oz)

• ANNOTATING rather than changing data

• Credibility defined through user typeView Query Annotate Change

Public Selected parts of

common

No No No

Supervisor Common Yes No No

Scribe Common Yes No Yes

Teacher Common +private annotations

Yes Make public and private

No

Sys admin Common Yes Yes Yes


Target Outcomes

• Teachers would learn to define assessment– By articulating success– By voicing concerns about disconnect between

expectations and implementation

• Teachers would raise questions about curricular– Choice– Sequencing– Relevance


Curriculum Example


Annotations


Queries via Wizard of Oz


QUESTIONS WE ANTICIPATED

•What other grade levels teach rocks and minerals?•What grammar is taught in each grade level?•How many times are maps taught across the curriculum?•When is oral reading introduced and is it done at every grade

level?•Where is the overlap in regards to types of writing?•When in second grade do students revisit telling time to the

five minute interval?•When is division mastered?•Where are 5 paragraph essays introduced?•What money skills are taught in 1st grade?•Are mammals taught in any other grade level?•How thoroughly do our fifth graders investigate Matter?


GOOD QUESTIONS!This isn't a query, but I do have a question. How could I ask the 5th grade teachers to switch the "American Tour" section

of the Everyday Math Program with the "World Tour" in the 4th grade edition?

Are the Kindergarten skills verbiage consistent with other grade levels?Is it possible to have some flexibility when teaching science and social

studies topics in a different order than what is presented on the map?We really need to track how Native Americans are studied K-5...we are

supposed to cover it in 4th grade, but quite honestly, I never get to spend much time on it. Is there a way to query all lessons related to Native Americans.

Can you modify my search on multiplication to find out where a traditional approach is taught for multiplication? I don't think the map reflects it, but it should be somewhere in 4th grade, and also in 5th


OUTCOMES• Teachers were better able to articulate assessment• Annotations did lead to queries that motivated

proposals for curricular change• New literacy curriculum is:

– Articulated with state standards– Tied to assessments– Defined by student skills not calendar– Organized in a table that allows vertical views


Segue

• Hopewell captures static credibility through user type, human decision-making mediates change to the artifact

• rMusic promotes dynamic credibility that influences automatic decision-making in an artifact


The rMUSIC Challenge

• In ad-hoc local networks security and privacy must be managed “on the fly”

• How to allocate fixed resources:– Printers– Scanners– Large screen video– Shared sound !!!!!


Overview

• rMUSIC is a way to collaboratively design a music play list at a small gathering

• Aims to reinvent the way groups of people share music


The “Modern Sock-Hop”

• A group of people gathering at a social function where music is to be played.

• Each individual contributes his/her own music to the group’s collective library.

• There is no specific DJ – everyone is involved in choosing music.

• Everyone has a different taste in music.• Small nightclubs/discotheques, dorm rooms,

apartments, karaoke bars, etc.


What rMUSIC Does

• The songs residing on each mobile device are registered with the server (NOT copied).

• Songs can then be chosen by anyone else at the gathering by selecting them on their mobile device.

• Position of songs on the play list can be altered by using the mobile device.


How The Dynamic Play list Is Formed

• Problem:– First-In First-Out models are subject to abuse– Deterministic voting isn’t that fair (nor does it

capture the natural dynamic of credibility).

• Solution:– A better “mix” evolves from group decisions that

are based on some measure of credibility.


rMUSIC Architecture

Mediation ServerClients

Fixed Resource


Voting• Users post songs, other users vote those songs up

or down.• User credibility is measured by how songs survive

referenda.• Linear voting potentially produces:

– Isolation where a user’s songs are never played– Dominance where only one (or a few) user’s songs are

played.

• A non-linear model is used to prevent domination/isolation


Graph of Change WeightChange Weight(w/ maxChange = 4.47)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Target User's Rating

Ch

ang

e W

eig

ht


Current Implementation• Java 2 Standard Edition desktop client.

– Moving towards a J2ME mobile client on Sharp Zaurus.

• MySQL database used to manage play list and users.

• Wired ethernet connection.


User Testing Has Just Begun

• 6 - 12 users

• Shills to force dominance or isolation

• Food!

• Music loaded by hand

• In a lab setting


The Current Interface


Preliminary Outcomes

• Party-goers had fun, they– Tried to get their songs to the top– Got competitive about their ratings– Appreciated the need for credibility

• The software worked (!), we need to – Migrate to a more informal locale– Get the handheld software running


Next Steps• Would dynamic credibility be useful for curriculum

analysis?– Whose comments wake up an administrator?– Voting on comments (comments on comments?)– Would competitiveness be useful?

• Could dynamic ordering of a performance be applied to visual media?

– A virtual, collaborative museum– Single screen, multiple screens

• Voting Dynamics– Who adjudicates change?– The Baseball problem (three scorekeepers, who got it right)


Summary

Static Credibility• Provides safety• Potential for creating

implicit isolation or dominance

• Appears to encourage cooperative collaboration

Dynamic Credibility • Models natural behavior

• Raises interesting concerns about value of opinion

• Appears to encourage competitive collaboration


Hopewell CreditsThe DB Developers (over the years)

Peter BabinskiStacy BrasolRyan Gladysiewics Michael HulmeMichael LocastoMichael MassimiGreg Nice Kultida Tangwongchai

Consultants:Dr. Miroslav MartinovicDr. Nobo Komogata

The Hopewell StaffStephen CochraneDr. Nick LorenzettiClaire BrusseauPatricia CoatsThe teachers:

Ann Marie AnthonyMarianne CalvoJennifer DutkaLori GeraldSusanne KingFrances M. MaguireDeborah M. NuttElizabeth M. RichtmyerClaudia P. ScottiAnnette R. StayEileen W. SweeneyDawn F. WheelerW. Lee White

Find more information at http://hopewell.tcnj.edu


rMUSIC CreditsThe Software Developers

Mike Massimi

Eric Tarn

Scott Carpenter

Eric Thul

Consultants

Prof. Phil SandersProf. Kim Pearson

All of our friends and colleagues who are willing to come to our very weird parties!

Find more information at:http://www.tcnj.edu/~rmusic

So what does it stand for?resource Mediation by User-Supported Initiative in Communities

new media consortium,social computing conference nov.19, 2004 - 1 two examples of credibility in...

Documents

sequential curriculum

curriculum development

formal curriculum

collaborative decisionmaking

implementation teachers

individual credibility

grade levels

collective decisionmaking