Using Networked Multimedia to Improve Educational Access for Deaf and Hard of Hearing Students

Anna C. CavenderGeneral ExamUniversity of WashingtonAdvisor: Richard Ladner

2

Enabling Access to Education

Better include deaf and hard of hearing students in mainstream universities

byintegrating existing technology with a deaf-centered design

3

Problems:

Deaf and hard of hearing students in mainstream classrooms are often: Overloaded with visual information Excluded from content Isolated from peers

Proposed Solutions: Modify existing technology to best suit deaf and

hard of hearing student by: Reducing visual dispersion Enhancing classroom collaboration Preserving missed content for later retrieval

4

DHH Cyber-Community

Enabling access to STEM* education High bandwidth connections between

universities Networked classrooms allow students to

control learning environment

Enabling ASL to grow in STEM* Online video forum (vlog) to facilitate

discussion about signing for STEM

* STEM = Science, Technology, Engineering, Math

5

Outline

Background on Deafness

Thesis Overview

Related Work

Thesis Proposal

6

DHH Identities

Deaf people tend to prefer sign language may be active in the deaf community

Hard of hearing people tend to speak and lip-read may rely on hearing aids or cochlear implants may prefer real-time text captions may know sign language and be active in the deaf community

Hearing impaired audiological term elderly people who lose hearing later in life

Group Association ≠ Preferred Accommodation

7

Demographics

25,000 deaf and hard of hearing students enrolled in ~4,000 post-secondary institutes in U.S.

95% of colleges/universities serve 1 or more deaf or hard of hearing student Students are dispersed thinly

Increased enrollment at mainstream universities

National Center for Education Statistics (NCES) 1999.

8

Public Law 94-142 (for K-12)

Individuals with Disabilities Education ACT (IDEA)

“All children with disabilities are assured a free appropriate public education”

Shift from centralized residential schools to programs within mainstream schools 85% of d/hh students at mainstream schools

Trickle through to post-secondary enrollment

M. E. Ross and M. A. Karchmer. Demographics of deaf education: Morestudents in more places. (151:2):95–104, 2006.

9

Summer Academy for DHH 2007 – Intro to Programming

Current Accommodations

Interpreters

Real-time captionists

Hearing aids FM systems

Note takers

10

Accommodation of Choice

Depends on:experiences and education backgroundstrength in sign languagecomfort with Englishprior accommodationsmodes of study: in-class vs. review

11

Accommodation of Choice

Also depends on course content: lecture-based lots of new vocabulary

spatial or relative information focused on discussion

→ Captions

→ Sign language

12

Interpreter Matching

Limited number of interpreters at a given university

Matching interpreter/captionist who is knowledgeable on course content is crucial

13

[1] www.viabletechnologies.com[2] www.hovrs.com[3] http://main.wgbh.org/wgbh/access/access.html

Accommodation Opportunities

Challenge: Utilize remote interpreters and captionists to increase the poolViable TechnologiesHandsOn Video Relay ServiceMedia Access Group at WGBH

DHH Cyber Community – pooling resources

14

Accommodation Overload

In person: Text + interpretation = great info loss [1]

On computer screen: Text + interpretation = reduced info loss [2]

Accommodation is best utilized when it does not increase visual overload

[1] Mayer et al. Cognitive Constraints on Multimedia Learning: When Presenting More Material Results in Less Understanding. Journal of Educational Psychology. 2001

[2] Marschark et al. Benefits of Sign Language Interpreting and Text Alternatives for Deaf Students' Classroom Learning. Journal of Deaf Studies and Deaf Education. 2006

15

Attrition of Deaf and Hard of Hearing Students

Only 25% graduate Causes include:

Missed content in classroom due to• Lack of skilled interpreters and captionists• Multiple visual tasks

Classroom ParticipationSocial Isolation

Harry G. Lang. Higher education for deaf students: Research priorities in the new millennium. Journal of Deaf Studies and Deaf Education (7:4):267–280, 2002.

16

Outline

Background on Deafness

Thesis Overview

Related Work

Thesis Proposal

17

Thesis Overview

Goal:

Address challenges faced by deaf and hard of hearing students in mainstream classrooms

by

integrating technology in an accessible and unobtrusive way.

18

Design Challenges

1. Reducing Visual Dispersion

2. Enabling Student Flexibility

3. Enhancing Classroom Interaction

4. Preserving Missed Content

19

Design Challenge 1.Reduce Visual Dispersion

Problem: “Deaf Whiplash”

20

Design Challenge 1.Reduce Visual Dispersion

Consolidate using video, text, sharing of materials

21

Design Challenge 2.Enable Student Flexibility

Problem: Different students have different preferences

22

Design Challenge 2.Enable Student Flexibility

Accommodation & Layout Choices

23

Design Challenge 3.Enhance Classroom Interaction

Problem: Participation is strained due to language barriers

24

Design Challenge 4.Preserve Missed Content

Problem: Students still miss class content

Student-initiated capture for later retrieval

25

Design ConsiderationInstructor Buy-in

Problem: Technology should not burden instructors

Proposed technology is compatible with: many types of classroom technology many types of pedagogy

• lecture-style, group work, study session

Place power and choice with the student

26

Design Challenges

1. Reducing Visual Dispersion

2. Enabling Student Flexibility

3. Enhancing Classroom Interaction

4. Preserving Missed Content

+. Considering Instructor Buy-in

Addressing these challenges integrating existing technologies and related

work with student needs in mind

27

Outline

Background on Deafness

Thesis Overview

Related Work Educational Technology in General Educational Technology for Deaf and Hard of Hearing Enabling Technology – a comparison and a demo

Thesis Proposal

28

Educational Technology (in general)

Education technology – electronically facilitates active learning

Active learning = active engagement promotes learning

29

Instructor initiates activity Students submit responses

multiple choice or number Instructor can summarize

results

Clicker, Classtalk, Pebbles

Ed. Tech. Classroom Response Systems

3. Enhancing Classroom Interaction

30

Ed. Tech.ActiveClass

ActiveClass polling and short answer

questions student initiated questions students can rate questions

of other students

ActiveClass: Ratto CSCL 2003.

3. Enhancing Classroom Interaction

31

Ed. Tech.Digital Classroom Interaction

Classroom Presenter and DyKnow Instructor initiates

activity

DyKnow

Classroom Presenter

Student submit ink responsesdirectly on slides with TabletPCs

Instructor can display and discuss responses

Classroom Presenter: Anderson et al. SIGCSE 2004. DyKnow: www.dyknow.com

2. Enabling Student Flexibility

3. Enhancing Classroom Interaction

32

Ed. Tech.Collaborative Note-taking

LiveNotes Digital ink on lecture slides Encourages group note-

taking

Livenotes: Kam et al. CHI 2005.

1. Reducing Visual Dispersion

3. Enhancing Classroom Interaction

4. Preserving Missed Content

33

Ed. Tech. Capture & Retrieval

eClass: capture and synchronization of video digital ink presentation materials

for later retrieval

Post-class access helps reduce missed content

eClass

eClass: Brotherton and Abowd. CHI 2004

4. Preserving Missed Content

34

Educational Technology DHH(for Deaf and Hard of Hearing)

Similar goals: encourage active learning

Focus on:interactionaccess to speechreduce visual overload

35

Ed. Tech. DHHNetworked Activities

Western PA School for the Deaf

SMART board and networked laptops

Teacher can “grab” student screens

Students keep digital notes

Participation == Note-takingBurik., NTID, 2003.

1. Reducing Visual Dispersion

3. Enhancing Classroom Interaction

4. Preserving Missed Content

36

Educational Technology DHH MultiChat

MultiChat Face-to-face chat between

deaf and hearing students Concurrent

(preserves timing)

MultiChat: Schull. ASSETS 2006.

3. Enhancing Classroom Interaction

37

Educational Technology DHHFacetop Tablet

Facetop Tablet

Transparent interpreter video

Student notes visually closer to focus of attention

Facetop Tablet: Miller et al. ASSETS 2006.

1. Reducing Visual Dispersion

4. Preserving Missed Content

38

Educational Technology DHHPhotoNotes

PhotoNotes Lecture is recorded Computer vision

techniques used to make best snapshot

For student review

PhotoNotes: Hughes and Robinson. ASSETS 2007

1. Reducing Visual Dispersion

4. Preserving Missed Content

39

Educational Technology DHH Technology aiding participation

Accommodations that include student participation have best effect on learning [1]

Technological classrooms may lower the barrier to participation [2]

[1] Dowaliby and Lang. Adjunct aids in instructional prose: a multimedia study with deaf college students. Journal of Deaf Studies and Deaf Education.

[2] Richardson et al. Academic Engagement in Students with a Hearing Loss in Distance Education. Journal of Deaf Studies and Deaf Education.

40

Educational Technology Summary

Design Challenges

EdTech in General

EdTech for DHH

1. Reduce Visual Dispersion

2. Enable Student Flexibility

3. Classroom Interaction

4. Capture & Retrieval

ClassroomPresenter

Clickers

DyKnow

ActiveClass

eClass

LiveNotesWPSD + Smartboards

WPSD + Smartboards

Facetop Tablet

MultiChat

PhotoNotesFacetop Tablet

PhotoNotes

41

Outline

Background on Deafness

Thesis Overview

Related Work Educational Technology in General Educational Technology for Deaf and Hard of Hearing Enabling Technology – a comparison and a demo

Thesis Proposal

42

Enabling Technology

Adobe Connect and ConferenceXPvideo/audio conferencing toolsremote sharing of

• presentation slides• application windows• desktop view• video/audio from webcam

offer archiving Both have shared source* versions

* open for academic use

43

Enabling TechnologyAdobe Connect

Formerly Macromedia Breeze Browser-based, multi-platform interface

uses Flash Intended for online meetings –

one webcam per person Already used by many universities* for

distance learning. Extensions available for captioning

Alliance with ColoradoCaption and WGBH

* Including NTID (National Technical Institute for the Deaf)

44

Enabling TechnologyAdobe Connect - Scenario

Bobby is hard of hearing Prefers captions and uses voice Learning sign language

Sally is deaf of deaf parents Prefers sign language

Instructor uploads slides wears headset uses Connect like PowerPoint uses Connect to archive for all students

Students pass around microphone

45

Enabling TechnologyAdobe Connect - Demo

1. Reduce Visual Dispersion

2. Enable Student Flexibility

3. Enhance Classroom Interaction

4. Preserve Missed Content

46

Enabling TechnologyConference XP

Developed at Microsoft Research Used for networking Tablet PCs in

Classroom Presenter Geared toward multi-cast, high-

bandwidth connections between universitiesmany universities already onboardmulti-institutional instruction

47

Enabling TechnologyConference XP

48

Enabling Technology Summary

Adobe Connect Conference XP

Basic conferencing tools

shared source

Uses available bandwidth

Platform Independent

Browser Accessible

Good for Prototyping

Requires high-bandwidth

Platform Dependent

Already at Universities

Can Port Later

49

Related WorkSummary

Summary EdTech in General

EdTech for DHH

EnablingTech

1. Reduce Visual Dispersion

2. Student Flexibility

3. Classroom Interaction

4. Capture & Retrieval

ClassroomPresenter

Clickers

DyKnow

ActiveClass

eClass

LiveNotesWPSD + Smartboards

WPSD + Smartboards

Facetop Tablet

Facetop Tablet

MultiChatConfXP Adobe

Connect

PhotoNotes

PhotoNotes

50

Outline

Background on Deafness

Thesis Overview

Related Work

Thesis Proposal

Identified challenges faced by dhh students

Proposed design challenges

Integration and evaluation of classroom technology

51

Proposed Design Space DHH Cyber-Community

ReduceVisual Dispersion

Enable Student Flexibility

Enhance Class Interaction

Preserve Missed Content

52

Design Proposal 1.Reduce Visual Dispersion

CamCutsOn-the-fly video modificationsCut, size, zoom, transparency, arrangement

53

Design Proposal 1.Reduce Visual Dispersion

SceneCuts Student control of

interface and layout

Techniques inspired by:WinCuts

WinCuts: Tan, Meyers, Czerwinski. CHI 2004.

54

Design Proposal 2.Enable Student Flexibility

LookingGlassPersonalized view of learning environmentIndependent choice of feeds:

• slides, video, accommodation, etc.

De-coupled from instructor and other students

55

Design Proposal 3.Enhance Class Interaction

ProjectUnitsIncrease channels of communicationBetter support group work

• Small group chat, whiteboard, project

Techniques inspired byLiveNotesMultichat

Livenotes: Kam et al. CHI 2005.Multichat: Schull. ASSETS 2006.

56

Design Proposal 4.Preserve Missed Content

SceneSaveHelp students find missed informationStudent-driven video segmentation

Techniques inspired byeClass (Classroom 2000)

eClass: Brotherton and Abowd. CHI 2004

57

Summary of Proposed Design

Challenges: Proposed Solutions:

1. “Deaf Whiplash” 1. CamCuts & SceneCuts:Reduce Visual Dispersion

2. Diverse Needs and Prefs 2. LookingGlass: Enable Student Flexibility

3. Isolation from Peers 3. ProjectUnits: Enhance Class Interaction

4. Missed Content in Class 4. SceneSave: Preserve Missed Content

58

Evaluation Techniques

Challenges:Small sample size

• e.g.: 21 deaf or hard of hearing students enrolled at UW Fall 2007

“In-the-wild” evaluationsKnowledge “gains” may be

unreliable and unrealistic measures• (e.g. test scores or grades)

59

Evaluation Techniques

Quantitative Data Attendance and/or participation Effects on note-taking Effects on seating Increased or continued use post-study

Qualitative Data Students’ reflections Effects on engagement/interest Students’ perception of usefulness in class Students’ perception of usefulness in studying

60

Summer Academy for DHH

10 studentshigh school seniors / college freshmen

9 weeks at UW Intro Computer Science and Animation Test bed for classroom technology

2008 – feedback on prototype2009 – formal evaluation

61

Research Timeline

Spring 2008 - Summer 2008 Prepare working prototype for the DHH Cyber Community

Summit gathering in June 2008. Implement and evaluate initial version locally at DHH Summer

Academy.

Fall 2008 - Spring 2009 Use feedback from summer release to improve design. Evaluate with interpreters/captioners at RIT and UW students. Spring: Investigate longitudinal use and any novelty factors.

Summer 2009 – Winter 2010 Evaluate at Summer Academy - compare to previous summer. Finish remaining analysis and research. Prepare dissertation and defend.

62

Conclusion

Anna C. Cavendercavender@cs.washington.eduUniversity of Washington

Identified challenges faced by dhh students in mainstream academics

Proposed integration and modification of existing tech to reach design goals:

1. Reduce Visual Dispersion

2. Enable Student Flexibility

3. Enhance Classroom Interaction

4. Preserve Missed Content

Objective: create more inclusive, easily accessible classroom environments

63

DHH Cyber-Community Summit

June 25-27, 2008 at RIT/NTID Goal: Energize the DHH Cyber-Community

members Participants:

Students Interpreters Captioners Sign language researchers Educational technology researchers Cyber infrastructure experts

Demonstration of prototype with feedback

64

Classroom layout

Typical Classroom Deaf Classroom

65

Outline

Background on Deafness Terms and Demographics Shift from Centralized to Mainstreamed Education Accommodations in the Classroom

Design Goals Reducing Visual Dispersion Broadening Opportunities for the Best Services Reducing Barriers to Classroom Participation

Related Work Educational Technology in General Educational Technology for Deaf and Hard of Hearing

Enabling Technology – a comparison and a demo

Thesis Proposal User Control of the Interface Collaboration and Group Work Capture and Retrieval Evaluation Techniques