Music Performamatics: Interdisciplinary Interaction

Music Performamatics: Interdisciplinary InteractionJesse Heines • Gena Greher • Sarah

KuhnDepts. of Computer Science, Music, and Regional Economic & Social Development

University of Massachusetts Lowell

Jesse Heines • Gena Greher • Sarah Kuhn

Depts. of Computer Science, Music, and Regional Economic & Social Development

University of Massachusetts Lowell

ACM SIGCSE 2009Chattanooga, TNACM SIGCSE 2009Chattanooga, TN March 7, 2009March 7, 2009

22

• connecting Computer Science to Art, Music, and Theatre through interdisciplinary courses that highlight computational thinking

• originally conceived to attract and retain CS majors by connecting theory to practice

• but also exposes non-CS majors to computing at a higher level than typically seen in General Education (“GenEd”) computer literacy courses

• connecting Computer Science to Art, Music, and Theatre through interdisciplinary courses that highlight computational thinking

• originally conceived to attract and retain CS majors by connecting theory to practice

• but also exposes non-CS majors to computing at a higher level than typically seen in General Education (“GenEd”) computer literacy courses

33

Performamatics:Interdisciplinary Course Models

Performamatics:Interdisciplinary Course Models• Synchronized

– pairings of upper-level courses for majors– joint project developed within the two

courses– courses remain independent

• Hybrid– courses open to all students across the

university– General Education (“GenEd”) credit– integrated, two instructors in the

classroom

• Synchronized– pairings of upper-level courses for majors– joint project developed within the two

courses– courses remain independent

• Hybrid– courses open to all students across the

university– General Education (“GenEd”) credit– integrated, two instructors in the

classroom

44

Performamatics:Synchronized Courses To Date

Performamatics:Synchronized Courses To Date• Graphical User Interface Programming

+ Web Art and Design– artistic & human factors program

enhancements

• Graphical User Interface Programming + General Music Education Methods– software for creative music notations

• Software Engineering + Introduction to Theatrical Design– software for theatrical lighting and scenic

design

• Graphical User Interface Programming + Web Art and Design– artistic & human factors program

enhancements

• Graphical User Interface Programming + General Music Education Methods– software for creative music notations

• Software Engineering + Introduction to Theatrical Design– software for theatrical lighting and scenic

design

55

Performamatics:Hybrid Courses To DatePerformamatics:Hybrid Courses To Date• Artbotics (predecessor, model)

– the use of robotics to create new media art

• Tangible Interaction Design– exploring how people interact with

designed and computational objects in their daily lives

• Sound Thinking– the art and science of digital audio

• Artbotics (predecessor, model)– the use of robotics to create new media

art

• Tangible Interaction Design– exploring how people interact with

designed and computational objects in their daily lives

• Sound Thinking– the art and science of digital audio

66

Performamatics:Focus for Today: CS+Music

Performamatics:Focus for Today: CS+Music• GUI Programming + Music Methods• Sound Thinking

• GUI Programming + Music Methods• Sound Thinking

77

Interdisciplinary Courses Must Benefit Both Disciplines

Interdisciplinary Courses Must Benefit Both Disciplines• Benefits for Music Ed Students

– Getting a feel for what’s involved in building technology applications for the classroom

– Gaining insight into students growing up under the influence of media

– Seeing the interdependence of sound, images, and technology

• Benefits for Music Ed Students– Getting a feel for what’s involved in

building technology applications for the classroom

– Gaining insight into students growing up under the influence of media

– Seeing the interdependence of sound, images, and technology

88

Interdisciplinary Courses Must Benefit Both Disciplines

Interdisciplinary Courses Must Benefit Both Disciplines• Benefits for CS Students

– Applying CS concepts– Working in an interesting

application domain– Interacting with students

who think differently• CS view: code-centric• Music view: usability

– Gaining a strong exposure to human factors

• Benefits for CS Students– Applying CS concepts– Working in an interesting

application domain– Interacting with students

who think differently• CS view: code-centric• Music view: usability

– Gaining a strong exposure to human factors

99

Interdisciplinary Courses Must Benefit Both Disciplines

Interdisciplinary Courses Must Benefit Both Disciplines• Benefits for both Music Ed and CS

Students– Addressing creative challenges– Finding a common language– Getting out of their comfort zones

• Benefits for both Music Ed and CS Students– Addressing creative challenges– Finding a common language– Getting out of their comfort zones

1010

Found Instruments:A Synchronized ProjectFound Instruments:A Synchronized Project• Conceived by Prof. Gena Greher,

Dept. of Music, & Coordinator, Music Ed

• Conceived by Prof. Gena Greher, Dept. of Music, & Coordinator, Music Ed

1111

Found Instruments:Music Methods Assignment

Found Instruments:Music Methods Assignment• Using only household object(s), create a

musical “instrument” that can produce several pitches or timbres.

• Create a composition for your instrument.• Devise a system of creative notation that

others will be able to understand well enough to perform your composition without specific instruction from you. – Your system must not resemble standard

musical notation in any way, shape, or form.

• Using only household object(s), create a musical “instrument” that can produce several pitches or timbres.

• Create a composition for your instrument.• Devise a system of creative notation that

others will be able to understand well enough to perform your composition without specific instruction from you. – Your system must not resemble standard

musical notation in any way, shape, or form.

1212

Found Instruments: Create InstrumentFound Instruments: Create Instrument• Chris (CS), Joe (Music), and Sophanna

(CS) playing their found instruments• Chris (CS), Joe (Music), and Sophanna

(CS) playing their found instruments

1313

Found Instruments: Create NotationFound Instruments: Create Notation

1414

Found Instruments: Create NotationFound Instruments: Create Notation• Maggie explaining

her notation forplaying a steam iron

• Maggie explaining her notation forplaying a steam iron

1515

Found Instruments: Test NotationFound Instruments: Test Notation• Maggie introducing Sophanna to her

steam iron instrument and its notation• Maggie introducing Sophanna to her

steam iron instrument and its notation

1616

Found Instruments: Test NotationFound Instruments: Test Notation• Sophanna trying to play the steam

iron using Maggie’s notation• Sophanna trying to play the steam

iron using Maggie’s notation

Click Click picture to picture to play videoplay video

Click Click picture to picture to play videoplay video

1717

Found Instruments:GUI Programming Assignment

Found Instruments:GUI Programming Assignment• Create a computer program to

implement one of the music students’ found instruments.

• Your program must be designed so that others can understand it well enough to create compositions without specific instruction from you.

• Create a computer program to implement one of the music students’ found instruments.

• Your program must be designed so that others can understand it well enough to create compositions without specific instruction from you.

1818

Found Instruments: Create ProgramFound Instruments: Create Program• Sophanna’s computer program for

writing Maggie’s steam iron notation• Sophanna’s computer program for

writing Maggie’s steam iron notation

1919

Found Instruments: Test ProgramFound Instruments: Test Program• Maggie and Mike trying out

Sophanna’s program and recommending revisions

• Maggie and Mike trying out Sophanna’s program and recommending revisions

2020

Mike’s Found InstrumentJacket NotationMike’s Found InstrumentJacket Notation

2121

Mike’s Found Instrument“Eine Kline Jacket Music”Mike’s Found Instrument“Eine Kline Jacket Music”

Click Click picture to picture to play videoplay video

Click Click picture to picture to play videoplay video

2222

Chris’s Composing Program Initial Screen

Chris’s Composing Program Initial Screen

2323

Chris’s Composing Program Set Beat and Drag Icons

Chris’s Composing Program Set Beat and Drag Icons

2424

Chris’s Composing Program Select Icon & Position Cursor

Chris’s Composing Program Select Icon & Position Cursor

2525

Chris’s Composing Program Delete Icon, Where’s Cursor?

Chris’s Composing Program Delete Icon, Where’s Cursor?

2626

Chris’s Composing Program Insert Icon Keyboard Shortcut

Chris’s Composing Program Insert Icon Keyboard Shortcut

2727

Chris’s Composing Program Where Did Icon Get Inserted?

Chris’s Composing Program Where Did Icon Get Inserted?

2828

Performamatics:Music Student Observations

Performamatics:Music Student Observations• “I thought that their systems depicted

a much more technical view of things than what we had in mind.”

• “It’s very productive to have them come to class with us to share different ideas.”

• “They are in a creative process just as much as we are when we create music… I saw a lot of similarities between what they were doing and what we were doing.”

• “I thought that their systems depicted a much more technical view of things than what we had in mind.”

• “It’s very productive to have them come to class with us to share different ideas.”

• “They are in a creative process just as much as we are when we create music… I saw a lot of similarities between what they were doing and what we were doing.”

2929

Performamatics:Music Student Observations

Performamatics:Music Student Observations• “I love hearing different perspectives

from people in totally different areas of study.”

• “It is always just interesting to see the differences – but similarities – between both of the majors.”

• “... somebody else on campus [who] has nothing to do with us [that is, a CS student] has everything to do with us.”

• “I love hearing different perspectives from people in totally different areas of study.”

• “It is always just interesting to see the differences – but similarities – between both of the majors.”

• “... somebody else on campus [who] has nothing to do with us [that is, a CS student] has everything to do with us.”

Click to play Click to play videovideo

Click to play Click to play videovideo

3030

Sound Thinking:A Hybrid CourseSound Thinking:A Hybrid Course• Dual-listed elective with “GenEd” credit

– CS students get Arts & Humanities credit– Arts students get Technology credit

• Dual-taught: two professors in the classroom– Sustainability issues

• FTE and, more importantly, P &T “credit”

– Scheduling issues in tightly packed curricula• difficult for professors as well as students

– Campus geography issues

• Dual-listed elective with “GenEd” credit– CS students get Arts & Humanities credit– Arts students get Technology credit

• Dual-taught: two professors in the classroom– Sustainability issues

• FTE and, more importantly, P &T “credit”

– Scheduling issues in tightly packed curricula• difficult for professors as well as students

– Campus geography issues

3131

Campus GeographyCampus Geography

3232

Sound Thinking:Course ProjectsSound Thinking:Course Projects• Found Instruments

– Manipulating with Audacity• SoundScapes

– Audio Art: a soundtrack of an image– Audio Ethnography: a soundtrack of your life

• Computer Manipulation– Static webpages that incorporate sound– Dynamic webpages that can manipulate

sounds• Final Project

– Integrating sound & video or complex webpages

• Found Instruments– Manipulating with Audacity

• SoundScapes– Audio Art: a soundtrack of an image– Audio Ethnography: a soundtrack of your life

• Computer Manipulation– Static webpages that incorporate sound– Dynamic webpages that can manipulate

sounds• Final Project

– Integrating sound & video or complex webpages

3333

Sound Thinking:... and Expanded ThinkingSound Thinking:... and Expanded Thinking

3434

Sound Thinking:... and Expanded ThinkingSound Thinking:... and Expanded Thinking• Bach to Lachenmann to Sciarrino• Demonstration by violinist Ari

Streisfeld

• Bach to Lachenmann to Sciarrino• Demonstration by violinist Ari

Streisfeld

Click to play Click to play videovideo

Click to play Click to play videovideo

3535

Performamatics:BenefitsPerformamatics:Benefits• Grounding of theory in real applications• Exposure to other ways of thinking

– through interaction with other majors

• Attraction of majors with new interests– consideration of BA in CS to complement BS

• Revitalization of faculty as well as students– attraction of new faculty in related disciplines

• New interdisciplinary research opportunities– invitations to work with other faculty

• Grounding of theory in real applications• Exposure to other ways of thinking

– through interaction with other majors

• Attraction of majors with new interests– consideration of BA in CS to complement BS

• Revitalization of faculty as well as students– attraction of new faculty in related disciplines

• New interdisciplinary research opportunities– invitations to work with other faculty

3636

Performamatics:FuturePerformamatics:Future• Address issues at University level

– our co-PIs are now on Univ.-level committees

• Expand successes into pipeline programs– develop closer ties with high and middle schools– coordinate with NSF-funded CAITE project

(Commonwealth Alliance for Information Technology Education)

• Build on alliances with other institutions– refinement and replication of our efforts

(Artbotics replication is already underway)

• Address issues at University level– our co-PIs are now on Univ.-level committees

• Expand successes into pipeline programs– develop closer ties with high and middle schools– coordinate with NSF-funded CAITE project

(Commonwealth Alliance for Information Technology Education)

• Build on alliances with other institutions– refinement and replication of our efforts

(Artbotics replication is already underway)

Jesse Heines, Ed.D.Gena Greher, Ed.D.Sarah Kuhn, Ph.D.

http://www.performamatics.org

video editing by Grad. Asst. Daniel Gabriel

Jesse Heines, Ed.D.Gena Greher, Ed.D.Sarah Kuhn, Ph.D.

http://www.performamatics.org

video editing by Grad. Asst. Daniel GabrielACM SIGCSE 2009Chattanooga, TNACM SIGCSE 2009Chattanooga, TN March 7, 2009March 7, 2009

This work is supported by the National Science Foundation CPATH Program under Grant No. 0722161.

This work is supported by the National Science Foundation CPATH Program under Grant No. 0722161.