MUSI 6001: Music Perception and Cognition

October 19, 2012

Details• Course number: MUSI 6001

• Instructor: Sylvain Le Groux (sylvain.legroux@fulbrightmail.org)

• Time & Location: M&W, 15:05-16:25pm / room 102

• Office hours: after classes or by appointment

OverviewThe course will examine how humans process musical sound beginning withthe basics of the human auditory system and building from this to address theexperience of musical sound. The first part of the course will examine how thefundamental percepts of sound, such as loudness, pitch and timbre, arise fromthe audio signal while the second part of the course will discuss higher-levelissues specific to music, such as melody, rhythm and music-evoked emotion aswell as evolutionary and developmental perspectives on music.

PrerequisitesThere is no formal requirements apart from some interest in music, psychologyor neuroscience.

Presentation & Final PaperThe goal will be to conduct research in a topic area that you choose (e.g. rhythmperception, timbre, emotion etc). The first part will involve a literature surveywhere you select the most relevant research articles and present them to theclass. You will then conduct a behavioral experiment and prepare a researchpaper on this basis in standard APA journal format.

1

WorkloadThe beginning of the class will involve mostly reading, but rapidly, in additionto reading you will be spending a lot of time finding a research topic, conductinga literature survey, and designing and running your experiment.

Texts• Required (approximate paperback prices)

– Plack 43$– Huron 16$– Snyder 30$

• Used in class:

– Deutsch 85$

• Recommended:

– Levitin 10$– Sacks 10$

Grading PolicyAll assignments, quizzes, and tests will be graded by points. The final grade forthe course will be determined by dividing the total points earned by the numberof points possible for each of the categories listed in Method of Evaluation.These numbers will be converted into a grade according to the following scale:A=100-90%, B=89-80%, C=79-70%, D= 69-60%, F= 59% and below.

• Midterm: 20%

• Research paper presentation: 10%

• Term project presentation: 20%

• Term project report: 45%

• Class participation: 5%

Homework assignments are due ON THE DUE DATE. A penalty of one lettergrade per day will be applied to all late assignments. Documented illnesses andfamily emergencies are excepted, of course. Quizzes and exams cannot be madeup unless you have a valid, documented excuse.

2

Course communicationWe will use google group (MUSI 6001) for communication and announcements.All emails to Sylvain Le Groux concerning the course should contain MUSI 6001in the subject.

Honor codeStudents must do their own work on assignments, projects, and tests unlesscollaboration is previously specified and approved by the instructor. Studentscaught cheating will receive zero credit for that assignment/quiz/test and maybe subject to further sanctions through the Office of Student Integrity. Studentsare expected to abide by the Georgia Tech Honor Code and avoid any instancesof academic misconduct, including but not limited to:

• Possessing, using, or exchanging improperly acquired written or oral in-formation in the preparation of a paper or for an exam.

• Substitution of material that is wholly or substantially identical to thatcreated or published by another individual or individuals.

• False claims of performance or work that has been submitted by the stu-dent.

Please refer to the published Georgia Institute of Technology Academic HonorCode for further information: http://www.honor.gatech.edu.

Students with disabilitiesIn accordance with the Americans with Disabilities Act, students with bonafide disabilities will be afforded reasonable accommodation. The ADAPTS Of-fice will certify a disability and advise faculty members of reasonable accom-modations. The web site for a student requesting accommodation is: http:

//www.adapts.gatech.edu/faculty_guide/sturespon.htm

Changes in course requirementsSince all classes do not progress at the same rate, it may be necessary to modifythe above requirements or their timing as circumstances dictate. For exam-ple, the number and frequency of exams may be changed, or the number andsequence of assignments will be altered. In either of these cases, adequate noti-fication will be given in writing and be discussed in class.

3

Course outlineThis syllabus will be subject to changes and updates throughout the course ofthe semester. There is also the possibility to accommodate student interests inspecific subjects. Main readings are listed below and I will propose additionalpapers for each topics.

4

Week Monday Wednesday Reading Project1 (08/20) Introduction,

courseoverview

The natureof sound

Plack 1-3

2 (08/27) Theauditorysystem

Frequencyselectivity &

Masking

Plack 4-5

3 (09/3) Holiday Researchmethodology

4 (09/10) Introductionto statistical

analysis

Loudness Plack 6-8

5 (09/17) Pitch 1 Pitch 2,Timbre

Deutsch 5

6 (09/24) Spatialhearing

Auditoryscene

Plack 9, 10 Projectproposalsdue (a fewparagraphs)

7 (10/01) Consonance,roughness

Scales,tuning

Deutsch 4, 7

8 (10/8) Midterm Melody,harmony,tonality

Deutsch 9,Snyder 11,Deutsch 13

Midterm

9 (10/15) GT Break Rhythm Snyder 3-5,Deutsch 9,Huron 1-6

10 (10/22) Memory,grouping

Emotion

11 (10/29) Emotion/AnticipationHalloween Huron 7-11,Plack 11,

Projectoutline due

(a fewpages)

12 (11/5) Neurology Musictherapy

Schlaug,Peretz,Sacks,Berger

13 (11/12) Developmentalpsychologyof music

Evolutionarypsychologyof music

Bamberger,Trehub,Merker

14 (11/19) Music andlanguage

TBD Patel

15 (11/26) TBD Presentations Termproject due(about 10

pages)16 (12/ 3) Presentations Presentations17 (12/10) Presentations Presentations

5

MUSI 6202: Music DSP

January 21, 2013

Details• Course number: MUSI 6202

• Instructor: Sylvain Le Groux (sylvain.legroux@coa.gatech.edu)

• Time & Location: T&Th, 10:05-11:25pm / room 102

• Office hours: after classes or by appointment

OverviewResearch in music, as well as music production and composition increasinglyrelies on sophisticated digital signal processing techniques. This course will re-view fundamental elements of digital audio signal processing, such as sinusoids,spectra, digital filters, and Fourier analysis and their application to the fun-damental music analysis problems of modeling and synthesis. We will discussaudio effects and techniques such as sinusoidal modeling, phase vocoder, reverb,chorus / flanger, pitch-shifting, time compression, etc. The class will includepractical lab sessions as well as presentations of state of the art papers andstudent’s projects.

PrerequisitesBasic signals and systems + some programming experience. Labs & projects willinvolve implementing audio processing algorithms using Python and Max/MSP.

Presentation & Final PaperThe goal will be to develop a project in a topic area that you choose (e.g.synthesizer, audio effects, filtering, ...). The first part will involve a literaturesurvey where you will select the most relevant research articles related to yourproject and present the survey to the class. You will then develop your ownapproach and algorithms and prepare a ~10 pages report on your work.

1

WorkloadThe class will involve some reading, regular practical exercises/labs, midterm &final exams, as well as a final project consisting of an audio dsp implementationand a written report.

Texts• Recommended (approximate paperback prices)

– “Real sound synthesis for interactive applications” by Cook, Perry40$

– “Signal Processing First” by James H. McClellan, Ronald W. Schafer,Mark A. Yoder 130$

• Extra

– “DAFX: digital audio effects” by Udo Zolzer 80$

Grading PolicyAll assignments, quizzes, and tests will be graded by points. The final grade forthe course will be determined by dividing the total points earned by the numberof points possible for each of the categories listed in Method of Evaluation.These numbers will be converted into a grade according to the following scale:A=100-90%, B=89-80%, C=79-70%, D= 69-60%, F= 59% and below.

• Midterm: 20%

• Final: 20%

• Homework/labs: 20 %

• Research paper presentation: 10%

• Term project 30%

Homework assignments are due ON THE DUE DATE. A penalty of one lettergrade per day will be applied to all late assignments. Documented illnesses andfamily emergencies are excepted, of course. Quizzes and exams cannot be madeup unless you have a valid, documented excuse.

Course communicationWe will use google group (MUSI 6202) for communication and announcements.All emails to Sylvain Le Groux concerning the course should contain [MUSI6202] in the subject.

2

Honor codeStudents must do their own work on assignments, projects, and tests unlesscollaboration is previously specified and approved by the instructor. Studentscaught cheating will receive zero credit for that assignment/quiz/test and maybe subject to further sanctions through the Office of Student Integrity. Studentsare expected to abide by the Georgia Tech Honor Code and avoid any instancesof academic misconduct, including but not limited to:

• Possessing, using, or exchanging improperly acquired written or oral in-formation in the preparation of a paper or for an exam.

• Substitution of material that is wholly or substantially identical to thatcreated or published by another individual or individuals.

• False claims of performance or work that has been submitted by the stu-dent.

Please refer to the published Georgia Institute of Technology Academic HonorCode for further information: http://www.honor.gatech.edu.

Students with disabilitiesIn accordance with the Americans with Disabilities Act, students with bonafide disabilities will be afforded reasonable accommodation. The ADAPTS Of-fice will certify a disability and advise faculty members of reasonable accom-modations. The web site for a student requesting accommodation is: http:

//www.adapts.gatech.edu/faculty_guide/sturespon.htm

Changes in course requirementsSince all classes do not progress at the same rate, it may be necessary to modifythe above requirements or their timing as circumstances dictate. For exam-ple, the number and frequency of exams may be changed, or the number andsequence of assignments will be altered. In either of these cases, adequate noti-fication will be given in writing and be discussed in class.

Course outlineThis syllabus will be subject to changes and updates throughout the course ofthe semester. There is also the possibility to accommodate student interests inspecific subjects. Main readings are listed below and I will propose additionalpapers for each topics.

3

Week Tuesday Thursday Reading Project1 (01/08) Introduction,

courseoverview

X C1

2 (01/15) Sinusoids,Python

DigitalAudio

C 2, 3

3 (01/22) Spectrum,Fourier

Samplingsynthesis,DFT lab

C 4

4 (01/29) DigitalFilter

Modalsynthesis

C 5

5 (02/05) STFT STFT lab C56 (02/12) Spectral

modeling &Additivesynthesis

Spectralmodeling lab

C 6

7 (02/19) Sinusoidal +residualmodeling

Sinusoidal +noise lab

C6

8 (02/26) Vocoder +filterbanks

Subtractivesynthesis &

LPC

C 7,8 Projectproposalsdue (a fewparagraphs)

9 (03/05) Strings &Bars

Non-linearity,waveshap-

ing,FM

C 9, 10

10 (03/12) Review Midterm11 (03/19) Spring break Spring break12 (03/26) Sound FX

labSound FX

labProject

outline due(a fewpages)

13 (04/02) Tubes & aircavities

2 & 3D C 11, 12

14 (04/09) FOF,wavelets,particles

Excitation &control

C 13, 14

15 (04/16) Music DSPapplicationsto MIR, ML,

Cognitivesystems

Presentations Termproject due(about 10

pages)

16 (04/23) Presentations Presentations17 (04/30) Final exam Final exam

4

Advanced interface design - 30853

(Universitat Pompeu Fabra)

December 19, 2014

1 Description

This course focuses on paradigms, methods and tools used in the constructionof complex multimodal interfaces between users and artefacts

• instructors: Marti Sanchez (marti.sanchez@upf.edu) and Sylvain Le Groux(sylvain.legroux@fulbrightmail.org)

2 Course objectives

Students will learn to build and use interfaces and artefacts that can engagesubjects during perceptual/behavioural task and will be given the tools to beable to capture and measure different characteristics of the performed tasks anddeliver feedback. Covering the different phases of this closed loop experience wewill learn:

the basics of sensing technologies and we will learn how to use existingdevices and how to build our own how transfer data through communicationprotocols and how to process it with examples of measures that we can extracthow to deliver feedback through actuation using motors, displays, leds Robotsare great tools to learn and practice sensor actuator loops. We will learn basicprinciples of robotic systems using a mobile Arduino based robotic platformthat we developed at SPECS group.

3 Readings and materials

In the course we provide tutorials for each weekly session as well as source ex-ample code. We will also provide hardware material to use. The main hardwaretools that will be covered in the course are Arduino, RaspberryPi, Kinect, Con-sole Devices like the Wiimote, Eye tracking, Physiological signals. Processingand Python will be the main programming languages used during the course.The course is project oriented: from an early stage a project will be structuredand developed in accordance with the teacher

1

3.1 Class attendanceRegular attendance to the classes is mandatory.

4 Evaluation Criteria

Evaluation of the progress of the students is carried on during the differentphases of the project development.

• Initial project draft: 15 %

• Work in class, group: 15%

• Final presentation, group: 40%

• 4-page final report, one per group: 30%

Individual contributions have to be indicated explicitly.

5 Course Structure

5.1 Week 1 and Week 2Introduction and Arduino / Processing basics

We will give a broad overview of the state of the art in interface technolo-gies. Arduino electronics and programming will be introduced. The Processingprograming is also used for rapid prototyping of applications interfacing withsensors and actuators which may need basic visualization capabilities. The ad-vantage of using processing is that programs can be very easily transferred toAndroid mobile phones.

5.2 Week 3 and Week 4Raspberry Pi, Communication protocols

Project guidelines will be presented and students will need to start focusingon which direction they want to take. During week 4 projects start to bedeveloped. Raspberry Pi is a linux based mini computer with the capabilityof interfacing with sensors and actuators and Arduino itself. We will introduceit in the course so that we realize the similarities and differences with Arduino.Communication protocols are the basic tools to plug together sensors of differentnature or systems.For this purpose we introduce TUIO and OSC. An example isdeveloped through the class which consists of a video sequencer and controllerthat can be interfaced with Arduino sensor data.

2

5.3 Week 5Robotic and control applications

We will learn the basics of robotic systems through different examples. We willuse an Arduino/RaspberryPi robotic platform developed in SPECS. We will alsoexplain a case study of an Arduino self-balancing robot that can be controlledvia bluetooth through processing. Projects will be developing through the restof the weeks with feedback and support.

5.4 Week 6Computer Vision and RGB-D Applications

We will deal in this class with a variety of camera sensing technologies includinguse of normal cameras and Kinect devices.

5.5 Week 7Interfacing with physiology signals: Arduino E-Health Sensor board

This is a class dedicated to learn how to interface with physiology signals andhow to compute arousal and valence measures from them. The class covers adiverse variety of physiological signals : from complex EEG recordings to heartrate and galvanic skin response.

5.6 Week 8Audio Processing

We deal with several audio processing applications like beat detection, frequencybased analysis and others. We will learn how to control interfaces with differentaudio extracted parameters. Projects start reaching

5.7 Week 9Acquiring and sharing data through internet and the internet of things

Examples of how to share sensor data through internet applications are provided.Being the last class before the presentations the main part of the class will bedevoted to finalizing projects.

5.8 Week 10Project Presentations

3

Project presentations include a demo of the built device. Examples of previousyears projects:

A sonar sensor based sensitive stick with haptic feedback for blind naviga-tion. The vibrating belt : an 8 motor vibrating belt based on Arduino A learninggestural frequency selector and filtering system An arduino sensing skate usedfor monitoring skateboard tricks A fitness system using multimodal interfaces

4