acm sigcse 2003: multimedia construction projects mark guzdial college of computing georgia...
TRANSCRIPT
![Page 1: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/1.jpg)
ACM SIGCSE 2003: Multimedia Construction Projects
Mark Guzdial
College of Computing
Georgia Institute of [email protected]
http://www.cc.gatech.edu/~mark.guzdial
![Page 2: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/2.jpg)
Plan
7-7:15: Introduction What’s on your CD Why media computation
7:15-7:45: Picture manipulations in Python 7:45-8:15: Sound manipulations in Python 8:15-8:30: Break 8:30-8:45: How to do this in Java and Squeak 8:45-9:45: You Play 9:45-10: Wrap-up
![Page 3: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/3.jpg)
What’s on your CD
Materials from our Introduction to Media Computation course Jython Environment for Students (JES) Book Course slides MediaTools: Squeak-based media exploration tools
Material for this workshop Java and Squeak API slides Java source code Squeak computer music essays
![Page 4: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/4.jpg)
Computer science is more important than Calculus In 1961, Alan Perlis argued
that computer science is more important in a liberal education than calculus Explicitly, he argued that all
students should learn to program.
Calculus is about rates, and that’s important to many.
Computer science is about process, which is important to everyone
![Page 5: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/5.jpg)
How close are we to being able to teach everyone CS?
Not very CS1 is one of the most despised courses for non-majors
At many departments, CS retention rates are lower than the rest of campus At Georgia Tech: 65% for 1995 cohort, vs. 73% for
Engineeering
Drop-out rates near 50% at many institutions Female enrollment in CS has been dropping nationally
![Page 6: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/6.jpg)
Why?
Several recent studies and books claim that CS instruction tends to dissuade anyone but white males “Tedious,”
“taught without application relevance,”“boring,” “lacking creativity,” “asocial”
![Page 7: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/7.jpg)
The potential of computing
How can we realize the potential impact of computing if future professionals in other disciplines despise computer science?
To realize that potential, we need practitioners in other disciplines to understand computing. At least some of those practitioners need to understand
it as a creative, exciting venture—as we do!
![Page 8: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/8.jpg)
The best uses for computing technologies will come from others
Thomas Edison vs. D.W. Griffith If we want computing technologies to become
useful, they have to get out of our hands. It can’t be just through applications
That presumes that the current technologists know how everyone else wants to do things and should do things
Suggestion: D.W. Griffith knew things that Edison didn’t.
![Page 9: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/9.jpg)
An attempt at relevant, creative computing: Introduction to Media Computation
A course for non-CS and non-Engineering majors International Affairs, Literature, Public Policy,
Architecture, Management, Biology, etc.
120 students this semester,planning 400-600 in the Fall 2/3 female in this semester’s CS1315
Focus: Learning programming within the context of media manipulation and creation
![Page 10: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/10.jpg)
Motivating the Computing
As professionals, these students will often the use the computer as a communications medium.
All media are going digital,and digital media are manipulated with software.
Knowing how to program, then, is a communications skill.
![Page 11: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/11.jpg)
Python as the programming language Huge and contentious issue Use in commercial contexts legitimatizes the choice
Industrial Light & Magic, Google, Nextel, etc.
Minimal syntax Looks like other programming languages
Potential for knowledge transfer for students “Executable pseudocode” for this workshop
Actually using Jython (http://www.jython.org) for Java class libraries
![Page 12: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/12.jpg)
JES - Jython Environment for Students
![Page 13: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/13.jpg)
Simple Python in JES
>>> print 34 + 5690>>> print 34.1/46.50.7333333333333334>>> print 22 * 33726>>> print 14 - 15-1>>> print "Hello"Hello>>> print "Hello" + "Mark"HelloMark
![Page 14: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/14.jpg)
Command Area Editing
Up/down arrows walk through command history You can edit the line at the bottom
Just put the cursor at the end of the line before hitting Return/Enter.
![Page 15: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/15.jpg)
Basic Picture Functions
makePicture(filename) creates and returns a picture object, from the JPEG file at the filename
show(picture) displays a picture in a window We’ll learn functions for manipulating pictures
later, like getColor, setColor, and repaint
![Page 16: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/16.jpg)
Basic Sound Functions
makeSound(filename) creates and returns a sound object, from the WAV file at the filename
play(sound) makes the sound play (but doesn’t wait until it’s done)
blockingPlay(sound) waits for the sound to finish We’ll learn more later like getSample and
setSample
![Page 17: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/17.jpg)
Demonstrating simple JES>>> print pickAFile()/Users/guzdial/mediasources/barbara.jpg>>> print makePicture(pickAFile())Picture, filename /Users/guzdial/mediasources/barbara.jpg height 294 width 222>>> print pickAFile()/Users/guzdial/mediasources/hello.wav>>> print makeSound(pickAFile())Sound of length 54757>>> print play(makeSound(pickAFile()))None>>> myfilename = pickAFile()>>> print myfilename/Users/guzdial/mediasources/barbara.jpg>>> mypicture = makePicture(myfilename)>>> print mypicturePicture, filename /Users/guzdial/mediasources/barbara.jpg height 294 width 222>>> show(mypicture)
![Page 18: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/18.jpg)
Writing a recipe: Making our own functions To make a function, use the
command def Then, the name of the function,
and the names of the input values between parentheses (“(input1)”)
End the line with a colon (“:”) The body of the recipe is indented
(Hint: Use two spaces) Your function does NOT
exist for JES until you load it
![Page 19: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/19.jpg)
Functions for picking and playing/showing files
def pickAndShow(): myfile = pickAFile() mypict = makePicture(myfile) show(mypict)
def pickAndPlay(): myfile = pickAFile() mysound = makeSound(myfile) play(mysound)
![Page 20: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/20.jpg)
Functions that take input
def playNamed(myfile): mysound = makeSound(myfile) play(mysound)
def showNamed(myfile): mypict = makePicture(myfile) show(mypict)
![Page 21: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/21.jpg)
CS1315:Introduction to Media Computation
Picture encoding and manipulation
![Page 22: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/22.jpg)
Pictures and Pixels
A picture is a matrix of pixels A picture has two dimensions: Width and Height
Pixels are picture elements Each pixel object knows its color using RGB
encoding It also knows where it is in its picture
![Page 23: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/23.jpg)
RGB
In RGB, each color has three component colors:
Amount of redness Amount of greenness Amount of blueness
Each does appear as a separate dot on most devices, but our eye blends them.
In most computer-based models of RGB, a single byte (8 bits) is used for each
So a complete RGB color is 24 bits, 8 bits of each
![Page 24: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/24.jpg)
Encoding RGB
Each component color (red, green, and blue) is encoded as a single byte
Colors go from (0,0,0) to (255,255,255) If all three components are
the same, the color is in greyscale
(50,50,50) at (2,2) (0,0,0) (at position (1,2) in
example) is black (255,255,255) is white
![Page 25: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/25.jpg)
Manipulating pixels
>>> pixel=getPixel(picture,1,1)>>> print pixelPixel, color=color r=168 g=131 b=105>>> pixels=getPixels(picture)>>> print pixels[0]Pixel, color=color r=168 g=131 b=105
getPixel(picture,x,y) gets a single pixel.
getPixels(picture) gets all of them in an array. (Square brackets is a standard array reference notation—which we’ll generally not use.)
![Page 26: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/26.jpg)
What can we do with a pixel?
• getRed, getGreen, and getBlue are functions that take a pixel as input and return a value between 0 and 255
• setRed, setGreen, and setBlue are functions that take a pixel as input and a value between 0 and 255
![Page 27: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/27.jpg)
We can also get, set, and make Colors getColor takes a pixel as input and returns a Color object with the
color at that pixel setColor takes a pixel as input and a Color, then sets the pixel to that
color makeColor takes red, green, and blue values (in that order) between
0 and 255, and returns a Color object pickAColor lets you use a color chooser and returns the chosen
color We also have functions that can makeLighter and makeDarker an
input color
![Page 28: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/28.jpg)
We can change pixels directly…>>> file="/Users/guzdial/mediasources/barbara.jpg">>> pict=makePicture(file)>>> show(pict)>>> setColor(getPixel(pict,10,100),yellow)>>> setColor(getPixel(pict,11,100),yellow)>>> setColor(getPixel(pict,12,100),yellow)>>> setColor(getPixel(pict,13,100),yellow)>>> repaint(pict)
But that’s really dull and boring…That’s the subject of the next lecture
![Page 29: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/29.jpg)
If you make something you like…
writePictureTo(picture,”filename”) Writes the picture out as a JPEG
Be sure to end your filename as “.jpg”!
If you don’t specify a full path,will be saved in the same directory as JES.
![Page 30: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/30.jpg)
How do you find out what RGB values you have? And where?
Use the MediaTools!
![Page 31: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/31.jpg)
Use a loop!Our first picture recipe
def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Used like this:>>> file="/Users/guzdial/mediasources/barbara.jpg">>> picture=makePicture(file)>>> show(picture)>>> decreaseRed(picture)>>> repaint(picture)
![Page 32: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/32.jpg)
getPixels returns a sequence of pixels Each pixel knows its color and its original picture Change the pixel, you change the picture So the loop below assigns the index variable p to
each pixel in the picture picture, one at a time.
def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
![Page 33: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/33.jpg)
Let’s walk that through slowly…def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Here we get a picture object in as input and call it picture
picture
![Page 34: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/34.jpg)
Now, get the pixelsdef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
We get all the pixels from the picture, then make p be the name of each one one at a time
picture
Pixel, color r=168 g=131 b=105
Pixel, color r=160 g=131 b=105
Pixel, color r=168 g=132b=106
…
p
getPixels()
![Page 35: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/35.jpg)
Get the red value from pixeldef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
We get the red value of pixel p and name it value
picture
Pixel, color r=168 g=131 b=105
Pixel, color r=168 g=132b=106
…
pvalue = 168
Pixel, color r=160 g=131 b=105
![Page 36: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/36.jpg)
Now change the pixeldef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Set the red value of pixel p to 0.5 (50%) of value
picture
Pixel, color r=84 g=131 b=105
Pixel, color r=168 g=132b=106
…
pvalue = 168
Pixel, color r=160 g=131 b=105
![Page 37: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/37.jpg)
Then move on to the next pixeldef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Move on to the next pixel and name it p
picture
Pixel, color r=84 g=131 b=105
Pixel, color r=168 g=132b=106
…
pvalue = 168
Pixel, color r=160 g=131 b=105
![Page 38: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/38.jpg)
Get its red valuedef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Change value to the new red value at the new p
picture
Pixel, color r=84 g=131 b=105
Pixel, color r=168 g=131 b=105
Pixel, color r=168 g=132b=106
…
value = 160p
Get its red valuedef decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)
Change value to the new red value at the new p
picture
Pixel, color r=84 g=131 b=105
Pixel, color r=168 g=132b=106
…
p
Pixel, color r=160 g=131 b=105
![Page 39: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/39.jpg)
And change this red value
def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5) Change the red value at
pixel p to 50% of valuepicture
Pixel, color r=84 g=131 b=105
Pixel, color r=80 g=131 b=105
Pixel, color r=168 g=132b=106
…
value = 160p
![Page 40: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/40.jpg)
And eventually, we do all pixels
We go from this… to this!
![Page 41: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/41.jpg)
Increasing Red
def increaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*1.2)
What happened here?!?
Remember that the limit for redness is 255.
If you go beyond 255, all kinds of weird things can happen: Wrap around
![Page 42: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/42.jpg)
Clearing Blue
def clearBlue(picture): for p in getPixels(picture): setBlue(p,0)
![Page 43: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/43.jpg)
Combining into a sunset function
How do we turn this beach scene into a sunset?
What happens at sunset? At first, I tried increasing the
red, but that made things like red specks in the sand REALLY prominent.
That can’t be how it really works
New Theory: As the sun sets, less blue and green is visible, which makes things look more red.
![Page 44: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/44.jpg)
A Sunset-generation Functiondef makeSunset(picture): for p in getPixels(picture): value=getBlue(p) setBlue(p,value*0.7) value=getGreen(p) setGreen(p,value*0.7)
![Page 45: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/45.jpg)
Creating a negative
Let’s think it through R,G,B go from 0 to 255 Let’s say Red is 10. That’s very light red.
What’s the opposite? LOTS of Red! The negative of that would be 245: 255-10
So, for each pixel, if we negate each color component in creating a new color, we negate the whole picture.
![Page 46: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/46.jpg)
Recipe for creating a negative
def negative(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor( 255-red, 255-green, 255-blue) setColor(px,negColor)
![Page 47: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/47.jpg)
Converting to greyscale
We know that if red=green=blue, we get grey But what value do we set all three to?
What we need is a value representing the darkness of the color, the luminance There are lots of ways of getting it, but one way that works reasonably well is
dirt simple—simply take the average:
![Page 48: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/48.jpg)
Converting to greyscaledef greyScale(picture): for p in getPixels(picture): intensity = (getRed(p)+getGreen(p)+getBlue(p))/3 setColor(p,makeColor(intensity,intensity,intensity))
![Page 49: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/49.jpg)
But that’s not really the best greyscale
In reality, we don’t perceive red, green, and blue as equal in their amount of luminance: How bright (or non-bright) something is. We tend to see blue as “darker” and red as “brighter” Even if, physically, the same amount of light is coming
off of each
Photoshop’s greyscale is very nice: Very similar to the way that our eye sees it B&W TV’s are also pretty good
![Page 50: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/50.jpg)
Building a better greyscale
We’ll weight red, green, and blue based on how light we perceive them to be, based on laboratory experiments.
def greyScaleNew(picture): for px in getPixels(picture): newRed = getRed(px) * 0.299 newGreen = getGreen(px) * 0.587 newBlue = getBlue(px) * 0.114 luminance = newRed+newGreen+newBlue setColor(px,makeColor(luminance,luminance,luminance))
![Page 51: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/51.jpg)
Comparing the two greyscales:Average on left, weighted on right
![Page 52: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/52.jpg)
Let’s try making Barbara a redhead!
We could just try increasing the redness, but as we’ve seen, that has problems. Overriding some red spots And that’s more than just her hair
If only we could increase the redness only of the brown areas of Barb’s head…
![Page 53: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/53.jpg)
Making Barb a redhead
def turnRed(): brown = makeColor(57,16,8) file = r"C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\barbara.jpg" picture=makePicture(file) for px in getPixels(picture): color = getColor(px) if distance(color,brown)<50.0:
redness=getRed(px)*1.5 setRed(px,redness) show(picture) return(picture)
Original:
![Page 54: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/54.jpg)
Tuning our color replacement
If you want to get more of Barb’s hair, just increasing the threshold doesn’t work Wood behind becomes within the threshold value
How could we do it better? Lower our threshold, but then miss some of the hair Work only within a range…
![Page 55: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/55.jpg)
Introducing the function range
Range returns a sequence between its first two inputs, possibly using a third input as the increment
>>> print range(1,4)[1, 2, 3]>>> print range(-1,3)[-1, 0, 1, 2]>>> print range(1,10,2)[1, 3, 5, 7, 9]
![Page 56: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/56.jpg)
Replacing colorsin a range
def turnRedInRange(): brown = makeColor(57,16,8) file=r"C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\barbara.jpg" picture=makePicture(file) for x in range(70,168): for y in range(56,190): px=getPixel(picture,x,y) color = getColor(px) if distance(color,brown)<50.0: redness=getRed(px)*1.5 setRed(px,redness) show(picture) return(picture)
Get the range using MediaTools
![Page 57: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/57.jpg)
Mirroring
Imagine a mirror horizontally across the picture,or vertically
What would we see? How do generate that digitally?
We simply copy the colors of pixels from one place to another
![Page 58: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/58.jpg)
Mirroring a picture
Slicing a picture down the middle and sticking a mirror on the slice Do it by using a loop to measure a difference
The index variable is actually measuring distance from the mirrorpoint
Then reference to either side of the mirror point using the difference
![Page 59: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/59.jpg)
Recipe for mirroring
def mirrorVertical(source): mirrorpoint = int(getWidth(source)/2) for y in range(1,getHeight(source)): for x in range(1,mirrorpoint): p = getPixel(source, x+mirrorpoint,y) p2 = getPixel(source, mirrorpoint-x,y) c = getColor(p2) setColor(p,c)
![Page 60: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/60.jpg)
Can we do it with a horizontal mirror?
def mirrorHorizontal(source): mirrorpoint = int(getHeight(source)/2) for y in range(1,mirrorpoint): for x in range(1,getWidth(source)): p = getPixel(source,x,y+mirrorpoint) p2 = getPixel(source,x,mirrorpoint-y) setColor(p,getColor(p2))
![Page 61: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/61.jpg)
Of course!
![Page 62: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/62.jpg)
What if we wanted to copy bottom to top? Very simple: Swap the p and p2 in the bottom line
Copy from p to p2, instead of from p2 to p
def mirrorHorizontal(source): mirrorpoint = int(getHeight(source)/2) for y in range(1,mirrorpoint): for x in range(1,getWidth(source)): p = getPixel(source,x,y+mirrorpoint) p2 = getPixel(source,x,mirrorpoint-y) setColor(p2,getColor(p))
![Page 63: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/63.jpg)
Messing with Santa some more
![Page 64: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/64.jpg)
Copying pixels
In general, what we want to do is to keep track of a sourceX and sourceY, and a targetX and targetY. We increment (add to them) in pairs
sourceX and targetX get incremented together sourceY and targetY get incremented together
The tricky parts are: Setting values inside the body of loops Incrementing at the bottom of loops
![Page 65: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/65.jpg)
Copying Barb to a canvasdef copyBarb(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying targetX = 1 for sourceX in range(1,getWidth(barb)): targetY = 1 for sourceY in range(1,getHeight(barb)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) targetY = targetY + 1 targetX = targetX + 1 show(barb) show(canvas) return canvas
![Page 66: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/66.jpg)
Transformation = Small changes in copying Making relatively small changes in this basic
copying program can make a variety of transformations. Change the targetX and targetY, and you copy wherever
you want Cropping: Change the sourceX and sourceY range, and
you copy only part of the program. Rotating: Swap targetX and targetY, and you end up
copying sideways Scaling: Change the increment on sourceX and sourceY,
and you either grow or shrink the image.
![Page 67: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/67.jpg)
Copying into the middle of the canvas
def copyBarbMidway(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying targetX = 100 for sourceX in range(1,getWidth(barb)): targetY = 100 for sourceY in range(1,getHeight(barb)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) targetY = targetY + 1 targetX = targetX + 1 show(barb) show(canvas) return canvas
![Page 68: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/68.jpg)
Rotating the copydef copyBarbSideways(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying targetX = 1 for sourceX in range(1,getWidth(barb)): targetY = 1 for sourceY in range(1,getHeight(barb)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetY,targetX), color) targetY = targetY + 1 targetX = targetX + 1 show(barb) show(canvas) return canvas
![Page 69: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/69.jpg)
Cropping: Just the facedef copyBarbsFace(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying targetX = 100 for sourceX in range(45,200): targetY = 100 for sourceY in range(25,200): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) targetY = targetY + 1 targetX = targetX + 1 show(barb) show(canvas) return canvas
![Page 70: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/70.jpg)
Scaling the picture downdef copyBarbsFaceSmaller(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)/2)): sourceY = 25 for targetY in range(100,100+((200-25)/2)): color = getColor(getPixel(barb,sourceX,sourceY)) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 2 sourceX = sourceX + 2 show(barb) show(canvas) return canvas
![Page 71: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/71.jpg)
Scaling the picture updef copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor(getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 0.5 sourceX = sourceX + 0.5 show(barb) show(canvas) return canvas
![Page 72: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/72.jpg)
Background subtraction
Let’s say that you have a picture of someone, and a picture of the same place (same background) without the someone there, could you subtract out the background and leave the picture of the person?
Maybe even change the background? Let’s take that as our problem!
![Page 73: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/73.jpg)
Person (Katie) and Background
![Page 74: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/74.jpg)
Background Subtraction Codedef swapbg(person, bg, newbg): for x in range(1,getWidth(person)): for y in range(1,getHeight(person)): personPixel = getPixel(person,x,y) bgpx = getPixel(bg,x,y) personColor= getColor(personPixel) bgColor = getColor(bgpx) if distance(personColor,bgColor) < 10: bgcolor = getColor(getPixel(newbg,x,y)) setColor(personPixel, bgcolor)
![Page 75: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/75.jpg)
Putting Katie in a Jungle
![Page 76: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/76.jpg)
But why isn’t it alot better?
We’ve got places where we got pixels swapped that we didn’t want to swap See Katie’s shirt stripes
We’ve got places where we want pixels swapped, but didn’t get them swapped See where Katie made a
shadow
![Page 77: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/77.jpg)
Another way: Chromakey
Have a background of a known color Some color that won’t be on
the person you want to mask out
Pure green or pure blue is most often used
I used my son’s blue bedsheet
This is how the weather people seem to be in front of a map—they’re actually in front of a blue sheet.
![Page 78: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/78.jpg)
Chromakey recipedef chromakey(source,bg): # source should have something in front of blue, bg is the new background for x in range(1,getWidth(source)): for y in range(1,getHeight(source)): p = getPixel(source,x,y) # My definition of blue: If the redness + greenness < blueness if (getRed(p) + getGreen(p) < getBlue(p)): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,x,y)))
![Page 79: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/79.jpg)
Can also do this with getPixels()def chromakey2(source,bg): # source should have something in front of blue, bg is the new background for p in getPixels(source): # My definition of blue: If the redness + greenness < blueness if (getRed(p) + getGreen(p) < getBlue(p)): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,getX(p),getY(p))))
![Page 80: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/80.jpg)
Example results
![Page 81: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/81.jpg)
Just trying the obvious thing for Red
def chromakey2(source,bg): # source should have something in front of red, bg is the new background for p in getPixels(source): if getRed(p) > (getGreen(p) + getBlue(p)): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,getX(p),getY(p))))
![Page 82: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/82.jpg)
Doesn’t always work as you expect
![Page 83: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/83.jpg)
Let’s try that with green
def chromakeyGreen(source,bg): # source should have something in front of green, bg is the new background for x in range(1,getWidth(source)): for y in range(1,getHeight(source)): p = getPixel(source,x,y) # My definition of green: If the greenness > redness + blueness if getGreen(p) > getBlue(p) + getRed(p): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,x,y)))
![Page 84: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/84.jpg)
The same definition of green doesn’t work
![Page 85: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/85.jpg)
Tweaking Chromakey
def chromakeyGreen(source,bg): # source should have something in front of green, bg is the new background for x in range(1,getWidth(source)): for y in range(1,getHeight(source)): p = getPixel(source,x,y) # My definition of green: If the greenness > redness and blueness if getGreen(p) > getBlue(p) and getGreen(p) > getRed(p): #Then, grab the color at the same spot from the new background setColor(p,getColor(getPixel(bg,x,y)))
![Page 86: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/86.jpg)
That looks better
![Page 87: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/87.jpg)
New functions for drawing on pictures
addText(pict,x,y,string) puts the string starting at position (x,y) in the picture
addLine(picture,x1,y1,x2,y2) draws a line from position (x1,y1) to (x2,y2)
addRect(pict,x1,y1,w,h) draws a black rectangle (unfilled) with the upper left hand corner of (x1,y1) and a width of w and height of h
addRectFilled(pict,x1,y1,w,h,color) draws a rectangle filled with the color of your choice with the upper left hand corner of (x1,y1) and a width of w and height of h
![Page 88: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/88.jpg)
Making it easier to deal with the media folder
>>> setMediaFolder()New media folder: /Users/guzdial/mediasources/>>> print getMediaPath("barbara.jpg")/Users/guzdial/mediasources/barbara.jpg>>> print getMediaPath("sec1silence.wav")/Users/guzdial/mediasources/sec1silence.wav
![Page 89: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/89.jpg)
Example picturedef littlepicture(): canvas=makePicture(getMediaPath("640x480.jpg")) addText(canvas,10,50,"This is not a picture") addLine(canvas,10,20,300,50) addRectFilled(canvas,0,200,300,500,yellow) addRect(canvas,10,210,290,490) return canvas
![Page 90: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/90.jpg)
CS1315:Introduction to Media Computation
Sound Encoding and Manipulation
![Page 91: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/91.jpg)
How sound works:Acoustics, the physics of sound
Sounds are waves of air pressure Sound comes in cycles The frequency of a wave is
the number of cycles per second (cps), or Hertz
(Complex sounds have more than one frequency in them.)
The amplitude is the maximum height of the wave
![Page 92: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/92.jpg)
Volume and pitch:Psychoacoustics, the psychology of sound
Our perception of volume is related (logarithmically) to changes in amplitude If the amplitude doubles, it’s about a 3 decibel (dB) change
Our perception of pitch is related (logarithmically) to changes in frequency Higher frequencies are perceived as higher pitches We can hear between 5 Hz and 20,000 Hz (20 kHz) A above middle C is 440 Hz
![Page 93: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/93.jpg)
“Logarithmically?”
It’s strange, but our hearing works on ratios not differences, e.g., for pitch. We hear the difference between 200 Hz and 400 Hz, as
the same as 500 Hz and 1000 Hz Similarly, 200 Hz to 600 Hz, and 1000 Hz to 3000 Hz
Intensity (volume) is measured as watts per meter squared A change from 0.1W/m2 to 0.01 W/m2, sounds the same
to us as 0.001W/m2 to 0.0001W/m2
![Page 94: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/94.jpg)
Decibel is a logarithmic measure
A decibel is a ratio between two intensities: 10 * log10(I1/I2) As an absolute measure, it’s in comparison to threshold
of audibility 0 dB can’t be heard. Normal speech is 60 dB. A shout is about 80 dB
![Page 95: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/95.jpg)
Digitizing Sound: How do we get that into numbers? Remember in calculus,
estimating the curve by creating rectangles?
We can do the same to estimate the sound curve Analog-to-digital conversion
(ADC) will give us the amplitude at an instant as a number: a sample
How many samples do we need?
![Page 96: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/96.jpg)
Nyquist Theorem
We need twice as many samples as the maximum frequency in order to represent (and recreate, later) the original sound.
The number of samples recorded per second is the sampling rate If we capture 8000 samples per second, the highest frequency
we can capture is 4000 Hz That’s how phones work
If we capture more than 44,000 samples per second, we capture everything that we can hear (max 22,000 Hz)
CD quality is 44,100 samples per second
![Page 97: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/97.jpg)
Digitizing sound in the computer
Each sample is stored as a number (two bytes) What’s the range of available combinations?
16 bits, 216 = 65,536 But we want both positive and negative values
To indicate compressions and rarefactions. What if we use one bit to indicate positive (0) or negative (1)? That leaves us with 15 bits 15 bits, 215 = 32,768 One of those combinations will stand for zero
We’ll use a “positive” one, so that’s one less pattern for positives
Each sample can be between -32,768 and 32,767
![Page 98: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/98.jpg)
Working with sounds
We’ll use pickAFile and makeSound as we have before. But now we want .wav files
We’ll use getSamples to get all the sample objects out of a sound
We can also get the value at any index with getSampleValueAt
Sounds also know their length (getLength) and their sampling rate (getSamplingRate)
Can save sounds with writeSoundTo(sound,”file.wav”)
![Page 99: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/99.jpg)
Demonstrating Working with Sound in JES
>>> filename=pickAFile()>>> print filename/Users/guzdial/mediasources/preamble.wav>>> sound=makeSound(filename)>>> print soundSound of length 421109>>> samples=getSamples(sound)>>> print samplesSamples, length 421109>>> print getSampleValueAt(sound,1)36>>> print getSampleValueAt(sound,2)29
![Page 100: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/100.jpg)
Demonstrating working with samples>>> print getLength(sound)220568>>> print getSamplingRate(sound)22050.0>>> print getSampleValueAt(sound,220568)68>>> print getSampleValueAt(sound,220570)I wasn't able to do what you wanted.The error java.lang.ArrayIndexOutOfBoundsException has occuredPlease check line 0 of >>> print getSampleValueAt(sound,1)36>>> setSampleValueAt(sound,1,12)>>> print getSampleValueAt(sound,1)12
![Page 101: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/101.jpg)
Working with Samples
We can get sample objects out of a sound with getSamples(sound) or getSampleObjectAt(sound,index)
A sample object remembers its sound, so if you change the sample object, the sound gets changed.
Sample objects understand getSample(sample) and setSample(sample,value)
![Page 102: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/102.jpg)
Example: Manipulating Samples>>> soundfile=pickAFile()>>> sound=makeSound(soundfile)>>> sample=getSampleObjectAt(sound,1)>>> print sampleSample at 1 value at 59>>> print soundSound of length 387573>>> print getSound(sample)Sound of length 387573>>> print getSample(sample)59>>> setSample(sample,29)>>> print getSample(sample)29
![Page 103: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/103.jpg)
Recipe to Increase the Volume
def increaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 2)
Using it:>>> f="/Users/guzdial/mediasources/gettysburg10.wav">>> s=makeSound(f)>>> increaseVolume(s)>>> play(s)>>> writeSoundTo(s,"/Users/guzdial/mediasources/louder-g10.wav")
![Page 104: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/104.jpg)
Decreasing the volume
def decreaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 0.5)
This works just like increaseVolume, but we’re lowering each sample by 50% instead of doubling it.
![Page 105: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/105.jpg)
Maximizing volume
How do we get maximal volume? It’s a three-step process:
First, figure out the loudest sound (largest sample). Next, figure out a multiplier needed to make that sound
fill the available space. We want to solve for x where x * loudest = 32767 So, x = 32767/loudest
Finally, multiply the multiplier times every sample
![Page 106: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/106.jpg)
Maxing (normalizing) the sound
def normalize(sound): largest = 0 for s in getSamples(sound): largest = max(largest,getSample(s) ) multiplier = 32767.0 / largest
print "Largest sample value in original sound was", largest print "Multiplier is", multiplier
for s in getSamples(sound): louder = multiplier * getSample(s) setSample(s,louder)
![Page 107: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/107.jpg)
Could also do this with IF
def normalize(sound): largest = 0 for s in getSamples(sound): if getSample(s) > largest: largest = getSample(s) multiplier = 32767.0 / largest print "Largest sample value in original sound was", largest print "Multiplier is", multiplier for s in getSamples(sound): louder = multiplier * getSample(s) setSample(s,louder)
![Page 108: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/108.jpg)
Increasing volume by sample indexdef increaseVolumeByRange(sound): for sampleIndex in range(1,getLength(sound)+1): value = getSampleValueAt(sound,sampleIndex) setSampleValueAt(sound,sampleIndex,value * 2)
This really is the same as:def increaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 2)
![Page 109: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/109.jpg)
Recipe to play a sound backwards
def backwards(filename): source = makeSound(filename) target = makeSound(filename)
sourceIndex = getLength(source) for targetIndex in range(1,getLength(target)+1): sourceValue = getSampleValueAt(source,sourceIndex) setSampleValueAt(target,targetIndex,sourceValue) sourceIndex = sourceIndex - 1
return target Note use of return for returning the processed sound
![Page 110: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/110.jpg)
Recipe for halving the frequency of a sound
def half(filename): source = makeSound(filename) target = makeSound(filename)
sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex,
getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex + 0.5
play(target) return target
This is how a sampling synthesizer works!
Here are the pieces that do it
![Page 111: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/111.jpg)
Compare these twodef half(filename): source = makeSound(filename) target = makeSound(filename)
sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex,
getSampleValueAt( source, int(sourceIndex)))
sourceIndex = sourceIndex + 0.5
play(target) return target
def copyBarbsFaceLarger(): # Set up the source and target pictures barbf=getMediaPath("barbara.jpg") barb = makePicture(barbf) canvasf = getMediaPath("7inX95in.jpg") canvas = makePicture(canvasf) # Now, do the actual copying sourceX = 45 for targetX in range(100,100+((200-45)*2)): sourceY = 25 for targetY in range(100,100+((200-25)*2)): color = getColor(
getPixel(barb,int(sourceX),int(sourceY))) setColor(getPixel(canvas,targetX,targetY), color) sourceY = sourceY + 0.5 sourceX = sourceX + 0.5 show(barb) show(canvas) return canvas
![Page 112: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/112.jpg)
Both of them are sampling
Both of them have three parts: A start where objects are set up A loop where samples or pixels are copied from one
place to another To decrease the frequency or the size, we take each
sample/pixel twice In both cases, we do that by incrementing the index by
0.5 and taking the integer of the index Finishing up and returning the result
![Page 113: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/113.jpg)
Recipe to double the frequency of a sounddef double(filename): source = makeSound(filename) target = makeSound(filename) targetIndex = 1 for sourceIndex in range(1, getLength(source)+1, 2): setSampleValueAt( target, targetIndex,
getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1 #Clear out the rest of the target sound -- it's only half full! for secondHalf in range( getLength( target)/2, getLength( target)): setSampleValueAt(target,targetIndex,0) targetIndex = targetIndex + 1 play(target) return target
Here’s the critical piece: We skip every other sample in the source!
![Page 114: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/114.jpg)
What happens if we don’t “clear out” the end? Try it!
def double(filename): source = makeSound(filename) target = makeSound(filename) targetIndex = 1 for sourceIndex in range(1, getLength(source)+1, 2): setSampleValueAt( target, targetIndex, getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1 play(target) return target
![Page 115: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/115.jpg)
Splicing Sounds (page 81-85)
Splicing gets its name from literally cutting and pasting pieces of magnetic tape together
Doing it digitally is easy, but not short We find where the end points of words are We copy the samples into the right places to make the
words come out as we want them (We can also change the volume of the words as we
move them, to increase or decrease emphasis and make it sound more natural.)
![Page 116: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/116.jpg)
Finding the word end-points
Using MediaTools and play before/after cursor, can figure out the index numbers where each word ends
![Page 117: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/117.jpg)
Now, it’s all about copying
We have to keep track of the source and target indices.
targetIndex = Where-the-incoming-sound-should-startfor sourceIndex in range(startingPoint,endingPoint) setSampleValueAt( target, targetIndex,
getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1
![Page 118: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/118.jpg)
def splicePreamble(): file = "/Users/guzdial/mediasources/preamble10.wav" source = makeSound(file) target = makeSound(file) # This will be the newly spliced sound targetIndex=17408 # targetIndex starts at just after "We the" in the new sound for sourceIndex in range( 33414, 40052): # Where the word "United" is in the sound setSampleValueAt(target, targetIndex, getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1 for sourceIndex in range(17408, 26726): # Where the word "People" is in the sound setSampleValueAt(target, targetIndex, getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1 for index in range(1,1000): #Stick some quiet space after that setSampleValueAt(target, targetIndex,0) targetIndex = targetIndex + 1 play(target) #Let's hear and return the result return target
The Whole Splice
![Page 119: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/119.jpg)
What’s going on here?
First, set up a source and target. Next, we copy “United” (samples 33414 to
40052) after “We the” (sample 17408) That means that we end up at 17408+(40052-
33414) = 17408+6638=24046
Where does “People” start?
Next, we copy “People” (17408 to 26726) immediately afterward. Do we have to copy “of” to? Or is there a pause in there that we can make
use of?
Finally, we insert a little (1/441-th of a second) of space – 0’s
![Page 120: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/120.jpg)
What if we didn’t do that second copy? Or the pause?
def spliceSimpler(): file = "C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\preamble10.wav" source = makeSound(file) target = makeSound(file) # This will be the newly spliced sound targetIndex=17408 # targetIndex starts at just after "We the" in the new sound for sourceIndex in range( 33414, 40052): # Where the word "United" is in the sound setSampleValueAt( target, targetIndex, getSampleValueAt( source, sourceIndex)) targetIndex = targetIndex + 1 play(target) #Let's hear and return the result return target
![Page 121: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/121.jpg)
Can we generalize shifting a sound into other frequencies?This way does NOT work:def shift(filename,factor): source = makeSound(filename) target = makeSound(filename)
sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex, getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex + factor
play(target) return target
![Page 122: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/122.jpg)
Watching it not work
It’ll work for shifting down, but not shifting up. Why?>>> hello=pickAFile()>>> print hello/Users/guzdial/mediasources/hello.wav>>> lowerhello=shift(hello,0.75)>>> higherhello=shift(hello,1.5)I wasn't able to do what you wanted.The error java.lang.ArrayIndexOutOfBoundsException has occuredPlease check line 7 of /Users/guzdial/shift-broken.py
![Page 123: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/123.jpg)
We need to prevent going past the end of the sound
def shift(filename,factor): source = makeSound(filename) target = makeSound(filename)
sourceIndex = 1 for targetIndex in range(1, getLength( target)+1): setSampleValueAt( target, targetIndex,
getSampleValueAt( source, int(sourceIndex))) sourceIndex = sourceIndex + factor if sourceIndex > getLength(source): sourceIndex = 1
play(target) return target
![Page 124: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/124.jpg)
Now we have the basics of a sampling synthesizer
For a desired frequency f we want a sampling interval like this:
Useful exercise: Build a shift function that takes a frequency as input.
![Page 125: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/125.jpg)
Modules: Moving towards movies and animations The OS module offers a number of powerful
capabilities for dealing with files, e.g., renaming files, finding out when a file was last modified, and so on.
We start accessing the OS module by typing: import os
The function that knows about directories is listdir(), used as os.listdir() listdir takes a path to a directory as input.
![Page 126: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/126.jpg)
Using os.listdir
>>> import os>>> print getMediaPath("barbara.jpg")C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\
barbara.jpg>>> print getMediaPath("pics")Note: There is no file at C:\Documents and Settings\Mark Guzdial\My
Documents\mediasources\picsC:\Documents and Settings\Mark Guzdial\My Documents\mediasources\
pics>>> print os.listdir("C:\Documents and Settings\Mark Guzdial\My
Documents\mediasources\pics")['students1.jpg', 'students2.jpg', 'students5.jpg', 'students6.jpg',
'students7.jpg', 'students8.jpg']
![Page 127: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/127.jpg)
Writing a program to title pictures
We’ll input a directory We’ll use os.listdir() to get each filename in the
directory We’ll open the file as a picture. We’ll title it. We’ll save it out as “titled-” and the filename.
![Page 128: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/128.jpg)
A Working Titling Program
import os
def titleDirectory(dir): for file in os.listdir(dir): print "Processing:",dir+"//"+file picture = makePicture(dir+"//"+file) addText(picture,10,10,"This is from CS1315 Spring 2003") writePictureTo(picture,dir+"//"+"titled-"+file)
![Page 129: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/129.jpg)
Showing it work>>> titleDirectory("C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics")Processing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students1.jpgProcessing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students2.jpgProcessing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students5.jpgProcessing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students6.jpgProcessing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students7.jpgProcessing: C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics//students8.jpg>>> print os.listdir("C:\Documents and Settings\Mark Guzdial\My Documents\mediasources\pics")['students1.jpg', 'students2.jpg', 'students5.jpg', 'students6.jpg', 'students7.jpg', 'students8.jpg', 'titled-
students1.jpg', 'titled-students2.jpg', 'titled-students5.jpg', 'titled-students6.jpg', 'titled-students7.jpg', 'titled-students8.jpg']
![Page 130: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/130.jpg)
Comparing before and after
![Page 131: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/131.jpg)
What if you want to make sure you’ve got JPEG files?import os
def titleDirectory(dir): for file in os.listdir(dir): print "Processing:",dir+"//"+file if file.endswith(".jpg"): picture = makePicture(dir+"//"+file) addText(picture,10,10,"This is from CS1315 Spring 2003") writePictureTo(picture,dir+"//"+"titled-"+file)
![Page 132: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/132.jpg)
Video Processing
Burst a movie into a series of JPEG frames MediaTools or QuickTime Pro will do this for you.
Process each frame using os.listdir
![Page 133: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/133.jpg)
Lightening every frame of a movie
def lightenMovie(folder): import os for file in os.listdir(folder): picture=makePicture(folder+file) for px in getPixels(picture): color=getColor(px) makeLighter(color) setColor(px,color) writePictureTo(picture,folder+“L"+file)
![Page 134: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/134.jpg)
Creating animations
Simply draw on JPEG frames And save each out as a consecutively numbered
picture
![Page 135: ACM SIGCSE 2003: Multimedia Construction Projects Mark Guzdial College of Computing Georgia Institute of Technology guzdial@cc.gatech.edu mark.guzdial](https://reader035.vdocuments.site/reader035/viewer/2022062519/5697bfac1a28abf838c9bb84/html5/thumbnails/135.jpg)
Simplest Animationdef AnimationSimple():
frames = 50rx = 0ry = 0rw = 50rh = 50for f in range(1,frames+1):
pic=makePicture(getMediaPath("640x480.jpg")) addRectFilled(pic,rx,ry,rw,rh,red)if(f < 10):
writePictureTo(pic,'test0%d.jpg'%(f))else:
writePictureTo(pic,'test0%d.jpg'%(f))rx = 5 + rxry = 5 + ry