music processing with mpeg-7 low level audio …part 2 music processing with mpeg-7 low level audio...
TRANSCRIPT
![Page 1: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/1.jpg)
Part 2Music Processing with MPEG-7
Low Level Audio Descriptors
Dr. Michael CaseyCentre for Computational Creativity
Department of ComputingCity University, London
![Page 2: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/2.jpg)
MPEG-7 Software Tools• ISO 15938-6 (Reference Software C++)
» http://www.lis.ei.tum.de/research/bv/topics/mmdb/e_mpeg7.html
• Audio Only Reference Software (Matlab)» http://ccc.soi.city.ac.uk/mpeg7 (City University Mirror)
![Page 3: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/3.jpg)
Audio DescriptionsHeader
![Page 4: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/4.jpg)
Audio Descriptions
Segments
![Page 5: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/5.jpg)
Audio Descriptions
Descriptor
![Page 6: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/6.jpg)
Containment Hierarchy for Audio Descriptors
SeriesOfScalarType
AudioSegmentType
AudioDSType
AudioLLDScalarType
AudioDType
SeriesOfVectorType
AudioLLDVectorType
ScalableSeriesType
![Page 7: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/7.jpg)
Audio LLD DataTypes
![Page 8: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/8.jpg)
Some Useful Descriptors for Music Processing
• AudioSpectrumEnvelopeD
• AudioSpectrumBasisD
• AudioSpectrumProjectionD
• SoundModelDS
• SoundModelStatePathD
• SoundModelStateHistogramD
![Page 9: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/9.jpg)
Other Useful Descriptors for Music Processing
• AudioSpectrumFlatnessD
• AudioHarmonicityD
• AudioSpectrumCentroidD
![Page 10: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/10.jpg)
AudioSpectrumEnvelopeD
• Log frequency scale spectral power coefficients• Total power preserved across logarithmic bands
62.5 1K 16000
1 coeff 8 coeffs 1 coeff
total power
within-bandbelow-band above- band
![Page 11: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/11.jpg)
AudioSpectrumEnvelopeD
[AudioSpectrumEnvelope, attributegrp, map, XMLFile] = AudioSpectrumEnvelopeType(audioFile,hopSize,attributegrp,writeXML,XMLFile,map)
This function determines an AudioSpectrumEnvelopeand also returns the map from linear to log bands.
% EXAMPLE 1: AudioSpectrumEnvelopeD extractionag.octaveResolution='1/4';ag.loEdge=62.5;ag.hiEdge=8000;hopSize='PT10N1000F';fname='e:\Beatles\1\000100.wav';
[ASE,ag]=AudioSpectrumEnvelopeD(fname,hopSize,ag,1,'ase.xml');
![Page 12: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/12.jpg)
AudioSpectrumEnvelopeD
. . .
![Page 13: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/13.jpg)
AudioSpectrumEnvelopeD
![Page 14: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/14.jpg)
AudioSpectrumBasisD
SVD / ICABasis Rotation
AudioSpectrumProjectionD
AudioSpectrumBasisD
![Page 15: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/15.jpg)
AudioSpectrumBasisDAudioSpectrumBasisType -independent components of a spectrum matrix
[V,env]=AudioSpectrumBasis(X, k, DDL_FLAG)
Inputs:X - spectrum data matrix ( t x n, t=time points, n=spectral channels)k - number of components to extractDDL_FLAG - 1=write XML output. [0]
OutputsV - n x k matrix of basis functionsenv - L2-norm envelope of log Spectrogram data (required for MPEG7)
% EXAMPLE2: AudioSpectrumBasisD[ASB,env]=AudioSpectrumBasisD(ASE,10,'asb.xml');
![Page 16: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/16.jpg)
AudioSpectrumBasisD
![Page 17: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/17.jpg)
AudioSpectrumBasisD
![Page 18: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/18.jpg)
AudioSpectrumBasisD: Block Form
![Page 19: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/19.jpg)
AudioSpectrumProjectionDAudioSpectrumBasisD
SVD / ICABasis Rotation
AudioSpectrumProjectionD
![Page 20: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/20.jpg)
AudioSpectrumProjectionD
[P,maxenv] = AudioSpectrumProjectionD(X, V, XML)
InputsX = t x n matrix containing AudioSpectrumEnvelopeD values:
t=timepoints,n=frequency binsV = n x k matrix containing AudioSpectrumBasisD values
n=frequency bins, k=basis functions
DDL_FLAG XML file name [optional]
Output
P = t x (1 + k) matrix where each row contains 1 x L2-norm envelopecoefficient and k x spectral projection coefficients.
% EXAMPLE3: AudioSpectrumProjectionD extraction[ASP,maxEnv]=AudioSpectrumProjectionD(ASE,ASB,'asp.xml');
![Page 21: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/21.jpg)
AudioSpectrumProjectionD
![Page 22: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/22.jpg)
IndependentSpectrum Basis Features
Time Function
Reconstruction
1 Component
Spectral Feature
High Channel Spectrogram
Basis Reduction 4 Components
10 Components
![Page 23: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/23.jpg)
Outer Product Spectrum ReconstructionIndividual Basis Component
![Page 24: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/24.jpg)
4 Component Reconstruction
![Page 25: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/25.jpg)
10 Component Reconstruction
![Page 26: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/26.jpg)
Music Unmixing
• Linear basis projection using SVD and ICA• spectrum subspace separation • fast computation of subspace ICA• full-rate filterbank masking
• Blocked ICA functions• subspace reconstruction Y = XVV• cluster subspaces to identify “tracks”• sum masked filterbank output to create audio
+j j j
![Page 27: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/27.jpg)
Music Unmixing Example 1
dB
Drum Mixture
![Page 28: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/28.jpg)
Music Unmixing Example 1
![Page 29: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/29.jpg)
Music Unmixing Example 1
![Page 30: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/30.jpg)
Music Unmixing Example 1
![Page 31: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/31.jpg)
Music Unmixing Example 2(Pink Floyd: stereo -> 9 subspace tracks)
![Page 32: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/32.jpg)
SoundModelDS
![Page 33: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/33.jpg)
Sound Model DSand related descriptors
1 3 3 2 2 3 4 4 4 4 .
1
2 34
ContinuousHiddenMarkovModelDS
AudioSpectrumBasisD
T(i,j)
x
AudioSpectrumEnvelopeD
AudioSpectrumProjectionD
SoundModelStatePathD
![Page 34: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/34.jpg)
SoundModelDS - Bayesean inference of HMM parameters from training data
Y = SoundModelDS(TrainingDataListFile, nS, nB [,OPTIONAL ARGUMENTS...])
INPUTS:TrainingDataList - filename of training data list: WAV file names (one per line).
nS - number of states in hidden Markov model [10]nB - number of basis components to extract [10]
The following variables are optional, and are specified using['parameter', value pairs] on the command line.
'hopSize' 'PT10N1000F' - AudioSpectrumEnvelopeD hopSize'loEdge' 62.5, - AudioSpectrumEnvelopeD low Hz'hiEdge' 16000, - AudioSpectrumEnvelopeD high Hz'octaveResolution' '1/8' - AudioSpectrumEnvelopeD resolution'sequenceHopSize' '', - HMM data window hop [whole file]'sequenceFrameLength' '' - HMM data window length [whole file]'outputFile' '' - Filename for Model output [stem+mp7.xml]'soundName' '' - Model identifier name
OUTPUTS:
outputFile.dat = matlab struct Y.{T,S,M,C,X,maxenv,V,p}
T - state transition matrixS - initial state probability vectorM - stacked means matrix (1 vector per row)C - stacked inverse covariancesV - AudioSpectrumBasis vectors
maxenv- scaling parameter for model decodingp - training cycle likelihoods
outputFile.mp7 = XML file containing MPEG-7 SoundModel description scheme
![Page 35: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/35.jpg)
SoundModelDS - Bayesean inference of HMM parameters from training data
Y = SoundModelDS(TrainingDataListFile, nS, nB [,OPTIONAL ARGUMENTS...])
INPUTS:TrainingDataList - filename of training data list: WAV file names (one per line).
nS - number of states in hidden Markov model [10]nB - number of basis components to extract [10]
The following variables are optional, and are specified using['parameter', value pairs] on the command line.
'hopSize' 'PT10N1000F' - AudioSpectrumEnvelopeD hopSize'loEdge' 62.5, - AudioSpectrumEnvelopeD low Hz'hiEdge' 16000, - AudioSpectrumEnvelopeD high Hz'octaveResolution' '1/8' - AudioSpectrumEnvelopeD resolution'sequenceHopSize' '', - HMM data window hop [whole file]'sequenceFrameLength' '' - HMM data window length [whole file]'outputFile' '' - Filename for Model output [stem+mp7.xml]'soundName' '' - Model identifier name
OUTPUTS:
outputFile.dat = matlab struct Y.{T,S,M,C,X,maxenv,V,p}
T - state transition matrixS - initial state probability vectorM - stacked means matrix (1 vector per row)C - stacked inverse covariancesV - AudioSpectrumBasis vectors
maxenv- scaling parameter for model decodingp - training cycle likelihoods
outputFile.mp7 = XML file containing MPEG-7 SoundModel description scheme
Process Small Chunks= Local Dynamics Model
![Page 36: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/36.jpg)
SoundModelDS
![Page 37: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/37.jpg)
SoundModelDS
![Page 38: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/38.jpg)
SoundModelStatePathDA simplified representation of spectral dynamics
State Path
![Page 39: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/39.jpg)
SoundModelStatePathD[Path,loglike]=SoundModelStatePathD(soundfilename, arg2 [,OPTIONAL ARGS])
Compute HMM State Path and log likelihood of sequence data
Inputs:soundfilename - filename of input sound (.wav or .au)arg2 - SoundModelDS structure or filename of binary SoundModelDS instance
(.mat)
The following variables are optional, and are specified using'parameter' value pairs on the command line.
'hopSize' 'PT10N1000F''loEdge' 62.5, 'hiEdge' 16000, 'octaveResolution' '1/8''sequenceHopSize' '','sequenceFrameLength' ''
% EXAMPLE 5: SoundModelStatePathD extraction[Path,ll]=SoundModelStatePathD(fname,Y,'octaveResolution','1/4','hiEdge',8000);
![Page 40: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/40.jpg)
SoundModelStatePathD
![Page 41: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/41.jpg)
SoundModelStatePathD
seconds
0.01s Frames
state index
state index
BEATLES: A Hard Day’s Night
![Page 42: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/42.jpg)
SoundModelStateHistogramDSoundModelStateHistogramD(Path, Nstates, [segSkip], [segLen])
Extract normalized segmental state-path histograms
Inputs:Path - SondModelStatePathD outputNstates - Number of states in SoundModel[segSkip] - hop size in samples[segLen] - histogram length in samples
Outputs:H - t x n matrix containing segmented state occupancy histograms
t=time points, n=states
% EXAMPLE 6: SoundModelStateHistogramD extractionH=SoundModelStateHistogramD(Path,10,100,1000);
![Page 43: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/43.jpg)
SoundModelStateHistogramD
seconds
state index
state index
0.01s Frames
![Page 44: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/44.jpg)
S-Matrix
• Similarity Function• Segmented Histograms are Unit Norm• Outer Product Computes Similarity Matrix
>>size(H)
ans =
137 10
>>S = H * H’ ; % Similarity Matrix>>imagesc(S);>>D = real(acos(S)); % Dissimilarity Matrix
![Page 45: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/45.jpg)
S-Matrix
![Page 46: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/46.jpg)
Sound Replacement and Audio Mosaics
• Find segments similar to target segment• Similarity Scores Computed between Histograms.• Cluster with k-means or pair-wise clustering.
• Replace with similar (but different) material• Segmentation boundaries (beat alignment)
• EXAMPLES
![Page 47: Music Processing with MPEG-7 Low Level Audio …Part 2 Music Processing with MPEG-7 Low Level Audio Descriptors Dr. Michael Casey Centre for Computational Creativity Department of](https://reader034.vdocuments.site/reader034/viewer/2022042401/5f1049b97e708231d4485db6/html5/thumbnails/47.jpg)
Acknowledgements• International Standards Organisation
• ISO/IEC JTC 1 SC29 WG11 (MPEG)
• Mitsubishi Electric Research Labs• Massachusetts Institute of Technology
• Music Mind Machine Group (formerly Machine Listening Group)
• Paris Smaragdis, Youngmoo Kim, Brian Whitman• Iroro Orife, John Hershey, Alex Westner, Kevin Wilson
• City University • Deparment of Computing• Centre for Computational Creativity