musical instruments identification musical instruments and identifywhich musical instruments are...

Download Musical Instruments Identification musical instruments and identifywhich musical instruments are playing

If you can't read please download the document

Post on 08-Sep-2020

6 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • 1

    Queen Mary University of London

    Department of Electronic Engineering

    Xiang LI

    Musical Instruments Identification System Master of Science Project

    Project supervisor: Dr Mark Plumbley

  • 2

    Disclaimer

    This report is submitted as part of requirement for the degree of MSc in Digital Signal Processing at the University of London. It is the product of my own labour except where indicated in the text. The report may be freely copied and distributed provided the source is acknowledged.”

  • 3

    Acknowledgements

    Foremost I would like to acknowledge Dr. Mark Plumbley for supervising me on this project; you always gave me inspirations and provided guidance, advice and support for all kinds of this work. Special thanks to Andrew Nesbit who provided me very helpful advice to this project and the musical instrument sample resource, help me to keep the project in a correct direction. I would also like to thank all of my MSc classmates for supporting me in the project. Thanks to the Department of Electronic Engineering office staffs for assistance with the project. Finally, I would like to give a big thanks to my parents, though I am far away from home, they always gave me encouragements and confidence.

  • 4

    Abstract

    This project presents the identification of musical instruments, where the idea is to construct a computer system which can listen to the musical piece mixing several musical instruments and identify which musical instruments are playing. In this project, we research an efficient method to separate the musical instrument mixture and implement the different classifiers on the pre-processed signals which are the separated individual sources. Several different feature extractions are implemented and features are selected for a higher recognition rate during design a pattern recognition system. The performance of the system is evaluated in several experiments.

  • 5

    List of Contents:

    Disclaimer ........................................................................................................................ 2 Acknowledgements............................................................................................................ 3 Abstract............................................................................................................................ 4 List of Contents:................................................................................................................ 5 Glossary: .......................................................................................................................... 6 List of Tables and Figures: .................................................................................................. 7 1. Introduction................................................................................................................... 8 2. Background and related work........................................................................................... 9

    2.1 Blind Source Separation......................................................................................... 9 2.2 Musical Instrument Recognition ............................................................................ 16

    2.2.1 Recognition of single tones ......................................................................... 16 2.2.2 Spectral shape ........................................................................................... 19 2.2.3 Onset and offset transients, and the amplitude envelope.................................. 19 2.2.4 Pitch features............................................................................................. 20 2.2.5 Amplitude and loudness features.................................................................. 20

    3. System Design ............................................................................................................. 22 4. System implementation................................................................................................. 23

    4.1 DUET algorithm.................................................................................................. 23 4.2 Feature extraction................................................................................................ 25

    4.2.1 Mel-Frequency Cepstral Coefficient (MFCC)................................................ 25 4.2.2 Root Mean Square (RMS)........................................................................... 28 4.2.3 Spectral Centroid (SC)................................................................................ 28 4.2.4 Zero Crossing Rates (ZCR)......................................................................... 29 4.2.5 Spectral rolloff:.......................................................................................... 29 4.2.6 Bandwidth: ............................................................................................... 29

    4.3 Classification ...................................................................................................... 30 4.3.1 The K Nearest Neighbour classifier .............................................................. 30 4.3.2 The GMM classifier ................................................................................... 32

    4.4 Summary............................................................................................................ 34 5. Results........................................................................................................................ 35

    5.1 Musical Instrument Database ................................................................................ 35 5.2 Experiments........................................................................................................ 37 5.3 Discussion.......................................................................................................... 41

    6. Conclusion .................................................................................................................. 43 6.1 Summary............................................................................................................ 43 6. 2 Future Work....................................................................................................... 43

    Reference ....................................................................................................................... 45

  • 6

    Glossary:

    BASS – Blind Audio Source Separation BSS – Blind Source Separation ICA – Independent Component Analysis SDR – Source-to-Distortion Ratio SAR – Source-to-Artifacts Ratio SIR – Source-to-Interference Ratio STFT – Short-time Fourier Transform DUET – Degenerate Unmixing Estimation Technique IID – Inter-channel Intensity Difference IPD –Inter-channel Phase Difference MDCT – Modified Discrete Cosine Transform MFCC – Mel-Frequency Cepstral Coefficient RMS – Root Mean Square SC – Spectral Centroid ZCR – Zero Crossing Ratio k-NN or KNN– k-nearest neighbour GMM – Gaussian Mixture Model EM – Expectation Maximization

  • 7

    List of Tables and Figures:

    Table 1. Vocabulary used to classify mixture types Table 2. Summary of recognition percentages of isolated note recognition systems using only one example of each instrument Table 3. The training database Table 4. The testing database Table 5. wav files to be tested Table 6. SDR of estimated sources Table 7.Classification Result Table 8 Results of experiments when number of source is three Table 9 Results of experiments when number of source is four Table 10 Results of experiments when number of source is five Figure 1: Some usual ways of obtaining audio mixtures: live recording, studio recording and synthetic mixing Figure 2: Separation of a three-source instantaneous stereo mixture using IID cues and binary masking Figure 3 . Block diagram of implemented music instrument identification system Figure 4. Block diagram of the MFCC feature extractor Figure 5. Mel frequency scale respect with Hertz scale Figure 6 The K nearest neighbourhood rule ( K=5) Figure 7 Representation of an M component mixture model Figure 8. Magnitudes of mixtures in MDCT domain Figure 9 Estimated sources Figure 10. Original sources

  • 8

    1. Introduction

    The purpose of this project is to build a musical instrument identification system based on blind source separation and pattern classification. Musical instrument identification plays an important role in musical signal indexing and database retrieval [27]. When musical content is concerned, complex musical mixtures could be labelled and indexed in terms of musical events which meet the need for multimedia description. That means people can search music by the musical instruments instead of the type or the author. For instance, user is able to query ‘find piano mono parts of a musical database’.

    Musical instrument identification is a crucial task and many problems remain unsolved given the current state of the arts. My approach consists of two stages, where the first stage is blind source separation and the second one is source recognition. Most musical recognition studies mainly focus on the case where isolated notes are played [27]. The task of blind source s