android multimedia framework
DESCRIPTION
This slide shows how the StageFright framework works on the Android.TRANSCRIPT
![Page 1: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/1.jpg)
Android Multimedia Framework
on Jelly Bean
Author: Picker
![Page 2: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/2.jpg)
Introduction to OpenMAX
![Page 5: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/5.jpg)
fromMultimedia Framework
toOpenMAX
![Page 8: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/8.jpg)
Operating SystemOperating SystemOperating SystemOperating System
Multimedia FrameworkMultimedia Framework
OpenMAX ILOpenMAX IL
Software & Hardware CodecSoftware & Hardware Codec
Go through the OpenMAX
![Page 9: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/9.jpg)
![Page 10: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/10.jpg)
Developer API (MediaPlayer)Developer API (MediaPlayer)Developer API (MediaPlayer)Developer API (MediaPlayer)
EventHandlerEventHandler SurfaceHolderSurfaceHolder
StageFrightStageFright
OpenMAX InterfaceOpenMAX Interface
OpenMAX ILOpenMAX IL
Software & Hardware CodecSoftware & Hardware Codec
The simple stack architecture
![Page 11: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/11.jpg)
Big picture of the StageFright
![Page 12: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/12.jpg)
The Simple Workflow
![Page 13: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/13.jpg)
The StageFright Workflow
![Page 14: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/14.jpg)
Path of All the Components
![Page 15: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/15.jpg)
frameworks/av/media/libmedia
Path of the Source Files
frameworks/base/media/java/android/media
frameworks/av/media/libmediaplayerservice
frameworks/base/media/jni
frameworks/av/media/libstagefright
![Page 16: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/16.jpg)
Big PictureClass Diagram
![Page 17: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/17.jpg)
![Page 18: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/18.jpg)
HighLevel Applications Framework
![Page 19: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/19.jpg)
![Page 20: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/20.jpg)
StageFright Framework
![Page 21: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/21.jpg)
![Page 22: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/22.jpg)
OpenMAX Interface
![Page 23: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/23.jpg)
Four Steps for Playing the Media File
Step 1: MediaPlayer mp = new MediaPlayer();Step 2: mp.setDataSourcesetDataSource(mediaPath);Step 3: mp.prepareprepare();Step 4: mp.startstart();
![Page 24: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/24.jpg)
![Page 25: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/25.jpg)
How to set the data source?
![Page 26: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/26.jpg)
The workflow of the setDataSource
![Page 27: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/27.jpg)
The sequence flow of the setDataSource
![Page 28: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/28.jpg)
![Page 29: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/29.jpg)
How to prepare to play?
![Page 30: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/30.jpg)
The workflow of the prepare
![Page 31: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/31.jpg)
The sequence flow of the prepare
![Page 32: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/32.jpg)
![Page 33: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/33.jpg)
How to start the playing?
![Page 34: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/34.jpg)
The workflow of the start
![Page 35: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/35.jpg)
The sequence flow of the start
![Page 36: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/36.jpg)
Synchronization Architecture of StageFright
![Page 37: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/37.jpg)
Synchronization Flow of StageFright
![Page 38: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/38.jpg)
The Synchronization Formula of StageFright
mPositionTimeRealsUs=(
mNumFramesPlayed+sizedone
mFrameSizemSampleRate
)×1000000
mTimeSourceDeltaUs=mPositionTimeRealUs−mPositionTimeMediaUs
nowUs=RealTimeUs−mTimeSourceDeltaUs
latenessUs=nowUs−timeUs
mPositionTimeRealsUs: the time in real playingmPositionTimeMediaUs: the time which is defined in the media source
![Page 39: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/39.jpg)
The Synchronization ConditionsConditions of latenessUs1. > 500000ll2. > 400003. < -10000 1750 if (latenessUs > 500000ll
1751 && mAudioPlayer != NULL 1752 && mAudioPlayer->getMediaTimeMapping(1753 &realTimeUs, &mediaTimeUs)) {
1757 mVideoBuffer->release(); 1758 mVideoBuffer = NULL; 1763 postVideoEvent_l(); 1764 return;1765 }
1767 if (latenessUs > 40000) { 1768 // We're more than 40ms late.
1780 mVideoBuffer->release(); 1781 mVideoBuffer = NULL; 1787 1788 postVideoEvent_l();1789 return;1791 }
1793 if (latenessUs < -10000) {1794 // We're more than 10ms early.1795 postVideoEvent_l(10000);1796 return;1797 }
![Page 40: Android Multimedia Framework](https://reader037.vdocuments.site/reader037/viewer/2022102605/5557a62bd8b42a696c8b4785/html5/thumbnails/40.jpg)
Conclusion
●Easy for maintaining rather than OpenCore●Support Software/Hardware Codecs●Unstable Product
● Rough Mechanisms● Create the NuPlayer for playing the multimedia streaming