1rémi devinant dii5 / devices synchronization for modeling 3d plane
Post on 19-Dec-2015
214 views
TRANSCRIPT
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 1
Devices Synchronization for modeling « 3D plane »
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 2
Summary
Introduction Context Problem Modeling Devices Synchronization Computer vision Conclusion
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 3
Introduction
More and more augmented reality applications Development of more performant technologies
Best performances Reliability tools for users
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 4
Context
Smartphones More than 100 applications 2 known techniques▪ Geolocalization and camera▪ Camera, gyro, accelerometer▪ Embedded systems constraints
« EyeTape » prototype Head up display Display information superimposed on vision
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 5
Context
The project Based on a CCD
camera▪ Taking frames in
video▪ 2 kinds of
information to▪ Static (buildings, urban
environment…)▪ Dynamic (people,
cars…)
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 6
Context
Ubiquitous Computing Mobile systems around users Interacts with the environment Devices used in parallel Mobile system▪ Video treatment▪ 2 informations flow
Embedded systems problematic Light mobile system Light CPU power Memory constraints
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 7
Context
Smartphone (iPhone 4) composed of CCD 5mPixels (video 720p à 30fps) Gyro L3G4200D Digital 3-axis Accelerometer ST Micro LIS331DLH 3-axis CPUApple A4 APL0398 (ARM Cortex A8) DMA Memory RAM 512Mo DRAM Samsung
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 8
Context
Devices synchronisation Apple iOS Apple iPhone OS Based on a BSD Kernel Mac OS X with cellphone services
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 9
Context
Video frames overlap Video’s framerate preset Purpose Reduce or increase framerate Original 30fps Smartphone motion in space
Slow motion : video frames can overlap significantly need for suppressing useless ones
Need for fusioning devices datas Allow selection in video frames
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 10
Context
2 known methods Hardware▪ Can exist if it’s built-in▪ Smartphones dont' have this characteristic
Software▪ Can be implemented in the OS▪ Require an personalized algorithm
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 11
Summary
Introduction Context Problem Modeling Devices Synchronization Computer vision Conclusion
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 12
Problem modeling
Each devices have different time delays In retrieving datas In transmitting datas
Require modeling each delays representing the time between the datas are required and the time where they are in memory
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 13
Problem modeling
Basic system architecture
Gyroscope
Accéléromètre
Capteur CCD
Device Controller gyro
Device Controller accéléromètre
Device Controller CCD
Buffer mémoire DMA
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 14
Problem modeling
Accelerometer 3 datas send into 1 block 3 axis acceleration sensor
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 15
Problem modeling
Accéléromètre
DeviceController
Mémoire
CPU
Data BusAdresses BusControl Bus
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 16
Problem modeling
Accelerometer Retrieving datas delay
Devices bus output: fbus Datas volume: Vdata Datas acquisition delay: ∆tacqacc Bus acquisition delay (if busy): delay Writing in memory delay depending on datas
volume : ∆twrite
∆tacc = ∆tacq + (fbus / Vdata) + ∆twrite + delay
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 17
Problem modeling
Gyro 3 datas send into 1 block 3 axis rotations
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 18
Problem modeling
Gyroscope
DeviceController
Mémoire
CPU
Data BusAdresses BusControl Bus
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 19
Problem modeling
Gyro Retrieving datas delay
Devices bus output: fbus Datas volume: Vdata Datas acquisition delay: ∆tacqacc Bus acquisition delay (if busy): delay Writing in memory delay depending on datas
volume : ∆twrite
∆tacc = ∆tacq + (fbus / Vdata) + ∆twrite + delay
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 20
Problem modeling
CCD Camera Treatment of frames
before they are placed in memory
sRGB colors 720p Red/ Green/ Blue Heavy data volume DMA
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 21
Problem modeling
CCD
DeviceController
Mémoire
CPU DMA
Data BusAdresses BusControl Bus
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 22
Problem modeling
CCD Camera Retrieving datas delay
Devices bus output: fbus Datas volume: Vdata Datas acquisition delay: ∆tacqacc Bus acquisition delay (if busy): delay Writing in memory delay depending on datas
volume : ∆twrite
∆tacc = ∆tacq + (fbus / Vdata) + ∆twrite + delay
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 23
Summary
Introduction Context Problem Modeling Devices Synchronization Computer vision Conclusion
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 24
Devices synchronization
T0
T1 ∆t
Recording at 30fps
Windows of N scan in memory for finding frame
Start memory scan window
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 25
Devices synchronization
Frame Spotted at ti
- ti : date of scan
- ∆t : retrieveing image delay
- ti - ∆t = real date of the frame capture
∆t
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 26
Devices synchronization
In this window If a frame is spotted at pi S∆ = pi – (N - 1) / 2
Next window start at T1 + S∆
Window placed every time near the last image was spotted Real time and embedded constraints
context
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 27
Devices synchronization
When the window is started, creation of coordinate table Accelerometer▪ ta[] = tspotted - ∆tacc
Gyro▪ tg[] = tspotted - ∆tgyr
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 28
Devices synchronization
Coordinate are chosed tCoordonnees = tImgSpotted - ∆tImage We try to keep the nearest coordinate
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 29
Summary
Introduction Context Problem Modeling Devices Synchronization Computer vision Conclusion
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 30
Computer Vision
A frame = spatial coordinates Smartphone’s spatial localization
Spatial localization= frame plane Creates frame 3D plane
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 31
Computer Vision
Rémi Devinant DII5 / Devices synchronization for modeling 3D plane 32
Computer Vision
Modeling 3D plane