msc geodetic engineering msr-02: advanced techniques … · 2020. 6. 16. · msc geodetic...
TRANSCRIPT
![Page 1: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/1.jpg)
05: FROM IMAGES TO POINT CLOUDS (SFM)
MSC GEODETIC ENGINEERING
MSR-02: ADVANCED TECHNIQUES FOR MOBILE SENSING AND ROBOTICS (GEODESY TRACK)
DR. LASSE KLINGBEIL
INSTITUTE FOR GEODESY AND GEOINFORMATIONUNIVERSITY OF BONN
![Page 2: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/2.jpg)
ADVANCED TECHNIQUES FOR MOBILE SENSINGAND ROBOTICS – LECTURE CONTENT
(1) Mobile Laser Scanning
(2) Trajectory Estimation
(3) System Calibration
(4) Sensor Synchronisation
(5) From Images to Point Clouds (SfM)
(6) Accuracy of Point Clouds I
(7) Accuracy of Point Clouds II
(8) Deformation Analysis with Point Clouds I
(9) Deformation Analysis with Point Clouds II
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 2
![Page 3: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/3.jpg)
SO FAR: MOBILE LASER SCANNING
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 3
![Page 4: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/4.jpg)
NOW: WHAT IF THE THE SENSOR IS A CAMERA?
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 4
We can not calculate a 3D position of an object from a single camera observation. What to do?
2D image coordinate3D world coordinate ?
![Page 5: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/5.jpg)
STRUCTURE FROM MOTION/MULTI-VIEW STEREO
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 5
http://www.theia-sfm.org/sfm.html
![Page 6: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/6.jpg)
LINK TO STACHNISS‘ LECTURE
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 6
Cyrill Stachniss: Lecture ‚Direct Linear Transform: Camera Calibration and Localization’Now: Estimate - Intrinsics- Extrinsics- Object Pointsfrom multiple images
No detailed photogrammetric + mathematical background in this lecture!Focus on UAV mapping
![Page 7: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/7.jpg)
DEFINITIONS FROM FÖRSTNER‘S BOOK
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 7
Set of feature points in object space‚structure‘
Set of orientations (at times t) ‚motion‘
Camera calibration parameters (at times t)
Set of feature points in image space (‚image features‘)
Scene static, camera moves
Wolfgang Förstner, Bernhard P. Wrobel: Photogrammetric Computer Vision, Springer
![Page 8: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/8.jpg)
DEFINITIONS FROM FÖRSTNER‘S BOOK
• Orientation or motion from structure: get {Oi} from correspondences {Fi,fit}
• Calibration: get Ct from {fit} and possibly given {Fi}
• Reconstruction or structure from motion (SfM): get {Fi} from {fit}
• Relative Orientation: get relative pose between multiple images and a local scenedescription {Fi} from correspondingimage points {fi_a, fi_b, fi_c, …} in multiple images a, b, c, …
• Absolute Orientation: get global scene {Fi} from local scene {Fi} and 3D control features
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 8Wolfgang Förstner, Bernhard P. Wrobel: Photogrammetric Computer Vision, Springer
![Page 9: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/9.jpg)
DEFINITIONS FROM FÖRSTNER‘S BOOK
• Bundle Adjustment (BA): get {Fi} and {Oi} and Ct from {fit} global {Fi} also needs
3D control features ordirect measurements of some points/poses (e.g. via GNSS)
combines all points aboveclosely related to SLAM
• What most people actually mean when they say SfM: The above BA including all necessary steps to create the initial solution (relative orientation, sequentially adding images to createan image block, not necessarily scaled and in absolute coordinates) Result: Sparse point cloud {Fi} & image parameters {Oi}, Ct
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 9
Wolfgang Förstner, Bernhard P. Wrobel: Photogrammetric Computer Vision, Springer
![Page 10: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/10.jpg)
SFM/MVS WORKFLOW
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 10
From: Iglhaut, J., Cabo, C., Puliti, S. et al. Structure from Motion Photogrammetry in Forestry: a Review. Curr Forestry Rep 5, 155–168 (2019). https://doi.org/10.1007/s40725-019-00094-3
![Page 11: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/11.jpg)
SIFT OR SIMILAR
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 11
1. Extract featureso SIFT, SURF, BRISC, …
o Corner or Blob Features
o Also multi-scale approaches
2. Feature descriptiono Vector of numbers describing
gradients, intensities, …
3. Feature Matchingo Finding the same features in
multiples images
o Outlier rejectionFrom: Iglhaut, J., Cabo, C., Puliti, S. et al. Structure from Motion Photogrammetry in Forestry: a Review. Curr Forestry Rep 5, 155–168 (2019). https://doi.org/10.1007/s40725-019-00094-3
![Page 12: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/12.jpg)
STRUCTURE FROM MOTION (SFM)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 12
1. Initialize o use two images
o Initialize structure
2. Add new imageso Orient new image using existing
structure
o Run local BA
o Initialize new structure points
3. Optional: Georeferencing
o Include information from GPSor Ground control points
4. Run final BA on completeimage block
From: Iglhaut, J., Cabo, C., Puliti, S. et al. Structure from Motion Photogrammetry in Forestry: a Review. Curr Forestry Rep 5, 155–168 (2019). https://doi.org/10.1007/s40725-019-00094-3
![Page 13: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/13.jpg)
SPARSE POINT CLOUD
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 13
![Page 14: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/14.jpg)
MULTI VIEW STEREO (MVS)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 14
• Try to find much morecorrespondences betweenimages than using features
• Ideal case: match every pixel
• Match areas instead of features
• Use known orientations and epipolar constraints
• Use multi-resolution
• Use multi-image matching
• Use assumptions on surfacesmoothness(semi-global matching)
Result: dense cloudFrom: Iglhaut, J., Cabo, C., Puliti, S. et al. Structure from Motion Photogrammetry in Forestry: a Review. Curr Forestry Rep 5, 155–168 (2019). https://doi.org/10.1007/s40725-019-00094-3
![Page 15: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/15.jpg)
DENSE POINT CLOUD
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 15
![Page 16: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/16.jpg)
IMAGE BASED (UAV) VS LASER BASED (TLS)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 16
UAV TLS
UAV+TLS
![Page 17: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/17.jpg)
GEOREFERENCING
• Georeferencing is needed
o to get a scale OR
o to get the scene in a globalcoordinate system
• Option 1: Use known scene points (ground control points or 3D control features) to absolutely orient all images and the scene(indirect georeferencing, ‚classical‘ case)
• Option 2: Use onboard sensors to absolutely orient images and skip SfM step (direct georeferencing, rarely used)
• Option 3: Combine 1+2 (integrated sensor orientation, often used )
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 17
![Page 18: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/18.jpg)
INDIRECT GEOREFERENCING
• Deploy a set of Ground Control points and measure theircoordinates
• Use total stations: ~mm accuracy
• Use RTK GNSS: ~1-3cm accuracy
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 18
70m
![Page 19: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/19.jpg)
DIRECT GEOREFERENCING & INTEGRATED SENSOR ORIENTATION
• Determine the cameraorientation (position and rotation parameters) at thetime when the imagewas taken
• We need:
oA system/algorithm to determine the position and orientationof the UAV lecture Trajectory estimation
oThe relative orientation (rotation and translation) between theUAV and the camera lecture System calibration
oThe exact time (syncronized with trajectory estimation), when theimage was takenlecture Sensor Synchronization
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 19
![Page 20: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/20.jpg)
UAV NAVIGATION SENSORS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds
STANDARD UAV SENSORS
• Low-Cost Inertial sensors for in-flightstability control
• Low-cost code based GNSS (~severalmeters accuracy)
• Sensors used for navigation and mission control
• Sensors can also be used for imagetagging (time-sync and calibration isnot critical as accuracy is anyway ~m)
• Needs ground control points forprecise indirect georeferencing
20
![Page 21: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/21.jpg)
UAV NAVIGATION SENSORS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds
ON-BOARD RTK GNSS
• More and more commercial solution areavailable
• Real-time solution needs connection forcorrection data, otherwise offline processing
• Time-sync and calibration becomes critical,most solution work only with certain cameras
• RTK GNSS mostly used only for image taggingnot for navigation
• High precision Image coordinates can then beintegrated in the Bundle Adjustment (integratedsensor-orientation)
21
Mavinci Sirius Pro
SenseFly eBee RTK
DJI Phantom 4 RTK
![Page 22: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/22.jpg)
UAV NAVIGATION SENSORS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds
HIGH-END IMU/GNSS RECEIVER
• High-end IMU/GNSS usually used fordirect georeferencing of laser scans
• Same principle as in mobile laser scanning
22
![Page 23: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/23.jpg)
SUMMARY SFM PIPELINE
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 23
From: Farid Javadnejad. Small Unmanned Aircraft Systems (UAS) for Engineering Inspections and Geospatial Mapping, PhD Thesis, Oregon State University, 2018
![Page 24: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/24.jpg)
SFM SOFTWARE (NO RANKING, NOT TESTED, NO GUARANTEE)
• FreeoVisual SFM
oBundler + PMVS2
oOSM-Bundler
oMicMac
oMeshroom
oSF3M
o…
• Google and find more
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 24
• Commercial solutionsoAgisoft Metashape
oPix4D
oReality Capture
oDroneDeploy
oPhotomodeler
o…
![Page 25: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/25.jpg)
PROJECT EXAMPLE:DFG RESEARCH GROUP: MAPPING ON DEMAND
• Goal: Lightweight Unmanned Aerial Vehicle based 3D reconstruction of objects and environments
– Based on a semantic user request(e.g. ‚find all open windows!‘)
– On demand (no preparation of theenvironment)
– In Realtime (ideally no offline processing)
– Fully autonomous flight(trajectory planning, collisionavoidance)
– Interpretation of facade elements
Project started in 2012 and finished in 2019
modified, www.wikipedia.org
Example scenario:
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 25
![Page 26: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/26.jpg)
MAPPING ON DEMAND: SUB-PROJECTS
• Real-time Pose Determination(Uni Bonn, Inst. of Geodesy and Geoinformation, Geodesy)
• Incremental Mapping and Exploration(Uni Bonn, Inst. of Geodesy and Geoinformation, Photogrammetry)
• Local Perception and Obstacle Detection(Uni Bonn, Inst. of Computer Science, Intelligent Autonomous Systems)
• 3D Surface Reconstruction(TU München, Inst. of Computer Science, Computer Vision)
• Representation and Generation of 3D and 4D Maps(Uni Bonn, Inst. of Computer Science, Computer Graphics)
• Semantic Building Models(Uni Bonn, Inst. of Geodesy and Geoinformation, Geoinformation)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 26
![Page 27: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/27.jpg)
CUSTOM UAV
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 27
![Page 28: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/28.jpg)
CUSTOM GEOREFERENCING SYSTEMSensors• 3 axis accelerometer• 3 axis angular rate sensor• 3 axis magnetometer• barometer• dual frequency GPS receiver• single frequency GPS receiver• radio link for GPS correction data• input for external sensors
(e.g. stereo cameras)
• small size, small weight (~400g including antennas)• real-time (!) processing of position and orientation• high accuracy (~1cm, <1°)• custom multi sensor fusion algorithms for high robustness• high flexibility (use with any other mobile multi-sensor system)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 28
![Page 29: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/29.jpg)
TRAJECTORY ESTIMATIOPN /KALMAN FILTER
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds
SIMPLIFIED INERTIAL NAVIGATION
29
Angular rate
acceleration
Earth rateTransport rate
Gravity
Coriolis correction
IMUposition
velocity
attitude
![Page 30: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/30.jpg)
KALMAN FILTERMEASUREMENT MODELS
• GNSS
lever arm
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 30
![Page 31: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/31.jpg)
KALMAN FILTER
MEASUREMENT MODELS
• Earth magnetic field
Wikipedia
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 31
![Page 32: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/32.jpg)
KALMAN FILTERMEASUREMENT MODELS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point CloudsFolie
32
• Local GNSS Baseline
GNSS baseline
![Page 33: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/33.jpg)
ACCURACY ANALYSIS I• Acquisition of images @1Hz
• Onboard camera pose estimation using directgeoreferencing system
• offline bundle adjustment to estimatecamera poses based on images
• Comparision of both trajectories(up to a 7 parameter transformation)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 33
![Page 34: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/34.jpg)
ACCURACY ANALYSIS II
Difference between the camera positions as measured by onboard RTK and asdetermined using GCPs and photogrammetry ~cm
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 34
![Page 35: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/35.jpg)
INCREMENTAL BUNDLE ADJUSTMENT (VISUAL ODOMETRY)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 35
![Page 36: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/36.jpg)
INCREMENTAL BUNDLE ADJUSTMENT (VISUAL ODOMETRY)
185°
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 36
![Page 37: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/37.jpg)
GNSS/ STEREO CAMERA INTEGRATION
• Real-time Visual Odometry based on 4 fisheye cameras (incremental bundle adjustment)
• Tightly-coupled integration of GPS raw observations (double differences) into the adjustment
Red: full GPS, Blue: two Satellites, Black: no GPS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 37
![Page 38: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/38.jpg)
TESTFLIGHTS• 4 flights around „Versuchsgut
Frankenforst“ near Bonn
• ~5min per flight
• Close to buildings and trees
• Manual control
• On board real time georeferencing
• No control points
• High resolution images @1HzPostprocessing:
• Bundle adjustment on high resolution images• Transformation from photogrammetric trajectory to GPS trajectory• Dense image matching (PMVS: free software) with oriented images Georeferenced Point cloud without usage of control points
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 38
![Page 39: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/39.jpg)
GEOREFERENCED POINT CLOUDS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 39
![Page 40: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/40.jpg)
GEOREFERENCED POINT CLOUDS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 40
![Page 41: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/41.jpg)
GEOREFERENCED POINT CLOUDS
closer view (the window is open)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 41
![Page 42: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/42.jpg)
GEOREFERENCED POINT CLOUDS
merged point clouds from 4 flights (no scan registration!)
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 42
![Page 43: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/43.jpg)
UAV BASED MOBILE MAPPING
ACCURACY ANALYSIS
• Com
Point to Point distance
(neares neighbour)
Comparison withTerrestrial Laser Scan
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 43
![Page 44: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/44.jpg)
PROJECT EXAMPLECROPWATCH
• Goal: Observe plant parameters (e.g. canopy height) fromUAV flights during growing season
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 44
https://ars.els-cdn.com
![Page 45: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/45.jpg)
CREATE GEOREFERENCED POINT CLOUDS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 45
![Page 46: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/46.jpg)
UAV BASED HEIGHT MEASUREMENTS
• Measure before emergenceand at different dates
• Calculate difference
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 46
ground
![Page 47: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/47.jpg)
UAV BASED HEIGHT MEASUREMENTS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 47
Point cloud data
Week 1 Week 3
Week 5 Week 9
Height raster
Height (m)
![Page 48: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/48.jpg)
UAV BASED HEIGHT MEASUREMENTS
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 48
Intensive vs. extensivemanagement
From: Becirevic, D., Klingbeil, L., Honecker, A., Schumann, H., Rascher, U., Léon, J., and Kuhlmann, H.: On the derivation of crop heights from multitemporal UAV based imagery, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., IV-2/W5, 95-102, https://doi.org/10.5194/isprs-annals-IV-2-W5-95-2019, 2019.
![Page 49: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/49.jpg)
IMAGE BASED POINT CLOUDS EXAMPLE
• Building industry, progress documentation
• Surveying and Mapping
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 49
Leica
Leica
![Page 50: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/50.jpg)
IMAGE BASED POINT CLOUDS EXAMPLE
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 50
![Page 51: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/51.jpg)
IMAGE BASED POINT CLOUDS EXAMPLE
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 51
![Page 52: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/52.jpg)
WHAT YOU HAVE LEARNED TODAY
• How can point clouds be created from images?
• What are the processing steps in a SfM processingpipeline?
• What georeferencing options are there for UAV based mobile mapping?
• How can you calculate the pose of a UAV?
• What are applications for UAV based mapping?
Klingbeil: Advanced Techniques for Mobile Sensing and Robotics - Geodesy - 05 - From Images to Point Clouds 52
![Page 53: MSC GEODETIC ENGINEERING MSR-02: ADVANCED TECHNIQUES … · 2020. 6. 16. · msc geodetic engineering msr-02: advanced techniques for mobile sensing and robotics (geodesytrack) dr](https://reader036.vdocuments.site/reader036/viewer/2022071401/60ec3b5946906a3f880390f9/html5/thumbnails/53.jpg)
THANKS