[hci2017] 3d 증강현실 - 알고리즘과 사용방법
TRANSCRIPT
PowerPoint
3D SDK -
1
SDK
App
Tutorial
HCI 2017#
56789101112141516
P4P5P5
23432
17
13
18XXX
19
20
12221
12 : MAXST-
SDK Home > Sub PageMAXST AR SDK 3DSLAM(Simultaneous Localization And Mapping) 3D SDK
Monocular(RGB) 3 / Unity3d AndroidiOSWindowsMac Trainer App (Unity3d) App
HCI 2017#SDK 4
SDK Home > Sub Page (Virtual Guide) , /3D (VIVAR) + MAXST HMD(Revelio) 3D ,
HCI 2017#
5
SDK Home > Sub PageSLAM(Simultaneous Localization And Mapping) (Map) ,
RobotDroneAugmented RealityEtc.https://www.youtube.com/watch?v=5I5pbSs-yrUhttps://www.youtube.com/watch?v=RQhps_Sqa54
HCI 2017#
6
Home > Sub Page
ThreadRuntime Process
Windows, Mac, Android, iOS PlatformCamera Preview ThreadRendering ThreadReservedObject
LocalTrackedObject
Logic Thread(Mapping or Tracking)
Camera Pose Matrix, Track ID
Core LibraryShared MemoryRecognition
HCI 2017#
7
SDK
App
Tutorial
HCI 2017# Home > Sub Page
Motion PredictionInput ImageTrainingSuccess?Success?Add Key-frameStereo Initial MappingSuccess?Localization3D MapyesyesyesnononoTracking
HCI 2017# 9
Motion PredictionInput ImageTrainingSuccess?Success?Add Key-frameStereo Initial MappingSuccess?Localization3D MapyesyesyesnononoTracking Home > Sub Page
HCI 2017#Stereo mapping 10
Home > Sub PageStereo Initial MappingESM Tracker Motion Prediction
Key-frame ,Key-point Homography Homography Map-point Homography (SE3) SE3 Map-point Key-frame Rotation Threshold Key-frame Bundle AdjustmentTraining
HCI 2017#ESM Tracker 11
Home > Sub PageStereo Initial Mapping
ESM Tracker Motion Prediction
Key-frame ,Key-point Homography Homography Map-point Homography (SE3) SE3 Map-point Key-frame Rotation Threshold Key-frame Bundle AdjustmentTraining
HCI 2017#ESM Tracker 12
Home > Sub PageESM(Efficient Second-order Minimization) Tracker SSD(Sum of Squared Difference) Gradient Homography Hessian(2 ) Jacobian(1 ) 2
Global local minima Jacobian computing overhead
x, y Gradient Homography x, y Gradient x, y Gradient (delta) Homography Jacobian Homography Re-Projection error Re-Projection error Homography
HCI 2017#
13
Home > Sub PageStereo Initial MappingESM Tracker Motion Prediction
Key-frame ,Key-point Homography Homography Map-point Homography (SE3) SE3 Map-point Key-frame Rotation Threshold Key-frame Bundle AdjustmentTraining
HCI 2017#Bundle Adjustment 14
Home > Sub PageStereo Initial MappingESM Tracker Motion Prediction
Key-frame ,Key-point Homography Homography Map-point Homography (SE3) SE3 Map-point Key-frame Rotation Threshold Key-frame Bundle AdjustmentTraining
HCI 2017#Bundle Adjustment 15
Home > Sub PageBundle AdjustmentKey-frame Map-point , Projection Map-point Map-point Key-frame Map-point
Projection Map-point Map-point Re-Projection error
HCI 2017#
16
Home > Sub Page
HCI 2017#
17
Home > Sub PageStereo Initial MappingESM Tracker Motion Prediction
Key-frame ,Key-point Homography Homography Map-point Homography (SE3) SE3 Map-point Key-frame Rotation Threshold Key-frame Bundle AdjustmentTraining
HCI 2017#Stereo Mapping 18
Motion PredictionInput ImageTrainingSuccess?Success?Add Key-frameStereo Initial MappingSuccess?Localization3D MapyesyesyesnononoTracking Home > Sub Page
HCI 2017# Training Add Key-frame 19
Home > Sub PageTraining
Key-frame Map-point Key-frame Bundle Adjustment Map-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureLearning Key-frame ,Key-frame Map-point Key-frame Map-point MappingBundle AdjustmentTracking
HCI 2017#Add Key-frame Descriptor 20
Home > Sub PageTraining
Key-frame Map-point Key-frame Bundle Adjustment Map-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureLearning Key-frame ,Key-frame Map-point Key-frame Map-point MappingBundle AdjustmentTracking
HCI 2017#Descriptor 21
Home > Sub Page
HCI 2017#Home > Sub PageTraining
Key-frame Map-point Key-frame Bundle Adjustment Map-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureLearning Key-frame ,Key-frame Map-point Key-frame Map-point MappingBundle AdjustmentTracking
HCI 2017#Random forest decision tree 23
Home > Sub Page
HCI 2017#Home > Sub PageTraining
Key-frame Map-point Key-frame Bundle Adjustment Map-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureLearning Key-frame ,Key-frame Map-point Key-frame Map-point MappingBundle AdjustmentTracking
HCI 2017#Training 25
Home > Sub PageError Reduction
Key-frameKey-frameKey-frameKey-frameKey-frameKey-frameKey-frameKey-frameKey-frameKey-frameKey-frame
HCI 2017#
26
SDK
App
Tutorial
HCI 2017# Home > Sub Page App
TrackingMotion PredictionInput ImageLocalizationSuccess?3D Mapyesno
HCI 2017# 28
Home > Sub Page App
TrackingMotion PredictionInput ImageLocalizationSuccess?3D Mapyesno
HCI 2017#Localization 29
Home > Sub PageLocalization
Fast-12 Key-point Key-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureMatching P3P(Perspective 3 point) Pose Estimation SE3 TrackingESM Tracker Motion Prediction
HCI 2017#Fast 12 30
Home > Sub PageLocalization
Fast-12 Key-point Key-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureMatching P3P(Perspective 3 point) Pose Estimation SE3 TrackingESM Tracker Motion Prediction
HCI 2017#Fast 12 31
Home > Sub PageFast-12 Feature DetectionFeature Feature
12 Decision Tree Non-Maximal Suppression Repeatability Noise
p 3 16 p Fast-12 12
[] Rosten, Edward, and Tom Drummond. "Machine learning for high-speed corner detection."European conference on computer vision. Springer Berlin Heidelberg, 2006.
HCI 2017#Home > Sub PageLocalization
Fast-12 Key-point Key-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureMatching P3P(Perspective 3 point) Pose Estimation SE3 TrackingESM Tracker Motion Prediction
HCI 2017#P3p 33
Home > Sub PageLocalization
Fast-12 Key-point Key-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureMatching P3P(Perspective 3 point) Pose Estimation SE3 TrackingESM Tracker Motion Prediction
HCI 2017#P3p 34
Home > Sub Page
HCI 2017#
35
Home > Sub PageLocalization
Fast-12 Key-point Key-point Image Gradient Feature Patch Mobile SIFT Descriptor Random Forest FeatureMatching P3P(Perspective 3 point) Pose Estimation SE3 TrackingESM Tracker Motion Prediction
HCI 2017#Localization 36
Home > Sub Page App
TrackingMotion PredictionInput ImageLocalizationSuccess?3D Mapyesno
HCI 2017#Tracking 37
Home > Sub PageTracking
Map-point Homography frame 4 Warping Point Warping Point Perspective TransformTransformed Map-point frame Map-point Template Matching(NCC)Matching SE3 Localization
HCI 2017#Patch Warping Template Matching 38
Home > Sub PageTracking
Map-point Homography frame 4 Warping Point Warping Point Perspective TransformTransformed Map-point frame Map-point Template Matching(NCC)Matching SE3 Localization
HCI 2017#Patch Warping Template Matching 39
Home > Sub Page
HCI 2017#
40
Home > Sub PageTracking
Map-point Homography frame 4 Warping Point Warping Point Perspective TransformTransformed Map-point frame Map-point Template Matching(NCC)Matching SE3 Localization
HCI 2017#Tracking 41
SDK
App
Tutorial
HCI 2017#TutorialHome > Sub Page Webcam, Android or iPhone
Unity3d 5.4 64-bit
http://maxst.com/
MAXST AR SDK 3D
MAXST AR SDK 3D Developer Page
HCI 2017#
43
Trainer App DownloadSDK Download
TutorialHome > Sub Page https://arsdk3d.maxst.com
Trainer App() Android TraineriOS Trainer
MaxstAR3D.unitypackage
HCI 2017#
44
TutorialHome > Sub Page https://arsdk3d.maxst.com/Tutorial3D
Sample Tutorial Maxst Tutorial Box
Tutorial Box Download
HCI 2017#
45
TutorialHome > Sub Page
HCI 2017# 46
TutorialHome > Sub Page
HCI 2017# 47
TutorialHome > Sub Page
HCI 2017# 48
HCI 2017#
RecruitHome > Sub PageMAXST..2 3 6
.. 3
2 3
6MAXST
HCI 2017#
50
RecruitHome > Sub Page / Android/iOS SDK SDK Multi OS (C++)Android, iOS, Windows, Mac Handling OpenGL ES 3D App Unity3d Plugin SDK App WebRTC API
////?
! ( !)Android iOS OpenGL ( )Unity3d Asset bundle system ( )FBO, VBO ( ) ~ Only Java ~ , , , ~ .NET, Nodejs, Phython~ ~ ORM vs SP
HCI 2017#
51
RecruitHome > Sub Page - (Computer Vision Lab) MAXST /SDK /2D SDK: / /3D SDK : Visual SLAM /
/RGB : / , (VIVAR) RGB-D : Tango, Kinect Depth-Sensor / (Windows, Mac, Android, iOS), HMD(Head Mount Display) () MAXST HMD(Revelio) AR SDK /
/: / !!!/ : /
! - - - [email protected] - HCI - : 02-585-9566, 010-3043-2266- E-mail: [email protected]
HCI 2017#
52
56789101112141516
P4P5P5
23432
17
13
18XXX
19
20
12221
12 : MAXSTQ & A
HCI 2017#