lab 2 - forsiden - universitetet i · pdf fileπ= 27.3,−20.9, ... • samregistrer...
Post on 06-Feb-2018
227 Views
Preview:
TRANSCRIPT
Lab 2
04.02.2016
Ressurser
β’ OpenCV documentation: β http://opencv.org/documentation.html
β’ Eigen documentation : β http://eigen.tuxfamily.org/dox/ β Quick reference quide: http://eigen.tuxfamily.org/dox/group__QuickRefPage.html
β’ C++:
β http://en.cppreference.com/w/
β’ Image Watch: An image debugger plug-in for Visual Studio β Download directly from Visual Studio:
Tools Extensions and Updatesβ¦ Online Search for βImage Watchβ
2
Litt om cv::Mat
3
Huskeliste
β’ Lage prosjekt
β’ Konstruere en cv::Mat β Datatyper, lage Β«vinduerΒ» β MatCommaInitializer
β’ Hente og endre piksler
β at<β¦>() β forEach
β’ Regne pΓ₯ matriser
β Operasjoner, MatExpr, β¦
4
Perspektivkameramodellen
5
The perspective camera model
6
πΆ
Vehicle
π
π
π
Perspective Camera
World
πππ
ππΆ
ππΆπ
Point observed by the camera
π§π π¦π
π₯π
π₯π
π¦π π§π
π§πΆ
π₯πΆ π¦πΆ
π is a local NED coordinate system (North East Down)
πΆ is a standard coordinate system for a camera π₯πΆ - Right π¦πΆ - Down π§πΆ - Forward
π is a standard coordinate system for a vehicle π₯π - Forward π¦π - Right π§π - Down
The perspective camera model
β’ The point π β Position ππ = 27.3,β20.9,β11.5 π
β’ The vehicle
β 3m wide, 6m long β The origin of π is chosen to be at
the center of the vehicle, 1m above the ground
β Pose relative to the world: π₯ = 6.7π π¦ = β2.4π π§ = β14.0π ππππ = 3.7Β° πππππ = β9.3Β° πππππππ = 307.6Β°
β’ The camera β The optical center is 2m in front of
and 1m to the left of the vehicles center
β The optical center is 4m above ground
β The y-axis of πΆ is perpendicular to the xy-plane of π
β The optical axis, i.e. the z-axis of πΆ , is rotated 4.7Β° to the right of the x-axis of π
β The camera calibration matrix is
πΎ =1028 0 400
0 1028 3000 0 1
7
The perspective camera model
Problem In which pixel of the image will we observe the point π? Sub-problems 1. Represent πππ as a SE(3) object πππ 2. Represent ππΆπ as a SE(3) object ππΆπ 3. Represent ππΆπ as a SE(3) object ππΆπ 4. Determine the camera matrix π = πΎ π π 5. Determine the pixel π = π’,π£ that the
point π projects to according to the perspective camera model
8
Sub-problem 1
Represent πππ as a SE(3) object πππ β’ Sophus::SE3d objects can be initialized with a Eigen::Matrix3d rotation matrix π and a
Eigen::Vector3d translation vector π β’ Roll, pitch, heading relates to the zyx-rotation sequence, so π = π π§π π¦π π₯ β’ A basic rotation matrix like π π₯ π can be created by
Eigen::AngleAxisd(theta * M_PI / 180, Eigen::Vector3d::UnitX())
9
// Visualization of world and vehicle cv::Mat cv_t_W_V, cv_R_W_V; cv::eigen2cv(t_W_V, cv_t_W_V); cv::eigen2cv(R_W_V, cv_R_W_V); cv::Affine3d cv_T_W_V(cv_R_W_V, cv_t_W_V); cv::viz::Viz3d my_window("window 1"); my_window.showWidget("World-axes", cv::viz::WCoordinateSystem(3.0)); my_window.showWidget("vehicle-axes", cv::viz::WCoordinateSystem(3.0), cv_T_W_V); my_window.showWidget("vehicle", cv::viz::WCube(cv::Vec3d(-3.0, -1.5, -1), cv::Vec3d(3.0, 1.5, 1.0)), cv_T_W_V); my_window.spin();
Sub-problem 2 and 3
Represent ππΆπ as a SE(3) object ππΆπ β’ Which basic rotations must π undergo in order to coincide with πΆ ? β’ Two basic rotations is enough Represent ππΆπ as a SE(3) object ππΆπ β’ Recall that ππΆπ = πππ ππΆπ
10
// Visualization of camera frustum cv::Mat cv_t_W_C, cv_R_W_C, cv_K; cv::eigen2cv(K, cv_K); cv::eigen2cv(T_W_C.translation(), cv_t_W_C); cv::eigen2cv(T_W_C.rotationMatrix(), cv_R_W_C); cv::Affine3d cv_T_W_C(cv_R_W_C, cv_t_W_C); my_window.showWidget("camera_frustum", cv::viz::WCameraPosition(cv_K, 1.0, cv::viz::Color::red()), cv_T_W_C);
Sub-problem 4 and 5
Determine the camera matrix π = πΎ π π β’ Recall that in the perspective camera model π = π ππΆ and π = πππΆ , so we can not read π and
π directly from ππΆπ Determine the pixel π = π’,π£ that the point π projects to according to the perspective camera model β’ Recall that ποΏ½ = ππΏοΏ½
11
// Visualization of point (as a small sphere) my_window.showWidget("Q", cv::viz::WSphere({ Q_W(0), Q_W(1), Q_W(2) }, 0.1));
Laplace blending
12
Steg 1: Lag nytt prosjekt og vis frem bilder
β’ Kopier Β«opencv_project_templateΒ» og gi nytt navn β Husk Γ₯ endre Β«PROJECT_NAMEΒ» i CMakeLists.txt
β’ Lag nytt prosjekt med Cmake
β’ Skriv et program som leser to bilder
β img_1: free_cat.jpg β img_2: free_tiger.jpg β Bildene bΓΈr konverteres til flyttallsbilder:
cv::imread(β¦).convertTo(img_1, CV_32F, 1.0/255.0);
β’ Vis bildene frem β cv::namedWindow() β cv::imshow()
13
Steg 2: Enkel blanding av bilder
β’ Samregistrer bildene ved Γ₯ angi tre punktkorrespondanser
β’ Lag maske med rampe
14
cv::Point2f pts_1[] = {{321, 200}, {647, 200}, {476, 509}}; cv::Point2f pts_2[] = {{441, 726}, {780, 711}, {615, 1142}}; cv::Mat trans_mat = cv::getAffineTransform(pts_2, pts_1); cv::warpAffine(img_2, img_2, trans_mat, img_1.size());
cv::Mat mask = cv::Mat::zeros(img_1.size(), CV_32FC1); cv::rectangle(mask, cv::Rect{img_1.cols/2, 0, img_1.cols/2 + 1, img_1.rows}, 1, CV_FILLED); cv::blur(mask, mask, cv::Size{3, 3});
Steg 2: Enkel blanding av bilder
β’ Lag funksjon som gjΓΈr enkel blanding av to bilder vektet med masken
β Tips: cv::blendLinear()
β’ Bruk funksjonen og vis frem resultatet β PrΓΈv med forskjellige masker
β’ Andre sΓΈmmer, sirkler, stΓΈrre glattefilter
15
cv::Mat linear_blend(cv::Mat& img_1, cv::Mat& img_2, cv::Mat& mask)
Steg 3: Laplaceblanding
β’ Lag funksjonen β cv::pyrDown()
16
std::vector<cv::Mat> construct_gaussian_pyramid(cv::Mat& img)
Steg 3: Laplaceblanding
β’ Lag funksjonen β Bruk construct_gaussian_pyramid β cv::pyrUp()
17
std::vector<cv::Mat> construct_laplacian_pyramid(cv::Mat& img)
Steg 3: Laplaceblanding
β’ Lag funksjonen β cv::pyrUp()
18
cv::Mat collapse_pyramid(std::vector<cv::Mat>& pyr)
Steg 3: Laplaceblanding
β’ Lag funksjonen β For eksempel slik:
19
cv::Mat laplace_blending(cv::Mat& img_1, cv::Mat& img_2, cv::Mat& mask)
std::vector<cv::Mat> mask_pyr = construct_gaussian_pyramid(mask); std::vector<cv::Mat> img_1_pyr = construct_laplacian_pyramid(img_1); std::vector<cv::Mat> img_2_pyr = construct_laplacian_pyramid(img_2); std::vector<cv::Mat> blend_pyr(img_1_pyr.size()); for (int i = 0; i < img_1_pyr.size(); i++) { // TODO: Perform linear blend on each level. } return collapse_pyramid(blend_pyr);
Steg 3: Laplaceblanding
β’ Bruk laplace_blending(img_1, img_2, mask) β Vis frem resultat β Sammenlign med enkel blending
20
Steg 4: Moroplukk
β’ PrΓΈv andre bilder β Ta bilder med kameraet β Finn bilder pΓ₯ nett β Angi nye punkter for samregistrering
β’ Andre masker
β Last ned GIMP for Γ₯ tegne finere masker
21
Steg 5: Dypdykk
β’ Implementer pyramiden selv β Ikke bruk cv::pyrDown() eller cv::pyrUp()
β’ Ta en titt pΓ₯ cv::seamlessClone()
β’ PrΓΈv Γ₯ implementere warpingen selv
22
top related