computer graphics and image processing introduction · computer graphics and image processing...
TRANSCRIPT
Computer Graphics and Image ProcessingIntroductionPart 1 – Lecture 1
1
COMPSCI 373 Lecturers:
A. Prof. Patrice Delmas (303.391) Week 1, 5-8 Contact details:
[email protected]: 303 - 391 (3rd level CompSci building)
Office hours: Wed 2-3pm, also available right after each lectureProf. Georgy Gimel’farb (303.389) Week 2-4, 9-12
Tutors: Nick Stones-Havas, Trevor Gee Check https://www.cs.auckland.ac.nz/courses/compsci373s1c/tutorials/
Lecture time: Monday 1-2pm (260-092) Tuesday 1-2pm (260-092)
Friday 4-5pm (260-073) Marking:
25% assignments, 20% test, 55% examination Weekly Assignments (2.5 marks each):
Weekly assignments from week 3 Practise assignment on week 2
5 March 20152 COMPSCI 373 S1C
All assignments will be using CodeRunner https://www.coderunner.auckland.ac.nz
Type your solutions directly in the browser
Each assignment contributes
towards your final grade
Assignments
3
We will be using Piazza as a discussion forum You will receive an email with an activation link
Please feel free to answer each other’s questions
Piazza forum
4
We need to elect class representatives You get to attend the very exciting staff/student meetings (free
food!)
Find out a bit more about how the department runs!
Represent your classmates!
Class reps
5
Assessments Note: Students must obtain a pass in both the practical (assignments)
and non-practical work (test + exam) in order to pass as a whole Practical: (25%) Weekly Assignments (2.5 % each)
Theory: (75%) Test: 20%,
On Week 6 just after Easter break (during class time) Exam: 55%
TBA
6
Policy on Cheating and Plagiarism We uses many ways to check that the work students submit for marking is their
own and was not produced by, or copied from, someone else. A comparison program to automatically compare all submissions from students.
Note: All assignments deemed to be too similar are automatically allocated a zero mark. All students who submitted these assignments are entered in the duplicate assignment
register (a list of students whose work is known to have been copied). This list is maintained over many years.
Students who have been caught cheating will be notified by email. Offenders may be referred to the University Disciplinary Committee
Both the person who copied the work and the person whose work was copied are allocated a zero mark. It is important that you do not lend your assignments to others. Never give anyone a
copy of your assignment. It is the responsibility of each student to ensure that others do not copy their work.
Read: http://www.auckland.ac.nz/uoa/home/about/teaching-learning/honesty
7
Assessments (con’t)
Note: Students must obtain a pass in both the practical (assignments) and non-practical work (test + exam) in order to pass as a whole
Missed work If you miss the deadline for an assignment and have a valid reason, you
should see the course coordinator ASAP If you miss the test/exam for any valid reason, or you sit the test/exam
but believe that your performance was impaired for some reason, then you may be able to apply for an aegrotat, compassionate or special pass consideration. Refer to the online information on Missed Exams, Aegrotats and Compassionate
Consideration: http://calendar.auckland.ac.nz/regulations/academic/examination.html
8
Part I OverviewWeek 1: 2D and 3D Geometry
Week 2: Color Theory (Timo)
Week 3: Illumination and Shading
Week 4: Ray Tracing
9Raytracing image thanks to Tim Babb
Assignment 3
9
References F.S. Hill. Computer Graphics using OpenGL
(2nd or 3rd Edition). Prentice Hall. Nick Efford. Digital Image Processing:
A Practical Introduction Using Java.Addison Wesley.
C/C++:
C Reference Manualhttp://www.cs.bell-labs.com/who/dmr/cman.ps
Bruce Eckel - Thinking in C++ http://www.ibiblio.org/pub/docs/books/eckel/
C++ for Java Programmershttp://www.jgcampbell.com/cpp4jp/cpp4jp.pdf
See also CS373 resources page10
A bit more about me
14 years at Uni (lecturing 105,210,373,375,716,773,775)
What I dislike: noise during the class, cheating (test, exam, assignments)
What I like: engaging students, questions from students, ask questions to students (including the ones sleeping, talking, txting or disrupting the class)
What I do outside 373 this semester: I teach 105 and 773
I do research in 3D computer vision: www.ivs.auckland.ac.nz\
11
12
Email me forMistakes in the lecture notesDead-links on the webpageIssues not related to courses (personal problems-valid request for extension)
ForumCheck it as often as you want.I tend to read the forum (when close to assignment deadlines) and do answer but likely not as fast as the tutors.The tutors are taking care of the forum.
FacebookI am happy to post on the course Facebook page
EmailCheck email the night before each lecture
There might be special tutorials Other important information
TutorIs here to helpHas to help as many students as possible (be minded of others in need)
13
Miscellaneous We will have 3 lectures a week. You need to keep up with the
pace:
Each new lecture will require you to know and understand the content of the previous lectures.
You must do the exercises provides with the lecture notes to make sure you really understand the course content
Tutorials are a great way to supplement lectures
Learn as much as you can on your own, lectures and tutorials are a start not the end of the journey
Outside office hours and tutorials, tutors are not supposed to be at hand
14
How to progress while Learning
1. Read the lecture notes after each lecturea. Make a summary of what has been seen on the lecturesb. Redo examples already treated (during the lectures) or/and do the untreated examplesc. Do examples without refereeing to the lectures
2. Read the materials provided onlinea. To learn moreb. To complement lectures
3. If you have questions or do not understand somethinga. Do 1b. Do 2c. Attend the tutorialsd. Check the forume. Ask other 373 studentsf. Ask a tutor during office hoursg. Email me (only if points a to f are completed and did not bring relevant information)
4. How to prepare examsa. Do previous years examsb. Do exercises of the course/tutorials/exercise course booki. http://www.cs.auckland.ac.nz/compsci373s1c/exams/ii. Do 1 and 2
A glimpse of the research at the Intelligent Vision Systems Lab, Department of computer Science, The University of Auckland
5 March 2015 IVS research – Compsci 373 15
Low cost + advanced theory = improved performance
Online Interactive Web-based Stereo Vision
Build a portable /comprehensive website and mobile app: Accept off-the-lab real-life photos of different types:
Left + right
Anaglyph
Auto-stereogram
Automatically align images to epipolar stereo form
Reconstruct 3D information and display online
Returns different 3D results
URL: ww.ivs.auckland.ac.nz/quick_stereo Thousands of visitors from 47 countries around
the World First rank on Google Search on key words:
online stereo matching
online stereo vision
Stereo matching
(1) Input
(2) Find image correspondence set
(3) Estimate F matrix and epipolar lines (4) Rectify images
The development of a stereo‐photogrammetric setupS.Bertin, H.Friedrich, P. Delmas, .H. Chan, Trevor Gee
OBJECTIVES (CS only):
A high‐resolution, non‐proprietary, stereo‐photogrammetric setup for microtopography measurement in a laboratory
flume, and means to evaluate its accuracy.
Differ from previous photogrammetric applications in Hydraulics
Basis for 2‐media and underwater photogrammetry.
17
5 March 2015
From the depth map and the calibration results, a point cloud containing the 3D World Coordinates of each pixel is extracted.
Depth map 2D coordinates of corresponding pixels in the L and R images.
Calibration results 3D intersection of the 2 lines between the cameras’ centre and the corresponding pixels
• 210MB text file containing the X,Y,Z coordinates (in mm) of all measured points
• The left image is the basis for the coordinates
5 March 2015IVS research - Unam presentation
The point cloud is opened with MATLAB® , and a 3D DEM is plotted onto adjustable orthogonal grids…
Original Resolution ≃0.3‐mm 1‐mm resampling 2‐mm resampling
In Situ: The Ngongotaha river experiment ‐ 2
In Situ: The Ngongotaha river experiment ‐ 3
2015 Field trip
5 March 2015IVS research - Unam presentation 22
At Marine Sciences Institute
First week after the end of class (early June)
We may invite2‐3 373 students with interest in pos‐graduate studies
Interested in Post‐graduate studiesResearch projects
5 March 2015IVS research - Unam presentation 23
Talk to Me, Georgy, Nick or Trevor to know more about what is going in at the IVSLab
DIGITAL IMAGES
24
2D
3D
4D
How to Get Digital Images?1. 2D real 2D digital
2. 3D real 2D digital
3. 4D real animated 2D digital
4. 3D real 3D digital
5. 4D real 4D digital
6. Synthetic or computer generated25
2D slicesof 3D volume3D Scanner MRIStereoscopic Camera
Motion Capturing
What to Do with Digital Images? Image Processing: process them to get new images
Computer Vision: analyze them to get information about what is in the image
Computer Graphics: generate them
26
Descriptions Images
Descriptions Computer Graphics
Images Computer Vision Image Processing
OUTPUT
INP
UT
Pixels and Resolution Pixel or pel (picture element)
Position (x,y) + signal value v (greyscale or colour)
Origin (0,0) of pixel coordinates sometimes in top left corner
Resolution: how many pixels? width height Spatial resolution: image pixels per cm or inch (in x and y)
Can be used to convert pixel coordinates into physical coordinates
27
x
y
Position: (512, 456)Value: (10, 255, 8)
Encoding of Colors Bit-depth: number of bits used to represent each pixel's
value (typically 1, 8, 24 or 32) Binary image: bit depth is 1;
only code values 0 (black) and 1 (white) Scalar/monochrome/greyscale image: scalar code values
(e.g. just a single number per color)only grey values (from black to white)
and no colour Vector-valued image vector code values
(e.g. several numbers per color)All the colors can be represented
28
Defining Images MathematicallyImages can be defined on an M N arithmetic grid (or lattice)
Pixel coordinates x and ywith x 1,…,M ; y 1,…,N
Image as a function f: R V
V is a set of signal values,e.g. grey levels or colors
Example:pixel at position (100, 100)has value 255,i.e. f (100, 100) = 255
29
y
x
0 1 2 3 4 M01234
N
NyMxyxNM 11:,,R
Image Processing Geometric transformations:
resizing, rotation, deformations, …
Color transformations:quantization, conversion,color adjustment, …
Compositing:combination of two or more images
Many other operations
30
Computer Vision Use computer to do things similar to human vision, using
image processing, artificial intelligence, biology & physics
Usually dealing with 2D images of 3D scene
Often real-time and part of a larger system (e.g. robot)
E.g. scene reconstruction, image restoration, object recognition, tracking, motion estimation, event detection, …
31
A Typical computer vision project
5 March 2015 COMPSCI 773 S1C32
Stereo images- depth
map
Facesegmentation
Face features extraction
Features labelling
Face normalisation
PCAei ei+1 ... en
w1 w2 ... wnInput pose
Classification
ei ei+1 ... en
w11 w21 ... wn1
w12 w22 ... wn2
w13 w23 ... wn3
w14 w24 ... wn4
Trained informationRepresentative weightings
Class 1: OpenClass 2: GoClass 3: StopClass 4: Left
Mean image
Classified pose: Class 1
Calibration -rectification
Human vs. Computer VisionHuman Vision Subjective, unstable,
inaccurate in measurements Involves active interaction with
environment Exploits experience, knowledge,
context But unique capabilities to describe
and understand… Real-time
Computer Vision Objective accurate measurements But low capabilities to describe and
understand… Rarely works effectively real-time
33
Generating Digital ImagesNeed a 3D scene consisting of:
Objects (3D models)
Light sources
Camera (viewer, eye)
Parameters specifying how light interacts with the scene
Light parameterse.g. type of light (light bulb, spotlight), color, brightness
Material parameterse.g. color, texture, transparency
Rendering parameterse.g. algorithm, quality, resolution
34
Objects (3D Models)Objects are made up of parts (often also of other objects):
Vertices: 3D points used to define model
Edges: lines between vertices
Faces: polygons bounded by edges
Mesh: surface made up of connected polygons
35
Wireframe modelshowing mesh withvertices, edges andfaces
Smoothed and shaded solid model
View plane (screen image)
The Camera View point: where is the camera?
View direction: which direction does it point?
View orientation: where does the top of the camera point?E.g. is the photo taken upside down?
Projection: what type of lens does the camera have?
Rendering:
Project scene onto view plane in front of the camera
Compute the color of each pixel on the view plane36
SUMMARY
37
Summary Things we can do with images:
Image Processing: process them to get new images
Computer Vision / Image Analysis:analyze them to get higher-level information (context, emotional content, numbering/counting)
Computer Graphics: generate them
Images:pixels with encoded colors, (spatial) resolution,can be represented on arithmetic grid
For generating images:3D scene with objects (vertices, edges, faces, meshes), light sources, camera
38
Quiz1. What is the spatial resolution of an image?
2. What is Computer Vision?
3. What are the differences between Computer Vision and human vision?
4. What does a 3D scene consist of?
5. What are vertices, edges and faces?
39
Super Quiz1. What is camera calibration?
2. How much time did each CG still shots of Gollum-LOTR take to be generated in 2001?
3. Same question for the Hobbit
4. What is the cheapest 3D camera commercially available if any?
5. How many 3D mobile phone do you know?
6. Cite 3 famous (most-used) software packages for image processing/computer graphics?
7. What was the price of a medium ranged 2Mpixels digital camera in 2002 compared to today’s 10 Mpixels camera?
40