comp9517: computer vision introduction
TRANSCRIPT
![Page 1: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/1.jpg)
COMP9517: Computer Vision
Introduction
Week 1 COMP9517 2021 T3 1
![Page 2: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/2.jpg)
What is computer vision?
Week 1 COMP9517 2021 T3 2
![Page 3: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/3.jpg)
What is computer vision?
Computer science perspectiveComputer vision is the interdisciplinary field that develops theories and methods to allow computers extract relevant
information from digital images or videos
Computer engineering perspectiveComputer vision is the interdisciplinary field that develops
algorithms and tools to automate perceptual tasks normally performed by the human visual system
Week 1 COMP9517 2021 T3 3
![Page 4: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/4.jpg)
Computer visionautomates and integrates
many information processing and representation approaches
useful for visual perception
Every picture tells a story
Week 1 COMP9517 2021 T3 4
![Page 5: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/5.jpg)
Yes and no (but mostly no)
• Humans are much better at “hard” tasks
• Computers can be better at “easy” tasks
Can computers match (or beat) humans?
Week 1 COMP9517 2021 T3 5
![Page 6: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/6.jpg)
Human vision has its limitations…
Which objects are brighter?
Week 1 COMP9517 2021 T3 6
![Page 7: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/7.jpg)
Human vision has its limitations…
Which objects are brighter?
Week 1 COMP9517 2021 T3 7
![Page 8: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/8.jpg)
Human vision has its limitations…
Which side of this object is brighter?
Week 1 COMP9517 2021 T3 8
![Page 9: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/9.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 9
Which side of this object is brighter?
![Page 10: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/10.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 10
Which side of this object is brighter?
![Page 11: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/11.jpg)
Human vision has its limitations…
Are the cells popping in or out?Week 1 COMP9517 2021 T3 11
![Page 12: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/12.jpg)
Human vision has its limitations…
Are the cells popping in or out?Week 1 COMP9517 2021 T3 12
180o
![Page 13: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/13.jpg)
Human vision has its limitations…
What pattern do the squares form?
Week 1 COMP9517 2021 T3 13
![Page 14: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/14.jpg)
Human vision has its limitations…
What pattern do the squares form?
Week 1 COMP9517 2021 T3 14
![Page 15: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/15.jpg)
Human vision has its limitations…
What object do you see in this image?
Week 1 COMP9517 2021 T3 15
![Page 16: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/16.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 16
What object do you see in this image?
![Page 17: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/17.jpg)
Human vision has its limitations…
How do the main lines run with respect to
each other?
Week 1 COMP9517 2021 T3 17
![Page 18: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/18.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 18
How do the main lines run with respect to
each other?
![Page 19: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/19.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 19
In which direction are these particles moving ?
![Page 20: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/20.jpg)
Human vision has its limitations…
Week 1 COMP9517 2021 T3 20
https://www.youtube.com/watch?v=a7efEqgpIrE
![Page 21: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/21.jpg)
Course rationale
Week 1 COMP9517 2021 T3 21
Human vision has its limitations
• Intensities, shapes, patterns, motions can be misinterpreted• Is labor intensive, time-consuming, subjective, error-prone
Computer vision can potentially improve this
• Can work day and night without getting tired• Analyses information quantitatively and objectively• Is potentially more accurate, precise, reproducible
If the methods and tools are well designed!
![Page 22: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/22.jpg)
Project VarCity recreates 3D city models using social media photos
Week 1 COMP9517 2021 T3 22
Application: 3D shape reconstruction
![Page 23: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/23.jpg)
Google’s Show and Tell open-source image captioning model in TensorFlow
Week 1 COMP9517 2021 T3 23
Application: image classification and captioning
![Page 24: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/24.jpg)
Iris Automation provides safer drone operation with intelligent collision avoidance
Week 1 COMP9517 2021 T3 24
Application: intelligent collision avoidance
![Page 25: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/25.jpg)
Facebook’s DeepFace project nears human accuracy in identifying faces
Week 1 COMP9517 2021 T3 25
Application: face detection and recognition
![Page 26: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/26.jpg)
For improving image capture on digital cameras
Week 1 COMP9517 2021 T3 26
Application: face detection and recognition
![Page 27: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/27.jpg)
Week 1 COMP9517 2021 T3 27
Application: vision-based biometrics
Who is she?
How the Afghan girl was identified by her iris patterns
The remarkable story of Sharbat Gula, first photographed in 1984 aged 12 in a refugee camp in Pakistan by National Geographic photographer Steve McCurry, and traced 18 years later to a remote part of Afghanistan where she was again photographed by McCurry…
![Page 28: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/28.jpg)
Week 1 COMP9517 2021 T3 28
Application: logging in without a password
Fingerprint scanners onmodern laptops and
other devices
Windows Hello makes logging in as easy as looking at your PC
![Page 29: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/29.jpg)
Converting scanned documents or number plates to processable text
Week 1 COMP9517 2021 T3 29
Application: optical character recognition (OCR)
![Page 30: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/30.jpg)
Week 1 COMP9517 2021 T3 30
Application: object recognition in supermarkets
LaneHawk by Evolution Robotics Retailprovides a loss-prevention solution that detects bottom-of-basket (BOB) items in checkout lanes
![Page 31: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/31.jpg)
Week 1 COMP9517 2021 T3 31
Application: object recognition in phones
![Page 32: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/32.jpg)
Intel’s Mobileye makes cars safer and more autonomous
Week 1 COMP9517 2021 T3 32
Application: autonomous vehicles
![Page 33: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/33.jpg)
NASA’s Mars Exploration Rover Spirit autonomously captured this picture in 2007
Week 1 COMP9517 2021 T3 33
Application: space exploration
Vision systems used for panorama stitching, 3D terrain modeling, obstacle detection, position tracking
See Computer Vision on Mars for more information
![Page 34: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/34.jpg)
Week 1 COMP9517 2021 T3 34
Application: machine vision in robotics
RoboCupNASA’s Mars Spirit Rover
![Page 35: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/35.jpg)
Week 1 COMP9517 2021 T3 35
Application: medical imaging
Image Guided SurgeryComputer Aided Diagnosis
![Page 36: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/36.jpg)
Week 1 COMP9517 2021 T3 36
Application: video surveillance
• Traffic monitoring• Person tracking• Action recognition• Speed estimation• Object counting• ...
![Page 37: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/37.jpg)
Week 1 COMP9517 2021 T3 37
Goals and challenges of computer vision
• Extract useful information from images, both metric and semantic
• Data heterogeneity, ambiguity, and complexity are a big challenge
• Significant progress in recent years due to improvements in
processing power, memory, storage capacity, data availability
• Computer vision workflow: images > measurements > models >
algorithms for learning and inference
![Page 38: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/38.jpg)
Week 1 COMP9517 2021 T3 38
Computer vision tasks
• Obtain simple inferences from individual pixel values
• Group pixels to separate object regions or infer shape information
• Recognise objects using geometric or statistical pixel information
• Combine information from multiple images into a coherent whole
Requires understanding of the physics of imaging and the use of
mathematical and statistical models for information extraction
![Page 39: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/39.jpg)
Week 1 COMP9517 2021 T3 39
Critical issues in computer vision
• Sensing: how do sensors obtain images of the world?
• Encoding: how do images encode information of the scene?
• Representation: what are appropriate representations of objects?
• Algorithmics: what are appropriate algorithms to process image
information and construct scene descriptions?
![Page 40: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/40.jpg)
Week 1 COMP9517 2021 T3 40
Low-level computer vision
This is almost entirely digital image processing (image in > image out)
• Sensing: image capture and digitisation
• Preprocessing: suppress noise and enhance object features
• Segmentation: separate objects from background and partition them
• Description: compute features which differentiate objects
• Classification: assign labels to image segments (regions)
![Page 41: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/41.jpg)
Week 1 COMP9517 2021 T3 41
High-level computer vision
This is about knowledge construction, representation, and inference
• Recognition: identify objects based on low-level information
• Interpretation: assign meaning to groups of recognized objects
• Scene analysis: complete understanding of the captured scene
![Page 42: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/42.jpg)
Week 1 COMP9517 2021 T3 42
Assumed knowledge
To do this course successfully you should:
• Be able to program well in Python or willing to learn it independently
• Be familiar with data structures and algorithms and basic statistics
• Be able/learn to use and integrate software packages (OpenCV, Scikit-Learn, Keras)
• Be familiar with vector calculus and linear algebra or willing to learn it independently
Please self-assess before deciding to stay/enroll in the course
![Page 43: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/43.jpg)
Week 1 COMP9517 2021 T3 43
Student learning outcomes
After completing this course, you will be able to:
• Explain basic scientific, statistical, and engineering approaches to computer vision
• Implement and test computer vision algorithms using existing software platforms
• Build larger computer vision applications by integrating software modules
• Interpret and comment on articles in the computer vision literature
![Page 44: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/44.jpg)
Week 1 COMP9517 2021 T3 44
Course topics and lecturersWeek Topic Lecturer
1 Introduction & Image Formation Professor Erik Meijering
2 Image Processing Dr Wafa Johal
3 Feature Representation Dr Wafa Johal
4 Pattern Recognition Professor Erik Meijering
5 Image Segmentation Professor Erik Meijering
6 Flexible Week (No Lectures)
7 Motion and Tracking Professor Erik Meijering
8 Deep Learning Guest Lecturers TBC
9 Applications Guest Lecturers TBC
10 Project Demos Professor Erik Meijering
![Page 45: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/45.jpg)
Week 1 COMP9517 2021 T3 45
Weekly class structure
• Lectures: Tuesdays 6-8pm & Thursdays 6-7pm (live stream via BB Colab)
All lectures will be live online with an opportunity to ask questions
• Lab consultations: Thursdays 7-8pm in weeks 1-5 (online via BB Colab)
Software demo in week 1 and lab consultations with your assigned tutor in weeks 2-5
• Project consultations: Thursdays 7-8pm in weeks 6-9 (online via BB Colab)
All project consultations will be live online with your assigned tutor
• Project demos: Tuesday 6-8pm & Thursday 6-8pm in week 10 (online via BB Colab)
Detailed schedule will be announced on WebCMS3 web page of the course
![Page 46: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/46.jpg)
Week 1 COMP9517 2021 T3 46
Assessments
Late submission penalty: Unless you have received special dispensation from the Lecturer in Charge, work that is submitted after the deadline during term will incur a penalty of 10% per day, up to a maximum of 100%. For the final examination, university exam rules will apply.
Assessment Marks Release Due
Assignment 10% Week 2 Week 4
Lab Work (4x) 10% Weeks 2, 3, 4, 5 Weeks 3, 4, 5, 6
Group Project 40% Week 5 Week 10
Exam 40% Exam Period Exam Period
![Page 47: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/47.jpg)
Week 1 COMP9517 2021 T3 47
Communication modes and etiquette• Online forum (Moodle) is your first port of call for queries of wider interest on lectures,
assignment, labs, project, and assessments
• Contact the LIC for late submission, absence, assessment deadlines, and specific questions about the assignment, labs, project, and assessment contents
• Contact the course admin for issues with enrolment, file submission, group enrolment, or other administration related matters
• Team is committed to respond quickly to queries with a maximum turnaround of 24 hours
• Do observe standards of equity and respect in dealing with all students and staff, in person, emails, forum posts, and all other communication
• Language of communication is English
![Page 48: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/48.jpg)
Week 1 COMP9517 2021 T3 48
Special Consideration• If your work in this course is affected by unforeseen adverse circumstances, you should apply
for Special Consideration via the UNSW website
• UNSW handles Special Consideration requests centrally, so use the website and do not email
the Lecturer in Charge about Special Consideration requests
• Special Consideration requests must be accompanied by documentation
• Marks are calculated the same way as other students who sat the original assessment
• If you are awarded a Supplementary Exam and do not attend, your exam mark will be zero
See the course webpage on WebCMS3 for more detailed information and links
![Page 49: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/49.jpg)
Week 1 COMP9517 2021 T3 49
Plagiarism PolicyREAD the UNSW Policy and Procedure on this (links in the course outline on WebCMS3)
For the purposes of COMP9517, plagiarism includes copying or obtaining all, or a substantial part, of the material for your assignment, whether written or graphical report material, or software code, without written acknowledgement in your assignment from:
• A location on the internet
• A book, article or other written document (published or unpublished) whether electronic or on paper or other medium
• Another student, whether in your class or another class
• Someone else (for example someone who writes assignments for money)
![Page 50: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/50.jpg)
Week 1 COMP9517 2021 T3 50
Plagiarism Policy• If you copy material from another student or non-student with acknowledgement, you will
not be penalised for plagiarism, but the marks you get for this will be at the marker’s discretion and will reflect the marker’s perception of the amount of work you put into finding and/or adapting the code/text.
• If you use text found in a publication (on the internet or otherwise), the marks you get for this will be at the marker’s discretion and will reflect the marker’s perception of the amount of work you put into finding and/or adapting the text.
Assessments provide opportunities for you to develop important skills!
Use these opportunities!
![Page 51: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/51.jpg)
Week 1 COMP9517 2021 T3 51
Further information on WebCMS
Please be sure you are familiar with:
• Communication Etiquette
• Special Consideration
• Student Conduct
• Plagiarism Policy
• Academic Integrity
![Page 52: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/52.jpg)
Week 1 COMP9517 2021 T3 52
Further reading on lectured topicsIn the lectures we will be referring to various online resources for further reading such as:
• Richard Szeliski, Computer Vision: Algorithms and Applications, Second Edition, Springer, 2021
• Dana H. Ballard and Christopher M. Brown, Computer Vision, Prentice Hall, 1982
• Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press, 2016
• David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, 2011
• Simon J. D. Prince, Computer Vision: Models, Learning and Inference, Cambridge University Press, 2012
And other books, scientific articles, and other resources available online or via the UNSW Library
![Page 53: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/53.jpg)
Week 1 COMP9517 2021 T3 53
Further reading on today’s topics
• Chapter 1 of Szeliski for a general introduction to computer vision
• Appendix A of Szeliski for a recap of linear algebra and numerical techniques
![Page 54: COMP9517: Computer Vision Introduction](https://reader030.vdocuments.site/reader030/viewer/2022012721/61b372df0a59290cd7548504/html5/thumbnails/54.jpg)
Week 1 COMP9517 2021 T3 54
Acknowledgements
• Some images on applications taken from Szeliski with original sources
credited where possible
• Other images and videos credited where possible