cse 252c: advanced computer visioncseweb.ucsd.edu/~mkchandraker/classes/cse252c/...cse 252c, sp20:...

Lecture0:Introduction

CSE252C:AdvancedComputerVision

ManmohanChandraker

CSE252C,SP20:ManmohanChandraker

Virtualclassrooms

• VirtuallecturesonZoom

– Onlyhostsharesthescreen

– Keepvideoturnedoffincaseofbandwidthissues

– Microphonesmutedunlessspeakingtoreducenoise

– Butpleasedospeakup(remembertounmute!)

– Slidestobeuploadedonwebpagejustbeforeclass

– WhiteboardthroughcameraandiPadtowritethings

• LecturesrecordedanduploadonKaltura

– Availableunder“MyMedia”onCanvas


Virtualclassrooms

• VirtualinteractionsonZoom

– Askandanswerplentyofquestions

– Trytohavein-classdiscussions

– “Raisehand”featureonZoomwhenyouwishtospeak

– InstructororTAscanunmute

– Postquestionsonchatwindow

– TAwillhelpkeeptrackofraisedhandsandchatwindow

– Ifinstructorcannotnotice,TAwillinterruptatlogicalpause

• Happytotryothersuggestions


Enrollmentlogistics

• Waitlist

– Thereisalongwaitlistfortheclass

– Departmenthasnotagreedtoaddmorestudents

– Youarewelcometoattendlecturesevenifonwaitlist

– TolimitTAworkload,wecangradeonlyenrolledstudents

• Canvas

– Allenrolledandwaitlistedstudentsshouldhaveaccess

• AllannouncementswillbepostedonPiazza

– SendemailtoTAs(CCinstructor)ifdidnotgetPiazzainvite


ComputerVision


Definingcomputervision

Wall-E: Fact and Fiction (Minh Do, Princeton University)CSE252C,SP20:ManmohanChandraker

Studyingcomputervision

Source:2Pmedia

• Visionisafundamentalinterfacetotheworld



Source:2Pmedia



Humans

Machines

Virtual

(Cyborgs!)


• Weareallusersofcomputervision


Transportation Design

Communication E-CommerceCSE252C,SP20:ManmohanChandraker

Studyingcomputervision• Visionisafundamentalinterfacetotheworld

• Weareallusersofcomputervision

• Deepandattractivescientificproblems• Howdowerecognizeobjects?

• Whydonewbornbabiesrespondtoface-likeshapes?

[Farroni etal.,2005]




Ourproblemsbecamemorecomplex



Ourambitionsbecamehigher



Ourconceptschangedovertime



Ourtoolsadvancedovertime

[Mahony etal.,2019]

Gainingperspectiveoncomputervision


ImportantforAutopilot:Donothitapolice car!

Gainingperspectiveoncomputervision


ImportantforAutopilot:Donothitapolice car!

ImportantforRobots:Stayclearofhumans!

[Wired.com]

Autonomousdriving

Source:WiredCSE252C,SP20:ManmohanChandraker

AutonomousdrivingWhereisourcar?

StructurefromMotion

VisualSLAM

Whereareotheragents?

Objectdetection

3Dlocalization

Wherearesceneelements?

Semanticsegmentation

Whatisasafepath?

Behaviorprediction

Pathplanning


Afewtopicsinthiscourse

• StructureandMotion

• FacesandHumans

• ObjectsandStuff

• MaterialandLighting

• BiasandPrivacy


StructureandMotion

CSE252C,SP20:ManmohanChandraker [Agarwaletal.,ICCV2009]

StructureandMotion

CSE252C,SP20:ManmohanChandraker[SongandChandraker,CVPR2015]

FacesandHumans


[Blanz andVetter]

[Microsoftblog, 2018]

FacesandHumans

CSE252C,SP20:ManmohanChandraker [Kanazawaetal.,CVPR2019]

ObjectsandStuff

CSE252C,SP20:ManmohanChandraker [Choi,ICCV2015]

ObjectsandStuff


[Wangetal.,CVPR2019]

[Leeetal.,ICCV2017]

MaterialandLightingInputimage Augmented image

CSE252C,SP20:ManmohanChandraker [Lietal.,CVPR2020]

MaterialandLighting

CSE252C,SP20:ManmohanChandraker [Li,Yeh,etal.,CVPR2020]

BiasandPrivacy


Caucasian African-American East-Asian

Proportion in datasets: 80% Proportion in datasets: 10% Proportion in datasets: 5%

High accuracy Low accuracy Low accuracyTraining on biased data without domain adaptation

High accuracy High accuracy High accuracyUsing domain adaptation to address dataset bias

BiasandPrivacy


Noadaptation

Withadaptation

Labelsrequire1.5hours perimage!

Sourcedomain:goodweather,with labels

Targetdomain:rainyweather,no labels

Trainonsource,apply ontarget

Trainonsource,adapt totarget

[Tsaietal.,ICCV2019]

BiasandPrivacy


America

Collect

data

Label

data

Train

modelDeployExisting

solution

Newmarket

Newmarket

....

....

....

....

Asia

Collect

data

1week

Label

data

Adapt

model

1week

Deploy

Africa

Collect

data

Label

data

Adapt

modelDeploy

1week 1year 1week

1week 1week

BiasandPrivacy

CSE252C,SP20:ManmohanChandraker [Xiaoetal.,AAAI2020]

Overallgoalsforthecourse

• Introducethestate-of-the-artincomputervision

• Studyprinciplesthatmakethempossible

• Getunderstandingoftoolsthatdrivecomputervision

• Enableoneorallofseveralsuchoutcomes

– Pursuehigherstudiesincomputervision

– Joinindustrytodocutting-edgeworkinAI

– GainanappreciationofmodernAItechnologies


CourseDetails


Coursedetails• Eachclasswillcovertopicsincomputervision

• Examplesoftopics

• Correspondence

• Stereo

• Opticalflow

• Structurefrommotion

• Facerecognition

• Humanposeestimation

• Materialandlighting

• Semanticsegmentation

• Objectdetection

• Tracking

• Actionrecognition

• Domainadaptation

• Privacyandfairness


Coursedetails

• Topicsstructuredintoafewmodules

– Background

– StructureandMotion

– FacesandHumans

– ObjectsandStuff

– MaterialandLighting

– AdaptationandBias


Coursedetails

• “Lightning”presentations

– Fourstudentstopresentinoneclass

– Timelimit:5minutes

– Paperstobeassignedbyinstructor

– Orderofpresentation:alphabetic

• Sendpresentation1daybeforeclass

– Well-practicedandfluentpresentation

– Includenarrationifasynchronous

– Askandanswerquestionsafterpresentation


Coursedetails

• Presentationformat(1slideforeach):

1.Motivationandproblemdescription

2.Priorwork

3.Methodoverview

4.Methodanalysis

5.Experiments

6.Futureworkanddiscussion


Coursedetails• Classwebpage:

– http://cseweb.ucsd.edu/~mkchandraker/classes/CSE252C/Spring2020/

• Instructoremail:

– [email protected]

• TAs:Zhengqin LiandYou-YiJau

– Emails:[email protected] [email protected]

• Grading

– 10%presentation

– 60%assignments

– 30%finalexam

• Aimistolearntogether,discussandhavefun!


MyInterests


Sceneunderstandingforself-drivingLearningtosimulate

Large-scale,real-time,monocularSFM.

Accuracycomparabletostereosystems.

Teacher

CNN

Student

CNN

ImageHint

loss

Student

Detector

Teacher

Detector

Distillation

loss

GT

loss

Distillationnetworksforfastandaccurateobjectdetection

DistillationforcompressedCNN(student) tomimicuncompressed

CNN(teacher),toachievegreateraccuracyatthesamespeed.

MonocularSFMLearningtopredictuncertainfuturebehavior

DESIRE:DeepStochasticIOCRNNEncoder-Decoder

• DeepCVAE(autoencoder) togeneratediversehypotheses.• RNNtorankpredictions basedonmotion,sceneandinteractions.• Deepinversereinforcementlearningforlong-termfuturerewards.

Reinforcementlearningforsimulations.

Ensuringdiversity andcoverage.

NeurIPS 2017

CVPR2017CVPR2015

ICLR2019


Geometricandsemantic3Dreconstruction

WeaklysupervisedsemanticreconstructionMetriclearningforcorrespondence

OcclusionreasoningandlargetransformationsDeepsupervisionforocclusion-reasonedparts

CVPR2016

CVPR2017,PAMI2018

NeurIPS 2016,ECCV2018

ECCV2018,CVPR2019

Input

Occlusion-reasoned

top-viewlayout


Physically-basedlearningforshapeandmaterial

Materialsand

global

illumination

Refractive

interfaces

Shapeand

motion

WarpNet ColorNet

SIGGRAPHAsia2018

WACV2018,2019

ICCV2017ICCV2017CSE252C,SP20:ManmohanChandraker

UnsupervisedadaptationtonewdomainsReconstructionwithunaligneddataFacerecognitionforprofileinputs

FromrainytogoodweatherCarrecognitionacrosscameraandlightingconditions

CVPR2019

3DV2017

ICCV2017

CVPR2018CSE252C,SP20:ManmohanChandraker

Take-homemessage

• ComputervisionisakeybranchofAI

• Enablesseveralmodernapplicationsaroundus

• Alotofhighlyvisibleandhigh-impactactivity

• Hugeindustryinterest

• Thisisagreattimetostudycomputervision!


cse 252c: advanced computer visioncseweb.ucsd.edu/~mkchandraker/classes/cse252c/...cse 252c, sp20:...

Documents