practical computer vision-- a problem-driven approach towards learning cv/ml/dl
TRANSCRIPT
![Page 1: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/1.jpg)
Practical Computer Vision A problem-driven approach towards learning CV/ML/DL
Albert Y. C. Chen, Ph.D.Vice President, R&D
Viscovery
![Page 2: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/2.jpg)
Albert Y. C. Chen, Ph.D.
• Experience 2017-present: Vice President of R&D @ Viscovery 2016-2017: Chief Scientist @ Viscovery 2015: Principal Scientist @ Nervve Technologies 2013-2014 Computer Vision Scientist @ Tandent 2011-2012 @ GE Global Research
• Education Ph.D. in Computer Science, SUNY-Buffalo M.S. in Computer Science, NTNU B.S. in Computer Science, NTHU
![Page 3: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/3.jpg)
1. W. Wu, A. Y. C. Chen, L. Zhao, and J. J. Corso. Brain tumor detection and segmentation in a CRF framework with pixel-wise affinity and superpixel-level features. International Journal of Computer Assisted Radiology and Surgery, 2015.
2. S. N. Lim, A. Y. C. Chen and X. Yang. Parameter Inference Engine (PIE) on the Pareto Front. In Proceedings of International Conference of Machine Learning, Auto ML Workshop, 2014.
3. A. Y. C. Chen, S. Whitt, C. Xu, and J. J. Corso. Hierarchical supervoxel fusion for robust pixel label propagation in videos. In Submission to ACM Multimedia, 2013.
4. A.Y.C. Chen and J.J. Corso. Temporally consistent multi-class video-object segmentation with the video graph-shifts algorithm. In Proceedings of IEEE Workshop on Applications of Computer Vision, 2011.
5. D.R. Schlegel, A.Y.C. Chen, C. Xiong, J.A. Delmerico, and J.J. Corso. Airtouch: Interacting with computer systems at a distance. In Proceedings of IEEE Workshop on Applications of Computer Vision, 2011.
6. A.Y.C. Chen and J.J. Corso. On the effects of normalization in adaptive MRF Hierarchies. In Proceedings of International Symposium CompIMAGE, 2010.
7. A.Y.C. Chen and J.J. Corso. Propagating multi-class pixel labels throughout video frames. In Proceedings of IEEE Western New York Image Processing Workshop, 2010.
8. A. Y. C. Chen and J. J. Corso. On the effects of normalization in adaptive MRF Hierarchies. Computational Modeling of Objects Represented in Images, pages 275–286, 2010.
9. Y. Tao, L. Lu, M. Dewan, A. Y. C. Chen, J. J. Corso, J. Xuan, M. Salganicoff, and A. Krishnan. Multi-level ground glass nodule detection and segmentation in ct lung images. Medical Image Computing and Computer-Assisted Intervention, 2009.
10. A.Y.C. Chen, J.J. Corso, and L. Wang. Hops: Efficient region labeling using higher order proxy neighborhoods. In Proceedings of IEEE International Conference on Pattern Recognition, 2008.
![Page 4: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/4.jpg)
Some work done before I caught the startup fever
Freestyle Sketching Stage
AirTouch waits in background for the initialization signal
Initialize
Terminate
Output
imagedatabase
Start:Results
CBIRquery
Airtouch HCI interface for Content-based Image Retrieval
![Page 5: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/5.jpg)
Interactive Segmentation & Classification• Segmentation then classification:
• computationally more efficient, • results in much higher classification accuracy.
• Pioneered the “pixel label propagation” field. • First to utilize superpixels and supervoxels for the task.
FG
Traditional Spatial Propagation
Pixel label map
Label a subset of pixels
BG
Spatio-temporal Propagation
time
![Page 6: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/6.jpg)
Image/Video Object Recognition and Content Understanding
approaches
person carries
gives
recieves
Ontology
object
Person 1 Person 1Person 2
High-Level
Mid-Level
approachactivity
receives givescarries
activityactivity activity
Time
Reasoning
xx
x
Low-Level
x x
x
x
![Page 7: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/7.jpg)
Learning and Adapting Optimal Classifier Parameters
subspace B
subsp
ace A
subspace C
Image-level feature space
priors
Patch-level feature space
posteriorprobability
suggest optimal parameter configuration
![Page 8: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/8.jpg)
Graphical Models and Stochastic Optimization
A
(a) The space-time volume of a video showing the objects (A--F) and their appearing time-span.
spac
e
time
AB
C
D
E F
B E
F
C
D
(b) The temporal relationship graph. An edge between two vertices mean that the two objects overlap in time.
(c) The goal is: cover all objects with the smallest number of "ground truth key frames".
spac
e
time
AB
C
D
E F
key 1 key 2
A
B E
F
C
D
(d) This translates to: iteratively solving the max clique problem until all vertices belong to a clique.
A
B E
F
C
Dkey 2
key 1frame t-1 frame t
layer n layer n
layer n+1 layer n+1
TemporalShift
Shift
µ
![Page 9: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/9.jpg)
Medical Imaging and Geospatial Imaging
GNN detection and segmentation
in Lung CT geospatial imaging: building detection
Brain tumor detection and segmentation in MR images.
![Page 10: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/10.jpg)
Why are we here today?
To make a better change for our future.
![Page 11: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/11.jpg)
Change is the only constant-Heraclitus (535 BC - 475 BC)
![Page 12: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/12.jpg)
Change is the only constant-Heraclitus (535 BC - 475 BC)
![Page 13: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/13.jpg)
Why Risk Innovating?
• Good business model NEVER last forever.
• Average “shelf life” on S&P 500: 20 years.
• 100-year old companies constantly reinvent themselves every 10-20 years
• Startups contribute to 20% of USA’s GDP.
![Page 14: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/14.jpg)
The Death of a Good Business Model
• Foxconn 20 year revenue v.s. net profit (now at 5%)
![Page 15: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/15.jpg)
What do 100 year old corporations do?
GE Schenectady, 1896
![Page 16: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/16.jpg)
History of change at GE• 1886: one of the 12 original companies on the Dow
Jone Industrial Average (also the only one remaining). • 1889: lightbulbs • 1919: radios • 1927: TV • 1941: jet engine • 1960: nuclear power • 1971: room AC units • 1995: MRI
![Page 17: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/17.jpg)
History of change at IBM• 1960s: mainframe computer • 1980s: personal computer • 2000s: integrated solutions • 2020s: AI, Watson
![Page 18: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/18.jpg)
How about the leading Semiconductor companies?
![Page 19: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/19.jpg)
NVidia reinventing itself —2 times in 20 years
![Page 20: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/20.jpg)
“Bad money drives out good” in the desktop GPU market
![Page 21: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/21.jpg)
The rise of mobile computing, and how NVidia missed the boat!
![Page 22: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/22.jpg)
NVidia’s Tegra mobile processors never took off
then, the market saturated…
![Page 23: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/23.jpg)
NVidia not just survived. NVidia is thriving!
![Page 24: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/24.jpg)
Meet the new NVidia: Deep Learning, Deep Learning, and still, Deep Learning
![Page 25: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/25.jpg)
![Page 26: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/26.jpg)
The king is dead, long live the king!
![Page 27: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/27.jpg)
Now, again, do we want to do OEM/ODM forever?
Optimizing an old business model is just delaying its eventual death.
![Page 28: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/28.jpg)
Computer Vision, it can’t be that hard, right?
hmm… grayscale color can’t work alone… maybe color works better?
![Page 29: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/29.jpg)
Computer Vision, it can’t be that hard, right?
White and Gold or
Blue and Black?
The Dress 2015/02/26
![Page 30: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/30.jpg)
Computer Vision, it can’t be that hard, right?
![Page 31: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/31.jpg)
Even if we can auto-correct all lighting and color temperature
[w w w w] [w r r w] [w r r w] [w w w w]
and force all apples to be encoded as:
we’d still have all these “affine transformation” issues:
![Page 32: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/32.jpg)
Even if lighting, color, affine transformation are not an issue
• Our 3D world can’t simply be represented by fixed 2D encoding:
![Page 33: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/33.jpg)
Brief History
Marvin Minsky
“In 1966, Minsky hired a first-year undergraduate student and assigned him a problem to solve over the summer: connect a television camera to a computer and get the machine to describe what it sees.”
Gerald SussmanThe student never worked on Computer Vision problems again.
![Page 34: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/34.jpg)
Brief History• 1960’s: interpretation of synthetic worlds • 1970’s: some progress on interpreting selected images • 1980’s: ANNs come and go; shift toward geometry and increased
mathematical rigor • 1990’s: face recognition; statistical analysis in vogue • 2000’s: broader recognition; large annotated datasets available; video
processing starts
Guzman ‘68 Ohta Kanade ‘78 Turk and Pentland ‘91
![Page 35: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/35.jpg)
What was in our arsenal?
• Image filters
• Feature descriptors
• Classifiers
![Page 36: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/36.jpg)
Filters: blurring
![Page 37: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/37.jpg)
Filters: sharpening
![Page 38: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/38.jpg)
Filters: Canny Edge Detector
![Page 39: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/39.jpg)
Filters: straight lines
![Page 40: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/40.jpg)
Features: a compact and (hopefully) invariant representation
![Page 41: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/41.jpg)
Features: Gabor
![Page 42: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/42.jpg)
Features: Harris Corners
![Page 43: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/43.jpg)
Features: Laplacian of Gaussian (LoG; scale detection)
![Page 44: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/44.jpg)
Features: OrientationHow to compute the rotation?
Create edge orientation histogram and find peak.
![Page 45: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/45.jpg)
Features: SIFT
![Page 46: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/46.jpg)
Features: SIFT
![Page 47: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/47.jpg)
Classifier Training in Machine Learning
Classification Clustering
Regression DimensionReduction
supervised unsupervised
cont
inuo
usdi
scre
te
![Page 48: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/48.jpg)
Classifiers: SVM
![Page 49: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/49.jpg)
Classifiers: Ensemble
![Page 50: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/50.jpg)
Classifiers: Random Fields
![Page 51: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/51.jpg)
Classifiers: Deformable Parts Model (DPM)
![Page 52: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/52.jpg)
Meta-Learning• Different use
cases calls for different ML algorithms.
• Meta-Learning: learning how to learn.
• Requires plenty of domain-specific know-how.
![Page 53: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/53.jpg)
Neural Network (NN) Why didn’t it work; why now?
• MNIST digit data 28x28 • LeCunn’s 3 layer NN:
1170 variables. • Require tens of
thousands of samples. • Only learn simple line/
curve combinations
![Page 54: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/54.jpg)
AI Winter (1970-1980, 1990-2000) • Early NN problems:
• redundant structure, • slow learning speed • need too much data • bad learning
stability.
![Page 55: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/55.jpg)
What’s in a NN
( )zσ+
( )zσ+
( )zσ+
( )zσ+Input
weights
bias
activation function
![Page 56: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/56.jpg)
NN breakthroughs since 1970’s 1. Better Network Structure
• Convolutional Neural Network greatly reduces the number of variables in NN’s designed for images and videos. —> Improved convergence speed, reduced data requirements.
Upper-left corner Bird Beak Detector
Center Bird Beak Detector
Almost identical, can be shared across regions
![Page 57: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/57.jpg)
NN breakthroughs since 1970’s 1. Network Structure
![Page 58: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/58.jpg)
NN breakthroughs since 1970’s 2. Improved Activation Functions
Large
Small
1x
2x
……
Nx
……
……
……
……
……
……
……
y1
y2
yM
![Page 59: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/59.jpg)
NN breakthroughs since 1970’s 3. Effective Backpropagation
w1
w2
Clipping
[Razvan Pascanu, ICML’13]
![Page 60: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/60.jpg)
NN breakthroughs since 1970’s 4. Efficient Training Methods
• Mini-batch
• Adaptive Learning Rate
• Dropout, Batch-normalization
minibatchminibatch
1 epoch
![Page 61: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/61.jpg)
Deep Neural Networks (DNN) way more complex and capable!
![Page 62: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/62.jpg)
What do DNNs learn?
• Neurons act like “custom-trained filters”; react to very different visual cues, depending on data.
![Page 63: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/63.jpg)
What do DNNs learn?
• Neurons act like “custom-trained filters”; react to very different visual cues, depending on data.
![Page 64: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/64.jpg)
• Does not “memorize” millions of viewed images. • Extracts greatly reduced number of features that
are vital to classify different classes of data. • Classifying data becomes a simple task when
the features measured are “”good”.
What do DNNs learn?
![Page 65: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/65.jpg)
Mature/Maturing Computer Vision Applications
![Page 66: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/66.jpg)
• Final inspection cells • Robot guidance and
checking orientation of components
• Packaging Inspection • Medical vial inspection • Food pack checks • Verifying engineered
components[5] • Wafer Dicing • Reading of Serial
Numbers • Inspection of Saw
Blades
• Inspection of Ball Grid Arrays (BGAs)
• Surface Inspection • Measuring of Spark
Plugs • Molding Flash Detection • Inspection of Punched
Sheets • 3D Plane
Reconstruction with Stereo
• Pose Verification of Resistors
• Classification of Non-Woven Fabrics
1970s-now: Machine Vision for Industrial Inspection
• Automated Train Examiner (ATEx) Systems
• Automatic PCB inspection
• Wood quality inspection
• Final inspection of sub-assemblies
• Engine part inspection • \Label inspection on
products • Checking medical
devices for defects
![Page 67: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/67.jpg)
Industrial Inspection: turbofan jet engine blade maintenance• Some seemingly daunting
machine vision tasks actually works with relatively simple image processing algorithms.
![Page 68: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/68.jpg)
Industrial Inspection: Cognex Omniview
![Page 69: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/69.jpg)
Industrial Inspection: Cognex Omniview
![Page 70: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/70.jpg)
License Plate Recognition (1979-now)
![Page 71: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/71.jpg)
License Plate Readers with Text Detection and Neural Networks
![Page 72: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/72.jpg)
Biometrics
![Page 73: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/73.jpg)
Automated Fingerprint Identification (1970s-now)
![Page 74: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/74.jpg)
Face Recognition (1990s-now)
• Face Detection (Viola and Jones, 2001)
• Face Verification (1:1) v.s. Identification (1:N)
![Page 75: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/75.jpg)
Face Verification and Identification, Labeled Faces in the Wild (LFW)
Recognition Accuracy: • 1 to 1: 99%+ • 1 to 100: 90% • 1 to 10,000:
50%-70%. • 1 to 1M: 30%.
LFW dataset, common FN↑, FP↓
![Page 76: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/76.jpg)
![Page 77: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/77.jpg)
Sports—NFL first down line (1995-now)
![Page 78: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/78.jpg)
Sports—NFL first down line
minus
equals
![Page 79: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/79.jpg)
3D Reconstruction(As old as CV; became practical since SIFT)
![Page 80: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/80.jpg)
3D Reconstruction with Feature Matching, Structure from Motion
![Page 81: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/81.jpg)
3D Reconstruction with Feature Matching, Structure from Motion
![Page 82: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/82.jpg)
Image Panoramas (1980s - now)
![Page 83: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/83.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 84: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/84.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 85: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/85.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 86: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/86.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 87: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/87.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 88: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/88.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 89: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/89.jpg)
Solving Panorama Problem with Markov Random Fields
Input:
![Page 90: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/90.jpg)
Solving Panorama Problem with Markov Random Fields
![Page 91: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/91.jpg)
Solving Panorama Problem with Markov Random Fields
ICM (Iterated Conditional Modes), 1986
![Page 92: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/92.jpg)
Solving Panorama Problem with Markov Random Fields
Belief Propagation (1980-2000)
![Page 93: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/93.jpg)
Solving Panorama Problem with Markov Random Fields
Graph-Cuts (alpha expansion), 2001
![Page 94: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/94.jpg)
Photosynthesis
![Page 95: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/95.jpg)
Solving Photosynthesis Problems with Alpha-matting (2000s-now)
![Page 96: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/96.jpg)
Object Detection & Classification state-of-the-art
• ImageNet Large Scale Visual Recognition Challenge (ILSVRC) • 1000+ classes, 1.2M images.
0
0.125
0.25
0.375
0.5
11 12 13 14 11 12 13 14classification
errorclassification
+localization error
![Page 97: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/97.jpg)
Image Scene Classification• MIT Places 401
dataset.
• top-5 accuracy rates >80%.
![Page 98: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/98.jpg)
Self-driving cars (2000s-now)
![Page 99: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/99.jpg)
DARPA Grand Challenge (2005)
![Page 100: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/100.jpg)
2005 winner, Stanley (Stanford), 3mph through desert
![Page 101: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/101.jpg)
![Page 102: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/102.jpg)
DARPA Urban Challenge (2007)
![Page 103: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/103.jpg)
2007 winner, Boss (CMU), 13mpg through the city
![Page 104: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/104.jpg)
Self Driving Cadillac, US congressman to airport, 2013
![Page 105: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/105.jpg)
Google Self Driving Car, 2015
![Page 106: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/106.jpg)
Google Self-Driving Car, 2016
![Page 107: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/107.jpg)
Google Self-Driving Car, 2016
![Page 108: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/108.jpg)
NVidia Self Driving Car, 2016
![Page 109: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/109.jpg)
How did we come this far? Race car drivers know the trick
![Page 110: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/110.jpg)
Focus on Free Space / Drivable Area, not Obstacles!
![Page 111: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/111.jpg)
Up-and-coming Computer Vision
Applications
![Page 112: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/112.jpg)
Structure from X, Floored
![Page 113: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/113.jpg)
Structure from X, PIX4D
![Page 114: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/114.jpg)
Object Recognition Blue River Technology
![Page 115: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/115.jpg)
Augmented Reality Magic Leap
![Page 116: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/116.jpg)
IMRSV
![Page 117: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/117.jpg)
Retail Insights
Source: Prism Skylabs
![Page 118: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/118.jpg)
Other Applications in Business Intelligence
• Measure brand exposure. • Measure sponsorship effectiveness. • Loss prevention and retail layout optimization.
![Page 119: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/119.jpg)
How about Smart Surveillance?
![Page 120: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/120.jpg)
Exciting applications many of you might be
attempting to SOLVE!!!
![Page 121: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/121.jpg)
Problem Solving WorkflowClassical Workflow: 1. Data collection 2. Feature Extraction 3. Dimension Reduction 4. Classifier (re)Design 5. Classifier Verification 6. Deploy
Modern Brute-force workflow 1. Data collection 2. Throw everything into a Deep Neural Network 3. Mommy, why doesn’t it work ???
![Page 122: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/122.jpg)
Classical Problem #1: Curse of Dimensionality
zesit
앉다
sentarse
• Number of Variables vs Number of Samples
Q. Who would make such naive mistakes? A. Many “newbies” repeatedly do so.
![Page 123: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/123.jpg)
Example 1-1: illegal parking detection
legal parking samples x100 illegal parking samples x100Let’s train a 150-layer Res-Net!!!What could possibly go wrong?
![Page 124: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/124.jpg)
Example 1-1: illegal parking detection
• Data: try cleaner data
• Feature: fine-tune with pre-trained model; don’t train from scratch
• Classifier overfitting: beware of statistical coincidences,
![Page 125: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/125.jpg)
Example 1-2: Smart Photo Album with Google Cloud Vision
![Page 126: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/126.jpg)
Example 1-2: Smart Photo Album with Google Cloud Vision
No effective distance measure for thousands, if not millions of dimensions (tags); would be
approximately zero most of the time.
![Page 127: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/127.jpg)
Classical Problem #2: Overfitting Data
• Make sure your deep learning algorithm is learning better features for data, not overfitting the data with complex classifiers.
![Page 128: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/128.jpg)
Deep Learning Cookbook
GoodResultsonTestingData?
GoodResultsonTrainingData?
YES
YES
Newactivationfunction
AdaptiveLearningRate
EarlyStopping
Regularization
Dropout
(credit: Prof. H.Y. Lee, NTU)
![Page 129: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/129.jpg)
Example: AOI breakthroughs with Deep Learning—Metal Inspection
D Weimer et al. 2017
![Page 130: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/130.jpg)
Example: AOI breakthroughs with Deep Learning—Textile Inspection
X
Funding Li et al. / IEEE Tran Automation Science and Engineer 2017 (to appear)
![Page 131: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/131.jpg)
![Page 132: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/132.jpg)
Example: AOI breakthroughs with Deep Learning—Laser Welding
Johannes Günther et al. / Procedia Technology 15 (2014) 474 – 483
![Page 133: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/133.jpg)
Example: AOI breakthroughs with Deep Learning—Laser Welding
Johannes Günther et al. / Procedia Technology 15 (2014) 474 – 483
![Page 134: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/134.jpg)
Example: AOI breakthroughs with Deep Learning—Serial Number Processing
S. N. Lim et al. / GE Global Research
![Page 135: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/135.jpg)
S. N. Lim et al. / GE Global Research
Example: AOI breakthroughs with Deep Learning—Serial Number Processing
![Page 136: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/136.jpg)
Example: AOI breakthroughs with Deep Learning—Corrosion Detection
S. N. Lim et al. / GE Global Research
![Page 137: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/137.jpg)
Example: Dermatologist-level Skin Cancer Diagnosis with DNN+Smartphones
• 5.4M cancer cases, 58M pre-cancer cases diagnosed every year in the US.
(Andre Esteva, Sebastian Thrun, 2017)
![Page 138: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/138.jpg)
Example: Dermatologist-level Skin Cancer Diagnosis with DNN+Smartphones
![Page 139: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/139.jpg)
Example: Dermatologist-level Skin Cancer Diagnosis with DNN+Smartphones
![Page 140: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/140.jpg)
Example: Hippocampus Segmentation in 7T MR Images
(Dinggang Shen, 2017)
![Page 141: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/141.jpg)
(Dinggang Shen, 2017)
Example: Hippocampus Segmentation in 7T MR Images
![Page 142: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/142.jpg)
(Dinggang Shen, 2017)
Example: Hippocampus Segmentation in 7T MR Images
![Page 143: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/143.jpg)
Example: Histopathological Image Classification w. DNN
Microscopic view of Breast malignant tumor
40x 100x
200x 400x(FA Spanhol, IJCNN 2016)
![Page 144: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/144.jpg)
Example: Histopathological Image Classification w. DNN
![Page 145: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/145.jpg)
Example: Histopathological Image Classification w. DNN
![Page 146: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/146.jpg)
![Page 147: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/147.jpg)
Example: DNN for Plant Disease Detection
(S Mohanty, 2016)
![Page 148: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/148.jpg)
Example: DNN for Plant Disease Detection
![Page 149: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/149.jpg)
Example: DNN for Plant Disease Detection
![Page 151: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/151.jpg)
Appendix 1: Startups• A company, partnership, or temporary
organization designed to search for a new, repeatable and scalable business model.
![Page 152: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/152.jpg)
Your Idea• Are you passionate about it? • Is it disruptive enough? • What is your business plan?
• What is it? • Can it make money? • What is the future of the idea?
• What is your competitive advantage? • How do you build up your entry barrier?
![Page 153: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/153.jpg)
A minimal startup team
• A hacker
• A hustler
• A hipster
![Page 154: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/154.jpg)
![Page 155: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/155.jpg)
Startup Timeline
![Page 156: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/156.jpg)
Prototype• Hack out a prototype
• Spend 2-10 weeks max.
• Investors are much more likely to fund you if you have a minimal initial version of your idea.
• Hackathons are a good place to start.
• Iteratively improve the prototype
![Page 157: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/157.jpg)
Money!
![Page 158: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/158.jpg)
![Page 159: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/159.jpg)
Buildup your entry barrier!
• Market (users)
• Speed
• Team
• Technology
![Page 160: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/160.jpg)
Building entry barrier with Technology!!
![Page 161: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/161.jpg)
Angel.co
![Page 162: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/162.jpg)
Appendix 2: My humble attempts at putting the latest Computer
Vision algorithms to work
![Page 163: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/163.jpg)
Intrinsic Imaging at Tandent Vision Science
Computer Vision would be half-solved without shadows!
LightOriginal Image Surface
![Page 164: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/164.jpg)
Tandent Lightbrush
Video Tutorial for Tandent Lightbrush: https://vimeo.com/47009123
![Page 165: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/165.jpg)
![Page 166: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/166.jpg)
Issues• Highly anticipated, highly acclaimed, but small
crowd at $500 a license.
• Adobe Photoshop monopoly and the “not invented here” syndrome.
• Adobe’s arch-rival, Corel (Corel Draw, Paint Shop Pro, Ulead PhotoImpact) was DYING and asked too much from the botched deal.
![Page 167: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/167.jpg)
Have fun scribbling out your shadows in photoshop!
Poor Bob from Adobe wasted 9 minutes removing just 1 shadow
![Page 168: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/168.jpg)
Intrinsic Imaging for improving the RGB signal in autonomous driving
![Page 169: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/169.jpg)
Intrinsic Imaging’s other applications
![Page 170: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/170.jpg)
Retrospect
• 20 researchers burned 25 million in 8 years; investors got 50 patents in return, period.
• Overestimated the total addressable market size, in a market with existing monopoly.
• Many missed opportunities. Counterexample of the lean startup model.
![Page 171: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/171.jpg)
Some SfM, SLAM startups
![Page 172: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/172.jpg)
Satellite/Aerial Imagery Analysis
• 40cm resolution at 30fps for 90 sec for any location on earth. • One LEO satellite revisits any place on Earth every 3 days. • Need 24 satellites to revisit any place on Earth every 3 hours.
![Page 173: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/173.jpg)
Challenges for Single satellite depth estimation and 3D reconstruction
• At 30fps, a LEO satellite travels 250m between two consecutive frames —> theoretically sufficient for cm-level depth estimation.
• Sources of Noise: • Camera distortions • Atmospheric Disturbance • Ground vegetation • Sub-pixel sampling noise
1 2
![Page 174: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/174.jpg)
What happened?
• B2B customers takes too long to strike deals.
• Google ate us alive in just 3 months, while we were still pitching for VC-funding with our prototype.
![Page 175: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/175.jpg)
Visual Search at Nervve
![Page 176: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/176.jpg)
Retrospect• Growth pains expanding from intelligence
community clients to advertisement clients. • Forming the right team of engineers and
researchers and moving at the right pace. • For any Computer Vision/Machine Learning
company: • Researchers that cannot program—> OUT • Engineers that don’t know math —> OUT
![Page 177: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/177.jpg)
Visual Search, Simply Smarter
![Page 178: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/178.jpg)
Once in a lifetime opportunity in China’s video streaming market
![Page 179: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/179.jpg)
What do we need?
Face MotionImage scene Text Audio Object
Semantics
![Page 180: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/180.jpg)
Viscovery VDS (Video Discovery Service)
![Page 181: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/181.jpg)
Viscovery VDS (Video Discovery Service)
![Page 182: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/182.jpg)
Viscovery VDS (Video Discovery Service)
![Page 183: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/183.jpg)
Challenges Encountered Along the Way
• From Product Recognition in Images, to Face, Logo, Object, Scene recognition in Videos. • Number of Categories • Recognition Accuracy • Recognition Speed
• System Architecture
• Business Model
![Page 184: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/184.jpg)
Viscovery’s Edge• Market: first mover’s advantage in China’s video
streaming market. • Speed: we built the whole VDS thing in a few months! • Team: You! Seriously! • Technology:
• Depth • Breadth • Cloud • Customizability • Self-Learning
![Page 185: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/185.jpg)
Life is not all rosy at startups
• High Risk, High Pressure, High Uncertainty!
• Resources are scarce, but you MUST DELIVER!
• Forming your all-star team is not that easy…
• Focus, and persistence.
![Page 186: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/186.jpg)
Appendix 3: What can Taiwan’s academia do to help bridge the gap?
HMM….
![Page 187: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/187.jpg)
Academia
IndustryGeneral Public
reputation and policy support
improved living standards
students
opportunity
well-trained graduates
grants and collaborations
A healthy cycle
![Page 188: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/188.jpg)
Academia
IndustryGeneral Public
unsupportive policies
stagnant wages
useless education
unemployable graduates
A vicious cycle
no grants
no students
![Page 189: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/189.jpg)
Where should we start? Maybe with a few more stories.
![Page 190: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/190.jpg)
Where should we start? Maybe with a few more stories.
![Page 191: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/191.jpg)
Where should we start? Maybe with a few more stories.
![Page 192: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/192.jpg)
The Goldilocks zone of innovation
![Page 193: Practical computer vision-- A problem-driven approach towards learning CV/ML/DL](https://reader030.vdocuments.site/reader030/viewer/2022020917/5a64eee97f8b9aa2548b4ad1/html5/thumbnails/193.jpg)
The Goldilocks zone of innovation
Business Relevance
Academic Relevance
plentiful resources; hierarchical organization
lack of resources; responsive organization
traditional corporations talking “innovation”
corporate research
startups struggling to survive
academic spinoffs
MSR