about this specialization - thecads.org · about this specialization ... learn to set up a machine...
TRANSCRIPT
About This Specialization
If you want to break into AI, this Specialization will help you do so. Deep Learning is one of the most highly sought after skills in tech. We will help you become good at Deep Learning.
In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, Batch-Norm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach.
You will also hear from many top leaders in Deep Learning, who will share with you their personal stories and give you career advice.
AI is transforming multiple industries. After finishing this specialization, you will likely find creative ways to apply it to your work.
We will help you master Deep Learning, understand how to apply it, and build a career in AI.
Week 1Introduction to deep learningBe able to explain the major trends driving the rise of deep learning, and understand where and how it is applied today.
Video · Welcome
Video · What is a neural network?
Video · Supervised Learning with Neural Networks
Video · Why is Deep Learning taking o�?
Video · About this Course
Reading · Frequently Asked Questions
Video · Course Resources
Reading · How to use Discussion Forums
Quiz · Introduction to deep learning
Video · Geo�rey Hinton interview
Week 2Neural Networks BasicsLearn to set up a machine learning problem with a neural network mindset. Learn to use vectorization to speed up your models.
Video · Binary Classification
Video · Logistic Regression
Video · Logistic Regression Cost Function
Video · Gradient Descent
Video · Derivatives
Video · More Derivative Examples
Video · Computation graph
Video · Derivatives with a Computation Graph
Video · Logistic Regression Gradient Descent
Video · Gradient Descent on m Examples
Video · Vectorization
Video · More Vectorization Examples
Video · Vectorizing Logistic Regression
Video · Vectorizing Logistic Regression's Gradient Output
Video · Broadcasting in Python
Video · A note on python/numpy vectors
Video · Quick tour of Jupyter/iPython Notebooks
Video · Explanation of logistic regression cost function (optional)
Video · Vectorization
Video · More Vectorization Examples
Video · Vectorizing Logistic Regression
Video · Vectorizing Logistic Regression's GradientOutput
Video · Broadcasting in Python
Video · A note on python/numpy vectors
Video · Quick tour of Jupyter/iPython Notebooks
Video · Explanation of logistic regression cost function (optional)
Quiz · Neural Network Basics
Reading · Deep Learning Honor Code
Reading · Programming Assignment FAQ
Other · Python Basics with numpy (optional)
Practice Programming Assignment · Python Basicswith numpy (optional)
Other · Logistic Regression with a Neural Network mindset
Programming Assignment · Logistic Regression with a Neural Network mindset
Video · Pieter Abbeel interview
Week 3Shallow neural networksLearn to build a neural network with one hidden layer, using forward propagation and backpropagation
Video · Neural Networks Overview
Video · Neural Network Representation
Video · Computing a Neural Network's Output
Video · Vectorizing across multiple examples
Video · Explanation for Vectorized Implementation
Video · Activation functions
Video · Why do you need non-linear activationfunctions?
Video · Derivatives of activation functions
Video · Gradient descent for Neural Networks
Video · Backpropagation intuition (optional)
Video · Random Initialization
Quiz · Shallow Neural Networks
Other · Planar data classification with a hidden layer
Programming Assignment · Planar data classification with a hidden layer
Video · Ian Goodfellow interview
Week 4Deep Neural NetworksUnderstand the key computations underlying deep learning, use them to build and train deep neural networks, and apply it to computer vision.
Video · Deep L-layer neural network
Video · Forward Propagation in a Deep Network
Video · Getting your matrix dimensions right
Video · Why deep representations?
Video · Building blocks of deep neural networks
Video · Forward and Backward Propagation
Video · Parameters vs Hyperparameters
Video · What does this have to do with the brain?
Quiz · Key concepts on Deep Neural Networks
Other · Building your Deep Neural Network: Step by Step
Programming Assignment · Building your deep neural network: Step by Step
Other · Deep Neural Network - Application
Programming Assignment · Deep Neural NetworkApplication
� � � � � � �� ��
Improving Deep Neural Networks: Hyperparameter tuning, Regularization and OptimizationUpcoming Session: Dec 19
Commitment
Subtitles
3 weeks, 3-6 hours per week
English, Chinese (Traditional), Chinese (Simplified)
About the CourseThis course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process being a black box, you will understand what drives performance, and be able to more systematically get good results. You will also learn TensorFlow.
After 3 weeks, you will: - Understand industry best-practices for building deep learning applications. - Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking, - Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence. - Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance- Be able to implement a neural network in TensorFlow.
This is the second course of the Deep Learning Specialization.
Week 1Practical aspects of Deep Learning
Video · Train / Dev / Test sets
Video · Bias / Variance
Video · Basic Recipe for Machine Learning
Video · Regularization
Video · Why regularization reduces overfitting?
Video · Dropout Regularization
Video · Understanding Dropout
Video · Other regularization methods
Video · Normalizing inputs
Video · Vanishing / Exploding gradients
Video · Weight Initialization for Deep Networks
Video · Numerical approximation of gradients
Video · Gradient checking
Video · Gradient Checking Implementation Notes
Quiz · Practical aspects of deep learning
Other · Initialization
Programming Assignment · Initialization
Other · Regularization
Programming Assignment · Regularization
Other · Gradient Checking
Programming Assignment · Gradient Checking
Video · Yoshua Bengio interview
Week 1Practical aspects of Deep Learning
Video · Train / Dev / Test sets
Video · Bias / Variance
Video · Basic Recipe for Machine Learning
Video · Regularization
Video · Why regularization reduces overfitting?
Video · Dropout Regularization
Video · Understanding Dropout
Video · Other regularization methods
Video · Normalizing inputs
Video · Vanishing / Exploding gradients
Video · Weight Initialization for Deep Networks
Video · Numerical approximation of gradients
Video · Gradient checking
Video · Gradient Checking Implementation Notes
Quiz · Practical aspects of deep learning
Other · Initialization
Programming Assignment · Initialization
Other · Regularization
Programming Assignment · Regularization
Other · Gradient Checking
Programming Assignment · Gradient Checking
Video · Yoshua Bengio interview
Week 2Optimization algorithms
Video · Mini-batch gradient descent
Video · Understanding mini-batch gradient descent
Video · Exponentially weighted averages
Video · Understanding exponentially weighted averages
Video · Bias correction in exponentially weighted averages
Video · Gradient descent with momentum
Video · RMSprop
Video · Adam optimization algorithm
Video · Learning rate decay
Video · The problem of local optima
Quiz · Optimization algorithms
Other · Optimization
Programming Assignment · Optimization
Video · Yuanqing Lin interview
Video · Tuning process
Video · Using an appropriate scale to pick hyperparameters
Video · Hyperparameters tuning in practice: Pandas vs. Caviar
Video · Normalizing activations in a network
Video · Fitting Batch Norm into a neural network
Video · Why does Batch Norm work?
Video · Batch Norm at test time
Video · Softmax Regression
Video · Training a softmax classifier
Video · Deep learning frameworks
Video · TensorFlow
Quiz · Hyperparameter tuning, Batch Normalization, Programming Frameworks
Other · Tensorflow
Programming Assignment · Tensorflow
Week 3Hyperparameter tuning, Batch Normalization and Programming Frameworks
� � � � � � �� ��
Structuring Machine Learning Projects
Upcoming Session: Dec 18
Commitment
Subtitles
2 weeks of study, 3-4 hours/week
English
About the CourseYou will learn how to build a successful machine learning project. If you aspire to be a technical leader in AI, and know how to set direction for your team's work, this course will show you how.
Much of this content has never been taught elsewhere, and is drawn from my experience building and shipping many deep learning products. This course also has two "flight simulators" that let you practice decision-making as a machine learning project leader. This provides "industry experience" that you might otherwise get only after years of ML work experience.
After 2 weeks, you will: - Understand how to diagnose errors in a machine learning system, and - Be able to prioritize the most promising directions for reducing error- Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance- Know how to apply end-to-end learning, transfer learning, and multi-task learning
I've seen teams waste months or years through not understanding the principles taught in this course. I hope this two week course will save you months of time.
This is a standalone course, and you can take this so long as you have basic machine learning knowledge. This is the third course in the Deep Learning Specialization.
Week 1ML Strategy (1)
Video · Why human-level performance?
Video · Avoidable bias
Video · Understanding human-level performance
Video · Surpassing human-level performance
Video · Improving your model performance
Reading · Machine Learning flight simulator
Quiz · Bird recognition in the city of Peacetopia (case study)
Video · Andrej Karpathy interview
Video · Why ML Strategy
Video · Orthogonalization
Video · Single number evaluation metric
Video · Satisficing and Optimizing metric
Video · Train/dev/test distributions
Video · Size of the dev and test sets
Video · When to change dev/test sets and metrics
Week 1ML Strategy (2)
Video · Carrying out error analysis
Video · Cleaning up incorrectly labeled data
Video · Build your first system quickly, then iterate
Video · Training and testing on different distributions
Video · Bias and Variance with mismatched data distributions
Video · Addressing data mismatch
Video · Transfer learning
Video · Multi-task learning
Video · What is end-to-end deep learning?
Video · Whether to use end-to-end deep learning
Quiz · Autonomous driving (case study)
Video · Ruslan Salakhutdinov interview
� � � � � � �� ��
Convolutional Neural Networks
Upcoming Session: Dec 18
Commitment
Subtitles
4 weeks of study, 4-5 hours/week
English
About the CourseThis course will teach you how to build convolutional neural networks and apply it to image data. Thanks to deep learning, computer vision is working far better than just two years ago, and this is enabling numerous exciting applications ranging from safe autonomous driving, to accurate face recognition, to automatic reading of radiology images.
You will:- Understand how to build a convolutional neural network, including recent variations such as residual networks.- Know how to apply convolutional networks to visual detection and recognition tasks.- Know to use neural style transfer to generate art.- Be able to apply these algorithms to a variety of image, video, and other 2D or 3D data.
This is the fourth course of the Deep Learning Specialization
Video · Computer Vision
Video · Edge Detection Example
Video · More Edge Detection
Video · Padding
Video · Strided Convolutions
Video · Convolutions Over Volume
Video · One Layer of a Convolutional Network
Video · Simple Convolutional Network Example
Video · Pooling Layers
Video · CNN Example
Video · Why Convolutions?
Quiz · The basics of ConvNets
Other · Convolutional Model: step by step
Programming Assignment · Convolutional Model:step by step
Other · Convolutional Model: application
Programming Assignment · Convolutional model: application
Week 1Foundations of Convolutional Neural NetworksLearn to implement the foundational layers of CNNs (pooling, convolutions) and to stack them properly in a deep network to solve multi-class image classification problems.
Video · Why look at case studies?
Video · Classic Networks
Video · ResNets
Video · Why ResNets Work
Video · Networks in Networks and 1x1 Convolutions
Video · Inception Network Motivation
Video · Inception Network
Video · Using Open-Source Implementation
Video · Transfer Learning
Video · Data Augmentation
Video · State of Computer Vision
Quiz · Deep convolutional models
Other · Keras Tutorial - The Happy House (not graded)
Other · Residual Networks
Programming Assignment · Residual Networks
Week 2Deep convolutional models: case studiesLearn about the practical tricks and methods used in deep CNNs straight from the research papers..
Video · Object Localization
Video · Landmark Detection
Video · Object Detection
Video · Convolutional Implementation of Sliding Windows
Video · Bounding Box Predictions
Video · Intersection Over Union
Video · Non-max Suppression
Video · Anchor Boxes
Video · YOLO Algorithm
Video · (Optional) Region Proposals
Quiz · Detection algorithms
Other · Car detection with YOLOv2
Programming Assignment · Car detection with YOLOv2
Week 3Object detectionLearn how to apply your knowledge of CNNs to one of the toughest but hottest field of computer vision: Object detection.
Video · What is face recognition?
Video · One Shot Learning
Video · Siamese Network
Video · Triplet Loss
Video · Face Verification and Binary Classification
Video · What is neural style transfer?
Video · What are deep ConvNets learning?
Video · Cost Function
Video · Content Cost Function
Video · Style Cost Function
Video · 1D and 3D Generalizations
Quiz · Special applications: Face recognition & Neuralstyle transfer
Other · Art generation with Neural Style Transfer
Programming Assignment · Art generation with Neural Style Transfer
Other · Face Recognition for the Happy House
Programming Assignment · Face Recognition for the Happy House
Week 4Special applications: Face recognition & Neural style transferDiscover how CNNs can be applied to multiple fields, including art generation and face recognition. Implement your own algorithm to generate art and recognize faces!.
� � � � � � �� ��
Neural Networks and Deep Learning
Upcoming Session: Dec 19
Commitment
Subtitles
4 weeks of study, 3-6 hours a week
4 weeks of study, 3-6 hours a week
About the CourseIf you want to break into cutting-edge AI, this course will help you do so. Deep learning engineers are highly sought after, and mastering deep learning will give you numerous new career opportunities. Deep learning is also a new "superpower" that will let you build AI systems that just weren't possible a few years ago.
In this course, you will learn the foundations of deep learning. When you finish this class, you will:- Understand the major technology trends driving Deep Learning- Be able to build, train and apply fully connected deep neural networks - Know how to implement efficient (vectorized) neural networks - Understand the key parameters in a neural network's architecture
This course also teaches you how Deep Learning actually works, rather than presenting only a cursory or surface-level description. So after completing it, you will be able to apply deep learning to a your own applications. If you are looking for a job in AI, after this course you will also be able to answer basic interview questions.
This is the first course of the Deep Learning Specialization.
� � � � � � �� ��
Sequence Models
Starts December 2017
Subtitles English
About the CourseThis course will teach you how to build models for natural language, audio, and other sequence data. Thanks to deep learning, sequence algorithms are working far better than just two years ago, and this is enabling numerous exciting applications in speech recognition, music synthesis, chatbots, machine translation, natural language understanding, and many others.
You will:- Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs.- Be able to apply sequence models to natural language problems, including text synthesis. - Be able to apply sequence models to audio applications, including speech recognition and music synthesis.
This is the fifth and final course of the Deep Learning Specialization.