Upload File

digits- interactive deep learning gpu training system · digits visualize dnn topology and how...

  • Home
  • Documents
  • DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU
63
Pradeep Kumar Gupta April 2016 DIGITS- Interactive Deep Learning GPU Training System

Upload: others

Post on 23-Aug-2020

12 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

Pradeep Kumar Gupta

April 2016

DIGITS- Interactive Deep Learning GPU Training System

Page 2: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

2

Agenda

Ø Introduction to DIGITS

Ø Install and start DIGITS sever

Ø CREATING DATASETS

Ø TRAIN A NETWORK

Ø MODIFY YOUR NETWORK

Ø Testing/Classification with DIGITS

Ø DIGITS Features at a Glance

Ø References

Page 3: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

3

“ DIGITS makes it way easier to design the best network for the job. The DIGITS interface makes it super easy to track key diagnostics during training. The field will definitely benefit from having tools like this for configuration and

introspection”

— Simon Osindero, AI Architect at Flickr

Page 4: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

4

INTRODUCTION TO DIGITS

Page 5: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

5

Process Data Configure DNN VisualizationMonitor Progress

Interactive Deep Learning GPU Training System

NVIDIA DIGITS

Page 6: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

6

NVIDIA DIGITS

Data Scientists & Researchers:

• Quickly design the best deep neural network (DNN) for your data

• Monitor DNN training quality in real-time

• Manage training of many DNNs in parallel on multi-GPU systems, and multi-GPU training

Who is DIGITS for?

54.69.199.108

Page 7: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

7

DIGITS

Available at http://developer.nvidia.com/digits

Free to use, Source Code available at Github, latest branch v3.0 https://github.com/NVIDIA/DIGITS

Current release supports classification on images

Future versions: More problem types and data formats (video, speech)

Deep Learning GPU Training System

Page 8: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

8

DIGITS

Visualize DNN topology and how training data activates your network

Manage training of many DNNs in parallel on multi-GPU systems

Simple setup and launch

Import a wide variety of image formats and sources

Monitor network training in real-time

Open source, so DIGITS can be customized and extended as needed

Key Features

Page 9: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

9

Tree

Cat

Dog

What is Deep Learning Software?

“turtle”

Forward Propagation

Compute weight update to nudgefrom “turtle” towards “dog”

Backward Propagation

Trained NeuralNet Model

“cat”

Repeat

Training

Inference

Page 10: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

10

Deep Learning

Creating a Dataset

Define the Network or use existing

Choose a given Framework

Selecting a preconfigured (“standard”) network – LeNet, AlexNet, GoogleNet

Previous network

Custom network

Training a Model

Classification

Steps with DIGITS

Page 11: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

11

Install and Start DIGITS Server

Page 12: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

12

DIGITS Installation

Installation with Pre-Built Packages

Deb packages are provided for easy installation on Ubuntu 14.04.

Packages are provided for major releases, but not minor ones (i.e. v3.0 and v4.0, but not v3.1).

Download the web installer - https://developer.nvidia.com/digits

Two Ways

Page 13: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

13

DIGITS Installation

Installation with Source

Get the latest branch from GitHub

Latest features but carries risk of untested features

Build NVIDIA Caffe branch - https://github.com/NVIDIA/caffe

Download DIGITS from github –https://github.com/NVIDIA/DIGITS

Two Ways

Page 14: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

14NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

DIGITS Installation

Hardware/software recommendations

GPU(s) with compute capabilities >=3.0 for cuDNN support

Ubuntu 14.04, CentOS

HW and SW

Page 15: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITS

Start DIGITS by

http://localhost/ (if installed from Deb packages),

http://localhost:5000/ (if using digits-devserver) or

http://localhost:34448/ (if using digits-server)

Main Console

Page 16: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITSLOGIN

Page 17: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

CREATING DATASETS

Page 18: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

18

NVIDIA DIGITS Creating your Dataset

Main Console

Page 19: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

19

NVIDIA DIGITSCreating your Dataset

In the Datasets section on the left side of the page, click on the blue Images button and select Classification which will take you to the "New Image Classification Dataset" page

Main Console

Page 20: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

20

NVIDIA DIGITS

Use the following command to download the MNIST dataset (for Deb package installations, the script is at/usr/share/digits/tools/download_data/main.py):

$ tools/download_data/main.py mnist ~/mnistDownloading url=http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz ...

Downloading url=http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz ...

Downloading url=http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz ...

Downloading url=http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz ...

Uncompressing file=train-images-idx3-ubyte.gz ...

Uncompressing file=train-labels-idx1-ubyte.gz ...

Uncompressing file=t10k-images-idx3-ubyte.gz ...

Uncompressing file=t10k-labels-idx1-ubyte.gz ...

Reading labels from /home/username/mnist/train-labels.bin ...

Reading images from /home/username/mnist/train-images.bin ...

Reading labels from /home/username/mnist/test-labels.bin ...

Reading images from /home/username/mnist/test-images.bin ...

Dataset directory is created successfully at '/home/username/mnist'

Done after 16.722807169 seconds.

Download MNIST dataset

Page 21: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

21

NVIDIA DIGITS

While the model creation job is running, you should see the expected completion time on the right side

When Model creation is done, you can also see completion time and duration

Users may download a copy of .txt files for reference

Creating your Dataset

Page 22: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

22

NVIDIA DIGITS Database results

*

• Validation data tests the performance of the network

• This data is only used for testing the generalization ability of the network

• Not used to teach/train network

• Prevents use of and identifies when network is overfit.

• In current example 17500 images used for validation.

• Training data is used to train our neural network.

• Teaches the network to classify object categories

• Training Data Set, Current example uses 52500 Images for training

Page 23: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITSDatabase Results

Database of Images can be explored via Exploring

“Explore DB” tab and you can see test images used for

training/validation

Page 24: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

24 24

DIGITS DEMO

Creating your Dataset

A new database isCreated as visible on Home Page of DIGITS

Page 25: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

25

TRAIN A NETWORK

Page 26: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

26

TRAINING

With DIGITS 3.0, two frameworks are integrated into DIGITS

- Caffe

- Torch

Choose Framework

Page 27: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

27

NVIDIA DIGITS

Training a network interface, please note

- Database selection

- Data Transformations

- Solver Options

- Different Network configurations

- Caffe

- Torch(experimental)

Train A network

Page 28: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

28

NVIDIA DIGITS

Select the Database

Provide a Model Name

Do any changes in Solver options

Start with a default Network like LeNet (Framework can be anyone Caffe/Torch)

Click on Create Button

Train a Network

Page 29: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

29

NVIDIA DIGITSTrain a Network-Advance Options

Page 30: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

LENETNetwork Configuration

Page 31: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

31

NVIDIA DIGITSTrain a Network-Advance Options

Page 32: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

32

NETWORK PARAMETERS

28

28 8

...8

88

...888

20@24x24

20@12x12

Page 33: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

33

NVIDIA DIGITS

• Training epochs – processing of all data

• Snapshot interval – saving trained network

• Validation interval – DNN test with the validation data

• Batch size – number of images processed together

• Solver type – SGD, ADAGRAD,NAG

• Learning rate and policy

Train a Network-Advance Options

Page 34: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

34

NVIDIA DIGITSSingle and multi-GPU training is easy

Singe GPU system Multiple GPU system

Select the number of GPUs you want to use.

Page 35: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITSTraining Results

If performance is poor, abort, modify, and retrain.

Monitor Accuracy

Monitor GPU, Memory usage and Temperature

Page 36: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITS

First Graph is about the Loss and Accuracy graphs

Second Graph is about the Learning rate, as the training progresses, learning rate goes down as model is getting more mature.

Training Results

Page 37: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

37

OVERFITTING AND UNDERFITTINGHow can I use DIGITS to tell me this is happening?

Overfitting

Loss continues to decrease with training

data but increases with validation

Loss is not increasing but the accuracy is not

improving

Underfitting

Validation data helps you identify when/if this occurs!

Page 38: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

38

Modifying your Network &Classification

Page 39: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

39

NVIDIA DIGITSSingle Image Classification

* Lab tasks

Page 40: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

40

SINGLE IMAGE CLASSIFICATION RESULTS

• Network response at each layer will display

• Visualize responses from different inputs

Page 41: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

41

NETWORK CONFIGURATIONVisualizing LetNet Network Responses

Page 42: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

42

NETWORK CONFIGURATIONS

• Modifying a network can improve performance

• There are many parameters - add or remove a layer, pooling, activation function, zero padding, increasing outputs

Modifying a Network

LetNetModification – added a convolutional layer and pooling

Page 43: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

43

MODIFYING YOUR NETWORKlayer {

name: "conv1"type: "Convolution"bottom: "scale"top: "conv1"param {

lr_mult: 1}param {

lr_mult: 2}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}bias_filler {

type: "constant"}

}}

8

28

28

Convolution

8

25

25 8

Zero padding, input is reduced by the 2*radius of the kernelInput[28x28]*filter[5x5]=FeatureMap[25x25]

Convolution filter5x5

Page 44: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

44

MODIFYING YOUR NETWORKlayer {

name: "conv1"type: "Convolution"bottom: "scale"top: "conv1"param {

lr_mult: 1}param {

lr_mult: 2}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}bias_filler {

type: "constant"}

}}

8

28

28

Convolution

8

25

25 8

Zero padding, input is reduced by the 2*radius of the kernelInput[28x28]*filter[5x5]=FeatureMap[25x25]

Convolution filter5x5

Page 45: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

45

MODIFYING YOUR NETWORKlayer {

name: "conv1"type: "Convolution"bottom: "scale"top: "conv1"param {

lr_mult: 1}param {

lr_mult: 2}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}bias_filler {

type: "constant"}

}}

8

28

28

Convolution

8

25

25 8

Zero padding, input is reduced by the 2*radius of the kernelInput[28x28]*filter[5x5]=FeatureMap[25x25]

Convolution filter5x5

Page 46: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

46

MODIFYING YOUR NETWORKlayer {

name: "conv1"type: "Convolution"bottom: "scale"top: "conv1"param {

lr_mult: 1}param {

lr_mult: 2}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}bias_filler {

type: "constant"}

}}

8

28

28

Convolution

8

25

25 8

Zero padding, input is reduced by the 2*radius of the kernelInput[28x28]*filter[5x5]=FeatureMap[25x25]

Convolution filter5x5

Page 47: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

47

NETWORK PARAMETERS

* Lab tasks

layer {name: "conv0"type: "Convolution"bottom: "data"top: "conv0"param {

lr_mult: 1.0}param {

lr_mult: 2.0}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}

layer {name: "relu0"type: "ReLU"bottom: "pool0"top: "pool0"}

layer {name: "pool0"type: "Pooling"bottom: "conv0"top: "pool0"pooling_param {

pool: MAXkernel_size: 2stride: 2

}}

Pooling/Subsampling ActivationConvolution

weight_filler {type: "xavier"

}bias_filler {

type: "constant"value: 0.9

}}}

Page 48: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

48

NVIDIA DIGITSModifying your Network

Edit your network

Page 49: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

49

MODIFYING YOUR NETWORKlayer {

name: "conv1"type: "Convolution"bottom: "scale"top: "conv1"param {

lr_mult: 1}param {

lr_mult: 2}convolution_param {

num_output: 20kernel_size: 5stride: 1weight_filler {

type: "xavier"}bias_filler {

type: "constant"}

}}

Edit your network

* Lab tasks

*

Input and output to the layer

Page 50: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

50

MODIFYING YOUR NETWORK

layer {name: "relu1"type: "ReLU"bottom: "conv1"top: "conv1"

}

Edit your network

* Lab tasks

*Input and output to

the layer

Page 51: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

51

MODIFYING YOUR NETWORK

Visualize Configuration Changes

Will return error

Page 52: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

52

ANOTHER WAY TO IMPROVE PERFORMANCE

• Sometimes training data is not a great representation of the field data

• MNIST data is grayscale, black text with white background

• Will these images to be classified correctly when the network is trained with this digit data?

Data Augmentation

Page 53: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

53

ANOTHER WAY TO IMPROVE PERFORMANCE

• Depending on the deployment scenario, simple modifications can be made to the training data to improve performance

• There are many ways to augment data

• Rotations, noise, color distortions, stretching, etc.

• Many ways to modify images – ImageMagick, Pillow, OpenCV

Data Augmentation

* Lab tasks

*

Page 54: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

54

USING AN AUGMENTED DATA SET

• train.txt

/home/user/train/0/0_1.jpg 0

/home/user/train/0/0_1_invert.jpg 0

/home/user/train/5/5_1.jpg 5

/home/user/train/5/5_1_invert.jpg 5

• val.txt

/home/user/mnist/val/7_1.jpg 7

/home/user/mnist/val/7_1_invert.jpg 7

Page 55: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

55

ANOTHER WAY TO IMPROVE PERFORMANCE

• Example augmentation - inverted copies of the input data

• Would a network trained with this data augmented, accurately classify these images?

Data Augmentation

Page 56: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

56

DEPLOYING YOUR NETWORK

• Deploy in the cloud

• Deploy on a mobile device

Download the model you want to use

Tegra

Page 57: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

57

DEPLOYMENT WITH TEGRARapid Classification Anywhere

➢Flexible

➢Low Power

➢Easy to use

➢GPU accelerated

Build Caffe on your portable platformDownload your trained network from DIGITS

Page 58: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

58

DIGITS Features at a Glance

Page 59: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

NVIDIA DIGITS ROADMAP

Release 1(Q4 2014)

Release 2(Q2 2015)

Release 3(Q4 2015)

Release 4(Q2 2016)

Initial Release

• Image classification

• Layerwise

visualization

• Single-GPU tasks

Usability / Workflow

• Multi-GPU tasks

• Workload

management• Select GPU for task

• Visualize GPU

utilization

• Example inference

code

Improve Workflowfor Rapid Iteration

• Job Management

• Results Browser

• General Neural Nets

• Embedded SupportUnder NDA

Frameworks- Caffe

Frameworks- Multi-GPU Caffe

Frameworks- Torch Preview

Page 60: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

60

WHAT’S NEW IN DIGITS 3?

Train neural network models with Torch support (preview)

Save time by quickly iterating to identify the best model

Easily manage multiple jobs to optimize use of system resources

Active open source project with valuable community contributions

Screenshot of new DIGITS

DIGITS 3 Improves Training Productivity with Enhanced Workflows

developer.nvidia.com/digits

New Results Browser!

Page 61: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

61

NVIDIA DIGITS

• Where to get DIGITS

• Easy to use web installer https://developer.nvidia.com/digits

• github - https://github.com/NVIDIA/DIGITS

• Remember to install NVIDIA’s Caffe branch - https://github.com/NVIDIA/caffe

• User support

• DIGITS Users Google group - https://groups.google.com/forum/#!forum/digits-users

• For more information on getting started with DIGITS

• Parallel forall - http://devblogs.nvidia.com/parallelforall/easy-multi-gpu-deep-learning-digits-2/

• Getting started guide - https://github.com/NVIDIA/DIGITS/blob/master/docs/GettingStarted.md

Resources

Page 62: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

62

HANDS-ON LAB

1. Create an account at nvidia.qwiklab.com

2. Go to “Getting Started with DIGITS” lab at bit.ly/dlnvlab2

3. Start the lab and enjoy!

▪ Only requires a supported browser, no NVIDIA GPU necessary!

▪ Lab is free until end of this Deep Learning Lab series

Page 63: DIGITS- Interactive Deep Learning GPU Training System · DIGITS Visualize DNN topology and how training data activates your network Manage training of many DNNs in parallel on multi-GPU

63

DEEP LEARNING SERIES

▪ Review the other seminars in series

▪ Seminar #3 - Getting Started with the Caffe Framework

▪ Seminar #4 - Getting Started with the Theano Framework

▪ Seminar #5 - Getting Started with the Torch Framework

▪ More information available at developer.nvidia.com/deep-learning-courses


NVIDIA FOR DEEP LEARNING - Center for Automotive Researchcargroup.org/wp-content/uploads/2018/02/Bill-Veenhuis... · NVIDIA’S DIGITS Interactive Deep Learning GPU Training System

DIGITS DEEP LEARNING GPU TRAINING SYSTEM · System Management . 10 cuDNN Accelerates key routines to improve performance of neural net training Routines for convolution and cross-correlation

Number Systems Decimal (Base 10) 10 digits (0,1,2,3,4,5,6,7,8,9) Binary (Base 2) 2 digits (0,1) Digits are often called bits (binary digits) Hexadecimal

MULTI-GPU TRAINING WITH NCCL - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s... · 2 MULTI-GPU COMPUTING Harvesting the power of multiple GPUs 1 GPU Multiple GPUs per system

Introduction to multi gpu deep learning with DIGITS 2 - Mike Wang

CPS Labor Extractsdata.nber.org/morg/docs/cpsx.pdf · h-id hhid 12 digits 79 - 95:8 all hrhhid 15 digits 95:9 - 1979 - 1995 Digits 1-2 - regional office number Digits 3-5 - PSU Digits

GPU Technology Conference - DIGITS DEEP …...2015 Using DIGITS Deep Learning GPU Training System How to start DIGITS Two options “digits-devserver” Starts a development server

Cygnus: GPU meets FPGA for HPC - RIKEN R-CCS · 2020. 2. 27. · FPGA-GPU DMA (FPGA ← GPU) FPGA-GPU DMA (FPGA → GPU) direction via CPU FPGA-GPU DMA GPU→FPGA 17 1.44 FPGA→GPU

Multi GPU Training - NVIDIA€¦ · ALL Reduce Benchmark. Distributed Deep Learning SINGLE GPU DATA PARALLEL MODEL PARALLEL DATA AND MODEL PARALLEL. 병렬101 수치해석병렬PI계산(적분법)

An Introduction to GPU Computing and CUDA Architecturedeveloper.download.nvidia.com/CUDA/training/GTC... · What is CUDA? CUDA Architecture Expose GPU computing for general purpose

NVIDIA TURING GPU ARCHITECTURE · NVIDIA GPUs have become the standard industry solution for deep learning training, and GPU- ... TURING FEATURES ENHANCE VIRTUAL REALITY Turing GPU

SuperNeurons: Dynamic GPU Memory Management for Training …web.eecs.umich.edu/~mosharaf/Readings/SuperNeurons.pdf · 2019-12-09 · SuperNeurons: Dynamic GPU Memory Management for

MANUFACTURING Catalog FOR INDUSTRIAL AUTOMATIONFEMA · Catalog 2015-2016 09 3 Digits 3 1/2 Digits 4 Digits 4 Digits 5 Digits 6 Digits 14 mm (digit height) OEM 72 x 36 mm (front size)

DIGITS- Interactive Deep Learning GPU Training …on-demand.gputechconf.com/gtc/2015/webinar/deep-learning... · 2016-04-01 · DIGITS- Interactive Deep Learning GPU Training System

Large Graph Convolutional Network Training with GPU ...Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture Seung Won Min UIUC [email protected]

NVIDIA Deep Learning GPU Training System (DIGITS)The Deep Learning GPU Training System™ (DIGITS) puts the power of deep learning into the hands of engineers and data scientists

Introduction to GPU Computing with OpenCL - Nvidiadeveloper.download.nvidia.com/CUDA/training/NVIDIA... · Introduction to GPU Computing with OpenCL. ... //nvdeveloper.nvidia.com/login.asp

Significant Digits

Check digits

Zico: Efficient GPU Memory Sharing for Concurrent DNN Training

To Cross-Validate or Not? · 2016-09-13 · Company Department Account Product Line Intercompany 3 digits Numeric 4 digits Numeric 5 digits Numeric 3 digits Alphanumeric 3 digits

689844 digits - PrimeGrid3*2 2291610 +1 689844 digits continued

NVIDIA A100 Tensor Core GPU Architecture › gpu-specs › docs › nvidia... · AI deep learning training and inference, data analytics, scientific com puting, genomics, edge video

1 Dividing 4 Digits by 2 Digits Division. Copyright © 2013 by Monica Yuskaitis 2 How to Divide 2 Digits into 3 Digits

CUDA and GPU Training Workshop April 21, 2014

Installation Operation Maintenance · E = Export (LAR) R = Export (Others) Digits 16, 17 - Service Digits A0 = Service digits A0 A1 = Service digits A1 (NOVO FORNECEDOR MOTOR) Digit

Monday – Multiply 4-digits by 2-digits

GPU ACCELERATED HIGH PERFORMANCE DATA ANALYTICS · GPU ACCELERATED HIGH PERFORMANCE DATA ANALYTICS ... Software Stack Python Data Preparation cuDF Visualization cuGRAPH Model Training

Training Program on GPU Programming with CUDA

GPU, GP-GPU, GPU computing

CUDA and GPU Training: Sessions 1 & 2 April 16 & 23, 2012

Section 1.5—Significant Digits. Section 1.5 A Counting significant digits

Binary Digits

Number Systems Decimal (Base 10) –10 digits (0,1,2,3,4,5,6,7,8,9) Binary (Base 2) –2 digits (0,1) Digits are often called bits (binary digits) Hexadecimal

Recognizing Hand-written Digits Using Hierarchical ...papers.nips.cc/...written-digits-using-hierarchical...The MNIST database contains an average of 6,000 training examples per digit,