Benedict DiederichTensorFlow Tutorial

- Python based (..as well as C++, Haskell, Java and Go APIs)

- Open Source library: machine learning

- by Google - used for research and production at Google products (Translation, Image

Recognition, etc.)

- License Apache 2.0 open source license

- multiple CPUs and GPUs

- Huge Community (https://github.com/tensorflow/tensorflow)

- Codes available (Github)

- „Model-ZOO“ pretrained networks available 2

What is TensorFlow?

1. provides primitives for defining functions on tensors

2. automatically computing their derivativesComputes math on tensors (like numpy but with GPU-support!)

What is a „Tensor“?

- multilinear maps from vector spaces to the real numbers- examples: Scalar, Vector, Matrix

What is TensorFlow?

TensorFlow vs. Numpy

Numpy Tensorflow

TensorFlow requires explicit evaluation!

• Express a numeric computation as a graph.• Graph nodes are operations which have any number of inputs and outputs• Graph edges are tensors which flow between nodes

Tensorflow Computation Graph

• “A Session object encapsulates the environment in which Tensor objects are evaluated”

• “TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.” - TensorFlow docs

Tensorflow Computation Graph - Session

Graph creation- builds graph sequentially- adds nodes

Graph computation

Graph evaluation (gives result)

never changes its value(s)

input 1 = tf.constant(3.0)input 2 = tf.constant(2.0)input 3 = tf.constant(5.0)intermed = tf.add(input2, input 3)mul = tf.multiply(intermed, input1)

result = mul.eval()

Mathematical operations:computations that will act on tensors• MatMul: Multiply two matrix values• Add: Add elementwise (with broadcasting)• ReLU: Activate with elementwise rectified linear function• etc.

Tensorflow Computation Graph

“When you train a model you use variables to hold and update parameters. Variables are in-memory buffers containing tensors” -

TensorFlow Variables

Was previously a constant

TensorFlow variables must be initialized before they havevalues!

- nodes which output their current value- State is retained across multiple executions

of a graph- e.g. parameters, gradient stores, eligibility

traces, …- value(s) can be updated

tf.placeholder variables - dummy nodes that provide entry points for data to

computational graph- value is fed in at execution time- inputs, variable learning rates, …

feed_dict- python dictionary mapping from tf.placeholder vars to

data (numpy arrays, lists, etc.)

Tensorflow Placeholders and FeedDictionairies

Tensorflow Placeholders and FeedDictionairies

Google Example: Graph Creation

Google Example: Graph Evalution

Example: Linear Regression in TensorFlow

Example: Linear Regression in TensorFlow

Example: Linear Regression in TensorFlow

Example: Linear Regression in TensorFlow

Example: Linear Regression in TensorFlow

Example: Linear Regression in TensorFlow

• Linear regression example computed L2 loss for a linear regression system.

• tf.train.Optimizer creates an optimizer:• tf.train.AdagradOptimizer, tf.train.AdadeltaOptimizer, tf.train.MomentumOptimizer,

tf.train.FtrlOptimizer, etc.

• tf.train.Optimizer.minimize(loss, var_list) adds optimization operation to computation graph.

• Automatic differentiation computes gradients without user input• TensorFlow nodes in computation graph have attached gradient operations

• Use backpropagation (using node-specific gradient ops) to compute required gradients for all variables in graph.

Concept: Auto-Differentiation

1. Build a graph§ Graph contains:

- parameter specifications- model architecture- optimization process, …

2. Initialize a session3. Fetch and feed data with Session.run� Compilation� Optimization� Auto-Differentiation etc.

Workflow for TensorFlow

• Lecture Slides CS224d: TensorFlow Tutorial Bharath Ramsundar• https://en.wikipedia.org/wiki/TensorFlow• Talk: Introduction to TensorFlow -

https://docs.google.com/presentation/d/1oB_U_JagxWQdQJlLD80XlNk6fuv42bHV0hfDAPGAbrc/edit#slide=id.gd2d423435_0_112

• tensorflow.org• Tensorflow Short intro presentation: bit.ly/stanford-tf-tutorial• Great Cours: http://cs231n.github.io/• Great Cours: https://de.udacity.com/course/deep-learning--ud730/

Sources