sentiment analysis using apache mxnet and gluon - aws online tech talks
Post on 21-Jan-2018
530 Views
Preview:
TRANSCRIPT
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vikram Madan, Sr. Product Manager
November 14, 2017
Sentiment Analysis Using
Apache MXNet and Gluon
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sentiment analysis
Sentiment analysis is a natural language processing (NLP) task for classifying
text or speech as some specified sentiment
Brand Sentiment Political Sentiment Customer Satisfaction
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Performing sentiment analysis
Labelled Data
for Training
Word
Embedding
Train Deep
Neural Network
Deploy Model
for Prediction
Focus of Today’s Discussion
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IMBD dataset for movie reviews
Positive
“The impetuous pace of the film
is at one with its moral
shamelessness, and, without
thinking, we sign up for both”
“What a magical movie”
“It is a film so awe-inspiringly
wooden that it’s basically a fire-risk”
“An explosion in a stupid factory”
“Love him, hated Hur”
Negative
25,000 movie reviews: 12,500 positive & 12,500 negative
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Two Major Approaches: word2vec & GloVe
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Training Co-Occurrence Data Output Vectors
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Example: 3-Dimensional Word Vectors
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GloVe: Global Vectors for Word Representation
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Neural Networks
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recurrent Neural Networks (RNNs)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Long Short-Term Memory (LSTM) networks
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 1 of 2)
Edwin Chen Blog (http://blog.echen.me/)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 2 of 2)
Edwin Chen Blog (http://blog.echen.me/)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API Specification
Gluon is a technical specification providing a programming
interface that: (1) simplifies development of deep learning models,
(2) provides greater flexibility in building neural networks, and (3)
offers high performance
Simplicity Flexibility Performance
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API is Available in Apache MXNet
Others…
Development of the Gluon API specification was
a collaboration between AWS and Microsoft
Available Now
Coming Soon
Gluon API is implemented in Apache
MXNet and soon will be in CNTK
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Advantages of the Gluon API
Simple, Easy-to-
Understand Code
Flexible, Imperative
Structure
Dynamic Graphs
High Performance
Neural networks can be defined using simple, clear, concise code
Plug-and-play neural network building blocks – including predefined layers,
optimizers, and initializers
Eliminates rigidity of neural network model definition and brings together
the model with the training algorithm
Intuitive, easy-to-debug, familiar code
Neural networks can change in shape or size during the training process to
address advanced use cases where the size of data fed is variable
Important area of innovation in Natural Language Processing (NLP)
There is no sacrifice with respect to training speed
When it is time to move from prototyping to production, easily cache neural
networks for high performance and a reduced memory footprint
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simple, Easy-to-Understand Code
Use plug-and-play neural network building blocks, including
predefined layers, optimizers, and initializers
# First step is to initialize your model
net = gluon.nn.Sequential()
# Then, define your model architecture
with net.name_scope():
net.add(gluon.nn.Dense(256, activation="relu")) # 1st layer (256 nodes)
net.add(gluon.nn.Dense(256, activation="relu")) # 2nd hidden layer
net.add(gluon.nn.Dense(num_outputs)) # Output layer
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Flexible, Imperative Structure
Prototype, build, and debug neural networks more easily with a
fully imperative interface
epochs = 10
for e in range(epochs):
for i, batch in enumerate(train_data):
with autograd.record(): # Start recording the derivatives
output = net(data) # the forward iteration
loss = softmax_cross_entropy(output, label)
loss.backward()
trainer.step(data.shape[0])
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dynamic Graphs
Build neural networks on the fly for use cases where neural
networks must change in size and shape during model training
def forward(self, F, inputs, tree):
children_outputs = [self.forward(F, inputs, child)
for child in tree.children]
#Recursively builds the neural network based on each input sentence’s
#syntactic structure during the model definition and training process
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
High Performance
Easily cache the neural network to achieve high performance
when speed becomes more important than flexibility
net = nn.HybridSequential()
with net.name_scope():
net.add(nn.Dense(256, activation="relu"))
net.add(nn.Dense(128, activation="relu"))
net.add(nn.Dense(2))
net.hybridize()
top related