realtime predictive analytics using rabbitmq & scikit-learn
DESCRIPTION
In this talk, AWeber's Michael Becker describes how to deploy a predictive model in a production environment using RabbitMQ and scikit-learn. You'll see a realtime content classification system to demonstrate this design.TRANSCRIPT
RealtimePredictive AnalyticsUsing scikit-learn and RabbitMQ
Michael BeckerPyData Boston 2013
Who is this guy?Software Engineer @ AWeberFounder of the DataPhilly Meetup [email protected]
These slides and more @ github.com/mdbecker
What I'll cover• Model Distribution
What I'll cover• Model Distribution• Data flow
What I'll cover• Model Distribution• Data flow• RabbitMQ
What I'll cover• Model Distribution• Data flow• RabbitMQ• Demo
What I'll cover• Model Distribution• Data flow• RabbitMQ• Demo• Scalability
What I'll cover• Model Distribution• Data flow• RabbitMQ• Demo• Scalability• Other considerations
Supervised Learning
38 top wikipediasArabic العربيةBulgarian БългарскиCatalan CatalàCzech ČeštinaDanish DanskGerman DeutschEnglish EnglishSpanish EspañolEstonian EestiBasque EuskaraPersian فارسیFinnish SuomiFrench FrançaisHebrew עבריתHindi हि�न्दी�Croatian HrvatskiHungarian MagyarIndonesian Bahasa IndonesiaItalian Italiano
Japanese 日本語Kazakh ҚазақшаKorean 한국어Lithuanian LietuviųMalay Bahasa MelayuDutch NederlandsNorwegian (Bokmål) Norsk (Bokmål)Polish PolskiPortuguese PortuguêsRomanian RomânăRussian РусскийSlovak SlovenčinaSlovenian SlovenščinaSerbian Српски / SrpskiSwedish SvenskaTurkish TürkçeUkrainian УкраїнськаVietnamese Tiếng ViệtWaray-Waray Winaray
The model
Distributing the model
Data input
The client
Message loss
Enter RabbitMQ
Reliability
Flexible RoutingClustering
HA Queues
Many clients
AMQP
Reliability
Flexible RoutingClustering
HA Queues
Many clients
Data processing
The worker
The design
Demo time!
Demo time!
Demo time!
Scaling
Realtime vs batch
Monitoring
Load
Verify
Thank youAPI & Worker: Kelly O’Brien (linkedin.com/in/kellyobie)UI: Matt Parke (ordinaryrobot.com)Classifier: Michael Becker (github.com/mdbecker) Images: Wikipedia
My infoTweet me @beckerfuffleFind me at beckerfuffle.com
These slides and more @ github.com/mdbecker