big data science in the cloud from big data world conference 2013
DESCRIPTION
TRANSCRIPT
![Page 1: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/1.jpg)
„Big Data Science in the Cloud“
Markus Schmidberger
Big Data Analyst & Cloud Engineer
![Page 2: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/2.jpg)
Big Data gets Political
● New coalition agreement in Germany:– “Wir wollen die Informations- und Kommunikations-
Strategie (IKT-Strategie) für die digitale Wirtschaft weiterentwickeln. ...
– ... Wir werden die Forschungs- und Innovationsförderung für „Big Data“ auf die Entwicklung von Methoden und Werkzeugen zur Datenanalyse ausrichten ... “
![Page 3: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/3.jpg)
3. December 2013 - 3
Continuos Software delivery
“We change the rules!”
Curios, playful, agile, experienced, goal-oriented, love to detail, thinking differently ...
Big data &polyglot persistence
Lean & agile
![Page 4: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/4.jpg)
3. December 2013 - 4
Customer and Partners
![Page 5: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/5.jpg)
3. December 2013 - 5
Big Data
![Page 6: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/6.jpg)
3. December 2013 - 6
Big Data Science
● Data science seeks to use all available and relevant data to effectively tell a story that can be easily understood by non-practitioners.
![Page 7: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/7.jpg)
3. December 2013 - 7
Cloud Computing
● Wikipedia: “... describes a variety of computing concepts that involve a large number of computers connected through a real-time communication network such as the Internet. ...”
![Page 8: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/8.jpg)
3. December 2013 - 8
1) Put Apps & Data to best Place
![Page 9: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/9.jpg)
3. December 2013 - 9
AWS Zones at the right Place
![Page 10: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/10.jpg)
3. December 2013 - 10
Example: R and RStudio Server
● R: open-source statistical Software– www.r-project.org
● RStudio IDE– www.rstudio.org– IDE + web / server
version
![Page 11: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/11.jpg)
3. December 2013 - 11
2) Choose Cloud Resources carefully
● Instance type● EBS optimized● EBS provisioned
IOPS● Load Balancer● Availability Zones
http://media.amazonwebservices.com/AWS_NoSQL_MongoDB.pdf
![Page 12: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/12.jpg)
3. December 2013 - 12
● MongoDB hosting on Amazon EC2 (eu-west-1) and in Munich● 24x7 monitoring and support● Dedicated instances and shared hosting available● Replica Sets and Sharding available● SSL-enabled MongoDB
MongoSoup is the first German-based MongoDB cloud hosting solution!
Supported by a team of experts from MongoDB Inc. first German partner comSysto. You can have a running MongoDB database in virtually no time.
![Page 13: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/13.jpg)
3. December 2013 - 13
Performance <-> Costs
● scale up & out● scale down ?● monitor your resources
from the beginning
![Page 14: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/14.jpg)
3. December 2013 - 14
3) Use full Cloud Technology Stack
![Page 15: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/15.jpg)
3. December 2013 - 15
Example: AWS EMR with mapR
● Speed● Compression
– reduces disk and network I/O and increases performance
● Snapshots– data protection
![Page 16: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/16.jpg)
3. December 2013 - 16
4) Data Protection
● talk to the experts (e.g. Bitkom)
● use available mechanisms & services– EMR in VPC– Mongosoup.de
● be aware of the topic
![Page 17: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/17.jpg)
3. December 2013 - 17
More Big Data Events
● “Map-Reducing Everywhere”– https://hadoopsummit.uservoice.co
m
● Forum Big Data und Verantwortung u.a. mit Frank Schirrmacher– Di, 03.12. 19:00; Große Aula LMU
![Page 18: Big Data Science in the Cloud from Big Data World Conference 2013](https://reader034.vdocuments.site/reader034/viewer/2022042613/54c63f844a7959c9388b4790/html5/thumbnails/18.jpg)
3. December 2013 - 18
„Big Data Science in the Cloud“
- Yes We Can -
http://comsysto.com/events