concepts and technologies for distributed...

ConceptsandTechnologiesforDistributedSystemsand

BigDataProcessingGuidoSalvaneschi

1

People

GuidoSalvaneschi

PascalWeisenburgerMatthiasEichholz

2

Motivation

• Distributedsystemsareubiquitous• Emails,WWW,…

• Areaisquicklychanging- sorryforthebuzzwords:)• Bigdataanalytics• Complexeventprocessing/streamprocessing• Cloudcomputing• Newprogrammingmodels(streams,actors)• ...

• Yet,manyconceptsremainthesame• Abstractionoverlow-leveldetails• Faulttolerance• Performance:throughput,latency• Asynchonouscommunication• ...

3

ConceptsandTechnologiesforDistributedSystemsandBigDataProcessing

• ProvideanoverviewofrecentdevelopmentindistributedsystemsandBigDataprocessing

• Focusonconcepts,notontechnology.• Technologywasdifferent5yearsago,willbedifferentin5years• Wewilllookattechnologyonlyasawaytobettergrasptheconcepts

• Selectedtopics• Differenttopics,selectedamongthe“mostinteresting”ones• Guestlectures

4

Aboutthiscourse

WhatthiscourseIS• Anintroductiononselected topics• …ondistributedsystemsandbigdataprocessing• Acourseaboutconcepts…andabitabouttechnologies

WhatitisNOT• Itisnot(only)acourseaboutrecenttrendsinBigData.• Itisnotasystematic courseondistributedsystems

• DistributedSystems:PrinciplesandParadigmsAndrewS.Tanenbaum,MaartenvanSteen

• Itisnotatutorial onhowtoprogrambigdatasystems

5

April21- Intro,motivationApril28- Introtobigdata,mapreduceMay5- hadoop,hdfsMay12- Futures,actors,streamsMay19- Futures,actors,streamsMay26- ComplexEventProcessingJune2- SparkJune9- SparkstreamingJune16- Exampreparation…

JulyTBA- Exam

Tentativecourseschedule

Bigdataandcomplexeventprocessing

Architecturesandprogrammingmodelsfordistr.systems

Subjecttochange!Seecoursewebsiteforupdates

6

Exercises

• Nogradedexercises• Exerciseswillbeprovidedaftersomelectures.• Solutionsdiscussedinthenextlectureifneeded

• Why• Getmoreconfidentonthetopicsofthelectures• Getafeelingthetypeofquestionsthatcancomeupintheexam

• Examples• Readapaper• Answerquestionsbasedonthecontentofthelecture• Discussacasestudy• Smallcodingexercises

7

RegistrationandGrading

• RegisterinTucan forthecourse!

• Writtenexam– July,dateTBA• 90mins

• Questionsaboutthetopicscoveredinthelectures• Simple“programming”tasks• Understandacodesnippetthatisrelevantforoneofthecoveredtopics• Completeacodesnippet

8

Resources

• Website(slides,exercises)• http://stg-tud.github.io/ctbd/• Thisistheofficialplaceslides,exercises,updatedschedule,etc.

• Forum• https://www.fachschaft.informatik.tu-darmstadt.de/forum/viewforum.php?f=580• Pleaseusetheforumforyourquestions.Answerswillremainasareferenceforotherpeople.

• Pleaseaskassistantsforquestionsthatcannotbepostedontheforum• Butthinkifthisisreallythecase:)

9