[draft] workshop - technical introduction to joola.io

joola.ioWorkshopsTechnicalReviewofjoola.io

WorkshopDetailsandInstructions

Preface

AbouttheworkshopThisisahands-onworkshop

Labmaterialsareavailableforthisworkshop

ThisworkshopandmaterialsareavailableonourGitHubrepo

WorkshopGoalsUnderstandjoola.ioonallofitscomponents

Beabletodeployjoola.io

PushandQuerydatafromthesystem

DrawbasicVisualizations

LabDetailsDuringthisworkshopwe'llbeaccessinganonlinelab

PleaseSSHwiththefollowing$sshworkshop@lab-intro.joola.io

Password:password

*accesstotheabovelabis whitelis ted

Understandingjoola.io

Whatisjoola.io?$npminstalljoola.io

FeaturepackedDataAnalyticsandVisualizationFramework

WritteninNode.JS

Scalabletodealwithhighvolumesofdataandrealt imequeries

Secureandmulti-tenant

Embedandintegratedatavisualizationsintoexistingsitesandapps

OpenSource

Howdoesitwork?OperatesandmanagedviaasetofHTTPAPIendpoints

DatapushedusingJSONviajoola.io'sBeaconAPI

Storedincachinglayer,optim izedforquery

QueriesandVisualizationsprovidedbyJavascriptSDK

That'sallinanutshell!

DeveloperFriendlyByDataLoversforDevelopers

Easeofuse

Extend

EmbedandIntegrate

Sitespecificautomatedtesting

Richdocumentation

ITFriendlyMasterlessgrid

Easyinstallation

Zeroconfiguration

PowerfulCLI

Smallfootprint

Monitor,AlertsandLogs

DBAFriendlyEasyDataModelling

Schema-less

Flexible

Zerodowntimeupdates

Completecontrol

Dispatch

NodesEverywhereEachnodeisafullyfunctioningjoola.iosystem

CommunicationisbasedonPub/Sub

Anodeneedstoqualifyforexeuction

Qualificationcanvaryandbeextended

Geopriorit ization

DispatchRoutingEveryactionisamessage

Allmessagesaredistributed

Messagescontainsecuritycontext

Default`notme handlinglogic

Backlogprocessingofstuckmessages

CommandtheGridUseREPL,CLIorSDK

$joola.io.c li

joola.io#admin@localhost:8080>joolaio.system.lis tnodes();joola.io#admin@localhost:8080>joolaio.system.terminate('nodeuid');

Whenyou'reconnectedtoonenode,you'reconnectedtotheGrid

Noactionisexecutedonthelocalnode,butdispatched

Allnodessubscribetoconfigurationchanges

Node.JS'EventLoopBlockManyconsiderasNode.JS'achil lesheel,wedon't

TheGridandPub/Subapproachresolvesthecoreissue

ELBcountersandmonitors

Usingtherighttoolforthejob

Madeoursoftwarebetter

Accessingjoola.ioViaHTTPAPIendpoints

UseSDK,CLI,cURLorwriteyourown

ShippedwithWebAdminconsole

Out-of-the-boxfullyfunctioninganalyticssite

Richdocumentation

PushingData

PushingDataisEasyjoolaio.beacon.insert('collection-purchases',{timestamp:newDate(),Player:{Username:'Aplayer',Device:'Desktop',Browser:'Chrome'},ip:'127.0.0.1',Vis its :1,Vis itDuration:123,Clicks:23,Purhcases:{Amount:123Currency:'USD',Status:'Processed'}}});

DataModellingDescribeyourdataasaJSONdocument(s)

DocumentsarestoredinaCollection

Schema-less,documentscanbeofdifferentstructure

Supportforalldatatypes

SupportforStrong-Typedcollections

PushWhatYou'llNeedStoreonlydatarelatingtoAnalytics

Separateintologicalcollections

Describeeverymetricwithasmanyattributesaspossible

Passrawdata,don'taggregate

Pushanarrayofdocumentswhenpossible

CachingLayerBasedonleadingproviders,suchasMongoDB,Cassandraandmore

Extendtosupportaddit ional

Insert,updateanddeletedirectlyfromcache

Executequeriesdirectlyoncachinglayer

NormalizedaccesstounderlyingcacheviaBeaconAPI

Query

UnderstandingQueriesDescribewhatyouwishtodisplay,ratherthanhowtogetit (SQL)

SupportforRealt ime(WebSocket)

Joindatafrommultiplecollections

Relativet imeframes

Advancedfi lters

DimensionandMetrictransformations

Manyaggregationoperators,includingUniqueCount

BasicQueryDescribetherequireddimensionsandmetric,nothowtoquerythemjoolaio.query.fetch({timeframe:'last_month',interval:'days ',dimensions:['timestamp','device'],metrics:['v is its ',{key:'avgvis itduration',name:'Avg.Vis itDuration',dependsOn:'v is itduration',aggregation:'avg',suffix :'ms.'}]},function(err,result){console.log(result.documents);});

CalculatedMetricsComposevirtual,calculatedmetricson-the-fly

metrics:[{key:'avgamountperplayer',name:'Avg.AmountperPlayer',formula:{dependsOn:[{key:'avgamount',dependsOn:'Purchase.Amount',aggregation:'avg'},{key:'playercount',dependsOn:'Player.Username',aggregation:'ucount'}],run:'function(avgamount,playercount){returnavgamount/playercount;}'}}]

QueryFiltersMultiplefi lters-mutli-levelfi lters-permetricfi lters

timeframe:'last_month',interval:'days ',metrics:[{key:'playercount',dependsOn:'Player.Username',aggregation:'ucount',filter:[['Player.Browser','eq','Chrome']]}],filter:[['Player.Device','eq','Desktop']]

QueryResultsContainretrieveddocumentsandmeta-datadescribingthequeryanditst im ings.

Resultscontainbothrawandformattedvalues.results :[{values:{timestamp:'2014-03-09T14:03:34.412Z',avgduration:'123ms.'},formattedValues:{timestamp:'SunMar09201415:03:34GMT+0100(CET)',avgduration:123'}},{...}]

RealtimeBuilt-inintoallvisualizationsandintegralpartofquery.fetchjoolaio.query.fetch({//queryparams,suchastimeframe,metrics,etc...realtime:true});

Firstexecutionreturnsentiret imeframe

Followingpacketsreturnonlydelta

Dataconsistencyassurance

Visualization

PlottingDataPartoftheJavascriptSDK

Easyascopy-paste

Shapetothedesiredlook,feelandbehavior

OffersjQueryplugins

RichDocumentationandPlayground

VisualizationTypesMetricBox

Sparkline

PieChart

Timeline

Table

DrawingaVisualizationEasyasacopy-pastingfromthewiki

AcceptsaQueryobjectandaddit ionalformattingoptionsjoolaio.v iz .timeline({container:document.getElementById('drawhere'),query:{dimensions:['timestamp'],metrics:['value']});

UsesHTML5,supportsalldevicetypes(mobiles,tablets)

Integrate,EmbedandExtend

IntegrateDesignedandbuilt forintegration

Pushdatafromanydatasourcetypewhilekeepingcontroloflogic

Supportforhigh-ratedistributed,guaranteedwrites

JSONcollectionsprovideflexibil ityandsitespecificintegration

Collectdatafromallpartsofthebusiness

Leveragejoola.iotoreduceoverheadofoperationaldatabases

EmbedSeamlessdrop-invisualizationsinyoursiteandapps

Eventdrivencommunicationwithcontainerpage

Copy-pastefromwiki

Onlyfront-enddevelopersareneeded,reduceDBAoverhead

Controlthet iniestdetailoflook,feelandflow

Responsive,HTML5,CSS3standards

ExtendMissingafeature?

Easytolearninfrastructureandcode

Followsbestpracticesguidelinesandmethodology

Accesstosourcecode,issues,wikiandknow-how

Contributetoopensourceproject

Extendanythingfromcachinglayertovisualizations

EndlessPossibilities

Analytics-as-a-ServiceOfferanalyticsandvisualizationstoyourend-users

Scalableandredundanttosupportverylargeoperations

Multi-tenantandsecure

Easybrandingandclientintegration

WebAnalyticsEasytosetupandconfiguration

Gatherandjoinmultipledatasourcesintoasinglestorejoolaio.beacon.insert('performance',window.performance);

Setupalertsonspecificuseractivityorthreshold

AdServingNoneedtopre-determineaudiencesandsegments

Leveragesplit secondqueryt imes

Applyadvancedfi ltersandsegmentsinrealt ime

Determinebestmatchbyunlim itedparameters

Performfull/partialsegmentationon-the-fly

InfographicsOffer"l ive"infographics

Combinerealt imecalculationsintostaticcontent

Segmentedinfographics

Secure/multi-tenantinfographics

THANKYOUVisitourGitHubtoreadmoreaboutjoola.io