how to run splunk as a docker image?
TRANSCRIPT
Copyright©2016Splunk Inc.
MarcChénéITMarketsProductManager,Splunk
DenisGladkikhPrincipalDevEngineer(akaoutcoldman),Splunk
HowToRunSplunk AsADockerImage?
Disclaimer
2
Duringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose
containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor
functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.
Agenda Agenda
WhatisDocker?WhyrunSplunk inDocker?DemoScenarios1. SetupSplunk ClusterinDocker2. ScalingUpSplunk inDocker3. ClusterUpgrade:6.4.1to6.4.2Guidance&BestPractices
Docker,inoneSlide• Build - Ship- Runyourapplications
– ”Infrastructureascode”– Enablesmicroservicesarchitectures– Portable– EnablesCloudMigration
• OpenSourceandCommunityMinded– DockerEngineisOpenSource– Thousandsofappscanbe“pulled”in
Dockerhub– YourdevelopersLOVEDocker
7
Docker– It’snotVirtualization
8
• VMs– focusonOS• Docker– focusonapplications
• Docker– lightweightandFAST
• NOTmutuallyexclusivewithVMs
Docker– it’sabigdeal
9
OpenSourcedrivenecosystem
Massiveincreaseinadoption…
…Butthegrowthisjustgettingstarted
Goals&Benefits
12
ReduceManagementCostsTimetoValueHighAvailableReducetimetoUpgradeSimplifiedRollbackStandardConfigurationsEasiertoSupport
Splunk ReferenceArchitectureinDocker
13
Search
CollectionTier– HEC– x3
LoadBalancer
SplunkAdmin
SearchTier– x3
IndexingTier– x4
…
…
…
…
Data
SearchHeadClusterDeployer
IndexClusterMaster
LicenseMaster
DeploymentServer
Control
TargetEnvironmentArchitecture
14
Dockerv1.12DockerforAWS– https://beta.docker.com/docs/aws/#upgrading-docker-and-changing-
instance-sizes
DockerSWARM
Splunk SizingGuidelinesinAWS
15
Storage– EBS
ê HighAvailableê Reliableê Growupto16TB
– EBSGeneralPurpose(SSD):consistentperformance– EBSProvisionedIOPS(SSD):consistentperformanceupto4KIOPS*EBSvolumescanbedeployedinaRAIDarchitecture
ComputeRequirementsperSplunk Components– 4vCPUs– 8GBRAM
SplunkEnterprisedeploymentonAWSSearch Heads (8+ users)c4.4xlarge 16 vCPU, 30 GB RAM
c4.8xlarge 36 vCPU, 60 GB RAM
Indexers (50-250GB/day/indexer)c4.4xlarge 16 vCPU, 30 GB RAM
d2.4xlarge 16 vCPU, 122 GB RAM
c4.8xlarge 36 vCPU, 60 GB RAM
CloudFormation TemplatesConsistent, repeatable deployments for SplunkAbstract away details of configuring distributed SplunkExtensible and customizable to fit any need
Cloudformation Templates On GitHub
Workload = Searching + Indexing
Storage- Ephemeral or EBS- Data Retention Dependent
Compute- Best Available
Archiving- S3
Best Practices for Sizing Splunk on AWS Tech BriefSplunk Cloudformation TemplatesSplunk Admin Docs
DeliveringSplunkasaContainerImage• Splunkcontainerimages
– SplunkEnterprise6.4.1– SplunkUniversalForwarder6.4.1
• IncludesconfigurationandDockerAdd-Onforcontainermonitoringout-of-the-box
• Certifiedimage• ComingsoontotheDockerStore(http://store.docker.com)
18
docker run splunk/enterprise:6.4.1-monitordocker run splunk/universalforwarder:6.4.1-monitor
Splunk ScaleUp
Goal– ScaleuptheSearchHeadsby3à Total:5– ScaleuptheIndexersby4à Total:8– ScaleuptheCollectionlayerby2à Total:4
Splunk inDocker– ScaleUp
22
Search
CollectionTier– HEC– x5
LoadBalancer
SplunkAdmin
SearchTier– x5
IndexingTier– x8
…
…
…
…
Data
SearchHeadClusterDeployer
IndexClusterMaster
LicenseMaster
DeploymentServer
Control
Splunk UpgradeOrderofUpgrade– SearchHead– LicenseManager– ClusterMaster– IndexersRecommendations– Backupyourconfigurations– BackupyourdataGoal– Upgrade6.4.1to6.4.2
GuidanceandBestPracticesUnderstandthesizingfactors– Howmuchdata(rawsizes)?Daily,Peak,Retained(archivesize),Future– Howmuchsearching?UseCases,#ofpeople,Apps– Jobs:Summarization,alerting,reporting
StandardOperationProceduresDatavolumeSearchvolume
WhatNow?
29
ArchitectingSplunk forHighAvailabilityandDisasterRecovery,SessionID:74762ObservationsandRecommendationsonSplunk Performance,SessionID:74765MonitoringandTroubleshootingDockeracrossCloudandOn-PremEnvironments,SessionID:IT88095Splunking AWSforEnd-to-endVisibility,SessionID:87942– Track:Splunk Platform forOperational Intelligence
Relatedbreakoutsessionsandactivities…
# 1. Come visit us at our boothdocker run splunk/visitourboothvisitourbooth_1 | Booth IT Markets
# 2. Try out our docker images in Docker Storedocker run splunk/enterprise:6.4.1-monitordocker run splunk/universalforwarder:6.4.1-monitor
# 3. Demos will all be available on GitHub under Splunk!git clone https://github.com/splunk/docker-gettingstarted-conf2016-sf88089.git
# 4. Visit our site to learn more about containerscurl http://www.splunk.com/containers
CalltoAction…
Resources
31
Splunk EducationArchitectingandDeployingSplunk 6.4– Virtual
Splunk DocsUpgradeGuide,http://docs.splunk.com/Documentation/Splunk/6.4.2/installation/Upgradeto6.4onUNIXCapacityPlanningManual,http://docs.splunk.com/Documentation/Splunk/6.4.1/Capacity/ReferencehardwareDEPLOYINGSPLUNK®ENTERPRISEONAMAZONWEBSERVICES,http://www.splunk.com/pdfs/technical-briefs/deploying-splunk-enterprise-on-amazon-web-services-technical-brief.pdf
DockerDockerforAWS,https://beta.docker.com/docs/DockerStore,http://store.docker.com