cloud patterns beuth hochschule
Post on 18-Jul-2015
284 Views
Preview:
TRANSCRIPT
CLOUD ARCHITECTURESascha Möllering | zanox AG
Sascha Möllering
sascha.moellering@zanox.com
Lead Engineer / DevOps Hipster
zanox AG
http://autoscaling.io
@sascha242
ABOUT ME
JAVA MAGAZIN 11.14
ARCHITEKTUR IM KONTEXT DER CLOUD: PATTERNS UND BEST
PRACTICES
AWS VIRTUAL CLOUD SUMMIT
CONTINUOUS DELIVERY IN AWS USING DOCKER
ENTWICKLER SPEZIAL: DOCKER
ÜBER DEN WOLKEN: EIN ERFAHRUNGSBERICHT ZUM DOCKER-
EINSATZ IN DER AMAZON CLOUD
ABOUT ME
AGENDA
●AWS Overview
●Creating a cloud service
●Real World example
●Cloud deployment
●Q&A
HOW CAN WE SCALE?
HOW CAN WE SCALE?
AWS OVERVIEW
RegionUS-WEST (N. California) EU-WEST (Ireland)
ASIA PAC
(Tokyo)
ASIA PAC
(Singapore)
US-WEST (Oregon)
SOUTH AMERICA (Sao
Paulo)
US-EAST (Virginia)
GOV CLOUD
ASIA PAC
(Sydney)
EU-CENTRAL (Frankfurt)
AWS OVERVIEW
Availability
Zone
AWS OVERVIEW
EC2: Resizable compute capacity
AutoScaling: scale up or down
Route 53: Domain Name System
ELB: Load Balancing
Elastic IP: Static IP address
AWS OVERVIEW
RDS: Managed database service
DynamoDB: NoSQL implementation
Redshift: data warehouse solution
ElastiCache: Managed cache
Kinesis: Service for streaming data
AWS OVERVIEW
S3: Highly-scalable object storage
CloudFront: CDN implementation
CloudWatch: Monitoring service
IAM: Identity and Access Management
EMR: Managed Hadoop
STARTING POINT
●How can we leverage the Cloud?
●Our application:
● Simple Java based web application
● One application
● One database
● Runs in our own DC
CREATING A CLOUD SERVICE
●Initial setup:
●Route53 for DNS
●One Elastic IP (static IPs for the Cloud)
●One EC2 instance
● Web Application
● Database
●One AZ
CREATING A CLOUD SERVICE
Availability Zone
CREATING A CLOUD SERVICE
●Simple approach to scale:
●More RAM
●More CPU power
●More IOPS
●Different EC2 instance type
●Approach will hit a barrier
CREATING A CLOUD SERVICE
●Problems:
●No failover
●No redundancy
●Database and application on one instance
●Does not scale very well
CREATING A CLOUD SERVICE
●Separate Web Application from DB
●One EC2 instance for the Web Application
●One EC2 instance for the DB
●Or: Managed DB service (RDS)
●One AZ
CREATING A CLOUD SERVICE
Availability Zone
Database
CREATING A CLOUD SERVICE
●Problems:
●No failover
●No redundancy
●Does still not scale very well
CREATING A CLOUD SERVICE
●Add an ELB
●Add EC2 instance for Web Application
●Different AZs
●Standby DB instance
●Multi AZ setup in RDS
CREATING A CLOUD SERVICE
CREATING A CLOUD SERVICE
Availability Zone
Database
Availability Zone
Database
CREATING A CLOUD SERVICE
●Problems:
●Still limited scaling capabilities
● If one EC2 instance crashes, 50% less
capability
CREATING A CLOUD SERVICE
●Add additional EC2 instances
●Add read replicas in RDS
CREATING A CLOUD SERVICE
Availability Zone
Database
Availability Zone
Database
CREATING A CLOUD SERVICE
●Problems:
●Naive implementation
●Does not leverage AWS services
●Limited scaling
●Can be quite expensive
CREATING A CLOUD SERVICE
●Refactoring
●Introduce caches to take load from DB
●Use S3 and CloudFront to ship static
content
●Store session data in DynamoDB
●Add Autoscaling
CREATING A CLOUD SERVICE
●Autoscaling
●Automatic resizing of compute clusters based
on demand
● Integrated to Amazon CloudWatch
●Maximum and minimum pool sizes
●Autoscaling policies triggered by CloudWatch
alarms
CREATING A CLOUD SERVICE
Availability Zone
Auto Scaling
group
CREATING A CLOUD SERVICE
●Started with a simple web application
●Added several Amazon services
●Managed to shift load from application to
services
●Added caches and CDN
●Added Autoscaling
CREATING A CLOUD SERVICE
●Next steps:
●SOA
●Loose coupling: decouple interactions
●Amazon services, don‘t reinvent the wheel
●Database federation
●Database sharding
REAL WORLD EXAMPLE
●Elastic architecture that grows and shrinks
●Multiple AZs
●Use case: streaming data
●Decoupling using Kinesis
● IAM Roles for managing credentials
●CloudWatch for monitoring
Internet
Auto Scaling group
Auto Scaling group
Availability Zone
Availability Zone
ARCHITECTURE
ARCHITECTURE
“Vert.x is a lightweight, high performance application
platform for the JVM that's designed for modern
mobile, web, and enterprise applications.”
Vert.x
ARCHITECTURE
●Main framework Vert.x
●mod-kinesis (Kinesis-Adapter for Vert.x)
●AWS Java SDK
● IAM roles for Amazon EC2 instances
●Coda Hale metrics and CloudWatch reporter
●Jedis (Redis client for Java)
●…
DEPLOYMENT
WorkstationGitHub
Jenkins
Python/Fabric
Nexus
Datacenter
Artefakt
DEPLOYMENT
EC2 instance with Docker
registry (port 5000)
S3 bucket to store
Docker images
DEPLOYMENT
Amazon Cloud
WorkstationGitHub
Jenkins
Python/Boto
Docker Image
Docker Registry S3 bucket with
Docker Images
DEPLOYMENT
DEPLOYMENT
EC2 INSTANCES AND ELB
python start_docker_instance.py \
-r <myregistry>:5000 \
-i tracking-ppv \
-t 95 \
-q 2 \
-s quality \
-d '-d -p 8080:8080'
https://github.com/SaschaMoellering/aws-docker-
scripts
EC2 INSTANCES AND ELB
python start_elb.py \
-r <myregistry>:5000 \
-i tracking-ppv \
-t 95 \
-s quality \
-d '-d -p 8080:8080'
https://github.com/SaschaMoellering/aws-docker-
scripts
EC2 INSTANCES AND ELB
EC2 INSTANCES AND ELB
DEPLOYMENT
●Amazon EC2 Container Service (ECS):
●Container management service
● Fast
●Highly scalable
●Supports Docker
DEPLOYMENT
●Cluster
●Container Instance
●Task Definition
●Task
●Container
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
#!/bin/bash
echo ECS_CLUSTER = \
your_cluster_name >> \
/etc/ecs/ecs.config
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
DEPLOYMENT
top related