new aws services

New AWS ServicesAWS PHOENIX MEETUP

Josh Padnick Gruntwork Monday, August 29, 2016

Today’s talk is about three recent updates in AWS.

Today’s talk is about three recent updates in AWS.‣ Application Load Balancer (ALB) ‣ EC2 Container Service (ECS) ‣ Kinesis Analytics

For each service, we’ll discuss…‣ The big idea ‣ What’s new ‣ Examples

Intended Audience

Executives Developers

We’ll start simple. But we’ll get progressively more technical.At a certain point, we’ll dive deep into the technical nuances of the topic. In such cases, look for the Nerd Alert ribbon.

Nerd Alert

Hi, I’m Josh Padnick.‣ Published A Comprehensive Guide

to Building a Scalable Web App on AWS. Received 500+ up votes on Hacker News.

‣ Consulted on DevOps & AWS with ~25 companies worldwide including Intel and Infusionsoft.

‣ Full-stack engineer for 10+ years

‣ Co-founder at Gruntwork.

‣ We setup software teams on AWS with DevOps best practices and world-class infrastructure.

‣ But we do it in about 2 weeks!

‣ The secret sauce is we offer battle-tested, pre-written “Infrastructure Packages” for common AWS needs.

‣ Plus consulting & support as needed.

http://gruntwork.ioI work at Gruntwork.

Application Load Balancer (ALB)

Let’s start by talking about the generic concept of a Load Balancer.

The Big Idea

Hi, I’m an EC2 Instance!

My App

The Big IdeaWith a single VM, users can connect directly to the VM.

The Big IdeaBut if that VM fails, our entire service goes down.

XX

The Big IdeaWith multiple VMs, we gain High Availability (HA)!

The Big IdeaIf one VM goes down, we can just serve traffic from the other.

XX

✅

The Big IdeaBut how do we route requests to more than one VM?

?

The Big IdeaWe use a Load Balancer. This is sometimes called a Reverse Proxy.

Load Balancer

The Big IdeaThere are a few properties we want out of this load balancer:

The Big IdeaThere are a few properties we want out of this generic load balancer:

‣ It should itself be HA!

‣ It should elastically scale as we get more traffic.

‣ It should do a Health Check on each VM.

The Big IdeaKeep going…

‣ It should support the latest protocols (TCP, UDP, HTTP(S) 1.1, HTTP/2, WebSockets).

‣ It should log all requests.

‣ It should emit helpful metrics.

The Big IdeaKeep going…

‣ It should allow routing a single user to the same VM, but spread different users across different VMs (sticky sessions).

‣ It should route a request for /apples to one set of VMs and /oranges to another (path-based routing).

The Big IdeaKeep going…

‣ It should have first-class support for routing to Docker containers in EC2 Container Service (ECS)

‣ Route to an app running in a container, not just to a VM.

‣ Route to multiple different containers on the same VM.

‣ Know about new containers when I launch them (service discovery).

Nerd Alert

In 2012, Amazon released the Elastic Load Balancer.

Elastic Load Balancer (ELB)

Nerd AlertOld ELB was a Layer 4 Load BalancerOpen Systems Interconnection (OSI) Network Model

Physical / Data Link1 / 2

Network (IP, ICMP)3

Transport (TCP, UDP)4

Session5

Presentation (TLS)6

Application (HTTP, FTP, DNS, SSH)7

But there’s a problem…

‣ Helpful metrics like “Sum HTTP 5XX errors” only apply to HTTP traffic.

‣ Path-based routing requires inspecting the HTTP traffic.

Some of our feature asks are HTTP-specific.

But there’s a problem…

‣ Route to more than one port on the same VM

Some of our feature asks are DOCKER-specific.

So AWS has released the new Application Load Balancer (ALB).

So AWS has released the new Application Load Balancer (ALB).An updated load balancer opinionated to: - modern apps built with HTTP- Docker

Updated Terminology

Elastic Load Balancing

Application Load Balancer (ALB) Classic Load Balancer (Sometimes called “ELB”)

Nerd AlertALB is a Layer 7 Load BalancerOpen Systems Interconnection (OSI) Network Model

Physical / Data Link1 / 2

Network (IP, ICMP)3

Transport (TCP, UDP)4

Session5

Presentation (TLS)6

Application (HTTP, FTP, DNS, SSH)7

Nerd AlertALB is a Layer 7 Load BalancerTranslation

‣ The ALB inspects HTTP traffic and makes routing decisions based on this.

‣ But the ALB doesn’t deal with “OSI Layer 3” forwarding, so no TCP or UDP forwarding.

Application (HTTP, FTP, DNS, SSH)7

New features in the ALB

Support for HTTP/2‣ Did you know HTTP 1.1 came out in 1999

when this was what the Web looked like?

Support for HTTP/2‣ The web of 2016 is different than the web of

1999:

HTTP/2 Benefits‣ Sends headers/cookies just once instead of on

every request.

‣ Encodes all data in binary versus a textual format.

‣ Transmits all data over a single, multiplexed TCP connection versus multiple blocking connections in HTTP/1.1.

Nerd Alert

Your Backend App Can Still Speak HTTP/1.1

Nerd Alert

HTTP/2 HTTP/1.x

Note that HTTP/2 requires that you use HTTPS on the ALB.

All modern browsers support HTTP/2

Nerd Alert

SOURCE: http://caniuse.com/#search=http2

Support for WebSockets‣ A long-time ask for ELBs has been WebSocket

support. ALBs now support this!

Nerd Alert

ws://…

ws://…

Content-Based Routing‣ Route /blue to one service.

‣ Route /green to another service.

‣ Previously, this required two load balancers. Now, it requires just one!

Content-Based Routing‣ LIMITATION

‣ We don’t get path rewriting.

‣ So you can’t send /blue to /hello/blue unless your backend app handles that.

Nerd Alert

New Concepts in Elastic Load Balancing‣ Target Groups

The Classic Load Balancer includes as part of its configuration which EC2 Instances it will route to.

ELB

New Concepts in Elastic Load Balancing‣ Target Groups

With ALBs, the concept of Load Balancer is separated from the concept of Target EC2 Instances.

ALB

Target Group

Target Group

New Concepts in Elastic Load Balancing‣ Target Groups

Our ALB needs a list of “targets” where it can send traffic. We’ll group all such targets into a Target Group.

Empty Target Group

New Concepts in Elastic Load Balancing‣ Target Groups

Let’s add one Target:

i-123Port 8000

Notice we have both an instance id and port number.

New Concepts in Elastic Load Balancing‣ Target Groups

Let’s add a second Target:

i-123Port 8000

i-123Port 8001

This target has the same instance id but a different port number.

New Concepts in Elastic Load Balancing‣ Target Groups

Let’s add a third Target:

i-123Port 8000

i-123Port 8001

i-789Port 3034

New Concepts in Elastic Load Balancing‣ Target Groups

Our ALB will send traffic to any Healthy Target in the Target Group.

i-123Port 8000

i-123Port 8001

i-789Port 3034

New Concepts in Elastic Load Balancing‣ Target Groups

Note that the Classic ELB does not use a Target Group and can only send to the same port on different EC2 Instances.

i-123Port 8000

i-789Port 8000

New Concepts in Elastic Load Balancing‣ Target Groups

The big takeaway is you can group your (micro)services into Target Groups, even if multiple target groups include the same EC2 Instance!

i-123 i-456 i-789

Service B

Service A

Nerd Alert

Content-Based Routing‣ Route /blue to one service Target Group.

‣ Route /green to another service Target Group.

‣ Previously, this required two load balancers. Now, it requires just one!

Support for Container-Based Apps

‣ We often want to run the same Docker image on the same EC2 Instance on different ports.

‣ Target Groups means the ALB can route to either to two different ports on the same server!

‣ This also means we can dynamically select our container ports in an EC2 Container Service Cluster!

Nerd Alert

Nerd AlertSupport for Container-Based Apps

Port8523

Port8000

Port4738

Port8713

Target Group metrics.‣ We get CloudWatch Metrics on Target Groups.

‣ This is a nice way to get metrics specific to a service.

Nerd Alert

Better metrics.‣ Many new metrics on the ALB!

Nerd Alert

‣ ClientTLSNegotiationErrorCount

‣ TargetTLSNegotiationErrorCount

‣ TargetConnectionErrorCount

‣ TargetResponseTime

‣ NewConnectionCount

‣ ActiveConnectionCount

‣ RejectedConnectionCount

‣ ProcessedBytes

Other Cool Features‣ Load-balancer generated sticky-session

cookies (client must support cookies).

‣ Slightly less expensive.

‣ Faster performance in general.

Nerd Alert

When to Use the ALB‣ When running any HTTP-based service.

‣ When using WebSockets with a load balancer.

‣ When using Docker, especially with EC2 Container Service.

When to Use the Classic ELB‣ You need OSI Layer 4 Routing (i.e. TCP / UDP)

‣ Your app listens on a protocol other than HTTP.

Alternatives to the ALB/ELB‣ Set up your own load balancer using Nginx or

HAProxy.

‣ But this means you need to build auto-scaling, auto-failover, automated DNS updates, configure metrics, configure logging, manage upgrades, and a few more items.

‣ Conclusion: don’t do this unless you have to.

Recent Updates to EC2 Container Service (ECS)

The Big Idea

Hi, I’m an EC2 Instance!

The Big Idea

I can offer you resource isolation.

And I can be launched in just minutes!

Limitations of a VMBut minutes could be an eternity.

If deploying multiple times a day,we’re just waiting for VMs to launch.

Building an Amazon Machine Imagealso takes on the order of minutes.

Limitations of a VMAnd I can’t run that AMI locally.

If I want to run the same “Golden Image” locally, I’m out of luck.

X

Sometimes a single app uses a tiny portion of available resources.

Mem Usage: 12%

CPU Usage: 7%

So it’d be nice if we could pack multiple apps in a single EC2 Instance.

Mem Usage: 85%CPU Usage: 90%

App 1 App 2

App 3

Introducing Docker

Why developers love containers.‣ A container is just an isolated OS process, so it runs

directly on your EC2 Instance.

‣ It’s similar to a “lightweight VM” and can start in milliseconds.

‣ You can run multiple containers on a single EC2 Instance.

‣ You can run the same docker image on any platform.

‣ You can download pre-built docker images for almost all custom software.

So we want to run our apps as containers.‣ But we don’t want to run containers on just a

single EC2 Instance.

If I go down, I’m taking all apps with me!

We want to run multiple containers across multiple EC2 Instances.

But running a “docker cluster” is hard.

‣ Way to bootstrap the cluster

‣ Container scheduler

‣ Service Discovery solution

‣ Load balance to containers

‣ Auto-restart failed containers

‣ Cluster-wide metrics

We need…

There are multiple options to solve this problem today.

But my favorite solution is Amazon EC2 Container Service (ECS)

Amazon EC2 Container Service

Benefits of ECS‣ Built-in cluster bootstrapping

‣ Built-in scheduler (with ability to use a custom scheduler)

‣ Built-in service discovery

‣ Built-in load balancer (ALB)

‣ Built-in auto-restart on failed containers

‣ NEW! Auto-scale your service

‣ NEW! Fine-grained AWS permissions on your service

What’s Missing from ECS‣ Service-to-service authentication

‣ Run background jobs within the cluster (you can still do this with Lamba’s run on cron schedules, though)

‣ DNS namespacing

‣ Built-in persistent volumes

‣ Built-in support for log aggregation (on services other than CloudWatch Logs)

Then why is it my favorite?‣ Because most teams don’t need those features.

‣ If you’re ok with the limitations, ECS is easier to setup than anything else.

‣ The new ALB plus the new features we’ll talk about make this even more compelling.

ECS Terminology

ECS Cluster

ECS Instance

ECS Task

ECS Task DefinitionDeclares what kind of ECS Task should be run.(e.g. docker run properties)

ECS Service

One other quick review…

EC2 Instances get permissions to AWS Resources via IAM Roles.

EC2 Instance

Authenticates to AWS via IAM Role

EC2 Instances get permissions to AWS Resources via IAM Roles.

EC2 Instance

S3 Bucket

New features in ECS

Feature #1: IAM Role for ECS Tasks

IAM Roles for EC2 InstancesPreviously, ECS Tasks could only get permission to other AWS resources (e.g. a file in S3) by using the IAM Role of the ECS Instance.

ECS Instance IAM Role

IAM Roles for EC2 InstancesThis meant that the BLUE and YELLOW app both got the same AWS permissions.

ECS Instance IAM Role

ECS Instance IAM Role

IAM Roles for ECS TasksWith IAM Roles for ECS Tasks, now each ECS Task can get its own IAM Role!

ECS Task IAM Role

ECS Task IAM Role

IAM Roles for ECS TasksThis means that each ECS Task can have its own set of permissions to other AWS resources.

ECS Task IAM Role

ECS Task IAM Role

Bucket A

Bucket B

How It Works‣ When we create an ECS Task Definition, we can

now specify a Task Role.

Feature #2: ECS Service Auto-Scaling

ECS Service Auto-ScalingPreviously, we could auto-scale the ECS Instances but not the ECS Tasks.

This meant that we could not auto-scale an ECS Service without lots of hackery.

ECS Service Auto-ScalingNow we can!

ECS Service Auto-ScalingWe define Scaling Policies just like with classic Auto Scaling.

We Scale on Just Two Metrics‣ CPUUtilization

‣ MemoryUtilization

Total CPU/Memory In Use

Total CPU/Memory Reserved=

Feature #3: EC2 Container Registry (ECR)

When you work with Docker, you need a place to store your Docker images.‣ Classic Docker build pipeline example:

Git Commit to Master Branch

Build Docker Image

Push to Docker Registry

There are a few options for the Docker Registry‣ Docker Hub

‣ Quay.io by CoreOS

‣ Artifactory by jfrog

But there are some challenges.‣ Docker Hub can sometimes be slow or

unreliable.

‣ Authenticating to any solution means you have to store the credentials somewhere.

‣ Download speeds and proximity to the service make a difference.

So Amazon has released EC2 Container Registry (ECR)

Amazon EC2 Container Registry

ECR Features‣ Fully managed by Amazon

‣ Relatively fast

‣ Accessible by a typical docker client

‣ Integrated with IAM Policies and IAM Users

ECR Limitations‣ You can only store up to 1,000 images per

docker repo.

‣ Pricing model requires you cull your unused docker images from the ECR repo.

‣ No hosting of public docker images.

‣ Docker repo names can be awkwardly long.

But I still prefer ECR.‣ One less vendor to deal with.

‣ One integrated security model.

‣ Repo limits are probably appropriate.

‣ Not hosting public repos gives clear separation of public and private repos.

Kinesis Analytics

Big Idea‣ As companies grow, they eventually evolve out

of the monolithic app and into a microservices architecture.

Microservice A Microservice B

‣ Usually, companies will start with two microservices.

‣ Then they’ll keep factoring out monolithic code into more and more microservices.

‣ Eventually, teams will want an individual microservice to publish an event stream.

‣ This way Microservice B can do something when Microservice A publishes a certain event.

‣ But if we have n services, and each service reads the event stream of the other n - 1 services, now we have a combinatorial explosion:

YUCK!

‣ What if instead all services published their event streams to a central service.

‣ And all services read event streams from that same central service.

‣ Now we have n connections, which is manageable!

‣ These are the insights that LinkedIn had around 2011 when it wrote Apache Kafka.

‣ The central “event publishing service” would need to be:

‣ scalable

‣ resilient

‣ temporarily persist data to support consumers that go down

‣ not lose any data, even as data volume surges

‣ The details are published in an epic blog post by LinkedIn engineer and Kakfa author Jay Kreps:

‣ It turns out the concept of a scalable, performant, resilient centralized event stream can apply to lots of domains!

‣ IoT events

‣ Logging events

‣ Social media clickstreams

‣ Basically, any real-time data source

‣ But running a Kafka cluster is highly non-trivial.

‣ So AWS introduced its own version of Kafka and offered it as a managed service.

Amazon Kinesis Streams

‣ At ReInvent 2014, Amazon shared a wicked cool example of how Major League Baseball was tracking data from the field and using it to generate stats, visualizations, and more:

‣ Here’s an excerpt from their architecture.

‣ But what happens after the data gets into Kinesis?

Amazon Kinesis

?

‣ The answer is that we can have Kinesis Consumers that periodically read the data.

Amazon Kinesis

Me Want Moar Data!

‣ The consumer can then do anything with it

‣ Store it in S3 for later retrieval.

‣ Store it in RedShift for later querying.

‣ Store it in a relational database.

‣ Or any other custom operation.

‣ Previously, we had to write our own custom worker to do any processing.

‣ But what if we just want to query windows of incoming data and write it to a database? Isn’t that pretty common?

‣ But now we don’t have to!

‣ That’s why Amazon has introduced:

Amazon Kinesis Analytics

Input - Query - Output‣ Inputs

‣ Streaming Data Sources: Kinesis Streams, Kinesis Firehose

‣ Reference Data Source: Data in S3

‣ Query

‣ Write ANSI SQL against the data stream

‣ Outputs

‣ S3

‣ Redshift

‣ Kinesis Firehose ( —> Amazon Elasticsearch)

‣ Kinesis Streams

Core Features‣ Use Standard SQL to query data streams.

‣ Kinesis will inspect your data stream and automatically create a baseline schema against which you can write your queries.

‣ Built-in live SQL editor to test queries against live data.

‣ Pre-written queries for common use cases.

‣ Query continuously, by Tumbling Windows, or Sliding Windows.

Let’s combine it all in a sample architecture!

Business Problem‣ Ice Cream shop

‣ IoT Enabled

‣ We track weight of each tub of ice cream continuously as a way to know in real-time how much ice cream we need to order.

‣ Our customer wants a slick real-time dashboard of everything.

Architecture

IoT Weight MonitorsKinesis Streams

Kinesis Analytics

S3 Bucket

ECS Cluster

App to Query S3 Data and return Dashboard data

App that serves static assetsfor a Single-Page App

ALB

Users get dashboard updates with WebSockets

RDS PostgreSQL

Caveats‣ If you had a low enough volume of data, you could just have

your sensors write directly to RDS Postgres and reduce lots of cost and complexity.

‣ But if you have enough data volume that you need the power of Kinesis, then this architecture makes sense.

‣ Querying S3 for real-time data is probably a bad idea, so it may make more sense to write a worker to read from S3 and write data to RDS Postgres or to use Redshift.

‣ Serving a static web app from an ECS app isn’t bad, but using S3 (+ CloudFront) is more efficient (but also more complex to setup).

Thank you!Want to keep up with the latest news on DevOps, AWS, software infrastructure, and Gruntwork?

http://www.gruntwork.io/newsletter/