dev ops on aws deep dive on continuous delivery - toronto
TRANSCRIPT
![Page 1: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mikhail Prudnikov, Solutions Architect
September 2016
DevOps on AWS:Deep Dive on Continuous Delivery
and the AWS Developer Tools
![Page 2: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/2.jpg)
https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?
![Page 3: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/3.jpg)
Software moves faster today
Software creation and distribution is
easier and faster than ever:
• Startups can now take on giants with little to
no funding ahead of time
• Getting your software into the hands of
millions is a download away
• Your ability to move fast is paramount to your
ability to fight off disruption
![Page 4: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/4.jpg)
Old software delivery model
The software delivery model has drastically changed
New software delivery model
![Page 5: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/5.jpg)
What tools do you need to move fast?
Releasing software in this new software driven world
requires a number of things:
• Tools to manage the flow of your software development
release process
• Tools to properly test and inspect your code for defects
and potential issues
• Tools to deploy your applications
![Page 6: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/6.jpg)
First, we need to
understand a little bit about
software release processes
https://www.flickr.com/photos/jurvetson/5201796697/
![Page 7: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/7.jpg)
• Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Release processes have four major phases
Source Build Test Production
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
container
images
• Deployment
to production
environments
![Page 8: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/8.jpg)
Release processes levels
Source Build Test Production
Continuous integration
Continuous delivery
Continuous deployment
![Page 9: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/9.jpg)
Release Processes levels
Source Build Test Production
Continuous integration
Continuous deployment
Our focus today
Continuous delivery
![Page 10: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/10.jpg)
Continuous Delivery Benefits
Improve developer
productivity
Find and address
bugs quickly
Deliver updates fasterAutomate the software
release process
![Page 11: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/11.jpg)
A look back at
development at
Amazon..
https://secure.flickr.com/photos/pixelthing/15806918992/
![Page 12: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/12.jpg)
2001
Development transformation at Amazon: 2001-2009
2009
monolithic
application + teams
microservices + 2 pizza teams
![Page 13: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/13.jpg)
Things went much
better under this
model and teams
were releasing faster
than ever, but we felt
that we could still
improve.
![Page 14: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/14.jpg)
In 2009, we
ran a study to
find out where
inefficiencies
might still exist
![Page 15: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/15.jpg)
We were just waiting.
WaitWrite Code WaitBuild
Code WaitDeploy to Test
Deploy to
Prod
![Page 16: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/16.jpg)
We were just waiting.
WaitWrite Code WaitBuild
Code WaitDeploy to Test
Deploy to
Prod
Mins Days Mins Days Mins Days Mins
![Page 17: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/17.jpg)
We were just waiting.
WaitWrite Code WaitBuild
Code WaitDeploy to Test
Deploy to
Prod
Weeks
Mins Days Mins Days Mins Days Mins
![Page 18: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/18.jpg)
We were just waiting.
WaitWrite Code WaitBuild
Code WaitDeploy to Test
Deploy to
Prod
Weeks
Mins Days Mins Days Mins Days Mins
![Page 19: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/19.jpg)
We built tools to
automate our software
release process
https://secure.flickr.com/photos/lindseygee/5894617854/
![Page 20: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/20.jpg)
Automated actions and transitions; from check-in to production
Development benefits:
• Faster
• Safer
• Simplification & standardization
• Visualization of the process
Pipelines
![Page 21: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/21.jpg)
This has continued to work out really well:
In 2014:
• Thousands of service teams across Amazon
• Building microservices
• Practicing continuous delivery
• Many environments (staging, beta, production)
50 million deploys
![Page 22: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/22.jpg)
We continue to survey our
software developers every year
and in 2014 results found only
one development tool/service
could be correlated statistically
with happier developers:
Our pipelines service!
![Page 23: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/23.jpg)
continuous delivery
==
happier developers!
https://www.flickr.com/photos/cannnela/4614340819/
![Page 24: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/24.jpg)
Continuous delivery service for fast and
reliable application updates
Model and visualize your software release
process
Builds, tests, and deploys your code every time
there is a code change
Integrates with 3rd party tools and AWS
AWS CodePipeline
![Page 25: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/25.jpg)
AWS CodePipeline Benefits
Configurable workflow Easy to integrate Improved quality
Rapid delivery Get started fast
![Page 26: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/26.jpg)
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
PipelineStage
Action
Transition
CodePipeline
MyApplication
![Page 27: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/27.jpg)
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
NotifyDevelopers
Lambda
CodePipeline
MyApplication
Parallel actions
![Page 28: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/28.jpg)
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
NotifyDevelopers
Lambda
TestAPI
Runscope
CodePipeline
MyApplication
Sequential actions
![Page 29: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/29.jpg)
Build
JenkinsOnEC2
Jenkins
Staging-Deploy
JavaApp
Elastic Beanstalk
Prod-Deploy
JavaApp
Elastic Beanstalk
QATeamReview
Manual Approval
CodePipeline
MyApplication
Manual ApprovalsReview
![Page 30: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/30.jpg)
You can add a manual approval at the point where you want the
pipeline to stop running until someone approves or rejects the
revision in progress.
Manual Approvals – New!
• Pipeline will stop executing when it has reached the point at which
you set the approval action.
• Pipeline execution resumes only when the action has been approved
• Approval action managed with AWS Identity and Access
Management (IAM) permissions
• Notify approvers in several ways including email, SMS, webhooks,
and more
• Useful for manual QA actions or as party of “Canary” deploy models
![Page 31: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/31.jpg)
8. Retrieve build artifact
EC2 instance
CodePipeline
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
JavaApp
Elastic Beanstalk
Source Artifact
S3
Build Artifact
S3
5. Get source artifact
1. Get Changes
6. Store build artifact
3. Poll for Job
4. Acknowledge Job
7. Put Success
9. Deploy build artifact
Elastic Beanstalk
Web container
Java App
MyApplication
![Page 32: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/32.jpg)
We have a strong partner list, and it’s growing
Source Build Test Deploy
*beta
![Page 33: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/33.jpg)
AWS service integrations
Source Invoke Logic Deploy
AWS Elastic Beanstalk
Amazon S3 AWS CodeDeployAWS Lambda
AWS CodeCommit
AWS OpsWorks
![Page 34: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/34.jpg)
Building your application
development release
pipeline
https://www.flickr.com/photos/seattlemunicipalarchives/12504672623/
![Page 35: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/35.jpg)
DEMO!
![Page 36: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/36.jpg)
![Page 37: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/37.jpg)
![Page 38: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/38.jpg)
Build & test your
application
https://secure.flickr.com/photos/spenceyc/7481166880
![Page 39: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/39.jpg)
Building Your Code
“Building” code typically refers to languages that
require compiled binaries:
• .NET languages: C#, F#, VB.net, etc.
• Java and related languages: Java, Scala,
JRuby
• Go
• iOS languages: Swift, Objective-C
We also refer to the process of creating Docker
container images as “building” the image.EC2
![Page 40: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/40.jpg)
No Building Required!
Many languages don’t require building. These
are considered interpreted languages:
• PHP
• Ruby
• Python
• Node.js
You can just deploy your code!
EC2
![Page 41: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/41.jpg)
Testing Your Code
Testing is both a science and an art form!
Goals for testing your code:
• Want to confirm desired functionality
• Catch programming syntax errors
• Standardize code patterns and format
• Reduce bugs due to non-desired application
usage and logic failures
• Make applications more secure
![Page 42: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/42.jpg)
Where to Start with Testing
1. Start with a generic syntax linter
• Are you missing brackets, commas, etc.?
2. Work on the top of the pyramid initially to
test user facing experience/functionality
• Are your pages rendering properly?
3. Move on to deeper UI/service checks
• Are these two components in sync?
4. Start unit tests against basic user functions
and move deeper into application logic
• Invest more time and effort here as you find
bugs or failures in production
![Page 43: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/43.jpg)
Deploying your
applications
https://secure.flickr.com/photos/simononly/15386966677
![Page 44: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/44.jpg)
Automates code deployments to any instance
Handles the complexity of updating your
applications
Avoid downtime during application deployment
Deploy to Amazon EC2 or on-premises
servers, in any language and on any operating
system
Integrates with 3rd party tools and AWS
AWS CodeDeploy
![Page 45: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/45.jpg)
appspec.yml Exampleversion: 0.0os: linuxfiles:
- source: /destination: /var/www/html
permissions:- object: /var/www/html
pattern: “*.html”owner: rootgroup: rootmode: 755
hooks:ApplicationStop:
- location: scripts/deregister_from_elb.shBeforeInstall:
- location: scripts/install_dependencies.shApplicationStart:
- location: scripts/start_httpd.shValidateService:
- location: scripts/test_site.sh- location: scripts/register_with_elb.sh
![Page 46: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/46.jpg)
appspec.yml Exampleversion: 0.0os: linuxfiles:
- source: /destination: /var/www/html
permissions:- object: /var/www/html
pattern: “*.html”owner: rootgroup: rootmode: 755
hooks:ApplicationStop:
- location: scripts/deregister_from_elb.shBeforeInstall:
- location: scripts/install_dependencies.shApplicationStart:
- location: scripts/start_httpd.shValidateService:
- location: scripts/test_site.sh- location: scripts/register_with_elb.sh
• Remove/add instance to ELB
• Install dependency packages
• Start Apache
• Confirm successful deploy
• More!
• Send application files to one
directory and configuration
files to another
• Set specific permissions on
specific directories & files
![Page 47: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/47.jpg)
v2 v2 v2 v2 v2 v2
one at a time
half at a time
all at once
v2 v2 v2 v1 v1 v1
v2 v1 v1 v1 v1 v1 Agent Agent
Dev Deployment group
ORProd Deployment group
Agent
AgentAgent
Agent Agent
Agent
Choose Deployment Speed and Group
![Page 48: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/48.jpg)
https://www.flickr.com/photos/spacex/16510243060/
Launching to
Production
![Page 49: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/49.jpg)
Launching to Production
After you’ve built and tested your code and
hopefully gone through a few pre-production
deploys, its time for the real thing!
You’ll want think about:
• Impact to customers
• Impact to infrastructure
• Impact to business
How can we track these and communicate
deploys?
![Page 50: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/50.jpg)
Extend AWS CodePipeline Using Custom Actions
Update tickets Provision resources
Update dashboards
Mobile testing
Send notifications Security scan
![Page 51: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/51.jpg)
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
Action
Custom Action
JavaApp
Elastic Beanstalk
MyApplication
CodePipeline
Deploy
JavaApp
Elastic Beanstalk Job Worker3. Perform Job
1. Poll for Job
2. Acknowledge Job
4. Put Success
Amazon
EC2
With custom actions,
the job worker drives the interaction
between AWS CodePipeline
and other applications or services
![Page 52: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/52.jpg)
Source
Source
GitHub
Build
JenkinsOnEC2
Jenkins
Deploy
MyAction
AWS Lambda
JavaApp
Elastic Beanstalk
2. Perform Job
1. Invoke Lambda function
3. PutJobSuccessResult
MyApplication
CodePipeline With AWS Lambda-based actions,
AWS CodePipeline
drives the integration with Lambda,
which then connects with other
applications or services
AWS
Lambda
3. PutJobSuccessResult w/
Continuation Token
4. Invoke Lambda function w/
Continuation Token
5. PutJobSuccessResult #3 & #4 repeat until no continuation
token is sent, signaling the action has
been completed (#5).
![Page 53: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/53.jpg)
What Extension Method Should I Use?
Lambda Custom Action
Short-running tasks are easy to build Can perform any type of workload
Long-running tasks need more work Control over links displayed in console
Node.js, Python, and Java support Any language support
Runs on AWS Can run on-premises
No servers to provision or manage Requires compute resources
![Page 54: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/54.jpg)
FIN, ACK
We’ve seen a quick run through today of the benefits of
continuous delivery on our software release process:
• Continuous integration (build/test) helps shrink our
feedback loop greatly
• We can get our software out in front of our users much
more rapidly
• By moving faster we can actually ensure better quality
• CodePipeline allows for integration with almost any
service or tool you can think of!
• Plus visualization of what’s going on!
![Page 55: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/55.jpg)
Try it out today
Test out CodePipeline and spin up a full continuous
delivery pipeline using the Starter Kit
bit.ly/AWSCodeStarterKit
![Page 56: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/56.jpg)
Resources to learn more:• Continuous integration: https://aws.amazon.com/devops/continuous-integration/
• Continuous delivery: https://aws.amazon.com/devops/continuous-delivery/
• CodePipeline
• https://aws.amazon.com/codepipeline/
• https://aws.amazon.com/documentation/codepipeline/
• CodeDeploy
• https://aws.amazon.com/codedeploy/
• https://aws.amazon.com/documentation/codedeploy/
• https://github.com/awslabs/aws-codedeploy-samples
• Code Services Starter Kit: http://bit.ly/AWSCodeStarterKit
But wait, there’s more!
![Page 57: Dev ops on aws deep dive on continuous delivery - Toronto](https://reader033.vdocuments.site/reader033/viewer/2022052405/587dc1491a28ab1b498b613d/html5/thumbnails/57.jpg)
Thank you!