continuous integration testing - dave stanke › img › davestanke-continuous... · continuous...
TRANSCRIPT
![Page 1: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/1.jpg)
Continuous Integration TestingFully test your microservices application, early and often
davidstanke@
![Page 2: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/2.jpg)
davidstanke@Continuous Integration Testing
Hi, I’m Dave.DevOps Advocate @ Google
davidstanke
![Page 3: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/3.jpg)
davidstanke@Continuous Integration Testing
Continuous Integration Testing1. Integration testing is hard
2. We can do it anyway
3. We should do it anyway
4. Some ways to do ita. Kubernetes / microservices example
![Page 4: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/4.jpg)
davidstanke@Continuous Integration Testing
Shift Left
code test deploy operate
Move critical processes more leftward
![Page 5: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/5.jpg)
davidstanke@Continuous Integration Testing
Shift all the things. Shift them left.
Merge
Security
Testing
![Page 6: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/6.jpg)
davidstanke@Continuous Integration Testing
source: Accelerate: State of DevOps 2018
Shifting left is good for your business
Continuous Testing
Continuous Delivery
Software Delivery Performance
Availability
![Page 7: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/7.jpg)
davidstanke@Continuous Integration Testing
Ideal test pyramid
Unit
Integration
End-to-End
![Page 8: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/8.jpg)
davidstanke@Continuous Integration Testing
Actual test pyramid
Unit
Integration
End-to-End
![Page 9: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/9.jpg)
davidstanke@Continuous Integration Testing
Integration testing is hardProduction runtimes != developer machines
Cost
Speed
Side effects
![Page 10: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/10.jpg)
davidstanke@Continuous Integration Testing
Why bother?API contracts / semver aren’t enough
Emergent properties of composed systems
Users experience an application
![Page 11: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/11.jpg)
davidstanke@Continuous Integration Testing
Don’t bother?
Test in Prod?!?
![Page 12: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/12.jpg)
davidstanke@Continuous Integration Testing
Don’t bother?
Test in Prod!And before prod.
![Page 13: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/13.jpg)
davidstanke@Continuous Integration Testing
How bad could it be?
No, seriously… how bad could it be?
![Page 14: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/14.jpg)
davidstanke@Continuous Integration Testing
They’re hard, but (potentially) worth it
Make informed risk analysis
Continuously optimize
We can still do integration tests
![Page 15: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/15.jpg)
davidstanke@Continuous Integration Testing
What are we optimizing for?Fidelity
Isolation
Scalability
Speed
Ephemerality
Cost
![Page 16: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/16.jpg)
davidstanke@Continuous Integration Testing
Fidelity
Illustrations by TRACEY LAGUERREIllustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 17: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/17.jpg)
davidstanke@Continuous Integration Testing Illustrations by TRACEY LAGUERRE
Isolation
Illustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 18: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/18.jpg)
davidstanke@Continuous Integration Testing
Scalability
Illustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 19: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/19.jpg)
davidstanke@Continuous Integration Testing
Speed
Illustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 20: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/20.jpg)
davidstanke@Continuous Integration Testing
Ephemerality
Illustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 21: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/21.jpg)
davidstanke@Continuous Integration Testing
Cost
Illustrations by TRACEY LAGUERRE
Integration Testing Goal:
![Page 22: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/22.jpg)
davidstanke@Continuous Integration Testing
An exampleCI pipeline with integration tests
/GoogleCloudPlatform/cloudbuild-integration-testing
![Page 23: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/23.jpg)
davidstanke@Continuous Integration Testing
Integration test flow
build
web db
provision
web
db
test.sh
curl | grep
deprovision
if pass
![Page 24: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/24.jpg)
davidstanke@Continuous Integration Testing
Technique 1: Docker Compose
![Page 25: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/25.jpg)
davidstanke@Continuous Integration Testing
Technique 2: K8s Staging Cluster Kubernetes Engine
Namespace Web DB
Namespace Web DB
Namespace Web DB
![Page 26: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/26.jpg)
davidstanke@Continuous Integration Testing
Technique 3: Self-destructing VM
![Page 27: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/27.jpg)
davidstanke@Continuous Integration Testing
Self-Destructing VM?
http://bit.ly/self-destructing-vm
![Page 28: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/28.jpg)
davidstanke@Continuous Integration Testing
Technique 3: Self-destructing VM!
![Page 29: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/29.jpg)
davidstanke@Continuous Integration Testing
Technique 4: K8s cluster per test Kubernetes Enginecluster: `test-123`
Web DB
gcloud gcloud
![Page 30: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/30.jpg)
davidstanke@Continuous Integration Testing
Integration testing techniquesMethod Fidelity Isolation Scalability Ephemerality Speed Cost
docker-compose ⬤ low ⬤ highest ⬤ highest ⬤ absolute ⬤ fastest ⬤ low
Shared “staging” K8s ⬤ high ⬤ medium ⬤ high ⬤ variable ⬤ fast ⬤ low
Single-node “k8s” ⬤ medium-high ⬤ high ⬤ high ⬤ variable ⬤ medium ⬤ medium
K8s per test ⬤ highest ⬤ high ? ⬤ variable ⬤ slow ⬤ high
![Page 31: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/31.jpg)
davidstanke@Continuous Integration Testing
Integration testing techniquesMethod Fidelity Isolation Scalability Ephemerality Speed Cost
docker-compose ⬤ low ⬤ highest ⬤ highest ⬤ absolute ⬤ fastest ⬤ low
Shared “staging” K8s ⬤ high ⬤ medium ⬤ high ⬤ variable ⬤ fast ⬤ low
Single-node “k8s” ⬤ medium-high ⬤ high ⬤ high ⬤ variable ⬤ medium ⬤ medium
K8s per test ⬤ highest ⬤ high ? ⬤ variable ⬤ slow ⬤ high
![Page 32: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/32.jpg)
davidstanke@Continuous Integration Testing
But Dave, what about...Lots of services?
Databases?
Test data?
![Page 33: Continuous Integration Testing - Dave Stanke › img › DaveStanke-Continuous... · Continuous Integration Testing davidstanke@ Yes we can (still) do integration testing There are](https://reader033.vdocuments.site/reader033/viewer/2022050715/5f0d50b57e708231d439bd20/html5/thumbnails/33.jpg)
davidstanke@Continuous Integration Testing
Yes we can (still) do integration testingThere are trade offs between different ways of doing it
It’s all about risk tolerance vs. reward of speed/cost/etc.
You have to find your own sweet spot. (When you do, tell me about it!)
davidstanke