eduardo piairo - deployment pipeline for databases (azure sql database, sql server)
TRANSCRIPT
![Page 1: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/1.jpg)
DEPLOYMENT PIPELINE FOR DATABASES Eduardo Piairo
@EdPiairo
#WinOpsAzure SQL Database, SQL Server
![Page 2: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/2.jpg)
ABOUT MEDeployment pipeline for databases
@EdPiairo, #WinOps
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/
Eduardo Piairo
Friend of Redgate
Operations Engineer
DevOps Porto Founder
![Page 3: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/3.jpg)
INDEX
1. Deployment pipeline
2. Database challenges
3. Deployment pipeline stages
4. Deployment pipeline scenarios
5. Final thoughts
Deployment pipeline for databases
@EdPiairo, #WinOps
![Page 4: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/4.jpg)
PIPELINE DEFINITION
In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines, functions, etc.), arranged so that the output of each element is the input of the next.
by Wikipedia
@ https://en.wikipedia.org/wiki/Pipeline_(software)
Deployment pipeline for databases
@EdPiairo, #WinOps
![Page 5: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/5.jpg)
DEPLOYMENT PIPELINE DEFINITION
A deployment pipeline should enable collaboration between the various groups involved in delivering software and provide everyone visibility about the flow of changes in the system, together with a thorough audit trail.
by Martin Fowler
@ https://martinfowler.com/bliki/DeploymentPipeline.html
Deployment pipeline for databases
@EdPiairo, #WinOps
![Page 6: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/6.jpg)
DEPLOYMENT PIPELINE STAGESDeployment pipeline for databases
Stage 1 – Change description (Source Control)
Stage 2 – Change validation (Continuous Integration)
Stage 3 – Change implementation (Continuous Delivery)
Source
ControlContinuous
Integration
Continuous
Delivery
@EdPiairo, #WinOps
![Page 7: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/7.jpg)
DATABASE & DEPLOYMENT PIPELINEDeployment pipeline for databases
Source
ControlContinuous
Integration
Continuous
Delivery
Database
+Application
@EdPiairo, #WinOps
![Page 8: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/8.jpg)
WHAT’S SO SPECIAL ABOUT DATABASES?Deployment pipeline for databases
@EdPiairo, #WinOps
![Page 9: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/9.jpg)
DATABASE CHALLENGESDeployment pipeline for databases
• Database and application changes (most of the time) are handled differently
• Different deployment pipeline
• Synchronization needed
• Database is not included (manual work)
@EdPiairo, #WinOps
![Page 10: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/10.jpg)
DATABASE CHALLENGESDeployment pipeline for databases
• Manual work …
• Lack of traceability of database changes (changes history)
• It’s the most expensive and risky work type
• Prevent CI and CD utilization in their full extent
• Promote the fear of changes
@EdPiairo, #WinOps
![Page 11: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/11.jpg)
DATABASE CHALLENGESDeployment pipeline for databases
Databases become a bottleneck in an agile delivery process
@EdPiairo, #WinOps
![Page 12: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/12.jpg)
AUTOMATIONDeployment pipeline for databases
@EdPiairo, #WinOps
![Page 13: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/13.jpg)
THE VALUE OF AUTOMATIONDeployment pipeline for databases
• Enable control over database development
• Increase speed of response to change
• Greater reliability of the release process
• Remove/reduce human intervention in the release processFrom
(Re)Learn and forget it
to
Improve and forget it
@EdPiairo, #WinOps
![Page 14: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/14.jpg)
THE VALUE OF AUTOMATIONDeployment pipeline for databases
Fearless database changes
@EdPiairo, #WinOps
![Page 15: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/15.jpg)
SOURCE CONTROLDeployment pipeline for databases
@EdPiairo, #WinOps
SQL Script
Fundamental resource
![Page 16: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/16.jpg)
SOURCE CONTROLDeployment pipeline for databases
• First step in your database deployment pipeline
• Traceability through change history
• SQL as documentation
• Shared code-base and shared process
• Enforceable standards to reduce conflicts
@EdPiairo, #WinOps
![Page 17: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/17.jpg)
• State based solutions
• How the database should be
• Migrations based solutions
• How the database should change
MIGRATIONS VS STATEDeployment pipeline for databases
SQL Source Control
@EdPiairo, #WinOps
Migration Migration
State
Delta
![Page 18: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/18.jpg)
SCRIPTING GUIDELINESDeployment pipeline for databases
• One script, one operation type, one object (small batches)
• Merge conflicts management
• Patterns identification
• File system scripts history search
@EdPiairo, #WinOps
![Page 19: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/19.jpg)
CONTINOUS INTEGRATIONDeployment pipeline for databases
• Integrate and validate changes
• Tests: Unit, Integration
• Small batches, less risk
• Considerations:
• What should be tested? And When?
• Can the application help on this?
@EdPiairo, #WinOps
![Page 20: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/20.jpg)
CONTINUOUS DELIVERYDeployment pipeline for databases
• Delivering change(s) in the target environment
• Considerations
• Downtime
• Time to recover
• Small batches – low risk
• Affected applications
@EdPiairo, #WinOps
![Page 21: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/21.jpg)
FAILED DEPLOYMENTDeployment pipeline for databases
• Rollback/roll forward scripts
• Very fast
• Please, do not make mistakes
• It’s like working backwards
• Backups (before deploy)
• Safer but slower
• Keep databases small
• Avoid share databases between applications
@EdPiairo, #WinOps
![Page 22: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/22.jpg)
SCENARIO #1Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Independent deployment pipelines
• High need of synchronization between DBA and development team
![Page 23: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/23.jpg)
SCENARIO #2Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Different code repositories
• Independent CI process
• The contact point will be the deployment moment
![Page 24: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/24.jpg)
SCENARIO #3Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APP
DB
@EdPiairo, #WinOps
• Different code repositories
• Connected/dependent CI process
![Page 25: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/25.jpg)
SCENARIO #4Deployment pipeline for databases
Source Control Continuous Integration Continuous Delivery
APPDB
@EdPiairo, #WinOps
• Share the some code repository, CI process and CD process
• No need of synchronization
• Promotes learning within teams (database code reviews)
![Page 26: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/26.jpg)
DEMODeployment pipeline for databases
@EdPiairo, #WinOps
![Page 27: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/27.jpg)
MY DEPLOYMENT PIPELINE DEFINITION
Pipeline: Cultural and technical tool for managing changes in the software development process (should contemplate databases, applications and infrastructure).
by me ;)
Show me your pipeline and I will tell you the way you work!
Deployment pipeline for databases
@EdPiairo, #WinOps
![Page 28: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/28.jpg)
DEVOPS AND DATABASESDeployment pipeline for databases
• DevOps way
• Increase flow visibility
• Increase feedback
• Increase knowledge base
@EdPiairo, #WinOps
![Page 29: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/29.jpg)
READING MATERIALDeployment pipeline for databases
@EdPiairo, #WinOps
![Page 30: Eduardo Piairo - Deployment pipeline for databases (Azure SQL Database, SQL Server)](https://reader031.vdocuments.site/reader031/viewer/2022021816/5aacd0f57f8b9a1f368b458b/html5/thumbnails/30.jpg)
Q&A
Deployment pipeline for databases
@EdPiairo, #WinOps
@EdPiairo
https://pt.linkedin.com/in/jesuspiairo
http://www.eduardopiairo.com/