active commerce product build automation - …€¦ · active commerce product build automation ......
TRANSCRIPT
Active Commerce Product Build AutomationNick Wesselman
April 26, 2016
Or…
“Anything is possible with enough time and
PowerShell.”
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 3
• What is Active Commerce?
• Goals and challenges of automating our product builds
• How we did it
• What does it look like
Agenda
What is Active Commerce?
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 5
Native Sitecore E-commerce
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 6
Quick to Market via Theming
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 7
Flexible Architecture
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 8
Integration and Extension
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 9
Targeted at the Mid-market
F100
Oracle/ATG
B2C B2B
SM
B
SAP/Hybris
JDA
Micros Retail
IBM
eBay
Enterprise
Demandware
Digital River
NetSuite
EPiServer
Magneto
NetSuite/Venda
Shopify
InSite
Sitecore
Commerce
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 10
Version Compatibility
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 11
Customers
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 12
Partners
UNITED STATES CANADA NORWAY
AUSTRALIA
For more information
Stop by our table during lunch or
one of the breaks today
http://help.activecommerce.com
13
Automation Challenges and
Goals
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
What was our product build process?
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 17
Everything in one Solution
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 18
Supporting multiple Sitecore versions
master (SC72)
master-SC75
master-SC80
19
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 20
• Reduce time and error setting up environments (development, testing)
• Reduce time and error creating product releases
• Support multiple versions of Sitecore without forking /branching
• Allow plugins to be versioned and released independently
• Automation of testing
Goals for Automation Project
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 21
Setup Development Environment
• One PowerShell command for any Sitecore Version
• Easily toggle between Sitecore versions
• No forking for version-specific Sitecore logic
Product and Plugin Debug Builds
• On every checkin
Setup Test Environments
• Via configuration file / PowerShell command
Desired Process
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 22
Development Environment
Product Build
Test Environment
Processes
Source
CodeLocal Install
Source
CodeBuild Install
Sitecore
Package
Sitecore
PackageTest Install
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 23
• Items are important
• Items need to be deployed
• Installing Sitecore and dependent modules
• Build dependencies / lack of Sitecore nuget
• Maintenance tasks (publishing, links database, index rebuild)
Challenges – Sitecore build automation
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 24
• Supporting and testing on multiple versions of Sitecore
• Creating Sitecore Zip packages for product installation
• Additional installation steps
• Plugin builds that are dependent on product builds
• Integration tests (NUnit Web Test Runner)
Challenges – Active Commerce build automation
25
How we did it(and what it can
do for you)
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
Visual Studio
MSBuild
Team Development for Sitecore
NUnit Web Test Runner
PowerShell
PowerShell
Octopus Deploy
Azure Virtual Machines
Azure Blob Storage
Atlassian Bamboo
PowerShell
PowerShell
ProGet
SlowCheetah
Sitecore Instance Manager
Sitecore PowerShell Extensions
PowerShell
PowerShell
Tools
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Separate solution for core product and each plugin
• Nuget for all dependencies
• Build properties for Sitecore version differences, populated by PowerShell scripts
• “Templated” solution structure for all plugins
What it can do for you
• Simplify dependency management via nuget
• Automate build activities using MSBuild
Tools – Visual Studio, Nuget and MSBuild
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 29
Using Build Properties
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 30
Using Build Properties
31
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Source control of serialized items
• Building Update Packages for environment setup and product builds
• Dev Root > Web Root Builds
• Codegen of content Templates for Glass Mapper
What it can do for you
• Allow distributed Sitecore development
• Automatic and GUI-based item synchronization with your Visual Studio solution
• Environment configuration replacement / management
• Automated validations against checked in templates and items
• Post-deployment operations (Publish, Links database, reindexing)
• Item merging
• Content file sync
• And much more…
Tools – Team Development for Sitecore
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Private nuget repository for Sitecoreassemblies and other dependencies not on public nuget servers
• Debug and release product build references for plugins
• Symbol server for debugging product issues with plugins
• Version-specific Sitecore files and Web.config
• Used with a customized Sitecore NugetPackage Generator (Alen Pelin)
What it can do for you
• Internal nuget server (or npm, or Bower, or chocolatey…)
Tools – ProGet
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 34
Nuget Packages
35
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Transform base Sitecore Web.config to a Web.config with Active Commerce additions for specific Sitecore versions
• Transform Active Commerce configuration files for specific Sitecore versions
• Transform Sitecore + Active Commerce Web.config with plugin-specific adds
What it can do for you
• XDT transforms for any config file
• Transform “unpatchable” base SitecoreWeb.config with your additions, before adding build-specific XDT transforms
Tools – SlowCheetah
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 37
SlowCheetah Use
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Execute NUnit tests with a full Sitecore and HTTP context
• Some legacy tests that should become unit tests
• Some Sitecore/SES dependencies that make unit testing difficult
• Some true integration tests
• Results available as JSON – pull in via NUnitTestCaseSource
What it can do for you
• Run integration tests
• Test code w/ Sitecore dependencies when FakeDB or similar tools not possible/practical
Tools – NUnit Web Test Runner
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 39
JSON Test Results
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Invoke via custom PowerShell wrapper to create Sitecore instances for development, product build, and testing
• Install Sitecore Zip packages
• Customize SIM pipelines with PowerShell-based steps
• Redirect SIM output to the shell
What it can do for you
• Manage Sitecore instances
• Install Sitecore Zip packages
• Backup/Restore Sitecore instances
• Export/Import Sitecore instances
• Tail Sitecore logs
• Generate Sitecore nuget packages
• Install MongoDB
• and more…
Tools – Sitecore Instance Manager
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 41
Wrapping and Extending SIM
42
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 43
• Invoking SIM classes directly
• Initializing SIM without its GUI
• Passing data to SIM steps via environment variables
• Deleting / cleaning up instances requires extra work
CAUTION: This turned out pretty hacky
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• SPE Remoting allows us to execute logic on a remote Sitecore instance
• Publishing
• Links DB Rebuild
• Index Rebuilds
• Initializing the Active Commerce database
• Script-based builds of Sitecore Zip packages
What it can do for you
• Change your life
• Content maintenance / manipulation
• Custom reporting
• Data exports / imports
• Content Editor extensions (context menu, ribbon buttons, gutter…)
• Automation of almost anything inside Sitecore
Tools – Sitecore PowerShell Extensions
45
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 46
Sitecore PowerShell = MAGIC
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Package our automation scripts, SIM, and build artifacts
• Securely deploy to product build or test environment using “tentacles” installed on the servers
• Invoke a deployment PowerShell script to create the environment
• Monitor script output
What it can do for you
• Deployment automation
• Continuous Deployment
• Automate IIS and much more
• Blue-green deployment
• Integrate with your existing build system
Tools – Octopus Deploy
48
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• VMs for product build and test environments
• Multiple environments per VM
• Scripted to shut down / start up only during business hours
• Install and run Sitecore from Temporary Storage (much faster)
What it can do for you
• VM option when you can’t run PaaS
Tools – Azure Virtual Machines (IaaS)
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Cloud file storage for build dependencies and artifacts
• Sitecore and module installers, license files
• Base product Sitecore Zip packages (debug and release)
• Plugin Sitecore Zip packages (debug and release)
• AzCopy provides scriptable upload/download including delta synchronization
What it can do for you
• Serving images or documents directly to a browser
• Storing files for distributed access
• Streaming video and audio
• Performing secure backup and disaster recovery
• Storing data for analysis by an on-premises or Azure-hosted service
Tools – Azure Blob Storage and AzCopy
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Glue everything together
• Functions for common developer tasks, build steps
• Structured into a PowerShell Module
• Scripts managed in their own git repository
• PowerShell Tools for Visual Studio
• JSON files for global and instance configuration
What it can do for you
• What can’t it do?
Tools – PowerShell PowerShell PowerShell
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 52
JSON Configuration
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved. 53
JSON Configuration
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
How we use it
• Triggered and scheduled builds
• Most steps just execute PowerShell
• Build dashboard
• Build reports in HipChat
What it can do for you
• A nice build / continuous integration server if you are already drinking the Atlassian kool-aid
Tools – Atlassian Bamboo
55
What’s it look like?
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
Setup a new development environment for a Sitecore version
Initialize-AcDev -SitecoreVersion 8.1
Switch the active Sitecore version
Switch-SitecoreVersion -SitecoreVersion 7.2
The same for plugins
Initialize-AcPluginDev -ActiveCommerceVersion 3.3 -SitecoreVersion 8.1
Switch-AcPluginVersion -ActiveCommerceVersion 3.3 -SitecoreVersion 7.2
Install a new Active Commerce instance based on a predefined config
New-AcInstance -ConfigPath .\AC33SC81.json -InstanceName Sherpa
PowerShell Commands
58
Development Environment
Setup
Product Build in Bamboo
Test Environment Setup
© 2016 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.
• TeamCity (Build templates for plugins)
• Enhance SIM for easier scripting
• Move version-specific Sitecore logic to its own assembly
• Docker containers for product builds
• Selenium integration tests
Future Improvements
Acknowledgements
Pavel Veller
Brian Beckham
Adam Najmanowicz
Michael West
Vincent Séguin
Alen Pelin
Alexander Doroshenko
Jorge Lusar
Sean Kearney
Charles Turano
Alex de Groot
and more…