case study: time warner cable's formula for maximizing adobe experience manager

AEM Maxed! = Agile + Automation Overcome three common challenges with AEM delivery

2015 Adobe Summit Cat Reusswig, Senior Director, Web Portals Engineering, Time Warner Cable Mark Kelley, Client Services, iCiDIGITAL

Presentation Agenda

1. Overview three common AEM delivery challenges 2. See how Time Warner Cable has overcome each of these

challenges and evolved over the years 3. Revealing of three open-source tools for AEM being

developed by TWC and iCiDIGITAL 4. Q&A

Questions? #ici #adobesummit

Presentation Contests

First to Answer the Question Wins.


Ask a Question on Twitter #ici #adobesummit

Most Favorites by 12PM, Thursday, wins.

Common Challenges When Implementing AEM

1. Lack of collaboration among business, IT, and marketing stakeholders leads to inefficiencies and possibly failure 2. Creating and scaling the AEM delivery team to serve multiple websites and business units 3. Inadequate automation & testing focus results in poor code

quality and site experience


Common AEM Challenge #1 Lack of collaboration among business, IT, and marketing stakeholders leads to inefficiencies


CULTURE IS KEY

A way of thinking, behaving or working that exists in an organization


2012

•  1 scrum team grew to 3 teams

• Definition of done •  Tedious manual deploy model

• Agile infancy, esp on business side

2013

•  3 scrum teams grew to 6 teams

•  Improved automation testing

• Maturing agile

•  Improved continuous integration

•  Improved user story development

2014 Remained at 6 teams for

major release Humming

along…

TWC Maturation over Three Years


TWC Definition of Done for Development:

l  Meets acceptance criteria

l  Is deployable

l  Been tested & code reviewed

Expanded later to: l  Test in both author & publish and multi Devices/Browsers

l  Generally code review & test review are +2

A Philosophy We Embrace


The model we began with in the early days

Development Cycle(s)

Wires / CSS Comps User Stories CQ5

Components Enhancements

Testing Dev Automation Acceptance OS /

Browsers Bug Fixes

Production Content Authoring QA Updates

Roll-out Strategy Prod Support Prioritization Future Phases

TWC Responsibility Model (2012)

Business Responsibility

Development Responsibility

Troubleshooting

Shared Responsibility


TWC Responsibility Model (Today)

Business

Development

Intake E-governance

UAT

Post Production Validation

Story/Defect Prioritization

Both

Story Review

Requirements/Design (Create Stories)

Backlog Management eCommerce Priority Review

Sprint Planning

Mid-Sprint Demo

Sprint Closure

Daily Stand-up Scrum of Scrums

Tasking

Dev / Test / CI

Production Release

Sprint Retro

Beta Release

Architecture

How and Why

DOTs Handoff

UAT Handoff

Go/No-Go

2-Week Sprint


– External and internal requests use Podio Request Form

- New requests are reviewed weekly

-  Request includes l  Categories (for routing request to appropriate team)

l  Description of effort

l  Justification for effort

l  High level deliverables and requirements

l  Requested release dates -  Request processes l  Requests for content authoring directed to Authoring Team

(no development necessary).

l  Requests for new functionality, features or projects are directed

to eGovernance team for review and prioritization.

l  Project status and details are tracked within Podio ticket.

Product Request Intake Process & Routing Questions? #ici #adobesummit

MVP – Minimal Viable Product, the smallest useful set of capabilities that will meet the business need in production

Minimum Viable Product Product Vision

This came later too – and is a critical success factor now

MVP and Release Planning


P Purpose O Outcomes W What’s in it for me

E Expectations from participants

R Result

Understand the business problem to be addressed Inform product backlog ROM Difficulty (Does story need to be split. Does it roughly match business expectations of difficulty)

Test the edges of the solution recommended: RWD, failure conditions, failure responses, success responses, do we have a spork that would fit the need, 508 considerations, patterns for reuse

Provide a readiness evaluation to the Product Manager, UX, Author, Dev Team, Testing, and Management so everyone knows what we’re building and why

•  Subset of Development and Testing Team •  Subset of Authoring Team •  Technical Product Owners (TPOs) •  Product Owners

Attendees:

Story Review (POWER)


1. Held weekly

2. Incoming stories

l  Reviewed for completeness

l  Multi-Story efforts categorized and ranked into Epic

l  Prioritized per eGovernance priority list

l  Prioritized agenda set for Story Review

3. Defect Review

l  Reviewed for duplication

l  Reviewed for reproducibility

l  Confirmed it is an Omega defect

-  If not, directed to proper team for resolution

l  Reviewed to determine there is enough detail

For the Team to work including repo steps,

Screen shots, browser version, environment, etc.

5. Cherry-Picks

l  Requested for gating or urgent issues.

Ranking, Categorization, and Review of Stories and Defects

Story Defect / Prioritization

4. Rank order prioritized within categorical groups:

l  Gating Defects (P1) – Defects which will prevent a Sprint release

l  Critical Defects (P1.5) – Defects that are critical, but will not hold up a Sprint release

l  Prioritized Defects (P2) – Defects which are considered high priority

l  Backlogged Defects (P3) – Defects which can remedied as resource availability allows

l  TechPriority / TechDebt Defects - Architectural or structural defects for prioritization by technical team =20%


Common AEM Challenge #2 Creating and scaling the AEM delivery team

to serve multiple websites and business units


Shake up the teams ~every 6 months forges

greater strength

& relationships

6 Scrum Teams l  Volunteer Scrum Master with Double Duty

(Dev/QA) l  Team commits to stories accepted &

pointed

l  Fibonacci pointing

l  Team Norms, responsible for DONE!

l  Generally 6 members per team

l  Embedded QA

l  Mixed geography & time zones

Team Alignment with Autonomy

Horizontal Team l  TPO

l  DevOps

l  QA Automation

l  Analytics

l  Architecture Forum

l  How & Why Forum

Support l  DOTS/DevOps = Support across all environments


P Purpose O Outcomes

W What’s in it for me E Expectations from

participants

R Result

Development team discuss and commit to work for sprint Discussion of story details and questions beyond what’s in the ticket. Re-calibrate views of complexity for roadmap. Identify missing information.

Clarity of skillset and team availability to inform selection of work Discussion of whether to roll or complete stories from previous sprint. Evaluate work critically. Ownership of expected work. Share learning from story refinement. Document decisions.

Clear picture of expected deliveries

Attendees: Scrum Team, Product Owner, Technical Product Owners, Cross functional team

Sprint Planning (POWER)


- Break Up Assigned Stories

•  Dissect stories/epics

•  Create as many sub-tasks as needed

•  Create investigation tasks if needed

- Collaborate As Needed

•  Contact TPO/Business with questions

•  Pull in other teams that have SMEs

•  Brainstorm on possible solutions

- Assign Work

•  Everyone should have at least 1 task

•  Stories should be worked by priority

•  Unassigned tasks get picked up as

bandwidth becomes available

Sprint Tasking


P Purpose O Outcomes W What’s in it for me


R Result

Expose blockers beyond individual team control Coordination among teams Share information across teams, received help for your groups needs, learn from others.

Bring a summary of the blockers & risks from your SoS, as well as any crossteam coordination needs not yet socialized

Informs risks to the sprint (business and technical)

SoS format is similar to a stand-up (short, focused on what is not yet known)

Daily Stand-Up / Scrum of Scrums (SoS)


P Purpose

O Outcomes W What’s in it for me


R Result

Avoid the problem of sprint closure being the first time Dev team gets feedback from the business

Confirm we are on the right path. Mid-sprint SOS with business Early preview of functionality for business. Discuss proposed solutions and ideas to solve problems. Potentially identify an immediate refactor candidate (up to team whether to digest in current sprint)

Development – Share in progress on stories Business – Have questions answered

Alignment and understanding across business and technical teams, scrum teams.

Mid-Sprint Demo


P Purpose O Outcomes

W What’s in it for me E Expectations from

participants

R Result

Demo the agreed upon sprint work to the stakeholders Inform progress against roadmap. Clarification on any possible blockers or incomplete work.

Confirmation that our sprint work is “done” Prepare to demo work and knowing sprint work is complete

The completion of the sprint

Sprint Closure


-  Reflect on previous sprint’s work -  Each team executes as works best for them -  Find out what was good, bad, or neutral -  Approximately 1-2 hours (as needed) -  Determine action items -  Evaluate action items from previous sprint -  Try to avoid repeating mistakes -  EVERYONE PARTICIPATES

Sprint Retrospective


O Outcomes W What’s in it for me


R Result

Share technical knowledge among technical team, across teams Get a common sense of best practices

Document via Wiki or web links Learn something new from others’ experiences

Don’t have to make the same review comments repeatedly Time is not wasted by presenting information that is common or not useful. Presented topics are documented well enough to discuss. Developers write better code and use tools more effectively

Attendees: Developers, QA, DevOps (Technical Folks)

P Purpose

How and Why


“Only thing we have to fear is fear itself” FDR

•  Don’t be afraid to take a big step/bite – we added close to 10 people at a time

•  We failed a bunch – don’t be afraid to try, fail, and adjust

•  Do retrospectives & actually take action

•  Create horizontal forums

•  We reorganized and added roles – TPO, Architect, DevOps, Tier1, ++

•  The business team is completely different & has a different set of roles

•  We consistently do more face-to-face whether virtual or live

Our Approach to the AEM Unknown Questions? #ici #adobesummit

-  Onboarding ease -  Horizontal consistency inducement -  Peer mentorship -  Team acceptance -  Team working agreements -  Training

Critical Success Factors to Growing the Team


Common AEM Challenge #3 Inadequate automation & testing focus results

In poor code quality and site experience


Tools used to help automate the agile process and develop test-driven culture


Deploy & Validate Environment

Deploy & Validate Environment

Integration Tests

Integration Tests

Build Unit Tests Sonar

Build Unit Tests Sonar

Functional Test (no UI)

Functional Test (no UI)

UI Tests (FF, IE, & Chrome)

UI Tests (FF, IE, & Chrome)

If successful If successful

If failure, email Everyone! If failure, email

DevOps If failure, email

Everyone! If failure, email

Everyone!

Dev/Test/Cl (CI)

If failure, email Everyone!


CR+1 Testing+1

GHE = +1 creates a culture of/for code


CI kicks off with developer commits Automated JIRA Comments

Testing Directed by Jenkins

SonarQube code quality

Performance with WebPageTest Private instance of WebPageTest (WPT) server

All Chrome UITests automatically save performance results to WPT server Visual breakdown of all page content load times

GHE = +1 creates a culture of/for code Questions? #ici #adobesummit

GebConfig.groovy

‘firefox’ { driver = “firefox”

} ‘ipad’ {

driver = “org.openqa.selenium.iphone.IPhoneDriver” } ‘android-grid’ { driver = { new RemoteWebDriver(new URL(seleniumGridUrl), DesiredCapabilities.android()) } ‘ie8-winXP’ { DesiredCapabilities caps = DesiredCapabilities.internetExplorer(); caps.setCapability(“platform”, “Windows 2003”); caps.setCapability(“version”, “8”); caps.setCapability(“name”, “regression-IE8-WinXP”) driver = { new RemoteWebDriver(new URL(seleniumGridUrl), caps) }

}Specifying Tests with Spock

Technologies

Coding with Groovy

Organizing with Geb

Specifying with Spock

Directing with Jenkins

Running with Gradle

Managing with Grid2 and Vagrant

Diversifying with SaucelLabs

}Diversifying tests w SauceLabs

Windows VirtualBox images with “vagrant up” attaches to 10 min

Local browser

Local emulator

Remote VMs Attached to Grid

SauceLabs VM Attached to Grid

31


Beta Release / UAT Handoff - UAT Handoff

•  Review Wikis (UAT Team)

•  Meet with DOTS team (or representative)

•  Ask Questions

•  Prepare to Test

- Beta Release

•  Cut Code and Create Release Candidate

•  Deploy to Beta (Test) Environment

•  Mark tickets for handoff

•  Provide UAT Wikis

32


- UAT includes:

•  Functional GUI testing of planned release effort

•  Content and image verification

•  Regression testing of existing functionality

•  Responsive testing on mobile devices

•  Cross browser validation

•  Defect validation from external reporters

•  Test and Target

•  Analytic and GTM/DTM testing coordination

- New defects are logged for Story/Defect Prioritization review

- Critical issues are tagged and escalated for immediate review by product team

User Acceptance Testing (UAT)

33


- Represented Teams

•  Residential

•  Checkout

•  Offer Configuration

•  Authoring

•  UAT

•  Analytics

•  GTM / DTM

•  Test & Target

Are we all on-board?

Go / No-Go

- Final release sign-off from all impacted teams - Review of UAT status

•  Confirm all tickets tested and status

•  Review impact of any defects being released

•  Confirm all gating issues resolved

34


What is AppDynamics? An application performance management software What does it for? Designed to help dev and ops troubleshoots problems in complex production app Key Benefits -Get complete visibility into application performance -Monitoring business Transaction in real time -Set up proactive alerting to find problems before they Hurt your business -Fix problem faster than ever before.

Any build to any environment Relic / Artifactory Artifactory is a Content Reponsitory which, uses the Apache Jackrabbit implementation. This natural compatibility makes it extremely easy to manage fully-indexed and searchable metadata Versions stored in artifactory Automatic Confluence updates – which code is where

Push Button Environment Deploys

DevOps

-Ever try to sift through log files across multiple servers trying to find an individual error or a trend of errors? -Splunk presents all your data from multiple sources and makes it available from a single location, making it easy to search for patterns. -Some of the things that we have leveraged splunk for: l  We Detecting Dos attack occurring on our rr.com site l  Used it for various analytics to learn about our customers l  Used if for capacity planning and finding load balancing problems. l  We have identified code defects by trending errors l  and many many more...

Puppet is IT automation software that help system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage changes, scaling from 10s of servers to 1000s, on-premise or in the cloud.

35

Production Release / DOTS Handoff - DOTS Handoff

•  Meet with New DOTS teams

•  Relay any issues that are unresolved

•  Note any Gotchas or Pitfalls

•  Update Wiki

- Production Release

•  Cut Code

•  Prepare Beta Environments for Test

•  Handle Beta and Prod Issues

•  Deploy Release

•  Provide DOTS Handoff

36


Did it work as expected?

Post Production Validation

- Validating tickets includes

•  Tickets included in the release are working as expected

•  Mini regression confirming core functionality is working as expected (not a full regression suite)

•  Mini browser compatibility and device testing suite

•  Validation of production content per copy decks (testing environment uses non-confirmed contents)

•  Offer validation

- New defects are logged for Story / Defect Prioritization review - Critical issues are tagged and escalated for immediate review by production team

37


38

Some Coming Open Source Attractions J


Grabbit Provide a fast and reliable way of copying content from

one Sling (specifically Adobe CQ / AEM) instance to another.

https://github.com/TWCable/grabbit


CQ Gradle Plug-ins A number of plugins to make it easer to work with

Adobe AEM from Gradle (Gradle.org), including CQ Package, SCR, Sling Bundle, and Bundlor.

https://github.com/TWCable/cq-gradle-plugin


Jackalope An in-memory implementation of the JCR with stubbing

capabilities for Apache Sling, better enabling unit testing of JCR representations with complex objects

and structures.

https://github.com/TWCable/jackalope


Questions

2015 Adobe Summit

Cat Reusswig, TWC & Mark Kelley, iCiDIGITAL


case study: time warner cable's formula for maximizing adobe experience manager

Technology

adobesummit twc definition

adobesummit external

adobesummit culture

qa questions

inefficiencies questions

years questions

week sprint questions

organization questions