symfony live san francisco 2017 - symfony @ opensky

Symfony @ OpenSkyPablo Godel

Symfony @ OpenSky

What is OpenSky?

Symfony @ OpenSky

What is OpenSky?• Multiple businesses powered by Symfony:

1. opensky.com 2. dotandbo.com 3. 55mulberry.com

http://opensky.com

http://dotandbo.com

http://55mulberry.com

Symfony @ OpenSky



• +100 employees

http://opensky.com

http://dotandbo.com


Symfony @ OpenSky



• +100 employees • Offices all around the world in New York,

Portland, Nashville, Miami, Chico, Bucharest and Los Angeles

http://opensky.com

http://dotandbo.com


Symfony @ OpenSky

HISTORY

Symfony @ OpenSky

• Early adopter of Symfony2 in 2010 before 2.0 stable was released

History

Symfony @ OpenSky


• Migrated from Magento -> symfony1 -> Symfony2

History

Symfony @ OpenSky


• Migrated from Magento -> symfony1 -> Symfony2

• Aspects of Magento cart utilized throughout migration process until completely phased out with proprietary cart/checkout technology

History

Symfony @ OpenSky

THE CURRENT STACK

Symfony @ OpenSky

• Symfony 2.8 • RequireJS + Backbone • PHP 7.1.10 • MySQL 5.6 + MongoDB 3.2 • Redis • Varnish + Nginx • VMWare • Vagrant + VirtualBox for Dev VMs

The Current Stack

Symfony @ OpenSky

• ~400 Controllers • +800 Routes • ~1,900 Services • 285 Commands • 374 Entities • 467 Documents • 808,532 LoC • 3,738 Classes • 156 Interfaces • 46 Traits • +10,000 Unit Tests

The Current Stack

Symfony @ OpenSky

The Current Stack

Symfony @ OpenSky

CODING WORKFLOW

Symfony @ OpenSky

Symfony at OpenSkyGitHub Enterprise

GitHub Enterprise

Coding workflow

Symfony @ OpenSky

• production branch (master)

Coding workflow

Symfony @ OpenSky

• production branch (master) • feature / fix branches

Coding workflow

Symfony @ OpenSky

• production branch (master) • feature / fix branches • Pull Requests linked to JIRA

Coding workflow

Symfony @ OpenSky

• production branch (master) • feature / fix branches • Pull Requests linked to JIRA • Integration with Jenkins for Unit/

Functional Tests

Coding workflow

Symfony @ OpenSky

• production branch (master) • feature / fix branches • Pull Requests linked to JIRA • Integration with Jenkins for Unit/

Functional Tests • Pre-commit Hooks for php-cs-fixer,

gulp-jshint, gulp-sass-lint

Coding workflow

Symfony @ OpenSky

• Symfony Coding Standards

Coding workflow

Symfony @ OpenSky

• Symfony Coding Standards • Short Array Syntax

Coding workflow

Symfony @ OpenSky

• Symfony Coding Standards • Short Array Syntax • Ordered Use Statements

Coding workflow

Symfony @ OpenSky

• Symfony Coding Standards • Short Array Syntax • Ordered Use Statements • Ensuring an EOL at end of file

Coding workflow

Symfony @ OpenSky

• PHP7 Type-Hints when possible

Coding workflow

Symfony @ OpenSky

• PHP7 Type-Hints when possible • DocBlocks only when adds clarity

Coding workflow

Symfony @ OpenSky

• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class

Coding workflow

Symfony @ OpenSky


• ‘\Namespace\ClassName’

Coding workflow

Symfony @ OpenSky


• ‘\Namespace\ClassName’ • In forms use class constant

Coding workflow

Symfony @ OpenSky



• $builder->add('name', TextType::class);

Coding workflow

Symfony @ OpenSky



• $builder->add('name', TextType::class); • Always use ‘use’ statement

• $time = new \DateTime()

Coding workflow

Symfony @ OpenSky

Before production deployment:

Coding workflow

Symfony @ OpenSky

Before production deployment: • Peer Reviews using GitHub PRs

Coding workflow

Symfony @ OpenSky

Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional)

Coding workflow

Symfony @ OpenSky

Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional) • Integration deploy/testing

Coding workflow

Symfony @ OpenSky

Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional) • Integration deploy/testing • UA / QA pass (tracked in JIRA)

Coding workflow

Symfony @ OpenSky

SYMFONY BUNDLES

Symfony @ OpenSky

• Split things into broad bundles. Don’t over bundle.

Symfony Bundles

Symfony @ OpenSky

• Split things into broad bundles. Don’t over bundle. • MainBundle

Symfony Bundles

Symfony @ OpenSky

• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra

services in the container)

Symfony Bundles

Symfony @ OpenSky


services in the container) • APIBundle

Symfony Bundles

Symfony @ OpenSky



• 3rd Party Bundles

Symfony Bundles

Symfony @ OpenSky



• 3rd Party Bundles • AvalancheImagineBundle

Symfony Bundles

Symfony @ OpenSky



• 3rd Party Bundles • AvalancheImagineBundle • FOSUserBundle

Symfony Bundles

Symfony @ OpenSky



• 3rd Party Bundles • AvalancheImagineBundle • FOSUserBundle • Using 3rd party bundles may impact

maintainability in the long term

Symfony Bundles

Symfony @ OpenSky

DOCTRINE

Symfony @ OpenSky

• MongoDB and MySQL together

Doctrine

Symfony @ OpenSky

• MongoDB and MySQL together • References between MongoDB and MySQL

Doctrine

Symfony @ OpenSky

• MongoDB and MySQL together • References between MongoDB and MySQL • Using the Doctrine ORM & MongoDB ODM to

make the models for the two databases transparent to the application.

Doctrine

Symfony @ OpenSky



• Never delete data due to referential integrity problems. Use soft delete (deletedAt)

Doctrine

Symfony @ OpenSky



• Never delete data due to referential integrity problems. Use soft delete (deletedAt)

• Traits for common functionality like createdAt/updatedAt/deletedAt

Doctrine

Symfony @ OpenSky

Symfony @ OpenSky

• How long did it take to run?

Doctrine Migrations

Symfony @ OpenSky

• How long did it take to run? • Is it backwards compatible?

Doctrine Migrations

Symfony @ OpenSky

• How long did it take to run? • Is it backwards compatible? • Up & Down

Doctrine Migrations

Symfony @ OpenSky

• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date

Doctrine Migrations

Symfony @ OpenSky

• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date • Add indexes

Doctrine Migrations

Symfony @ OpenSky

• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date • Add indexes • Remove data later in separate

migration

Doctrine Migrations

Symfony @ OpenSky

SYMFONY MISC

Symfony @ OpenSky

• Lean

Controllers

Symfony @ OpenSky

• Lean • No Business Logic

Controllers

Symfony @ OpenSky

• Lean • No Business Logic • Only Flow control

Controllers

Symfony @ OpenSky

• Lean • No Business Logic • Only Flow control • Extend Custom BaseController

Controllers

Symfony @ OpenSky

• Two tiers of authentication implemented using role voters.

Security

Symfony @ OpenSky

• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter

Security

Symfony @ OpenSky

• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter • AuthenticatedRecentlyVoter

Security

Symfony @ OpenSky


• When a user is recently authenticated, they are logged in to 2nd tier of authentication and have access to account, checkout, etc. After 15 minutes of inactivity, user is logged out of 2nd tier.

Security

Symfony @ OpenSky



• Admin integrates with LDAP (using zendframework/zend-ldap)

Security

Symfony @ OpenSky



• Admin integrates with LDAP (using zendframework/zend-ldap)

• Multiple firewalls/authentication providers for consumer facing, admin and api.

Security

Symfony @ OpenSky

Runtime Settings/Parameters• Parameters defined in config/parameters.yml

Symfony @ OpenSky

• Parameters defined in config/parameters.yml • Override stored in MongoDB

Runtime Settings/Parameters

Symfony @ OpenSky

• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime


Symfony @ OpenSky

• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache


Symfony @ OpenSky

• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache • When parameter value is changed, cache is

updated and queue workers restarted


Symfony @ OpenSky


updated and queue workers restarted • Allows for configuration changes without

deployment


Symfony @ OpenSky


updated and queue workers restarted • Allows for configuration changes without

deployment • Heavily used for feature flags and enable/

disable of new features


Symfony @ OpenSky


Symfony @ OpenSky

EVENTS

Symfony @ OpenSky

• Perform tasks after the response has been sent to client

Event kernel.terminate

Symfony @ OpenSky

• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection


Symfony @ OpenSky

• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection • Analyze request for further actions


Symfony @ OpenSky

• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection • Analyze request for further actions

• Only effective with FastCGI


Symfony @ OpenSky

• Queueing events for async functionality

Events system

Symfony @ OpenSky

• Queueing events for async functionality • Event serializer/unserializer

Events system

Symfony @ OpenSky

• Queueing events for async functionality • Event serializer/unserializer • Scheduling events

Events system

Symfony @ OpenSky

• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount

Events system

Symfony @ OpenSky

• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount • One queue per mongodb collection

Events system

Symfony @ OpenSky

• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount • One queue per mongodb collection • Queue workers managed with Supervisord

Events system

Symfony @ OpenSky

Events system

Symfony @ OpenSky

SLACK

Symfony @ OpenSky

• php-kip using yabot

Slack integration

Symfony @ OpenSky

• php-kip using yabot • Production deploy queue/line

Slack integration

Symfony @ OpenSky

• php-kip using yabot • Production deploy queue/line • Dev environment reservations & deploys

Slack integration

Symfony @ OpenSky

Symfony at OpenSkyPhp-kip using yubot

- Production deploy queue/line - Dev environment deploys

Slack integration

Symfony @ OpenSky

Slack integration

Symfony @ OpenSky

SEARCH

Symfony @ OpenSky

Search

• Integration with Algolia

Symfony @ OpenSky

• Integration with Algolia • algolia/algoliasearch-client-php

Search

Symfony @ OpenSky

• Integration with Algolia • algolia/algoliasearch-client-php • Update index through async Event

Listeners

Search

Symfony @ OpenSky

Payments• Integration with Braintree Payments • braintree/braintree_php

Symfony @ OpenSky

• NPM • jQuery • RequireJS • Backbone • TwigJS • Gulp / Less / Sass

The Frontend

Symfony @ OpenSky

• A/B Split Testing using Optimizely for testing UI/UX changes

• Send small % of traffic to new features • GoogleAnalytics tracking conversion rate

The Frontend

Symfony @ OpenSky

TESTING

Symfony @ OpenSky

• pr-nightmareTesting

Symfony @ OpenSky

• pr-nightmare • Runs on all commits and pull requests

Testing

Symfony @ OpenSky

• pr-nightmare • Runs on all commits and pull requests • Uses PHPChunkit to run 30 minutes of tests in 3

minutes by running groups of tests in parallel across multiple servers. https://github.com/jwage/phpchunkit

Testing

https://github.com/jwage/phpchunkit

Symfony @ OpenSky

• pr-nightmare • Runs on all commits and pull requests • Uses PHPChunkit to run 30 minutes of tests in 3

minutes by running groups of tests in parallel across multiple servers. https://github.com/jwage/phpchunkit

• Selenium for browser functional testing

Testing

https://github.com/jwage/phpchunkit

Symfony @ OpenSky

Testing

Symfony @ OpenSky

• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up

Testing with PHPUnit

Symfony @ OpenSky


• AutoDataCleanupListener keeps tracks of what collections and tables were inserted to in a test and cleans up afterwards. This used to be explicitly required by the developer but happens automatically now


Symfony @ OpenSky



• AbstractTwigLintTest: Twig linting. Loads up all twig files to make sure there are no parse errors


Symfony @ OpenSky




• Generating unit tests from class definitions


Symfony @ OpenSky




• Generating unit tests from class definitions • PHPUnit watch command


Symfony @ OpenSky


Symfony @ OpenSky

PHPUnit watch

Symfony @ OpenSky

Twig Lint with PHPUnit

Symfony @ OpenSky

DEPLOYMENT

Symfony @ OpenSky

Deployment• Use Customized Fabric • Manually started from Jenkins • First build task

• Merge feature/fix branch into master

• Create Tag

Symfony @ OpenSky

• Final build task • git checkout • composer install --no-dev —no-scripts • Build front controller • Run gulp, requires, less, sass • Warmup cache for each app • Assetic dump • Build artifacts file & distribute it to

servers • Run MySQL & MongoDB Migrations • Rotate logs • Restart php-fpm • Reload Varnish and nginx • Restart queue workers

Deployment• Use Customized Fabric • Manually started from Jenkins • First build task

• Merge feature/fix branch into master

• Create Tag

Symfony @ OpenSky

Questions?

Symfony @ OpenSky

Thank you!Pablo Godel

@pgodel

http://slideshare.net/pgodel https://joind.in/talk/bfd1e

http://slideshare.net/pgodel

https://joind.in/talk/bfd1e

symfony live san francisco 2017 - symfony @ opensky

Technology