Test Failed, Then...Optimizing communication between people

Toru Furukawa"@torufurukawa"

Agenda •  Introduction

Web app syncing with live TV show"•  Loose coupling"•  Testable components"•  Efficient communication"

Agile App Development on Concrete Backend Services

HTML App"

App Server

Backend Service(s)"

Project  dependent

Common

Because... •  Requirements keep changing"•  Not only Web, but also TV"•  Not only TV, but also Web"•  Spiky access traffic"•  Common fundamental features"

▼"•  Easy to change application"•  Well tested platform"

Ended up with…

HTML App"

App Server

Counter" Messenger"

High Communication Cost

"Communication overheads increase as the number of people increases." F. Brooks"

Minimal and Effective Communication

Optimize Communication •  Between Components"•  Between People

How to Simplify Dependency?

HTML App"

App Server Counter" Messenger"

PHP •  IMPORTANT: PHP is fine"•  Cannot be combined with Python

component in language layer easily

Loose Coupling "Write small services that speak HTTP and bridge them together with another application.""

Armin Ronacher (@mitsuhiko)" http://lucumr.pocoo.org/2010/12/24/common-mistakes-as-web-developer/

HTTP

Loose Coupling

HTML App"(JavaScript)

App Server"(whatever)

Counter"(Python)

Messenger"(Node.js)

HTTP

HTTP

Assign Engineer for Component

HTML App"(JavaScript)

App Server"(whatever)

Counter"(Python)

Messenger"(Node.js)

HTTP

Communication Paths

2 Projects Running

Wrap Engineers' Communication

•  New layers to simplify inter-component communication"

•  Assign new roles to simplify inter-people communication

HTML App"(JavaScript)

App Server"(whatever)

Counter"(Python)

Messenger"(Node.js)

Server/Server Communication"(Python)

Client/Server Communication

If a servicewise feature does not work, how to identify the cause?"•  Ask developer to check logs?"•  Ask developer to check DB?"

▼"Make component diagnosable"

Instrument Components

Log with fluentd

App Server"(whatever)

Counter"(Python)

Messenger"(Node.js)

fluentd

Mongo  DB Web App

Define CRUD APIs •  Make DB accessible from test and admin

•  REST /objects/:id"•  /get_objects

/get_object, /set_object, /delete_object"▼"

•  Diagnose 1-layer deeper from outside"

"requests" package for Web API Access

>>>  r  =  requests.get(url,  auth=(...),  ...)  >>>  r  =  requests.post(url,  data={...},  ...)  >>>  r.status_code  200  >>>  r.content  u'{"foo":"bar","x":true}'  >>>  r.json()  {u'foo':  u'bar',  'x':  True}  

Library for Productivity •  Use testing libraries to increase

productivity i.e. how many test you write per hour"

•  setup and teardown"– unittest"– nose"– py.test"–  testfixtures"– etc.

Translate Test Report Traceback  (most  recent  call  last):      File  "mytest.py",  line  7,  in  test          assert  result  ==  expected  AssertionError  

▼"•  How do YOU determine what is wrong?"•  How do you tell OTHERS what is wrong?"

▼"•  Need better way to communicate

Library for Readability class  MyTest(unittest.TestCase):          def  test(self):                  expected  =  {...}                  result  =  {...}                  self.assertEqual(x,  y)  

""

Traceback  (most  recent  call  last):      File  "mytest.py",  line  7,  in  test          self.assertEqual(x,  y)  AssertionError:  {'items':  ['spam',  'spam'],  'foo':  'bar'}  !=  {'items':  ['spam',  'ham'],  'foo':  'bar'}  -­‐  {'foo':  'bar',  'items':  ['spam',  'spam']}  ?                                                                      ^^    +  {'foo':  'bar',  'items':  ['spam',  'ham']}  ?                                                                      ^

Python vs HTTP

Python

HTTP

Test Fast

Loosely Couple

Develop Fast

If I stick to Python too much,my communication is

tightly coupled

Convert requests function to curl >>>  import  curledrequests  as  requests  >>>  requests.debug  =  True  >>>  r  =  requests.get(  ...      'http://www.google.com/',  ...      params={'q':'python'})  ...  $  curl  “http://www.google.com/?q=python”  -­‐w  '\n%{http_code}\n'  <!doctype  html><html  itemscope=""  itemtype="http://schema.org/WebPage"><head><me  ...  200  

h5p://goo.gl/hO579O  

Loosely Couple Components and Engineers

Share Your Practice

@torufurukawa or grab me"