google developer day 2007 - nestoria: from mash-up to business

this space left intentionally blank

why perl is awesome andall other languages suck

Mike Astleflamebait@lokku.com

from mash-up to business

Mike Astle - Lokku Ltd.Google Developer Day31 May 2007 - London

i like questions

Milestones

• Start dev April 2006• Launch London June 2006• Launch UK Sept 2006• Launch Spain May 2007

Numbers

• In the UK, have 850,000 listingsrefreshed continually

• Boils down to about 500,000 properties• 100,000+ users per month• 4 programmers• No dedicated operations staff

Houses on a map…so what?

this is harder than it looks

The Plan

• Talk about some of the technicaldecisions we made at Nestoria

• Go over a few challenges that we havefaced

• Hopefully there are lessons to learnfrom our experiences

first, you design

Things to worry about

• Architecture• Services vs. APIs• How to scale?• i18n• Future products

then you choose a platform

MS vs. Everything Else

Java?

That Leaves LAMP

PHP?

Ruby?

LAMR?

Python?

Perl

Perl

Utility Belt == CPAN

Apache

MySQL

GNU/Linux(Debian)

now choose adevelopment environment

Source Control

• SVN…duh• Does anybody use CVS anymore?• Love the trac

Coders

• A few veterans• As many interns / junior coders as fit in

the room

Highly Collaborative(but not XP or Agile)

Emphasis

• Collective decision-making• Individual responsibility for deadlines• Many small success rather than few

large projects

Why write tests?

• Find bugs, of course• Increase confidence to make changes

to unfamiliar systems or libraries• Promote good design / expose bad

design

Design

• Informal process (no UML, minimaldesign patterns)

• Concentrate on the things that will reallyhurt - e.g. i18n

• Not everything has to be an object

Documentation

• POD for interfaces• twiki for systems• Schedule regular refreshers

Outsource

• If you can spec it faster than you canwrite it, consider farming it out

• Many, many site full of hungry coders• Have had good results with

scriptlance.com but there are lots ofothers out there

how to deploy?

Virtualization?

• Sounds like voodoo to me• Extra layer to worry about• Easy enough to do everything in user

space

Wigwam

• 12 subdomains• Live data• 22 services• Many playpens per box• Releases several times a week

big ISP vs. small ISP

• Bigger - more consistent• Smaller - more personal• Bigger - discrete choices• Smaller - continuous advice

• We went small

Hardware

• Know your downtime limits• Be reasonable!• Load balancing - expensive and

complex - we do it with DNS through avendor

• Have a failure plan

TheHawkeyePrinciple

but is anybody using it?

Logging

• Need your own, and it needs to beflexible

• Use the free tools as a sanity check

Reporting

• An important area of expertise, not anannoying afterthought

• Glossary• Every mean needs a variance• A/B Testing (a.k.a Bucket Testing) is

hard to do right

Also…

• Build tools for ad hoc queries, they>will< get used

• Keep an audit trail of changes• Effective tools for drilling down on

sessions allows non-techs to helpdebugging

problems we have solved

http://www.nestoria.co.uk/covent-garden/flat/buy/bedrooms-2/maxprice-

1000000/minprice-250000/bathrooms-2

• Alderly Edge => Alderley Edge• Bayswater London => Bayswater• Berkhampsted => Berkhamsted• Birminham => Birmingham• Bletchingly => Bletchingley• Broxbourne Station => Broxbourne• Cambuslank => Cambuslang• Canary Warf => Canary Wharf• Claphamjuction => Clapham Junction• Covent Garden London => Covent

Garden

Geocoding

• Transform address into lat/longcoordinates

• 1,000+ calls a minute• Very high data variance (few cache hits)• Royal Mail data for UK• Teleatlas data for Spain

• Albert Villas, Gilbert Mews, LEIGHTON BUZZARD,• Bedfordshire, LU7 1NF• Lake Lock Drive, Wakefield West Yorkshire• Church Lane, London• C ELMINGTON ROAD, LONDON• Fountain Road (Flat 2), Edgbaston• London SE18, UK• Lee, London, UK• Free Trade Wharf, 340 The Highway,

Wapping/Limehouse London• Stunning 3 Storey Contemporary Barn Conversion In

The Popular Village Of Utkinton• A.4-1 GREAT NORTHERN TOWER,WATSON

STREET,CITY CENTRE,• MANCHESTER GREATER MANCHESTER• 15, ASTLEY HOUSE, LONDON

(((^[BEGLMNS][1-9]\d?) | (^W[2-9] ) | ( ^( A[BL] | B[ABDHLNRST] |C[ABFHMORTVW] | D[ADEGHLNTY] | E[HNX] | F[KY] | G[LUY] | H[ADGPRSUX] |I[GMPV] | JE | K[ATWY] | L[ADELNSU] | M[EKL] | N[EGNPRW] | O[LX] |P[AEHLOR] | R[GHM] | S[AEGKL-PRSTWY] | T[ADFNQRSW] | UB | W[ADFNRSV] |YO | ZE ) \d\d?) | (^W1[A-HJKSTUW0-9]) | (( (^WC[1-2]) | (^EC[1-4]) |(^SW1) ) [ABEHMNPRVWXY] ) ) (\s*)? ([0-9][ABD-HJLNP-UW-Z]{2}))

what next?

Future Projects

• Expose more data via API (e.g.historical aggregate data)

• Implicit personalisation (the systemlearns from users)

• More international expansion• Systemic testing to improve user

experience

like what you see?

http://www.lokku.com/jobs

thanks!

astle@lokku.comhttp://www.lokku.com

http://www.nestoria.co.ukhttp://www.nestoria.es

http://www.lokku.com/lpw06_geocoding/http://www.lokku.com/lpw06_wigwam/