PowerPoint Presentation

Startup Technology Pitfalls and How to Avoid them

BRYAN SHORTOriginally from CaliforniaGraduated from UCLA9 years at Microsoft

OfficeWindows Casual GamesXbox Live

Dont forget. Big problems that we are solving.3BIG BETS

BIG BETSArchitecture

Technical Stack

StorageBIG BETS - Architecture

BIG BETS - CLOUD

BIG BETS - CLOUD

EC2Route 53RDSS3GlacierCloudFront

MORE!!!BIG BETS - CLOUD

CloudFront

Pricing for Dedicated IP Custom SSL is simple. Because of the added cost associated with dedicating IP addresses per SSL certificate, we charge a fixed monthly fee of $600 for each custom SSL certificate you associate with your CloudFront distributions, pro-rated by the hour.BIG BETS - CLOUDIn Search of New CDN

BIG BETS - CLOUDRequirements

Host CDN powered by S3SSL custom cert required.Fast and PerformantPrice

Doesnt need to be fastestBIG BETS - Technical Stack

BIG BETS - Technical StackRequirements

Able to service http requestsCommunity of active developersOpen source friendlyEnables engineering excellenceModernBIG BETS Mongo?Node / Angular / Express all made total sense.

Our data is inherently very relational. What to do?

14BIG BETS Technical StackHow did we do?JavaPHPRubyJavascriptPython20098573357714513678311210201023888149102675227515194172011819218309382728819216605220124061972705834788174523202646692013328659255883405824529878210953201422031913244919691428097210828978, 701 Packages available on NPM!15BIG BETS Technical StackHow did we do?Not so good things:

NPM unreliable

Not all modules created equally!16BIG BETS Technical StackA Tale of Two ModulesModule 1Module 2Days since last commit60 20External DependenciesNoYesOpen Issues1741817BIG BETS Technical StackDeployment

If adding a single dependency to deployment overcomplicates it; than your deployment process is broken.18BIG BETS Technical StackDeployment

Like most engineering, its easiest to do the wrong thing.

19BIG BETS Technical StackIntegrated Pipeline

20BIG BETS Technical StackTest Code

Adds incredible value when we you have a product which has logic and behavior to validate.

Inside Social is lacking in test code.

21BIG BETS Technical StackTest Code

22BIG BETS Technical StackTest Code

How are we doing?23BIG BETS Technical StackTest Code

Corollary! Code that we put onto our clients sites needs to be tested, and ready for prime time production.

The worst thing that we can do is break a customers site.24BIG BETS Technical StackTest Code

Corollary! I am amazed at how many brands, lack any sort of test site / infrastructure and just drop us onto their production site.

25BIG BETS - Storage

BIG BETS - Storage

Short termBIG BETS - StorageShort term

Requirements:

Handle lots of writes quickly (implied in memory)Be able to serialize itself, such that state can be restored on another machineSDK

BIG BETS - StorageShort term

How did we do?

One month later Aerospike released V3.

December 16th, 2013 Amazon unveils Kinesis service.BIG BETS Data StoreData Store

30BIG BETS Data StoreHow did we do?

http://techcrunch.com/2014/03/25/google-launches-bigquery-streaming-for-real-time-big-data-analytics/31BIG BETS Data StoreHow did we do?

A UW big data study: BigQuery is very easy to setup and run queries and does not require any manual conguration of clusters; it automatically scales up according the dataset size. But this can be a disadvantage as well, since the user cannot tune the system according to his/her needs. However it has limited SQL language support and does not scale up well on complex queries involving multiple joins and nested subqueries.32BIG BETS Data StoreHow did we do?

33BIG BETS Data StoreHow did we do?

What are we missing?

Full SQLData Removal / Archival

What works well?

PricingNoSQL

34BIG BETS - Architecture

BIG BETS - ArchitectureHow are we doing?

Were able to accomplish our mediate goals!

Great Success!

BIG BETS - ArchitectureHow are we doing?

Can we solve our long term problems?

Real time?

Complex system that evaluates what social marketing tactics, and learns from them?

The road aheadOnce we have something stable, that people can utilize it is time to revisit some bets.

Inside Social will be reexamining short term data store + data store options. Thank You?Any Questions