infochimps: how we do it
DESCRIPTION
How a small company handles Big Data with agility.TRANSCRIPT
Big Data Made Easy
We use a lot of new-tech tools
We’ve written a lot of new tech tools
IronfanChef specialization for Big Data in the cloud
WonderdogHadoop interface for ElasticSearch
WukongRuby library for Hadoop
SwineherdWorkflow engine for Hadoop jobs
But it’s not about the technology…
… it’s about the culture
Hiring Process
Initial PassTechnical
Phone Interview
Team Interview
Initial Contract
Full Employment
Management Structure
Flat: I regularly talk with C-level folks
Open: Everyone has well-understood roles
Fair: Leadership leads, not orders about
Understanding: Problems are addressed, not blamed
Employee Support
Fully stocked kitchen
Daily group lunches
Employee joy fund and voting
Company outings, both impromptu and formal
Some fun and games, too
Employee Development
Employee’s focus is largely self-directed
Lack of experience is (almost) never a determining factor
Common language for problems and frustration
Make employees awesomely valuable, and totally uninterested in leaving
Technical Culture
Good ideas can come from anyone
Fail forward, not roll-back
Repeatability is your friend
Automate out of boredom or fear, not efficiency
Failure from audacity is better than failure from inaction
Infrastructure Choices
More assumed access for developers
Small, decoupled, late-binding wherever possible
Build anew, rather than repurpose an old
Actively pull unused code and data from production
Beyond DevOps
NoOps +1: everyone technically does part of Ops, it’s just my specialty
AllOps: product & marketing can help Ops, and vise versa
It’s all about removing barriers to being awesome, everywhere
What’s next?
How do we handle the impedance mismatch between our model and our clients’ models?
What do we do as the company grows beyond the size of the monkeysphere?
How should we tackle user segmentation and security as we build our Platform out?
