a slice of scala

Melbourne Scala User Group - August 2013Kevin O’Neill - CTO PlayUp - @kevinoneill

Kevin O’Neill CTO PlayUp

A Slice of Scala

@kevinoneill

August 2013


A little History


Engage & Entertain Around Live Sport


Mobile Games


SMS Based


Number of Runs


Complex


Carrier Integration


Smart Phone Revolution


Move from a Content Creator to a Content

Enabler


Top 10 in Engagement


Over 3 Million Likes


30% - 40% Active


Manually Curated


We Know there's an Opportunity


October 2011


Technologist at Large


Version 2


Rebuild Based on Hypermedia*


Same Feature Set


Different Skin


iOS


Android


Ruby


Version 3


Platform Extensions


Tiles


Third Party


Ticketing


Ruby


PHP


Version 4


Engagement Too Deep


Streams


Graph Storage


Story

League

Region

Team Story

Team

Story

League

TeamStory

Story

User

Story

about

about

about

aboutabout

follows competes in

competes in

follows

competes in

Region

follows

Team

Story

competes in

about

about

User

follows

follows

Story

Contest

about

plays in

plays in

Story

about


Value is in the Connections


Ruby


50 Seconds


Cypher Processing too much Data


Java plugin


5 Seconds


10 Times too Long*


Aiming Around 200ms


Time to Change


If you hear a voice within you say 'you cannot paint,' then by all means paint, and that voice will be silenced.

– Vincent Van Gogh


Scala


Just Couldn't Face Java


Some Experience with it on a Large Project


The Stack


Scala 2.10.x


akka 2.1.x*


Spray M8+


Play 2.1.x*


Other Bits


Neo4j 1.9.x


Scalaz 7.0.x


Blueprints 2.4.x


Scala Test 2.0.x


Scala Check 1.10.x


SBT 1.12.x*


Systems


Two akka Services*


Graph Service


Sports Connect


Fanbase Connect*


One Spray Server


Hypermedia API


One Play Application*


HTML Front End


Admin Front End*


JSON Generation


Problem : How do you isolate environment


Lambda: The Ultimate Dependency Injection

Framework


Did my Head in


So what if …


Environment Readers

type Settings = Set[EnvironmentSetting[_]] !type EnvironmentReader[+A] = scalaz.Reader[Settings, A] object EnvironmentReader { def apply[A](f: Settings => A):EnvironmentReader[A] = scalaz.Reader(f) }


Writing an Identity

protected def writeResourcePointer(identity: ResourceIdentity) : EnvironmentReader[JObject] = for { ! href <-‐ urlForIdentity(identity) contentType = identity.documentType representations <-‐ writeRepresentations(identity) views <-‐ viewsForIdentity(identity) ! } yield (":href" -‐> href) ~ (":type" -‐> contentType) ~ (":uri" -‐> identity.uri) ~ representations ~ (":views" -‐> views)


urlForIdentity

protected def urlForIdentity(identity:ResourceIdentity) : EnvironmentReader[JValue] = for { ! r <-‐ URLResolverSetting.resolve(identity) href <-‐ mapOrJNothing(r) { ref : String => pure(JString(ref)) } !} yield href


URLResolverSetting

case class URLResolverSetting(value: Resolver[String]) extends EnvironmentSetting[Resolver[String]] { val key = "url-‐resolver" } !object URLResolverSetting extends ResolverSetting ({ case setting: URLResolverSetting => setting.value })


ResolverSetting

class ResolverSetting[T]( finder : PartialFunction[ EnvironmentSetting[_], Resolver[T]]) { ! def resolve(identity : ResourceIdentity) : EnvironmentReader[Option[T]] = EnvironmentReader { settings => for { resolver <-‐ settings.collectFirst(finder) result <-‐ resolver.resolve(identity) } yield result } }


Wooh … That's far more

complex than my … blah blah in blah


Yup


Nope


Remember I mentioned Runars talk

did my head in.*


It's actually really easy to work with


Each components is responsible for one

thing


Combining components is trivial


You spend most of your time doing

simple, safe, composition


Spray


Elephants all the way down


The Language

val route = { language { lang => implicit val env = RequestEnvironmentProvider( EnvironmentConfig(language = Some(lang)) ) … !}


The Directive*

def language: Directive1[Lang] = { headerValue { ! case HttpHeader("accept-‐language", langString) => langString.split(',').headOption.flatMap(firstLanguage => Lang.get(firstLanguage)) ! case _ => None ! } | provide(Lang.defaultLang) }


Creating a Responsedef getStream(implicit env: RequestEnvironmentProvider): Route = { path("stream" / StreamPath) { case (stream, afterStory) => read(FetchStream(stream, afterStory)) { stream => val streamEnv = environmentForStreamWithTopics( env, stream.identity.references ) respondWithHeaders( `Cache-‐Control`(`public`, `max-‐age`(60))) { complete { implicit val criteriaMarshaller: Marshaller[StoryStream] = resourceMarshaller(streamEnv) stream } } } } }


A Little akka Helper


Reading the Graph

def read[T](message : GraphRequest[T])(f : T => Route):Route = { ctx => val response: Future[T] = Graph.reader.ask(message).asInstanceOf[Future[T]] response.map { case Error(messages) => { ctx.complete( InternalServerError, messages.mkString("\n") ) } case result => f(result)(ctx) } }


akka


Topic Management


Topic Actor

private class TopicActor (system : GraphSystem, topic : StreamComponentIdentity) extends GraphWorker(system) with Stash { ! override def preStart() { available(topic) onAvailable { become(active) } onMissing { topicManager ! UpdateTopic(topic) become(pending) } } !… }


Receive

def receive = PartialFunction.empty


Pending

def pending : Actor.Receive = { case message @ LinkStory(story, identity) if identity == topic => { stash() } ! case identity if identity == topic => { unstashAll() become(active) } }


Active

def active : Actor.Receive = { case LinkStory(story, identity) if identity == topic => { gs.write(associateWithSubject(story, topic)) } }


Baby Steps


Challenges


As the Carpenters said …


We've Only Just Begun


Data Load


Sparsity


Propagation


Friends


500 Topics


10,000,000 Nodes


1,000,000 Topics


10,000,000,000 Nodes


Time Series Data


Distribution


Need to change the way you think about

services


Actors are not Queues


Monitoring is Hard


"Real Time"


Balancing Load vs Caching


Two Hard Problems


Naming


Cache Invalidation


Off by One Errors


Caching Stuff is Easy


Invalidation is Hard


Limited Use


Just be Fast


Payments


Ticketing


À la carte


Cross Platform


Come Join Us


Disrupting


Looking for the 'X' factor


Hiring Selectively


Talk to Me or Andrea


You miss 100 percent of the the shots you don't take

– Wayne Gretzky


Kevin O’NeillCTO - PlayUp

[email protected]@kevinoneill

Questions?

mailto:[email protected]

a slice of scala

Technology

kevin oneill cto playup

rubykevin oneill cto

largekevin oneill cto

deepkevin oneill cto

likeskevin oneill cto

ioskevin oneill cto

tileskevin oneill cto

partykevin oneill cto