Web Facilitated Play
in the Real World

@ajfisher

Sketching in Hardware
Portland 20 July, 2012

Hi. My name's Andrew Fisher and my background is as an interaction developer and today I want to talk to about web facilitated play in the real world.

Play enhances our lives

Image (CC) flickr Duncan / kkseema / Adam Jones / mzagozda

Play is fundamental to our lives. From our earliest childhood through to being an adult. Play is a core part of what defines our humanity.

Technology and play are somewhat symbiotic and each one helps to drive the other forward and evolve.

http://www.flickr.com/photos/duncan/79106711/sizes/o/http://www.flickr.com/photos/kkseema/2042946052/sizes/o/http://www.flickr.com/photos/adam_jones/3794676268/sizes/o/http://www.flickr.com/photos/mzagozda/4242006639/sizes/l/

How we play today

Image (CC) flickr Henry Stratford

So I was on a train the other day going to work and saw someone that looked like this. Headphones in, iPhone out, Totally immersed in what was obviously a multiplayer game.

This is a familiar view of the modern world. At times I'm this person.

The technologist in me loves the fact we have the technical capability for someone to play a multiplayer game from their phone in Melbourne with others in Madrid and Moscow.

However, I can't help but feel we've had to give something up as a result as we tune out the world around us so totally.

Image: http://www.flickr.com/photos/henry_stradford/5348910688/sizes/o/

Civic spaces need an upgrade

Image (CC) flickr Jesus Leon

I think our shared spaces are causing some of this as well. How often do you stand on a train station platform and see advertising 6 months or in some cases 6 years out of date.

Our civic environment hasn't kept pace with technology and is becoming ever more boring just as our phones become ever more interesting is it any wonder we tune out?

Image: http://www.flickr.com/photos/jesusleon/6932806609/sizes/o/

Time to level up

Image (CC) flickr ipdegirl

We have hit an interesting time though.

With a nearly billion smartphones in people's pockets and a billion more not far away.

With physical computing devices increasing in capability and increasingly networked with all the benefits that confers

And all of this coupled with the web!

Now we have the technologies at our disposal to reshape the nature of play or at least of interaction in our shared spaces using digital tooling.

So what could this do for play as a result?

Image: http://www.flickr.com/photos/ipdegirl/5327066487/sizes/o/

1. Web Sockets overview

2. Real time web in a real time real world

3. Applications & demos

So today I want to show you how this can work.

I'll give a brief intro to web sockets, then show you the technical architecture for doing this and then round up with some examples and a demo at the end.

Websockets

So the technology that enables real world play is called web sockets.

Is anyone here familiar with it?Has anyone actually used it in a project?

Okay so very quickly web sockets is part of the suite of tech that makes up HTML 5. Thought it's not actually HTML.

http://www.flickr.com/photos/inertiacreeps/1414845543

Image (CC) flickr InertiaCreeps

Websockets

Web sockets is an upgrade to the standard HTTP connection Getting rid of the traditional request / response method.

Instead it opens a long lived, bi directional network connection that data can be passed along.

This lends itself well to scenarios where you are passing small packets of data very frequently over connections open for long periods like minutes or hours.

Often we use this for messaging use cases and the most frequent pattern is pub / sub.

PubSub messaging

Web SocketsServerPublisherSubscriber

{a:event, x:123,...}

{a:event, x:321,...}

Produces Messages

Consumes Messages

Processor / Broker

As you can see we have a publisher that produces a message and sends it to the web sockets server which acts as a broker and it passes that on to subscribers who consume the message and do something with it.

Now in web sockets world most messages are passed around using JSON because it's lightweight, many libraries exist, it's expressive and flexible.

Common pattern

Publishers

Subscribers

Broker

So this architecture scales well and you can have many publishers and subscribers just producing and consuming messages. Many topologies exist for different needs but this is one of the most common.

Now you understand a bit about web sockets let's talk about how we can use it to connect objects in the real world.

Image (CC) flickr rfranklinaz

Websocketing things

So what's interesting about web sockets is that it's a protocol and as such anything that can talk it can become a publisher or a subscriber and work with messages.

http://www.flickr.com/photos/88548643@N00/139617711/

Real time data capture

For example we could have an arduino with a temperature sensor on it that broadcasts temperature readings every 100 milliseconds to the broker.

The subscribers could be a real time logging database for later processing and a web client for real time graphing of the data.

Now scale the arduino out and have 10 sensor nodes, 100, 1000 all producing data in real time that could be used by a dozen subscribers all doing different things with it.

As interesting as that facet is, I came here to talk about play. For play, then, we can turn all of this on it's head. So here's a view of the stack I use to do this.

Architecture for play

DjangoSocketIO

Network connected

Something to control

Consolidation / processing

Websockets server

Interface

At the top we have web browsers across phone, tablet or desktop with interfaces that create messages.

These get sent to the web sockets server for processing. I use Django Socket IO as I'm a python guy but there's things like Node.JS which are conceptually similar.

The broker then forwards on messages to the subscribers which in this case is an arduino running Kevin Rohling's excellent WebSockets library and it then turns that data into some type of physical control.

Benefits

Web interfaces have wide reach

No app downloads

Mobile devices have numerous sensors

Can access other parts of the web app
(eg Authentication)

Web scaling well known problem with solutions

Debug in browser BEFORE going to hardware

There's a lot of benefits to using this stack which you can see here, however the main one I want to call out is that as the system is web based and messages are JSON it means I can prototype and simulate EVERYTHING in the browser.

This means I can get the web interaction rock solid and the networking totally debugged BEFORE going to hardware which then becomes a decoupled and more straightforward implementation on the physical side.

Image (CC) flickr kalexanderson

Time to play

Now you understand how the architecture works I have three examples I want to share, including one I've brought with me that you can play with.

http://www.flickr.com/photos/kalexanderson/5315471388/sizes/o/

Tank Tag

My first iteration of this was purely digital in order to assess the viability of the stack under real world conditions.

Tank tag uses the tilt sensor in a phone to drive a tank around on a screen; and the touch screen to fire, so you can tag other players.

The key is that the phone is now the controller and everyone plays together on a shared space on a screen or via a projector.

The largest scale I've had about 100 people playing this on a 40ft projected screen at once. I tried for bigger once and got nearly 150 before the network seized up for volumes of data being shifted.

Tug of war

For my next iteration, I wanted to incorporate something much more physical but also create a sense of team work and focus in the space amongst the players.

The tug of war concept was used for this and again players interact with their mobile phones.

After hitting a web page the players are divided into a red and green team with their own goals to pull a magnet towards.

Tug of war

This time the accelerometer is used for input and players have to shake their phones as hard as possible. The team that is collectively pulling the hardest at any time will pull the magnet towards their goal and eventually light it up.

This was a lot of fun and really starts to create a sense of shared use in the space given the focus and requirements for team work and it's all facilitated by web enabled devices that people already have in their pockets.

Multiplayer lighting

The third example I've actually brought with me from Australia. It's quite simple so it was travel proof but hopefully it will serve to illustrate to you the direction this can all go.

Play with the lights

Point a browser at

ajf.io/sketch12

Live demoed at conference

If you want to play with this then point a web browser on any device to 10.0.1.1

This won't work in Internet Explorer however so use something else.

Click on the circles and away you go.

As you all can have different colours they should mix somewhat if you hit the same lights at once.

As you can see the interaction is near instant and again there's almost no barrier to someone interacting with this.

----

HOPEFULLY PEOPLE CLICKING NOW!

Multiplayer lighting

So if you consider my original points about creating interactive spaces by bringing mobile, physical computing and web together this is how we can do it.

Imagine something like this scales up to say 10 metres across and hung on the side of a building and all you'd need is a phone with a web browser in order to interact with it.

Applications

This stack will work in any context where we want to create interaction on a semi mass scale but where physical proximity is important.

So it could be used for interactive advertising for shared user experiences, general civic space upgrades or for spaces like museums to create points of interaction with exhibits via smartphones without having to worry about downloaded apps.

Resources

Examples
github.com/ajfisher/sketching-conf-demoWebsocket Client for Arduino
github.com/krohling/ArduinoWebsocketClientDjango Socket IO
github.com/stephenmcd/django-socketio

Socket IO Library
socket.io

WebSockets Background
dev.w3.org/html5/websockets/

In summary, I've walked you through a tech stack that gives you the ability to harness the real time web and allow it to interact with the real world.

I've got some resources here which I'll send around as a link and all the code is in my git hub for the demos I've shown today. Obviously come and talk to me about this if you're interested and I can show you code etc if you want to see it.

Web Facilitated Play
in the Real World

@ajfisher
github.com/ajfisher
slideshare.net/andrewjfisher

Sketching in Hardware
Portland 20 July, 2012

I hope this gives a slightly different view on how we can bring together mobile with physical devices and use the web to facilitate play in the real world.

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline Level