tadhack london, be my friend, justin grierson & matt williams, tropo

Be My FriendJustin Grierson and Matt Williams

TADHack-mini London - April 9-10 2016

ProblemHow do we

Solve Mental

Health Issues

Without a

Therapist

SolutionUsing Tropo, Bluemix and

Javascript Libraries

To Determine

People’s Mental State

Demo 1

IVR platforms don’t generally support video

Architecture 2

Phone(jsSIP)

IMS Core(Clearwater)

App Server(Web Browser!)

SIP

(/WebSocke

t)SIP/WebSocket

WebRTC

“Server” side

Demo 2

Technology 2● Browser itself acts as application server!

● WebRTC● SIP.js● Media Capture from DOM Elements API● RecorderJS for recording to WAV file● clmtrackr for face and emotion tracking● three.JS and WebGL for 3D rendering● MakeHuman (for human model)

Futures● Integrated video and speech sentiment analysis● Speech tone sentiment analysis● RTP/WebRTC interworking using Dialogic● Video support built into IVR platform?● Other applications of video IVR

○ Mirroring to build trust○ Barcode recognition

Source code at https://github.com/matt-williams/browser-app-server