cs415 human computer interaction - mercury.pr.erau.edu
TRANSCRIPT
The Basic ElementsHuman – Models of Perception and Cognition
– Brain Models [Simulations]– Brain Scans [fMRI, EEG, fNIRS]– Brain Processing of Sensory Information [Perception]
Computer – Theory of Computation and Devices– What Can Be Computed? What Can’t? How Fast?– Traditional ALU, Stored Program Computer: Von Neumann
Architecture, Von Neumann Bottleneck– Turing Machines [Wikipedia], Church-Turing Thesis
[Wikipedia], Turing Test– Hypercomputation (Myth, Reality?)– Quantum Computing, D-Wave, ANN, Analog “Real”
Computing, Neuromorphic, Deep Learning (GPU)
Interaction – Physical Devices and Interfaces– Physics, Time and Space, Chemistry– Acoustics, Haptics, Optics, Olfaction, Gustation– 5 Senses – Hear, Feel, See, Smell, Taste– Fusion of Senses – Proprioception [Body movement]– Extended – Chronoception [Time], Nocioception [Pain]– 20+ Senses: Internal [interoceptive], External [exteroceptive]– Superhuman Sensing [E.g. Thermal imaging, Magnetic field]
Sam Siewert 2
Why is Human Vision > Computer?
Sam Siewert 3
Approximately 100+ Mega-Pixel(Rod/Cone Count)
James Cutting & Peter Vishton –Perceiving Layout
Cortex=10 Billion Neurons(High fan-out)
Total=100 Billion Neurons
Neuroscience. 2nd edition.Purves D, Augustine GJ, Fitzpatrick D, et al., editors.Sunderland (MA): Sinauer Associates; 2001.http://www.ncbi.nlm.nih.gov/books/NBK10848/
Red Epic 64563 Mega-Pixel
I/O Bus (x16 5Gbps = 8GB/sec)
CameraLink
Interface Card
Local Bus
CPU CPU
MemoryController
5 To 10 billion transistors1. Neuron > Transistor2. Better Programming? ROM?3. More Richly Interconnected4. Storage + Processing
> 1 Trillion Synapses
http://bluebrain.epfl.ch/Human Brain ModelsProject StatusVisualization
Biological Vision vs. Machine Vision(Why A Honey Bee is Better than HPC for CV)
Humans - 100 million Photoreceptors
– 10 billion Neurons (Cerebral Cortex)– Brain with 100 billion Neurons– Millisecond Transfer – Massively Parallel Analog + Digital Computation
Synapse Match is a Challenge– 7000 Connections from 10 Billion Neurons– 3 Year Olds Have 1015 Synapses
CPU to Digital Camera/HDD– Connects 10’s of millions of pixels – to Several Billion transistors – Through Sequential Logic and I/O Bus
Sam Siewert 4
960K Neurons in flight:Learns locations,complex odors,colors, and shapes;with high efficiency (500 Watt/Kg), 0.218g
Brain plasticity for learning, connectedness, concurrency, integrated sensing, power efficiency, and resiliency
2016 – 16 billion?
NVIDIA GK11028nm, (7.1 billion)
Pascal – 15 billion
Intel MICA 22nm(5 billion)
http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons
Moore’s Law Graphic - Wikipedia
3D PerceptionWe Live in an Animated 3D World
We still Compute with 2D Desktops (WIMP)– Tablets, MS Surface, Notebooks, Laptops,
Desktops– Mobile SmartPhones, iWatch, Android Gear– Enabled by Key Devices (Pointer – Mouse,
Stylus, Touch-screen, Raster Graphics – CRT, LCD, OLED, VR CAVE)
3D Limited to CAD, VR Worlds, Movies, Visualization So Far …
3D Batch Processing – Printers and Scanners
What About Interactive 3D?
Sam Siewert 5
https://en.wikipedia.org/wiki/OLED
https://en.wikipedia.org/wiki/3D_film
1950’s
Real World 3D PerceptionMany Visual Cues Provide 3D Perception – Open Research (Perceiving Layout)
HCI Inventors Want to Exploit What Matters Most– Stereopsis and Disparity (Left Eye and Right Eye View)– Accommodation (Near and Far Field)– Correspondence (Common Features Noted with Each Eye)
Sam Siewert 6
http://www.ibm.com/developerworks/library/bd-mdasecurity/, Sam Siewert
“Play” with RGB Depth MappersScene Depth Mapping + RGB Planar Image– Scene Segmentation (in Z as well as XY)– Skeletonization– Hand Tracking and Gesture Recognition– Interaction with 3D Virtual Worlds
Sam Siewert 7
Data Glove vs. RGB Depth MapperHand and Finger Tracking Based on SegmentationDepth (Skeletonization)Processing is Somewhat IntenseNothing to Wear
Data GloveRotational Rates, Accelerometer, Finger Flexure Sensors
Sam Siewert 8
CreativeCamWith Intel RealSense
3D SDK
M.S. research, CU Boulder,“A Hybrid Sign Language Recognition System”, Van Culver, 2004OpenCV and Monovision Camera with 5DT Data Glove
Interactive 3D Devices I/O?3D Input – Active Depth Mappers, Passive Stereopsis– ASUS Xtion Interactive Depth Mapper– PrimeSense Chip [Apple now] – Revitalized– Many Competing Chips and Devices Now– Passive – Compute Intensive, Less Robust
(Flat Wall Problem)– Time-of-Flight, LIDAR (Distance, Outdoor)
3D Output [Left, Right Eye Disparity]– 120 Hz, Left / Right Eye Parallax– Filtering, Polarization (Left/Right Eye)– Active Shutter Glasses– Entertainment and Visualization Success,
Interactive, Less So … Sam Siewert 9
“Fool the Eyes!”
3D Alternatives?Augmented Reality (Interactive Mix of Real/Virtual)
1. Project Virtual World onto Real World2. View Real World Through Goggles with Annotation3. Project Virtual Images into retina as Light-field Overlay
Heads-Up-Display – Aviation [Gunsight Generalization]
Pranav Mistry, MIT Media Lab, Samsung– Project onto Real World– We’ll Study Later
Google Glass VRD – Failed, “Glassholes”
MS Hololens [View Real World with Annotation]
Magic Leap [VRD] – MIT Tech Review Article
Oculus Rift – VR Headset [Dual OLED, Head Tracking]
Wearable AR/VR Doomed? Limited? Unhealthy?
Sam Siewert 10
C130 HUD
Before We Go FurtherStart with Basics
Move to 2D
3D Animation
Interaction Theory and Guidelines
Cognition and Perception Models
Then, Back to the Future! Sam Siewert 11
Linux How-ToUbuntu LTS 14.04 on Virtual Box and Jetson– Local How-To Resources
LinuxLinux-Development-Getting-Started.pdfLinux-Basic-Makefile-by-Example.pdfLinux-Setting-up-VBOXSF-Automount-from-Host.pdfLinux-Programming-Top-Errors.pdf
– https://ubuntu-manual.org/– https://www.youtube.com/watch?v=SGGj3hc
6R8Q
Sam Siewert 13
Virtual Box VM Installation ofa New OS Like Ubuntu Linuxis a Safe Way to Learn on thePlatform you Know Best
Server/Desktop, VM, EmbeddedA Key Advantage of Ubuntu is that One Linux Distribution Can be Used for Server/Desktop, VM and Embedded Use– Server: PRCLab (prclab.pr.erau.edu)– VM: Virtual Box with Ubuntu 14.04
LTS– Embedded: Jetson TK1 Developer
System
Linux Kernel supports the NDK Layer in the AOS (Android Operating System)– https://developer.android.com/ndk/inde
x.html– https://developer.android.com/sdk– https://developer.android.com/tools/sd
k/ndk/index.html
Sam Siewert 14
A few Comments on OptionsNative Linux (Jetson) – Best Performance for Interactive Applications
VM Linux – Nice Option for Convenient Development and Test
Server Linux – Safe Haven that is Highly Available, Backed-Up
Moving Code Can be Done with SFTP or SCP– SFTP Requires ZIP or Archive to Move Whole Directories– SCP – Moves Structure Automatically
Sam Siewert 15