![Page 1: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/1.jpg)
1Department of Electrical and Computer EngineeringDepartment of Electrical and Computer Engineering Advisor: Professor Goeckel
A Big Step Forward for Virtual Reality
Step
![Page 2: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/2.jpg)
2Department of Electrical and Computer Engineering
Team Members
Ryan DalyElectrical Engineer
Joseph RobertsElectrical Engineer
Jared RicciElectrical Engineer
Steven SoElectrical Engineer
![Page 3: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/3.jpg)
3Department of Electrical and Computer Engineering
Problem
▪ Modern smartphone VR isn’t immersive
• Discontinuity between virtual actions and physical actions
• Limited to sight and sound
▪ Less control in a virtual environment leads to a higher occurrence of disorientation [1]
[1] Sharples, Sarah. "Virtual Reality Induced Symptoms and Effects (VRISE): Comparison of Head Mounted Display (HMD), Desktop and Projection Display Systems." Displays 29.2 (2008): 58-69. Print.
![Page 4: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/4.jpg)
4Department of Electrical and Computer Engineering
Step
![Page 5: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/5.jpg)
5Department of Electrical and Computer Engineering
Societal Significance
▪ Potential for:
• Engaging exercise
• Educational experiences
• Infantry/first responder training
![Page 6: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/6.jpg)
6Department of Electrical and Computer Engineering
World Problems
▪ Obesity is a growing problem in America
▪ “obesity can lead to other illnesses from chronic to acute some very severe and deadly, such as diabetes, high blood pressure… and even cancer” -Arizona State University [2]
Obesity findings of the Center of Disease Control and Prevention [3][2] Begay, Dion. "What Can Obesity Lead To?" Obesity in the Latino Population. N.p., Spring 2005. Web. 13 Oct. 2016.[3] Nunes, Keith. "C.D.C.: Prevalence of Obesity among U.S. Adults at All-time High." Food and Beverage News, Trends, Ingredient Technologies and Commodity Markets Analysis. N.p., 26 Feb. 2016. Web. 13 Oct. 2016.
![Page 7: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/7.jpg)
7Department of Electrical and Computer Engineering
Conveniency and Educational Problems
▪ Being able to conveniently visit places, such as colleges, is inconvenient and costly
▪ Virtual tours are insufficient in thoroughly experiencing the environment
![Page 8: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/8.jpg)
8Department of Electrical and Computer Engineering
Incompetent Training
▪ “One of the study's findings was that Soldiers lack access to realistic TC3 simulation that could improve the individual and collective skills Soldiers and squads need to manage the complex environment of simultaneous combat and casualty management.” -Mike Casey, Combined Arms Center US ARMY [4]
▪ Prevent deaths and save lives, not just in the military, but also in first responder training
[4] Casey, Mike. January 25, 2016. "Improving Tactical Combat Casualty Care to save Soldiers' Lives."Www.army.mil. N.p., 25 Jan. 2016. Web. 13 Oct. 2016.
![Page 9: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/9.jpg)
9Department of Electrical and Computer Engineering
Overall Requirements
▪ User is able to freely move in virtual environment
▪ Hand and arm motion is translated to in-game action
▪ User does not have to wear any equipment beyond VR headset
▪ Maintain framerate at 60 FPS“You cannot drop below 60 FPS. Period. Ever.”-Chris Norden, Senior Staff Engineer, Sony [5]
[5] Hall, Charlie. "Sony to Devs: If You Drop below 60 Fps in VR We Will Not Certify Your Game." Polygon. Vox Media, 17 Mar. 2016. Web. 13 Oct. 2016. http://www.polygon.com/2016/3/17/11256142/sony-framerate-60fps-vr-certification
![Page 10: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/10.jpg)
10Department of Electrical and Computer Engineering
Design Alternatives (PC Based)
Oculus Rift
▪ User is static in reality but moving in the virtual world
▪ User input via game controller
HTC Vive
▪ User movement limited by room size
▪ Wire tether and handheld equipment limit immersion
![Page 11: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/11.jpg)
11Department of Electrical and Computer Engineering
Design Alternatives (Smartphone Based)
Google Cardboard
▪ User is unable to interact with the virtual environment via any kinesthetic movements
▪ Experience is strictly stationary
Google Daydream
▪ User can interact with the environment solely through the use of a controller
![Page 12: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/12.jpg)
12Department of Electrical and Computer Engineering
Our Solution▪ Create a system to immerse
the user in virtual reality
▪ Allows motion in endless environment
▪ Detects movement without added hardware
▪ Users movements will be reflected in the virtual world (walking, running, flailing)
![Page 13: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/13.jpg)
13Department of Electrical and Computer Engineering
Logical Block Diagram
Forward Motion
VR EnvironmentData ProcessingI/O System
Hand and Arm Movement
![Page 14: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/14.jpg)
14Department of Electrical and Computer Engineering
Physical Block Diagram
Smartphone and VR Headset
PC processor and data transmitter
Kinect Center Sensor
Kinect Left Sensor
Kinect Right Sensor
Hand and Arm Movement
Kinect Depth Processing
Forward Motion
Sensing Board
Raspberry Pi
Kinect Data
Speed Data
Data Compiler
and interfacing
Data Transmitter
via Wifi
Elliptical
![Page 15: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/15.jpg)
15Department of Electrical and Computer Engineering
Physical Block Diagram
Smartphone and VR Headset
PC processor and data transmitter
Kinect Center Sensor
Kinect Left Sensor
Kinect Right Sensor
Hand and Arm Movement
Kinect Depth Processing
Forward Motion
Sensing Board
Raspberry Pi
Kinect Data
Speed Data
Data Compiler
and interfacing
Data Transmitter
via Wifi
Elliptical
![Page 16: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/16.jpg)
16Department of Electrical and Computer Engineering
Forward Motion
Requirements
▪ Accurately read user speed within 0.5 MPH
▪ Transmit user movement speed to host PC with 150 ms latency
▪ Securely attach to equipment without interfering with user movement
![Page 17: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/17.jpg)
17Department of Electrical and Computer Engineering
Elliptical Sensor Implementation
▪ Hall Effect sensor to detect rotation speed
▪ Raspberry Pi attached to sensing board will send data to PC via WiFi
▪ Sensor will be mounted directly to elliptical
![Page 18: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/18.jpg)
18Department of Electrical and Computer Engineering
Physical Block Diagram
Smartphone and VR Headset
PC processor and data transmitter
Kinect Center Sensor
Kinect Left Sensor
Kinect Right Sensor
Hand and Arm Movement
Kinect Depth Processing
Forward Motion
Sensing Board
Raspberry Pi
Kinect Data
Speed Data
Data Compiler
and interfacing
Data Transmitter
via Wifi
Elliptical
![Page 19: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/19.jpg)
19Department of Electrical and Computer Engineering
Arm Motion Sensor
[6] S. ©2015, "Stimulant | Depth Sensor Shootout", Stimulant.com, 2016. [Online]. Available: https://stimulant.com/depth-sensor-shootout-2/. [Accessed: 13- Oct- 2016].
Choice: Xbox Kinect
- Simple interface with computer- Detects full body of motion
Alternatives- Leap Motion
- Valuable for processing a couple of hands, not entire upper body [6]
Requirements- Real-time processing and
transmitting- Depth within 1” tolerance for
arm movement
![Page 20: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/20.jpg)
20Department of Electrical and Computer Engineering
Kinect Problems
- Poor depth performance- “The depth data registered by the Kinect 1.0
[360] has poor quality … the structured light approach is not always robust enough to provide a high level of completeness.” [7]
[7] D. Pagliari and L. Pinto, "Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors", Sensors, vol. 15, no. 11, p. 27571, 2015.
![Page 21: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/21.jpg)
21Department of Electrical and Computer Engineering
Kinect Implementation Plan
- Add second Kinect at different angle
- Determine depth, leveraging pre-existing development kits
- Evaluate quality, implement additional Kinect(s) as needed
![Page 22: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/22.jpg)
22Department of Electrical and Computer Engineering
Physical Block Diagram
Smartphone and VR Headset
PC processor and data transmitter
Kinect Center Sensor
Kinect Left Sensor
Kinect Right Sensor
Hand and Arm Movement
Kinect Depth Processing
Forward Motion
Sensing Board
Raspberry Pi
Kinect Data
Speed Data
Data Compiler
and interfacing
Data Transmitter
via Wifi
Elliptical
![Page 23: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/23.jpg)
23Department of Electrical and Computer Engineering
PC and Communication Requirements
Requirements for PC
1. Supply power to the kinect
2. Process and compile the data from kinect and elliptical
3. Needs to be able to communicate and transfer data to the android efficiently
4. Latency Rate 200 ms
![Page 24: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/24.jpg)
24Department of Electrical and Computer Engineering
Data Compiler Implementation
▪ Elliptical input will be a value, the kinect data output that will contain user’s position
▪ Data can be compiled into one data set which will then be the data used to send
![Page 25: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/25.jpg)
25Department of Electrical and Computer Engineering
Communication Implementation Plan
Wifi▪ better than bluetooth▪ high transaction rates
• Latency 150ms vs 200ms• Bit-rate 600Mbps vs 2.1Mbps
Program▪ there are already ways to
interface PC-Android communication via Wifi
▪ Using eclipse and java to send data to android
![Page 26: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/26.jpg)
26Department of Electrical and Computer Engineering
Physical Block Diagram
Smartphone and VR Headset
PC processor and data transmitter
Kinect Center Sensor
Kinect Left Sensor
Kinect Right Sensor
Hand and Arm Movement
Kinect Depth Processing
Forward Motion
Sensing Board
Raspberry Pi
Kinect Data
Speed Data
Data Compiler
and interfacing
Data Transmitter
via Wifi
Elliptical
![Page 27: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/27.jpg)
27Department of Electrical and Computer Engineering
Smartphone and VR Headset Requirements
▪ No additional weight
▪ VR application is able to process data sent from Kinect and elliptical
▪ Frame rate = 60 FPS
![Page 28: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/28.jpg)
28Department of Electrical and Computer Engineering
Smartphone Application Implementation Plan
▪ Application will be developed in the Unity Game Engine environment with C# scripting• C# scripts will be capable of referencing data
sent from the Kinect and elliptical
![Page 29: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/29.jpg)
29Department of Electrical and Computer Engineering
MDR Deliverables
Interactive VR game in which the user can walk along a straight path whilst controlling their speed of travel, and using their arms to pop bubbles. ▪ Demonstrates:
a. precise speed control with ellipticalb. adequate kinect motion sensingc. data processing and transmitting to androidd. a programmed VR environment using inputs
![Page 30: A Big Step Forward for Virtual Reality · • Discontinuity between virtual actions and physical actions • Limited to sight and sound Less control in a virtual environment leads](https://reader033.vdocuments.site/reader033/viewer/2022052100/6039d9f40323592ba33f99a3/html5/thumbnails/30.jpg)
30Department of Electrical and Computer Engineering
Questions?