kinectballs: an interactive tool for ball throwing games

Download KinectBalls: An Interactive Tool for Ball Throwing Games

Post on 15-May-2015

499 views

Category:

Technology

2 download

Embed Size (px)

DESCRIPTION

Presentation by Jonathan Schoreels of KinectBalls during the INTETAIN 2013 conference in Mons, 4 July 2013. KinectBalls is an interactive computer game in which a player throws real balls toward a virtual scene, using the Kinect 3D sensor. This work is the result of a master student project at the Département d'Informatique, Faculté des Sciences, UMONS, Belgium

TRANSCRIPT

  • 1. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion KinectBalls An Interactive Tool for Ball Throwing Games Jonathan Schoreels, Romuald Deshayes, and Tom Mens UMons 4 July 2013 J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 1 / 17

2. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Table of contents 1 Introduction 2 Architecture of KinectBalls 3 Lessons learned and future work 4 Conclusion J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 2 / 17 3. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Introduction Context Recent interest for new games and entertainment applications using new HCI techniques like natural interaction : Nintendos Wii console Microsofts Kinect sensor Leap motion J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 3 / 17 4. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion KinectBalls Goal Bridge the gap between the real and virtual world. The aim of the game is to bring down a pile of virtual boxes by throwing a real ball towards them. For this, we exploited the raw information of the Kinects 3D sensor to track the moving ball. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 4 / 17 5. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Realisation J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 5 / 17 6. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Limitations The Kinect has a low frame-rate (30 Hz). This could be addressed by using an other sensor with better frame-rate. Ball moving too fast can be unseen by the device. Nothing can be seen at a distance below 50 centimetres. The shape of the ball can be distorted. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 6 / 17 7. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Table of contents 1 Introduction 2 Architecture of KinectBalls 3 Lessons learned and future work 4 Conclusion J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 7 / 17 8. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Implementation overview J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 8 / 17 9. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Object detection and tracking For each frame Fn, we have for each pixel the depth. We can compare a frame Fn with the previous frame Fn1. If that dierence exceeds a threshold T (to lter out noise) : Something has moved on that particular pixel. We create the matrix MoveMap. Creation of MoveMap MoveMap(i, j) = 1 if |Fn(i, j) Fn1(i, j)| > T 0 otherwise J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 9 / 17 10. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Object detection and tracking We search the biggest square of 1s in MoveMap. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 10 / 17 11. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Trajectory prediction At least 3 positions of the ball : Approximation with 3 second-degree polynomials. At each new frame where the ball is detected, a new approximation is computed. At the predicted time of impact, a virtual ball is created at the predicted position. Knowing the exact 3D position of the wall, we can approximate the collision between the trajectory approximated and the wall. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 11 / 17 12. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Trajectory prediction J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 12 / 17 13. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Table of contents 1 Introduction 2 Architecture of KinectBalls 3 Lessons learned and future work 4 Conclusion J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 13 / 17 14. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Lessons Learned Kids of 5 years and older tested the game for the Printemps des sciences 2013 : They were enthusiasts. The approximated impact point was good (between 1 to 5 centimetres of precision) Some adults had diculties to interpret the 3D virtual world. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 14 / 17 15. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Future work Use a sensor with a frame-rate of 60 Hz Exploit the Kinects RGB camera to create a virtual ball of the color of the thrown ball. Automate the calibration between the Kinect and the screen. Use an extra Kinect to track the head of the player and create a virtual window eect. Multi-player game. Performance improvements of algorithms. Use a 3D beamer. J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 15 / 17 16. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Table of contents 1 Introduction 2 Architecture of KinectBalls 3 Lessons learned and future work 4 Conclusion J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 16 / 17 17. Introduction Architecture of KinectBalls Lessons learned and future work Conclusion Conclusion KinectBalls requires only one very aordable 3D sensor. Developed algorithms are fast enough to run in real time on a standard computer. The solution worked ne in all tested indoor situations J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 17 / 17

Recommended

View more >