2017tabletop and tablet-based display system to support learner-centric ophthalmic anatomy...

Adam Dubrowski

Divisions of Emergency Medicine and

Pediatrics, Faculty of Medicine, Memorial

University, Canada

Associate

Professor Participant

Noriyuki Matsuda Faculty of Systems Engineering,

Wakayama University, Japan

Associate


Magdalina Todorova

Department of Computer Science, Faculty

of Mathematics and Informatics, Sofia

University, Bulgaria

Associate


Michael Vynnycky

Division of Metals Casting, Department of

Material Science Engineering, Royal

Institute of Technology, Sweden

Head of

Division Participant

Patrick Hung

Faculty of Business and Information

Technology, University of Ontario Institute

of Technology, Canada

Associate


Miguel Vargas Martin




Associate


Andrew Hogue




Associate


Sanshiro Sakai Faculty of Informatics, Shizuoka University,

Japan Professor Participant

Hiroshi Inokawa Research Institute of Electronics, Shizuoka

University, Japan Professor Participant

Kamen Kanev Faculty of Informatics, Shizuoka University,

Japan Professor Participant

Alvaro Joffre Uribe Multimedia and Industrial Engineering,

Nueva Garanada Mil. University

Assistant


Robert Shewaga




Graduate

Student Participant

Robert Codd-Downey

Department of Electrical Engineering and

Computer Science, Lassonde School of

Engineering, York University, Canada

Graduate

Student Participant

所要経費

旅費総額研究・会議費総額消耗品費総額

0円 0円 199,700円

生体医歯工学共同研究拠点内対

応教員

（共同研究をした教員名を記載）

Bill Kapralos, Kamen Kanev, Hiroshi Inokawa

共同研究継続の希望について

有・無

平成２９年度研究費

総額（千円） 200

※継続を希望される場合記入してください

研究成果

The purpose of the 2016 Cooperative Research Project (Research Institute of Electronics, Shizuoka University),

titled Cardiac auscultation skills development through medical simulation-based training, was to continue and

build upon (expand) an ongoing collaborative research initiative between researchers from several institutions in

Japan, Canada, United States, and Colombia, that is examining the application of immersive technologies (virtual

reality, and augmented reality in particular), to medical education and training.

Within the scope of this (year-long) project, our work has focused on completing the development of a

mobile-based application (“app”) for cardiac auscultation training. The app is intended to be used by novel

medical trainees (i.e., students in medical school) to provide them with an interactive and engaging method of

learning how to properly perform a cardiac auscultation. Auscultation refers to the act of listening, either directly

or through an instrument (commonly, a stethoscope) to sounds within the body as a method of medical diagnosis

to evaluate the frequency, intensity, duration, number, and quality of sounds associated with various parts of the

body (typically the heart, lungs, and intestines). Cardiac auscultation allows for the examination of the mitral,

tricuspid, pulmonary, and aortic valves to obtain information regarding the heart’s rate, rhythm, location timing,

intensity, quality, and shape. Cardiac auscultation requires skills that are hard to acquire. Traditional auscultation

teaching focuses on memorizing the heart sounds and their corresponding diagnosis. A variety of audio libraries,

websites and mobile apps that help familiarize trainees with the sounds of the heart under normal and diseased

conditions are available. More recently, virtual reality and manikin-based simulators have become widely used for

training as they allow trainees to practice in controlled scenarios where several procedures can be configured.

Thanks in part to the gaming and computer graphics “revolution” (hardware and software), multimedia tools were

developed to provide 2D and 3D animations, visualizations and simulations of the heart, providing enriched

context to better complement the cognitive development around the auscultation procedure. However, such

scenarios become repetitive and in most cases they are simply digital representations of the content found in

traditional books. That being said, there is a gap between memorizing and practicing with a manikin where

decision-making takes place to diagnose. Our developed app bridges this gap and takes advantage of game

elements supported by low fidelity multimodal cues as a tool to promote analytical thinking and decision making

while performing a virtual auscultation. Below is a summary of the results obtained during this funding period:

Development of the cardiac auscultation mobile app at the University of Ontario Institute of Technology. This

work was carried out by students supervised by Bill Kapralos and Kamen Kanev in addition to a student

(Sergio Prada) from Colombia supervised by Alvaro Quevedo who visited the University of Ontario Institute of

Technology for a four-month period to work at the lab of Bill Kapralos as part of his required university

internship.

Usability testing of the cardiac auscultation app in Canada and Colombia with plans to extend it to Japan. The

preliminary results of the usability testing indicate that the developed mobile app is engaging, easy to use, fun,

and shows promise as a viable training tool.

Expanding our collaboration by new efforts in examining the development of other engaging apps such as for

eye fundus examination using virtual reality technologies. This work is being carried out at multiple institutions

and more specifically, at the University of Ontario Institute of Technology (Bill Kapralos), York University

(Michael Jenkin), and Shizuoka University (Kamen Kanev).

使用した設備・資料・試料等

Research equipment available in the labs of the project members:

computing systems, interactive tabletops, VR and AR environments,

specialized optical readers for code extraction and analysis, and others.

本研究成果に関連する論文発表状況

（１） D. Acosta, A. Uribe-Quevedo, N. Jaimes, and B. Kapralos. Development of a smartphone augmented

reality eye examination tool. In Proceedings of the 11th Annual International Technology, Education and

Development Conference (INTED2017), March 6-8, 2017, Valencia, Spain.

（２） R. Codd-Downey, R. Shewaga, A. Uribe-Quevedo, B. Kapralos, K. Kanev, and M. Jenkin. A novel

tabletop and tablet-based display system to support learner-centric ophthalmic anatomy education. In Proceedings

of the 3rd International Conference on Augmented Reality, Virtual Reality and Computer Graphics (SALENTO

AVR 2016), Otranto, Lecce, Italy, June 15-18, 2016.

（３） R. P Barneva, K. Kanev, B. Kapralos, M. Jenkin, and B. Brimkov. Integrating technology-enhanced

collaborative surfaces and gamification for the next generation classroom. Journal of Educational Technology

Systems, 45(3):309–325, 2017.

（４） B. Kapralos, K. Collins, and A. Uribe-Quevedo. The senses and virtual environments. The Senses and

Society, 12(1):69-75, 2017.

（５） M. Nguyen, A. Uribe-Quevedo, B. Kapralos, M. Jenkin, K. Kanev, and N. Jaimes. An experimental

training support framework for eye fundus examination skill development. Computer Methods in Biomechanics

and Biomedical Engineering: Imaging & Visualization, Special Issue CompIMAGE. Submitted January 15, 2017

and currently under review.

（６） S. Prada, A. Uribe-Quevedo, N. Jaimes, and B. Kapralos. Development of a cardiac auscultation serious

game app. In Proceedings of the 11th Annual International Technology, Education and Development Conference

(INTED2017), March 6-8, 2017, Valencia, Spain.

（７） M. Nguyen, A. Uribe-Quevedo, M. Jenkin, B. Kapralos, K. Kanev, and N. Jaimes. An interactive virtual

reality environment for image-based eye fundus examination training. International Symposium CompIMAGE'16 -

Computational Modeling of Objects Presented in Images: Fundamentals, Methods, and Applications, September

21-23, Niagara Falls, NY, USA. (Short Communication).

（８） D. Rojas, B. Kapralos, and A. Dubrowski. The Role of Game Elements in Online Learning within

Health Professions Education. Studies in Health Technology and Informatics, 220:329-234, 2016.

（９） A. Uribe-Quevedo, B. Kapralos, A. Hogue, K. Kanev, M. Jenkin, and R. P. Barneva. A multi-user

tabletop display with enhanced mobile visuals for teaching and collaborative training. Consortium for Computing

Sciences in Colleges — Northeastern Region 2016 Conference, Clinton, NY, USA, April 29 – 30, 2016. Appears in

J. of Comp. Sciences in Colleges, 31(6):60-62, 2016.

（１０） A. Uribe-Quevedo, D. Rojas, B. Kapralos, F. Moussa, A. Dubrowski, and N. Jaimes. A virtual simulation

for cardiac auscultation training. 2016 Sim-One Healthcare Simulation Exposition. October 6-7, 2016, Toronto,

Canada. Abstract.

（１１） A. Uribe-Quevedo, D. Rojas, M. Usman, B. Kapralos, F. Moussa, A. Dubrowski, and N. Jaimes. Cardiac

auscultation serious game approach. In Proc. of the 4th Int.Conference on Serious Games and Applications for

Health, Orlando, FL, USA, May 11-13, 2016, pp. 1-2.

2017tabletop and tablet-based display system to support learner-centric ophthalmic anatomy...

Documents