Assessment of the use of the Virtual Orthodontic Case Studies in an Undergraduate Teaching

Program

Dr Jean –Marc Retrouvey

VOCS: An Interactive Teaching Tool in Orthodontics

What is CAL?Computer Assisted Learning

Any type of pedagogical material that is delivered through a computer software.

Can be very simple (PPTs) or very complex (Flight simulators)

Most effective is the interactive model where students participate to the learning process

Validity of CAL

Rosenberg study demonstrated that CAL usage was at least as effective in orthodontic undergraduate education (diagnosis) as standard classroom lectures One of the few orthodontically oriented pedagogical tools

build with instructional designers

Rosenberg, H., et al., Evaluation of computer-aided learning in orthodontics. American Journal of Orthodontics and Dentofacial Orthopedics, 2010. 138(4): p. 410-419.

Interactive Seminars are coming to Orthodontics….

Interactive Distance Learning in Orthodontic Residency Programs: Problems and Potential Solutions

Katherine P. Klein, D.M.D., M.S.; Wallace H. Hannum, Ph.D.; Henry W. Fields, D.D.S., M.S., M.S.D.; William R. Proffit, D.D.S., Ph.D. (2012)

Sharing resources through distance education has been proposed as one way to deal with a lack of full-time faculty members and maintain high-quality content in orthodontic residency programs. To keep distance education for orthodontic residents cost-effective while retaining interaction, a blended approach was developed that combines observation of web-based seminars with live post-seminar discussions.

Rationale for VOCS

Database of carefully selected patients used for pedagogical reasons (searchable)

Study and present cases anywhere ( home, school, lectures)

Avoid model storage and manipulations (paperless records)

Fast loading time on any computer (No 3D ) Capability to test students electronically

Introduction to VOCS

Objectives of this study

First objective: To evaluate students’ perceptions concerning a series of

aspects of the VOCS learning tool including quality and quantity of information provided and simplicity of use.

Objectives of the study

Second Objective: To determine if there was a difference in the results of

data collected from “conventional “ examination and measurements on study casts and the VOCS software.

Question : Is VOCS a reliable tool to study orthodontic cases?

Study Design: part 1

21 students Each student studied one case using the VOCS online system questionnaire was filled by same student to assess his/her

perception about the online method’s quality and likability. evaluating ‘general material’, ‘patient history’, ‘facial

images’, ‘intra-oral images’, ‘dental cast’, ‘panoramic’, ‘cephalometrics’, ‘quiz’, and ‘overall feedback’ were presented.

Each item had a 4 category scale ranging from ‘strongly disagree’ to ‘strongly agree’.

– Virtual Orthodontic Case Studies (VOCS) 

Date: ___________________ 

Instructions: Evaluations provide us with important feedback regarding our work. Please take a couple of minutes to answer the following questions candidly. We thank you for your time and input which will assist in preparing for future work developed for the VOCS. If you require additional room for your comments, please feel free to add an extra sheet of paper.

Strongly Disagree Agree Strongly Disagree AgreeInteractive Materials/Activities 1. Module objectives were clear. 2. Module materials were well written. 3. Module materials were well organized.

4. Materials/activities were helpful in learning new skills

5. Materials/activities were relevant and interesting

6. The module provided new information and/or sufficient depth to challenge me. 

Program Outcomes 7. The outlined module objectives were met. 8. I learned something I can immediately apply. 9. My time was well invested in this module. 10. I would recommend this module to other individuals.   

Evaluation – VOCS   

Strongly Disagree Agree Strongly Disagree Agree

Overall Usability of Website11. The website was easy to navigate.

12. It was simple to find information.

13. The radiographs were clear.

14. The photographs were clear.

15. The website models were sufficient for diagnosis.

  Website as a Teaching Method16. The website is a good teaching method.

17. The cases on the website are accessible anywhere/time.

18. The cases on the website can be reviewed easily.

19. The case material is easy to refer to during the quiz.

 Website Compared to other Teaching Methods20. The website is a better teaching method than the

student presentations.

21. The website is a better teaching method than the hands-on teacher-to-class technique.  

Evaluation – VOCS

You Tell Us

 

22. In your opinion, what were the advantages of the website over other teaching methods?

 

 

 

 

 

23. In your opinion, what were the disadvantages of the website over other teaching methods?

 

 

 

 

24. Additional comments (please feel free to add further remarks on another sheet).

 

 

History

TABS

Radiographs

Boley Gage Assessment

Strongly Disagree

Disagree Agree Strongly Agree

0

20

40

60

80

100

Boley Gage in-structions easy to

follow

Strongly Disagree

Disagree Agree Strongly Agree

0102030405060708090

100

Boley Gage is simple to use

Quizz

Diagnostic Capability

Strongly Agree

Agree Disagree Strongly Disagree

0

10

20

30

40

50

60

70

80

90

100

The website models were sufficient for diagnosis

All students either agreed that this 2 D method was sufficient to diagnose simple to moderately complex malocclusions.

Overall Feedback

Strongly Agree

Agree Disagree Strongly Disagree

0

10

20

30

40

50

60

70

80

90

100

My time was well invested in this module

Strongly Agree

Agree Disagree Strongly Disagree

0

10

20

30

40

50

60

70

80

90

100

I would recommend this module to other individuals

Comments for part 1

95% of the students responded ‘agree’ or ‘strongly agree’ to the questionnaire’s items (36 and 59%, respectively). Twenty of the 46 items had 100% of the students responding either ‘agree’ or ‘strongly agree’.

The only negatively phrased item (‘some questions are redundant’) had 65% of responses at ‘agree’ or ‘strongly agree’ and the remaining 35% at ‘disagree’ or ‘strongly disagree’.(quality of the quizz)

Biggest problem: The item concerning ‘intra-oral images being clear’ had 2 people (10%) responding ‘strongly disagree’ .

Midline determination is challenging

Part 2: Reliability measurements

Reliability of the VOCS measurements in relation to conventional measurements

Duplicated the methodology used in dental schools for an initial orthodontic case study (undergraduate level)

Assess if the students measured the same things and obtain the same diagnosis ( correctly or incorrectly).

Methods

21 students participated 13 completed the exercise correctly Analyzed conventional records and measured study

casts (test reliability and accuracy of the virtual tools) Teachers (3) corrections or evaluations were deemed

be worth 100% of the score (total agreement on data analysis)

Students Percentage score

Model VOCS

1 50,0 59,1

2 68,2 50,0

3 72,7 72,7

4 45,5 68,2

5 45,5 50,0

6 81,8 31,8

7 81,8 81,8

8 45,5 63,6

9 77,3 68,2

10 63,6 68,2

11 86,4 81,8

12 77,3 72,7

13 59,1 54,5

Table 1: Individual students percentage score in the orthodontic analysis evaluation using either models or the VOCS.

Table 2: Test statistics on the differences in average scores between groups using model or VOCS to complete the orthodontics analysis evaluation.

n Mean score Standard Deviation p

Model 13 65,7 15,2 0,63

VOCS 13 63,2 14,1

Results of the analyses from students

The results show that their was no significant difference between the analyses of the students if they used either the standard study casts, boley gages and rulers or if they used the virtual dental casts and tools (P=0.63)

The students who got the right or wrong diagnosis using either methods (except for one student 80 -30)

Discussion and Conclusions

Case studies can be made available 24/7 on any LMS (learning management system)

Quizzes and cases studies can be packaged by the instructor or the administrator

Repository of searchable cases Easy to load on any computer system Reliable and test at the same level any case analysis

Self paced teaching tool used now at McGill Canadian National Dental Board has used 4 of the

VOCS cases for its examination process Very positive response to the Virtual environment. All students from any group can be tested on the same

case

Better teaching tool?

Strongly Agree

Agree Disagree Strongly Disagree

0

10

20

30

40

50

60

70

80

90

100 The website is a better teaching method than the student pre-

sentations

Strongly Agree

Agree Disagree Strongly Disagree

0

10

20

30

40

50

60

70

80

90

100 The website is a better teaching method than the hands-on teacher-to-class

technique

More than 50% still preferred the one on one hands-on teaching technique (gold standard). Perfectly normal as feed back and assistance are not readily available with the on line method.It was preferred to the students presentations .

Thank you