RESULTSA total of 849 clinicians participated in the Do You Know JAK? web activity. Analysis was conducted of pretest and posttest responses for 490 matched participants. For all statistics described, the significance level was set at p<.001. The percent correct responses and Q values are summarized in Table 1. To demonstrate the effectiveness of the training, the ORs of providing a correct answer on the posttest compared with the pretest were determined. (Table 1) An OR greater than 1 indicates a positive effect. Increases in knowledge and competence are illustrated in Figures 4 and 5.

As another measure of competence, participants were asked to indicate on the postactivity evaluation, what changes they would make in their practice (Figure 6).

Learners had the opportunity to provide comments in a free-text item on the postactivity online evaluation (Figure 7).METHODS

Educational DesignUsing adult learning principles and recent educational research, Paradigm developed a unique and engaging CME activity featuring a quiz-style game called Do You Know JAK? which presents learners with knowledge-based and case-based quiz questions. (Figure 1)

The active learning design facilitates retention and application of information, resulting in increases in knowledge and competence, with the intention of improving clinical performance.4 (Figure 2) Moreover, fun has a positive effect on motivation levels, determining what we learn and how much we retain. Making learning fun increases its effectiveness and the likelihood of changing behavior.5,6

The web-based activity also includes a brief didactic primer introducing information about RA and JAK inhibition unfamiliar to the learners. The activity ends with a brief faculty discussion video summing up important take-home messages for clinical practice highlighted by the game.

A unique feature of the game is its animated visual scorecard using the MOA of JAK inhibitors: correctly answering a question results in successful JAK inhibition of the RA inflammatory cascade while answering incorrectly results in continued inflammation. (Figure 3) The animated scorecard not only tracks learners’ success in the game; it repeatedly reinforces one of the scientific concepts the activity is designed to teach. In addition to receiving feedback about their answers from the scorecard, participants receive additional feedback in the form of a short video rationale of the correct answer from one of the faculty members after each question.

DO YOU KNOW JAK?A CME GAMIFICATION CASE STUDY

Presented at the Alliance for Continung Education in the Health Professions (ACEhp) 2019 Annual Conference

Annette Schwind, MS, CHCPRobert L. Diehl, BATara E. Brant, MPHParadigm Medical Communications, LLC

INTRODUCTIONBackgroundRheumatoid arthritis (RA) affects an estimated 1.5 million adults in the United States with a substantial negative impact on quality of life and mortality.1 Despite advances in treatment over the past 2 decades, a significant percentage of RA patients either do not respond to therapy or lose their response to therapy over time.2 Janus kinase (JAK) inhibition represents a novel and effective way to treat patients with RA.3

Gaps in Clinician Knowledge and CompetenceOur survey of clinicians caring for patients with RA, including rheumatologists and primary care clinicians, demonstrated a lack of knowledge regarding the role JAK plays in RA pathogenesis and how JAK inhibitors work to treat the disease. Without this foundational knowledge, clinicians cannot appropriately use JAK inhibitors in practice. Published data and interviews with clinicians confirmed these gaps. It became clear that clinicians working as part of a multidisciplinary team to care for patients with RA would need education to enhance their understanding of oral JAK inhibitors’ multilayered mechanism of action (MOA) and how these innovative agents can be used to optimal effect in appropriate patients with RA.

FIGURE 1: AN EXAMPLE OF A CASE-BASED QUESTION FROM DO YOU KNOW JAK?

KNOWLEDGE RETENTION

What theyDO

What theySAY or WRITE

What theyHEAR and SEE

What theySEE

What theyREAD

MORE

LESS

GAMES

INTERACTIVEE-COURSES

E-COURSES WITHAUDIO AND VIDEO

FIGURE 2: GREATER RETENTION ASSOCIATED WITH ACTIVE LEARNING4

FIGURE 3: SCORING DO YOU KNOW JAK?

100%

80%

60%

40%

20%

0%Intracellular pathways of

JAK inhibition

Mechanism of JAK inhibition and effect on inflammation

JAK signaling Identifying patients for JAK inhibitor therapy

Pretest (n=490) Posttest (n=490) *p<.001

26%

89%

34%

86%

19%

88%

67%*

14%

***

FIGURE 4. PERCENTAGE OF PARTICIPANTS CORRECTLY RESPONDING TO PRETEST/POSTTEST QUESTIONS

FIGURE 7. PARTICIPANT FEEDBACK

242%INCREASE

222%INCREASE

379%INCREASE

Knowledge: Role of JAK in RA inflammatory cycle

Knowledge: Immune and anti-inflammatory effects of JAK inhibition

Competence: Appropriate patient selection for JAK inhibitor treatment

FIGURE 5. AVERAGE KNOWLEDGE AND COMPETENCE GAINS BY LEARNING OBJECTIVE

FIGURE 6. PERCENT INTEND TO MAKE PRACTICE CHANGES

LEARNING OBJECTIVESThree objectives were formulated for the activity, along with pretest/posttest questions to measure achievement:

1. Describe the role of Janus kinase in the chronic cycle of inflammation associated with rheumatoid arthritis.

Knowledge increase measured by question 1

2. Identify the intracellular immune and anti-inflammatory effects of Janus kinase inhibition on rheumatoid arthritis.

Knowledge increase measured by questions 2 and 3

3. Select appropriate candidates for treatment of rheumatoid arthritis with JAK inhibitors.

Competence increase measured by question 4

ANALYSESMatched pre/posttest results were analyzed to show increases in knowledge and competency. Cochran’s Q test was used to analyze the percentage of correct responses to each learning objective and to determine the statistical significance. The effectiveness of education was measured by calculating the Odds Ratio (OR) for providing a correct response on the posttest compared with the pretest.

Excellent activity!!!!!!

Excellent graphics. They will help me

remember the content.

Good review and explanation of a

challenging topic.

Great CME. Really enjoyed

the game.

The interactive game was good. Overall,

excellent CME.Thank you for such a great

activity.

DISCUSSIONWe found the Do You Know JAK? interactive game activity was associated with significant increases in knowledge and competence. These were accompanied by a large percentage of participants intending to make practice changes. Additionally, participants acknowledged that the activity was enjoyable and effective. The results of the Do You Know JAK? interactive game activity demonstrate such formats can be highly effective in improving clinician knowledge and competence (Moore’s Levels 3 and 4).7 To determine success in implementing their intended changes to practice, learners received a follow-up survey 2 months following their completion of the activity (Moore’s Level 5)7. The response rate (n=5) to the survey was too low to draw definitive conclusions, but the 20% of respondents who did implement practice changes suggests that the activity had some impact on clinical behavior.

CONCLUSIONSGaming formats engage participants in active learning while also appealing to innate senses of fun and competition, resulting in high knowledge and competence scores. In addition, learner feedback indicates clinicians enjoy and value the game format.

ACKNOWLEDGMENTSParadigm gratefully acknowledges Luciane Pereira-Pasarin, PhD, for statistical analysis of outcomes data.

This educational activity is supported by an educational grant from Pfizer Inc.

REFERENCES1. Singh JA, Saag KG, Bridges SL Jr, et al. 2015 American College of

Rheumatology Guideline for the Treatment of Rheumatoid Arthritis. Arthritis Rheumatol. 2016 Jan;68(1):1-26.

2. Yamaoka K. Janus kinase inhibitors for rheumatoid arthritis. Curr Opin Chem Biol. 2016;32:29-33.

3. Strand V, Kremer JM, Gruben D, et al. Tofacitinib in combination with conventional disease-modifying antirheumatic drugs in patients with active rheumatoid arthritis: patient-reported outcomes from a phase 3 randomized controlled trial. Arthritis Care Res. 2017;69(4):592-598.

4. Dale E. Audio-Visual Methods in Teaching. 3rd ed. New York, NY: Holt, Rinehart and Winston; 1969:108.

5. Tews MJ, Michel JW, Noe RA. Does fun promote learning? The relationship between fun in the workplace and informal learning. J Vocat Behav. 2017;98:46-55.

6. Garner LR. Humor in pedagogy: how ha-ha can lead to aha! College Teaching. 2006;54.177-180.

7. Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1-15.

Learning Objective

Question Number

Correct responses

pretest

Correct responses posttest

Q OR

1 1 26% 89% 295.78* 23.58

22 34% 86% 231.41* 12.00

3 19% 88% 321.73* 30.42

3 4 14% 67% 227.83* 12.58

*p<.001. All Q values were statistically significant.

TABLE 1: PRETEST/POSTTEST RESULTS FROM DO YOU KNOW JAK? ACTIVITY

79%OF RESPONDENTS WOULD MAKE A CHANGE TO THEIR PRACTICE

“Using or considering the use of JAK inhibitors in appropriate patients”was the top change indicated by respondents

(n=159)