using clicker technology in a clinically-based ... · pdf filestudy sample and group...
TRANSCRIPT
Using Clicker Technology in a Clinically-based Occupational Therapy Course: Comparing Student Perceptions of Participation and Learning
Thomas Mernar, PhD, OTR
Acknowledgements
I would like to thank
• The Office of the Provost, the Teaching, Learning, Technology Center, and Drs. Michael Vigorito and Theresa Bartolotta for the support of this program assessment proposal
• Research assistants Danielle Dorsi, OTS; Michelle Merciadez, OTS, Jennifer Ostaszewski, OTS, Janine Petito, and OTS; Rachel Sellers, OTS
Background
• Student–teacher interaction is ranked highly among the factors influencing learning performance (Bullock et al., 2002; Hake, 1998)
• Clicker technology has been changing how students and their instructor interact within the classroom by providing new opportunities to enhance in-class participation and learning in traditional, lecture hall classroom environments (Blasco-Arcas, et al., 2013).
Purpose
• Less is known about how clicker technology improves in-class participation and learning in more clinically content based courses
• The purpose of this program assessment was to examine how learning and in-class participation is effected by the use of clicker technology within a clinically based occupational therapy course
Research Design
• Nonequivalent pretest posttest control group design
Study Sample and Group Assignment
• Study sample
• 32 first year, first semester occupational therapy students
• 15 control group
• 17 experimental group
• All students enrolled into Occupational Therapy Practice Skills course; 2 sections (Monday and Thursday)
• Group assignment:
• Monday= Experimental group
• Thursday= Control group
Measurements
Pretest and Posttest ordinal data
• Student Assessment of Learning Goals (SALG)
• Self Consciousness Scale revised (SCSR)
Posttest only ordinal data
• Perceived Importance of Clicker Technology (PICT)
• Only given to experimental group at end of semester
Student Assessment of Learning Goals
• SALG instrument focuses exclusively on the degree to which a course has enabled student learning
• SALG asks students to assess and report on their own learning, and on the degree to which specific aspects of the course have contributed to that learning
• Instructor can customize questions
• The following 32 questions have been used in this program assessment: Understanding (6), Participation (5), Proficiency (8), Feelings about learning (4), Willingness to seek help (6), and Habits (3)
• Students rate questions as:
• “strongly disagree” (-2)
• “disagree” (-1)
• “agree” (1)
• “strongly agree” (2)
Self Consciousness Scale Revised
• SCSR is a 23- item questionnaire developed for the general public, which measures individual private self-consciousness, public self-consciousness, and social anxiety
• Students rate questions as:
• 3 = a lot like me
• 2= somewhat like me
• 1= a little like me
• 0= not like me at all
Perceived Importance of Clicker Technology
• Utilized SALG and reformatted all 32 questions to ask about how clickers have enhanced their understanding, participation, proficiency, feelings about learning, willingness to seek help, and habits of the content and skills learned throughout the course to create the PICT
• For example: • Under the SALG, a question would read as:
• Presently, I am… Able to demonstrate proficiency in exercising correct infection control techniques
• Under PICT, the same question would read as:
• Since using the clickers over the past few weeks, the use of clickers increased my… Proficiency in exercising correct infection control techniques
• Students rate questions as:
• “strongly disagree” (-2) • “disagree” (-1) • “agree” (1) • “strongly agree” (2)
Application of Independent Variable
• IV= use of clicker technology during class over 5 weeks during Fall 2013 semester to obtain polling of questions related to: • Assigned readings
• Content of lecture
• Past experiences related to weekly topic
• Confidence in performing lab skills
• Feelings about the class
• Classes included: • Transfer training
• Infection control
• Vital signs
• ADL Topics: Dressing, Toileting, and Bowel & Bladder
• ADL Topics: Eating & Feeding, Personal Hygiene, Bathing, and Sexuality
Which is the most common mode of infectious disease contamination?
1 2 3 4
58%
5%
32%
5%
For example
1. Contact
2. Droplet
3. Airborne
4. Common vehicle
Statistical Analyses
• All pretest and posttest questionnaires scored twice for accuracy
• Independent samples t-test used to distinguish differences in outcome scores at baseline
• Independent samples t-test used to distinguish differences in outcome scores from posttest to pretest (change scores)
Dependent Variables
Dependent Variable
Learning Gains Total Learning Gains in Understanding Learning Gains in Participation Learning Gains in Proficiency Learning Gains in Feelings Towards Learning Learning Gains in Willingness to Seek Help Learning in Habits Self-consciousness Total Private self-consciousness Public self-consciousness Social Anxiety Total Perceived importance of clicker technology *
DVs have pretest, posttest, and change score values, except for Total PICT*, as it was given only to experimental group at posttest
Descriptive Comparison of Groups at Baseline
Learning
Gains
Total
Learning
Gains in
Understan
ding
Learning
Gains in
Participation
Learning
Gains in
Proficiency
Learning
Gains in
Feelings
Learning
Gains in
Willingness
Learning
Gains in
Habits
Self-
conscious
ness Total
Private
Self-
conscious
ness
Public Self-
conscious
ness
Social
Anxiety
Mean 31.1333 10.1333 6.8667 -6.0000 6.0667 10.4667 3.6000 39.0667 19.2667 13.3333 6.4667
N 15 15 15 15 15 15 15 15 15 15 15
Std.
Deviation
8.45042 1.24595 1.50555 4.86973 2.25093 1.95911 1.29835 7.48777 1.98086 3.22195 4.98378
Mean 30.2353 9.9412 6.9412 -6.8824 6.5882 9.8824 3.7647 39.1176 18.2941 12.7647 8.0588
N 17 17 17 17 17 17 17 17 17 17 17
Std.
Deviation
9.25695 1.67595 1.81902 4.99853 1.62245 2.28808 1.75105 6.33327 3.53137 2.79574 3.94447
Mean 30.6563 10.0313 6.9063 -6.4688 6.3438 10.1563 3.6875 39.0938 18.7500 13.0313 7.3125
N 32 32 32 32 32 32 32 32 32 32 32
Std.
Deviation
8.75697 1.46979 1.65314 4.87908 1.92788 2.12678 1.53323 6.78404 2.90717 2.96740 4.46085
Descriptive Statistics of Control and Experimental Group
type of group
control group
experimental
group
Total
Inferential Comparison of Groups at Baseline
. Lower Upper
Equal variances
assumed
1.231 .276 .285 30 .777 .89804 3.14913 -5.53335 7.32943
Equal variances
not assumed
.287 29.956 .776 .89804 3.13070 -5.49610 7.29218
Equal variances
assumed
1.830 .186 .364 30 .719 .19216 .52811 -.88639 1.27070
Equal variances
not assumed
.371 29.220 .714 .19216 .51838 -.86770 1.25202
Equal variances
assumed
1.190 .284 -.125 30 .901 -.07451 .59514 -1.28995 1.14093
Equal variances
not assumed
-.127 29.895 .900 -.07451 .58800 -1.27555 1.12653
Equal variances
assumed
.002 .966 .504 30 .618 .88235 1.74956 -2.69073 4.45544
Equal variances
not assumed
.505 29.683 .617 .88235 1.74662 -2.68632 4.45102
Equal variances
assumed
1.235 .275 -.758 30 .454 -.52157 .68767 -1.92598 .88285
Equal variances
not assumed
-.743 25.153 .464 -.52157 .70187 -1.96665 .92352
Equal variances
assumed
1.104 .302 .770 30 .447 .58431 .75839 -.96453 2.13315
Equal variances
not assumed
.778 29.980 .443 .58431 .75089 -.94925 2.11787
Independent Samples Test
Levene's Test for
Equality of Variances t-test for Equality of Means
F Sig. t df
Sig. (2-
tailed)
Mean
Difference
Std. Error
Difference
95% Confidence
Interval of the
Learning Gains
Total
Learning Gains
in
Understanding
Learning Gains
in Participation
Learning Gains
in Proficiency
Learning Gains
in Feelings
Learning Gains
in Willingness
Inferential Comparison of Groups at Baseline
Equal variances
assumed
.393 .535 -.299 30 .767 -.16471 .55130 -1.29061 .96120
Equal variances
not assumed
-.304 29.196 .763 -.16471 .54106 -1.27097 .94156
Equal variances
assumed
.442 .511 -.021 30 .983 -.05098 2.44293 -5.04010 4.93814
Equal variances
not assumed
-.021 27.622 .984 -.05098 2.46925 -5.11213 5.01017
Equal variances
assumed
4.916 .034 .943 30 .353 .97255 1.03171 -1.13448 3.07958
Equal variances
not assumed
.975 25.709 .339 .97255 .99757 -1.07912 3.02422
Equal variances
assumed
1.344 .255 .535 30 .597 .56863 1.06351 -1.60335 2.74060
Equal variances
not assumed
.530 27.977 .600 .56863 1.07324 -1.62988 2.76714
Equal variances
assumed
1.468 .235 -1.008 30 .322 -1.59216 1.57984 -4.81862 1.63431
Equal variances
not assumed
-.993 26.634 .330 -1.59216 1.60346 -4.88431 1.70000
Learning Gains
in Habits
Self-
consciousness
Total
Private Self-
consciousness
Public Self-
consciousness
Social Anxiety
Discussion of Groups at Baseline
• A statistically significant difference was found between control group and experimental group at baseline for one dependent variable
• Private self-consciousness (p=0.034)
Inferential Comparison of Groups from Posttest - Pretest
.
Inferential Comparison of Groups from Posttest - Pretest Equal
variances
assumed
3.164 .085 1.316 30 .198 .86667 .65872 -.47863 2.21196
Equal
variances
not
assumed
1.355 26.968 .187 .86667 .63977 -.44610 2.17944
Equal
variances
assumed
.680 .416 -1.070 30 .293 -2.21569 2.07075 -6.44472 2.01334
Equal
variances
not
assumed
-1.060 27.860 .298 -2.21569 2.09082 -6.49950 2.06813
Equal
variances
assumed
1.273 .268 .429 30 .671 .44314 1.03361 -1.66778 2.55406
Equal
variances
not
assumed
.420 25.329 .678 .44314 1.05428 -1.72675 2.61303
Equal
variances
assumed
1.398 .246 -1.325 30 .195 -.92549 .69861 -2.35223 .50125
Equal
variances
not
assumed
-1.306 26.705 .203 -.92549 .70885 -2.38069 .52971
Equal
variances
assumed
.005 .944 -1.520 30 .139 -1.73333 1.14045 -4.06245 .59579
Equal
variances
not
assumed
-1.492 25.607 .148 -1.73333 1.16206 -4.12377 .65710
SCSRprivateSC
change
Public Self-
consciousness
Social Anxiety
Learning Gains
in Habits
Self-
consciousness
Total
Discussion of DV Group Comparisons
• A marginally statistically significant difference was found between control group and experimental group in change scores for one dependent variable
• Learning gains in habits (p=0.085)
Presently, I am in the habit of…
Connecting key ideas I learn in class with
other topics covered in this class
Strongly
Disagree
Disagree Agree Strongly
Agree
Connecting key skills I learn in class with
other skills covered in this class
Strongly
Disagree
Disagree Agree Strongly
Agree
Connecting key ideas I learn in class with
other classes in the curriculum
Strongly
Disagree
Disagree Agree Strongly
Agree
Descriptive Statistics for Perceived Importance of Clicker Technology
17 participants in experimental group scored the PICT at the end of the semester
32 questions were scored on an ordinal scale of
• “strongly disagree” (-2)
• “disagree” (-1)
• “agree” (1)
• “strongly agree” (2)
Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
Perceived Importance of Clicker Technology Total Score
17 -14.00 48.00 28.4118 17.12841
Valid N (listwise) 17
Discussion of Perceived Importance of Clicker Technology
• A mean score of 28.41 was obtained from this DV
• A mean score of 31 would suggest average agreement (1=agree) on items, conversely, a mean score of -31 would suggest average disagreement (-1=disagree)
Limitations
• Effect of IV was 5 weeks, a longer duration may have yielded different results
• Some classes were out of classroom, such as the community where clickers could not be used
• Participants themselves were not randomized into groups but class section instead
• Small sample size
Conclusion
• Clicker technology was trialed in a clinical (lab) occupational therapy course
• Although statistical significance (p≤0.05) was not obtained for DVs between control and experimental groups, the experimental group rated clicker technology has having agreeable importance
• Clicker technology may be better suited for lecture style instruction compared to laboratory style instruction
• Qualitative inquiry may be useful
• Utilization of clickers was reliable due to radio frequency
• Students using clickers typically display enthusiasm during in-class polling
References
Blasco-Arcas, L., Buil, I., Hernández-Ortega, B., & Sese, F. J. (2013). Using clickers in
class: The role of interactivity, active collaborative learning and engagement in
learning performance. Computers and Education, 62, 102-110.
Bullock, D.W., LaBella, V. P., Clingan, T., Ding, Z., Stewart, G., & Thibado, P. M. (2002).
Enhancing the student-instructor interaction frequency. Physics Teacher, 40, 535–541.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-
thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64–74.