Presented by Jennifer Fager For

University of Wisconsin-SuperiorEnhancement Day

1/19/2011

Advice for Assessing Student Learning in an Online EnvironmentMultiple Choice and Short Answer Questions

Not always the best technique in either online or ground formats

Lend themselves to cheating more so in onlineQuestions, regardless of ground or online,

must be written wellThe “test” has been overused and relied upon

unless the instructor is an expert in writing good test items

Advice continuedUse multiple forms of assessment

Has corroborating evidence of student achievement

More personalized assessments reduce the likelihood of plagiarism “Term paper” vs. application paper using personal

examples General research paper vs. select topics focus

Use examples more “authentic” in formatConsider problems of the fieldEngage in “conversation’s” online

Advice continuedSecurity options

Lockdown browser Student can’t go to another website during the test NOTE: Most students now have multiple devices so

they can go to their IPhones, blackberries, laptops, IPads, etc. to research the “answers”

Advice ContinuedSet test time limits

Even if the student gets help, time restricts the amount

Can punish the slower readers/processorsWhat do you want in terms of meaningful,

thoughtful responses

Advice continuedAssume any online test is, by definition, open

bookAnd perhaps open notes, and open neighbors

Don’t try to make it otherwise

Advice continuedQuiz/test software products are available

Can generate multiple versions of testsEach student can receive a “different” test

http://www.fau.edu/irm/instructional/respondus.php

Have students explain their answers25% have to explain ‘A’ answers, 25% ‘B’

answers, etc.

Advise continuedOther security ideas

“thumb” identificationWebcamsFace-to-face testing at a testing center

Online Assessment ResourcesCommercial Assessment Data Management Tools –

E.g. TracDat, WeaveOnline, Chalk & WireCurriculum Design Tools

E.g. WIDS – link outcomes to Performance Assessment Tasks

Survey InstrumentsSurvey MonkeyZoomerang

PortfoliosCourse Management Systems Quiz and Analysis

FunctionsBlackboard and others

Resources ContinuedGoogle DocsJingReflective Instruments

LeadershipPersonality StyleTeamwork

Merlot (has content template builder)Clickers AdobeConnect with polling featuresLock-down Browsers for online testing

Fundamental QuestionsWhat evidence do you have that students

achieve your stated learning outcomes?How have you articulated these outcomes to

students and adjunct faculty?In what ways do you analyze and use

evidence of student learning?

Assessment ProcessThe same process is used for online programs

and courses as for ground (traditional):Needs Assessment

Needs of the profession Needs of the learners

Articulate your expectationsMeasure achievement of expectationsCollect and analyze dataUse evidence to improve learningAssess effectiveness of improvement

Measure Online LearningDirect Methods

Mostly the same as groundCan use any student work products that can be

saved in an electronic formSome challenges exist with lab work, oral

performances, etc.Indirect Methods

Again, same as groundSurvey response rates are notoriously low

Advantages of Online AssessmentArtifact collection can be automated through

the course management softwareCan work for ground as well

Assignments, rubrics can be standardized and mapped to specific competencies/outcomesAgain, can work for ground as well

New Opportunities with OnlineAssess quality of discussions, group work

Particularly effective with asynchronousAssess individual learning in more depth

Student advising can be more robustUse of standardized rubrics can result in

greater consistency in gradingParticularly effective when combined with

periodic instructor norming and use of anchor samples

Discussion & Group WorkFaculty can isolate and organize

contributions to a threaded discussion and give feedback on how to improveCan archive individual contributions and group

discussionsFormative and summativeCourse, program, and institutional levels

Create a record of instructor effectivenessUseful for annual review, tenure

Individual Student LearningIndividual learning/achievements can be

tracked across the educational experienceE-format allows for wide dissemination of

collected dataCurrent software packages provide various

levels of data collection & dissemination

RubricsClear articulation of expectations that are

linked to specific course & program outcomesA means through which correspondence of

online & ground rigor can be ensured (alignment)

Increases consistency in grading across sections, courses, programs, even colleges

Holistic vs. Analytic

Example of a Successful Online Assessment StrategyDevelop course/program maps which

articulate learning outcomes and link them to specific courses and course assignments

Collect and archive artifacts through the course management software

Post rubrics through course management software for student and faculty use

Ensure Validity, ReliabilityTracking inter-rater reliability helps achieve

consistency in gradingPost rubrics, anchor samples for 24/7 faculty

useFull-time/core faculty should formulate

competencies, achievement standards & interpret results

Sample AssessmentsReview the Video Gaming Assignment

included at the end of this packet.Determine how this assignment could be

adapted for online learning

General Education Outcomes ExampleThe ability and inclination to think and make

connections across academic disciplinesHow would you assess this outcome in

ground or online courses?

Websites and Resourceshttp://fod.msu.edu/oir/TeachWithTech/online-

courses.asphttp://www.uwstout.edu/soe/profdev/assess.cf

mhttp://www.fgcu.edu/onlinedesign/index.htmlhttp://www.pvc.maricopa.edu/~lsche/resourc

es/onlineteaching.htm

http://sloanconsortium.org/

Questions and ConcernsWhat issues have you encountered when

assessing online learning?What are your concerns regarding the use of

online assessment techniques?What policies and/or procedures need to be

in place to reduce or alleviate these concerns?

Questions Still Burning?What else do you need to know?

The Bottom Line?Assessment in online learning can be the

same as assessment on ground—artifacts and tools can be the same even if the medium is different!