assessment using o nline a ssignment m anagement systems
DESCRIPTION
Assessment Using O nline A ssignment M anagement Systems. Glenn Ledder Department of Mathematics University of Nebraska-Lincoln [email protected]. Assessment Using O nline A ssignment M anagement Systems. Defining the terms OAM advantages and disadvantages - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/1.jpg)
Glenn LedderDepartment of MathematicsUniversity of [email protected]
Assessment Using Online Assignment Management
Systems
![Page 2: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/2.jpg)
1. Defining the terms
2. OAM advantages and disadvantages
3. Low-maintenance administration
4. Accurate and reproducible results
5. Getting meaningful results
Assessment Using Online Assignment Management
Systems
![Page 3: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/3.jpg)
Defining the Terms
![Page 4: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/4.jpg)
“Assessment”
• Conflicting meanings
![Page 5: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/5.jpg)
“Assessment”
• Conflicting meanings
– General meaning: measuring academic success in any context
![Page 6: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/6.jpg)
“Assessment”
• Conflicting meanings
– General meaning: measuring academic success in any context
– Specific meaning: measuring academic success in a broad context
![Page 7: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/7.jpg)
“Assessment”
• Conflicting meanings
– General meaning: measuring academic success in any context
– Specific meaning: measuring academic success in a broad context
(larger than a single course)
![Page 8: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/8.jpg)
• Dictionary definition:– “assess”: to determine the importance, size,
or value of– “evaluate”: to determine or fix the value of
Assessment vs Evaluation
![Page 9: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/9.jpg)
• Dictionary definition:– “assess”: to determine the importance, size,
or value of– “evaluate”: to determine or fix the value of
• Connotation:– “assessment” usually emphasizes the making
of summary judgments– “evaluation” usually emphasizes the
measurement of quantitative value
Assessment vs Evaluation
![Page 10: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/10.jpg)
Assessment vs Evaluation
• Assessment:– measuring knowledge and understanding
• Evaluation:– measuring achievement
![Page 11: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/11.jpg)
Assessment vs Evaluation
• Assessment:– measuring knowledge and understanding
– used for placement
• Evaluation:– measuring achievement
– used to determine course grades
![Page 12: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/12.jpg)
Assessment vs Evaluation
• Assessment:– measuring knowledge and understanding– used for placement
• Evaluation:– measuring achievement– used to determine course grades
• Test: an instrument used for assessment, evaluation, or both
![Page 13: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/13.jpg)
OAM Advantages and
Disadvantages
![Page 14: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/14.jpg)
OAM Advantages/Disadvantages
+ Immediate grading
(no wait for grading or machine scoring)
![Page 15: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/15.jpg)
OAM Advantages/Disadvantages
+ Immediate grading
+ General purpose testing center
(no need for direct faculty involvement)
![Page 16: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/16.jpg)
OAM Advantages/Disadvantages
+ Immediate grading
+ General purpose testing center
– Administrative difficulties
(How do you maintain the database?)
![Page 17: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/17.jpg)
OAM Advantages/Disadvantages
+ Immediate grading
+ General purpose testing center
– Administrative difficulties
+ Algorithmic Q’s and symbolic A’s
(individual answers, no prompts)
![Page 18: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/18.jpg)
OAM Advantages/Disadvantages
+ Immediate grading
+ General purpose testing center
– Administrative difficulties
+ Algorithmic Q’s and symbolic A’s
– Difficulties in writing questions
(How to avoid tricky, picky, sticky?)
![Page 19: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/19.jpg)
Low-Maintenance Administration
![Page 20: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/20.jpg)
Low-Maintenance Administration
• Lots of instructors and proctors– Should require minimal knowledge
![Page 21: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/21.jpg)
Low-Maintenance Administration
• Lots of instructors and proctors– Should require minimal knowledge
• One or two faculty manage the system– Should require minimal time
![Page 22: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/22.jpg)
Low-Maintenance Administration
• Lots of instructors and proctors– Should require minimal knowledge
• One or two faculty manage the system– Should require minimal time
• Questions are “permanent.”
• Student records are “transient.”
![Page 23: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/23.jpg)
106 COURSE
•Question banks•Gateway exam•Practice assignments
The 106 COURSE data is “permanent.” The only regular changes are to the Gateway exam dates.Maintained by faculty manager.
The Math 106 EDU folder structure
![Page 24: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/24.jpg)
The Math 106 EDU folder structure
106 COURSE
•Question banks•Gateway exam•Practice assignments
106 CLASS•Question banks•Gateway exam•Practice assignments
•Student records
The 106 COURSE data is “permanent.” The only regular changes are to the Gateway exam dates.Maintained by faculty manager.
Students register for their own 106 CLASS. Tests are inherited. Student records are local.Maintained by individual instructors/proctors.
![Page 25: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/25.jpg)
Accurate and Reproducible Results
![Page 26: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/26.jpg)
Reproducibility
• Tests should measure something.
– The test has to yield consistent results in multiple administrations for a single student.
![Page 27: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/27.jpg)
Reproducibility
• Tests should measure something.
– The test has to yield consistent results in multiple administrations for a single student.
• But questions must change when a test is given at different times.
– This is especially important for evaluations.
![Page 28: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/28.jpg)
Reproducibility
• Use template problems to get a great variety of answers.
• Use template problems to get uniformity of content and difficulty.
• Group problems into categories that are consistent in content and difficulty.
![Page 29: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/29.jpg)
The Math 106 Gateway Exam
1. Elementary functions: xn, sin(ax), cos(ax), tan(ax), eax, ln x, nx
2. Products 3. Quotients 4. Compositions
5. Compositions of compositions
6. Products with a composite factor
7. Compositions of products
8. Quotients with an embedded composition
9. Quotients with an embedded product
10. Functions defined by equations
10 questions, 8 correct to pass
![Page 30: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/30.jpg)
Category 4 - Compositions
A, C, N > 0; B ≠ 0; K ≠ 0, 1;
The algorithm chooses 5 random integers satisfying certain requirements.
![Page 31: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/31.jpg)
Category 4 - Compositions
A, C, N > 0; B ≠ 0; K ≠ 0, 1;
X = t, u, v, w, x, y, z;
The algorithm randomly chooses an independent variable.
![Page 32: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/32.jpg)
Category 4 - Compositions
X = t, u, v, w, x, y, z; A,C,N>0; B≠0; K≠0,1
P = XN+B, XN+BX
Q = AXN+B, AXN+BX, sqrt(X)+B
S = sin AX, cos AX, tan AX
T = e -CX+B, eKX+BX
U = Ae -CX+B, AeKX+BX, A ln X, ANX
The algorithm randomly creates functions P, Q, S, T, U, using some of the chosen integers.
![Page 33: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/33.jpg)
Category 4 - Compositions
X = t, u, v, w, x, y, z; A,C,N>0; B≠0; K≠0,1
P = XN+B, XN+BXQ = AXN+B, AXN+BX, sqrt(X)+BS = sin AX, cos AX, tan AXT = e
-CX+B, eKX+BXU = Ae
-CX+B, AeKX+BX, A ln X, ANX
F1 = sqrt(P), sqrt(S), sqrt(T)
F2 = SN, TN
F3 = ln Q, ln CS
F4 = eQ, eCS
F5 = sin Q, cos Q, sin U, cos U
The algorithm chooses one of 5 functions based on prior elements.There are 38 templates, each with 7 independent variables and at least one parameter
![Page 34: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/34.jpg)
Reproducibility
• Tests should measure something.
– The test has to yield consistent results in multiple administrations for a single student.
![Page 35: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/35.jpg)
Accuracy
• Tests should measure what they were intended to measure.
![Page 36: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/36.jpg)
Accuracy
• Tests should measure what they were intended to measure.
– Questions should have to be done right to be counted right.
![Page 37: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/37.jpg)
Accuracy
• Tests should measure what they were intended to measure.
– Questions should have to be done right to be counted right.
– Students who can do what the test is supposed to measure should get a good score.
![Page 38: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/38.jpg)
Counted right → Done right
• Savvy test takers can do well on multiple choice tests even when they don’t understand the material.
![Page 39: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/39.jpg)
Counted right → Done right
• Savvy test takers can do well on multiple choice tests even when they don’t understand the material.
• For calculations, students should have to type in the answer.
![Page 40: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/40.jpg)
Counted right → Done right
• Savvy test takers can do well on multiple choice tests even when they don’t understand the material.
• For calculations, students should have to type in the answer.
• For conceptual questions, use multiple selection. (Which of the following…)
![Page 41: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/41.jpg)
Can do → Good score
• Tricky: Misinterpretation or small mistakes account for many wrong answers.
![Page 42: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/42.jpg)
Can do → Good score
• Tricky: Misinterpretation or small mistakes account for many wrong answers.
Find the derivative of cos e2x
and
Find the derivative of e-2 cos x
are tricky
![Page 43: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/43.jpg)
Can do → Good score
• Tricky: Misinterpretation or small mistakes account for many wrong answers.
Find the derivative of cos e2x
and
Find the derivative of e-2 cos x
are tricky, compared to
Find the derivative of e2 cos x
![Page 44: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/44.jpg)
Can do → Good score
• Tricky: Misinterpretation or small mistakes account for many wrong answers.
Find the derivative of cos e2x
and
Find the derivative of e-2 cos x
are tricky, compared to
Find the derivative of e2 cos x
The problem is meant to test the chain rule, not elementary algebra.
![Page 45: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/45.jpg)
Can do → Good score
• Picky: Too many details connect minor errors with wrong answers.
Find the derivative of cos2(2x+3)+ 4 sin x.
Find the derivative of 4x5-2x cos (ex2).
![Page 46: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/46.jpg)
Can do → Good score
• Picky: Too many details connect minor errors with wrong answers.
Find the derivative of cos2(2x+3)+ 4 sin x.
Find the derivative of cos2(2x+3).
Find the derivative of 4x5-2x cos (ex2).
Find the derivative of ex5 cos (ex2).
![Page 47: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/47.jpg)
Can do → Good score
• Picky: Too many details connect minor errors with wrong answers.
Find the derivative of cos2(2x+3)+ 4 sin x.
Find the derivative of cos2(2x+3).
Find the derivative of 4x5-2x cos (ex2).
Find the derivative of ex5 cos (ex2).
The simpler problems are hard enough.
![Page 48: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/48.jpg)
Can do → Good score
• Sticky: Difficulties in transferring answer from paper to computer.
Find the derivative of —–. ————2x2
x+34x(x+3)-2x2
(x+3)2
![Page 49: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/49.jpg)
Can do → Good score
• Sticky: Difficulties in transferring answer from paper to computer.
Find the derivative of —–. ————
Find the derivative of —– at x = 2. —
2x2
x+34x(x+3)-2x2
(x+3)2
2x2
x+33225
![Page 50: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/50.jpg)
Can do → Good score
• Sticky: Difficulties in transferring answer from paper to computer.
Find the derivative of —–. ————
Find the derivative of —– at x = 2. —
2x2
x+34x(x+3)-2x2
(x+3)2
2x2
x+33225
Type 4x(x+3)-2x2 / (x+3)2, get 2x2
(x+3)24x(x+3)- ———
![Page 51: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/51.jpg)
Getting Meaningful Results
![Page 52: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/52.jpg)
Accuracy
• Tests should measure what they were intended to measure.
![Page 53: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/53.jpg)
Getting Meaningful Results
• Tests should measure what they were intended to measure.
• What should OAM evaluations and assessments be intended to measure?
![Page 54: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/54.jpg)
Don’t give up paper evaluations
![Page 55: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/55.jpg)
Don’t give up paper evaluations
• Use paper exams for questions that demand partial credit and questions where the answer is an integral, a graph, or an explanation.
![Page 56: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/56.jpg)
Don’t give up paper evaluations
• Use paper exams for questions that demand partial credit and questions where the answer is an integral, a graph, or an explanation.
• Use OAM evaluation for routine computations and basic concepts.
![Page 57: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/57.jpg)
Don’t give up paper evaluations
• Use paper exams for questions that demand partial credit and questions where the answer is an integral, a graph, or an explanation.
• Use OAM evaluation for routine computations and basic concepts.
• Testing takes time and effort. Decide what is important and then create the test to match.
• Set high standards and allow retakes.
![Page 58: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/58.jpg)
Do give up paper assessments
![Page 59: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/59.jpg)
Do give up paper assessments
• Assess routine computations and basic concepts.
![Page 60: Assessment Using O nline A ssignment M anagement Systems](https://reader036.vdocuments.site/reader036/viewer/2022081519/56812d77550346895d928c95/html5/thumbnails/60.jpg)
Do give up paper assessments
• Assess routine computations and basic concepts.
• Expect very few retakes. Qualified students should pass on the first try.
• Provide necessary formulas online or as a printed supplement. Test conceptual understanding and knowledge, not memory for trivia.