yearly assessment report – mathematics 2015-16 · pdf file ·...

32
page 1 Yearly Assessment Report – Mathematics 2015-16 The following prompt guides discussion regarding the history of the outcome and learning being assessed, evidence of improved learning, evidence of improved teaching/pedagogy, and evidence of improved assessment. Please address each heading separately with relevant examples. I. Student Learning Assessed Identify the student learning outcome(s) being assessed along with a description or definition of that learning outcome suitable for an outside audience. 1. Students will be able to construct and evaluate mathematical proofs. Description: Students will be able to write mathematical proofs and determine if mathematical proofs are correct. A mathematical proof is a deductive argument consisting of a formal series of statements showing that if one thing is true then something else necessarily follows from it. 2. Students will be able to solve complex, real-world problems using appropriate technologies. Description: A mathematical problem is a problem that is amenable to being represented, analyzed, and possibly solved, with the methods of mathematics. 3. Students will demonstrate an understanding of undergraduate Mathematics including Calculus, Linear and Modern Algebra. Description: Calculus is a branch of mathematics that deals mostly with rates of change and with finding lengths, areas, and volumes. Linear and Modern Algebra are branches of mathematics that are concerned with mathematical structures closed under the operations of addition and scalar multiplication. II. Changes to the Outcome(s), Program, and Assessment Identify changes made to the outcome(s), program, and/or the assessment of the outcome(s) since the last time it was assessed. Please note any changes made based on the results of the previous assessment. The programmatic outcomes of the math majors were re-written for the 2015- 16 academic year. This is the first time these re-written outcomes have been assessed.

Upload: doxuyen

Post on 19-Mar-2018

215 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

page 1

Yearly Assessment Report – Mathematics 2015-16 The following prompt guides discussion regarding the history of the outcome and learning being assessed, evidence of improved learning, evidence of improved teaching/pedagogy, and evidence of improved assessment. Please address each heading separately with relevant examples. I. Student Learning Assessed Identify the student learning outcome(s) being assessed along with a description or definition of that learning outcome suitable for an outside audience.

1. Students will be able to construct and evaluate mathematical proofs.

Description: Students will be able to write mathematical proofs and determine if mathematical proofs are correct. A mathematical proof is a deductive argument consisting of a formal series of statements showing that if one thing is true then something else necessarily follows from it.

2. Students will be able to solve complex, real-world problems using appropriate technologies. Description: A mathematical problem is a problem that is amenable to being represented, analyzed, and possibly solved, with the methods of mathematics.

3. Students will demonstrate an understanding of undergraduate Mathematics including Calculus, Linear and Modern Algebra. Description: Calculus is a branch of mathematics that deals mostly with rates of change and with finding lengths, areas, and volumes. Linear and Modern Algebra are branches of mathematics that are concerned with mathematical structures closed under the operations of addition and scalar multiplication.

II. Changes to the Outcome(s), Program, and Assessment Identify changes made to the outcome(s), program, and/or the assessment of the outcome(s) since the last time it was assessed. Please note any changes made based on the results of the previous assessment. The programmatic outcomes of the math majors were re-written for the 2015- 16 academic year. This is the first time these re-written outcomes have been assessed.

Page 2: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

page 2

III. Artifacts Used in Assessment Describe the artifacts and methods of assessing the learning and how they illustrate that knowledge set or skill. If multiple artifacts or methods are used, please describe each. Attach any assessment tools, such as rubrics and assignment sheets, used in the process. Provide the number of artifacts collected since the last time this outcome(s) was assessed, a breakdown of the amount used in the assessment process, and why those were used (e.g. 20 artifacts collected per year with seven randomly selected artifacts from each year used in the study).

OUTCOME

ARTIFACT FOR ASSESSMENT

NUMBER OF ARTIFACTS COLLECTED AND USED

#1 Final exam problems from Math 210 Math Skills Inventory from Math 400

19 9

#2 Calculus Mastery Exam from Math 307 Math Skills Inventory from Math 400

13 9

#3 Calculus Mastery Exam from Math 307 Math Skills Inventory from 400

13 9

The Calculus Mastery Exam is a test consisting of 15 calculus problems, 9 basic and 6 advanced. It is administered to all math majors taking MATH 307, which is required to be taken by all math majors. The Math Skills Inventory is a test consisting of 8 problems in a variety of areas of undergraduate mathematics. It is administered to all math majors taking MATH 400, which is required to be taken by all math majors. The final exam problems analyzed from Math 210 require students to evaluate and write mathematical proofs. The problems analyzed are problems 6, 7, 8, 9, and 10. The results on these problems are collected from all math majors taking MATH 210, which is required to be taken by all math majors.

In each case, learning is assessed and mastery of each knowledge set or skill is measured by how well the students performed on these tests and problems. Accompanying this document are the final exam problems from Math 210, the Calculus Mastery Exam, and the Math Skills Inventory.

Page 3: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

page 3

IV. Results Provide the results of the assessment.

Calculus Mastery Exam Results 2015-16 Average Score Basic Problems 75.21% Advanced Problems 34.62% Overall 58.97%

The average score on the Math Skills Inventory in 2015-16 was 66%. The average score on problems 6-10 on the MATH 210 final exam from Fall 2015 was 61.79%.

V. Conclusions Provide conclusions regarding student learning based on those results as well as how those artifacts led to those conclusions. Also, provide conclusions about the assessment process, particularly when using multiple artifacts and/or methods.

Students’ abilities to write proofs and solve advanced mathematical problems are satisfactory. Performance on the Math Skills Inventory and problems 6-10 on the MATH 210 final exam led to this conclusion since our goal was to have students get 60% of these problems correct.

Student’s abilities to solve calculus problems are abysmal. We expected students to be able to do 95% of the basic questions correctly and 60% of the advanced questions correctly.

We believe our assessment instruments do an adequate job of measuring what they were designed to measure.

VI. Assessment Use Provide the steps the program intends to take in regard to 1) the program and/or outcome itself, 2) improved student learning, and 3) the assessment process based on those conclusions.

Regarding the assessment process, the program is developing a Basic Proof Skills Test designed to measure how well math majors are learning and retaining the ability to write proofs and reason deductively. This tool will be administered to majors at the beginning of MATH 210, at the end of MATH 210, and in MATH 400. A

Page 4: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

page 4

preliminary version of this test was piloted in Fall 2015, and the test and a report can be found in the accompanying documents. The results of this pilot were not included in the results section of this assessment report because the Basic Proof Skills Test is still in the developmental stage. We do anticipate it being included in the results section in future years.

Regarding student learning, the program has a three-pronged plan to improve learning in the area of calculus. First, take steps to recruit better students as math majors. Second, study and improve the efficacy of the program’s teaching of calculus. Last, consider if the expectations of our students’ abilities to do calculus are too ambitious.

Page 5: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Skills Inventory of Mathematics MajorsMorningside College

Fall 2015

Please answer the following questions to the best of your ability. Show all work, and mentionwhen technology is used. All work and solutions should be written on a separate sheet ofpaper. Turn in this exam, along with your solutions, when you are done.

1. Give an example of a subset of the real numbers that is

(a) countable and infinite.

(b) uncountable.

2. Show that the sets A = N and B = Z have the same cardinality (have the samenumber of elements in them). Do this by exhibiting a one-to-one, onto function fromA to B.

3. Find the coordinates of the point of intersection of the functions f(x) = ex andg(x) = 2 + sinx, accurate to three decimal places.

4. Evaluate375∑n=1

1

n, accurate to five decimal places.

5. Suppose you are running a machine that both fills and drains a liquid from a container.You know that after 1 hour there are 12 L in the container, after 2 hours there are15 L, and after 3 hours there are 16 L. Find a quadratic polynomial that models thisbehavior. That is, find coefficients a0, a1, and a2 such that

a0 + a1(1) + a2(1)2 = 12a0 + a1(2) + a2(2)2 = 15a0 + a1(3) + a2(3)2 = 16.

Page 6: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

6. A rectangle has length x and width 20 − x.

(a) Find a formula for the area of this rectangle, as a function of x.

(b) What domain makes sense for this function?

(c) Of all such rectangles having dimensions x and 20 − x, what are the specificdimensions of the one that has the largest area?

7. Prove by induction that 1 + 2 + 3 + · · · + n = 12n(n + 1) for every natural number n.

8. On Day 1, you are given $1. On Day 2, you are given $2. On Day 3, you are given$4. On Day 4, you are given $8. On each day that follows, you are given twice asmuch as on the previous day. The table below summarizes the situation.

Day Amount You Are Given Total Amount You Have

1 $1 $12 $2 $33 $4 $74 $8 $155 $16 $31...

......

30 ? ?...

......

n ? ?

(a) How much money are you given on Day 6? Day 7? Day 8?

(b) What is the total amount of money that you have on Day 6? Day 7? Day 8?

(c) Find a formula for how much money you are given on Day n.

(d) Find a formula for the total amount of money you have on Day n.

(e) How much money are you given on Day 30? What is the total amount of moneyyou have on Day 30?

Page 7: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 1

NAME:

Please show all of your work on this paper. Guessed answers are NOT acceptable. Correct answerswithout work shown may receive no credit while incorrect answers with work shown may receivepartial credit. You may not use calculators. Good luck!

1. (12 pts) Let U = {a, b, c, ..., x, y, z} be the universal set and let A = {a, b, c, d, e, f, g, h, i} andB = {a, f, g, q, r}.

(a) Give an example of a partition, S, of A, such that |S| = 3.

(b) Give an example of a equivalence relation R on B that has 3 equivalence classes.

(c) Is there a function f : B → A that is surjective (onto)? Explain.

(d) Give an example of a function g : B → A that is injective.

(e) What is the cardinality of P(A)? What is the cardinality of A×B?

(f) Give an example of a set C such that C ⊂ A, but C * B

Page 8: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 2

2. (9 pts) Negate the following statements.

(a) Every real number is less than 100.

(b) If f is a polynomial function, then f(0) = 0.

(c) Every multiple of six is even and is not a multiple of four.

3. (12 pts) Consider the following statement: “If the square root of a positive integer is an eveninteger, then the integer itself is even”.

(a) Rewrite the statement in the form “for all ..., if ..., then ...” using symbols to representvariables.

(b) Write the negation of the statement, again using symbols.

(c) Prove or disprove the statement.

Page 9: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 3

4. (10 pts) Let A and B be sets contained in a universal set U . Which of the following statementsare equivalent to the statement A ⊆ B? No explanation required.

(a) ∀x ∈ U, x ∈ A and x ∈ B.

(b) ∀x ∈ U, if x /∈ B, then x /∈ A.

(c) ∃x ∈ U such that x ∈ A and x ∈ B.

(d) ∀x ∈ U, if x ∈ A, then x ∈ B.

(e) ∀x ∈ U, if x /∈ A, then x /∈ B.

5. (9 pts) For the function f : Z→ Z defined by

f(n) =

{n, if n is even;

n− 1, if n is odd

compute the inverse image of the following subsets of the codomain.

(a) W1 = E.

(b) W2 = {1}.

(c) W3 = {6}.

Page 10: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 4

6. (10 pts) Let A = {5k + 3|k ∈ Z} and B = {5k + 18|k ∈ Z}. Prove that A = B.

7. (10 pts) Choose one of the following proofs.

(a) Let f : A→ B, g : B → C and h : B → C be functions where f is a bijection. Prove thatif g ◦ f = h ◦ f , then g = h.

(b) Let A and B be sets and let X be a subset of A. Let f : A → B be a bijection. Provethat f(A−X) = B − f(X).

Page 11: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 5

8. (10 points) Choose one of the following proofs.

(a) Let A be the set of all people in the United States. Let R be the relation on A definedby: a is related to b if a and b were born on the same day of the week.

i. Prove that R is an equivalence relation on A.

ii. Describe the equivalence classes for R. How many are there?

(b) The < symbol defines a relation on Z.

i. Prove that the symbol < is transitive but not reflexive and not symmetric.

ii. A relation is antisymmetric if aRb and bRa, then a = b. Is < an antisymmetricrelation? Prove your answer.

Page 12: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 6

9. (10 points) Choose one of the following proofs.

(a) Use induction to prove that 1 + 3 + 5 + · · ·+ (2n− 1) = n2.

(b) Use induction to prove that 1 · 2 + 2 · 3 + 3 · 4 + · · · + n(n + 1) = n(n+1)(n+2)3

for everypositive integer n.

Page 13: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | Final Exam 7

10. (8 pts) Prove that the sequence{

nn−1

}converges to 1.

Page 14: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Preliminary Assessment Report: MATH 210Transition to Abstract Mathematics

Tessa Weinstein

June 28, 2016

1 Introduction

In my mind, proof and deductive reasoning are the primary activities that set mathe-matics apart from the other sciences. It is one of the cornerstones of any undergraduatemathematics program. As such, it is important that we develop the tools to asses howwell our students are learning and retaining this content. This document summarizes afirst effort at assessing the Mathematics Department’s program-specific outcome: “Stu-dents will be able to construct and evaluate mathematical proofs.” It is my hope thatthe Mathematics Department will consider carefully the results of this assessment, reviseand refine this assessment tool in future semesters to better suit our needs, and use theinformation to better educate our students with respect to this fundamental skill.

A proof and deductive reasoning course, commonly called a transition course, is aubiquitous course in modern undergraduate programs. Furthermore, all undergradu-ate mathematics programs have at least one student learning outcome associated withproof. Prior to the fall 2015 semester, during which I taught MATH 210, Morningside’stransition-to-proofs course, I began to research assessment tools related to this course andoutcome. Considering how ubiquitous it is, I found very little, with the exception of apdf authored by Merchant and Rechnitzer from the University of British Columbia. Theyhave been developing a pre/post-test for a similar course since 2010.

Ideally, such an assessment tool would be developed by interviewing mathematics fac-ulty about prerequisite skills for the course and determining where faculty think studentsencounter the most difficulty in the course. The next step in such a process would beto run the questions as open ended test items, and then use the responses to create dis-tractors. While this process would have been preferred, I needed an instrument that Icould use immediately. Thus, I developed a Basic Proof Skills Test (BPST) based on theinformation I gathered from Merchant and Rechnitzer [2].

The test consists of 16 multiple choice questions. Four questions cover pre-calculusmaterial concerning absolute value, algebraic expressions, and set theory. Similar materialwas included in the Merchant and Rechnitzer version, which is based on conversations theyhad with colleagues concerning what they felt was a disconnect between the prerequisite

1

Page 15: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Table 1: BPST Summary

Questions Area1, 3 Absolute Value2, 8, 9, 10 Algebraic Expressions4 Set Theory5, 6, 7 Logic11, 12 Proofs13, 14 Quantifiers15, 16 Definitions

skills that professors at their institution assumed, and the actual skills of students enteringtheir courses. The remaining 12 questions cover material specifically addressed in MATH210, and cover logic, algebraic expressions, proof evaluation, quantifiers, and definitions.Table 1 summarizes the area of mathematics that each question addresses.

The purpose of this initial assessment is three fold. The results and statistics providedin this document will

1. be used to identify vague and poorly worded questions so that the assessment toolcan be improved in future iterations

2. be used to identify areas of strength in MATH 210

3. be used to identify areas of weakness in MATH 210

4. be used to identify areas of weakness in the overall mathematics curriculum

2 Methods

The BPST was given to three groups of students. It was administered on the first day ofMATH 210 to n1 = 26 students and again at the end of the semester to n2 = 24 students.Additionally, the test was administered to the MATH 400 capstone class (n3 = 8), so thatwe might asses the long term retention of these skills.

3 Item Anaylsis

Because the instrument was developed with relative haste, the author feels a question-by-question item analysis of the instrument itself is necessary. To this end, the author willbegin by using Classical Test Theory to evaluate the instrument. This item analysis willconsist of the following measures.

The difficulty index of a question is simply the percentage of students who answer aquestion correctly. The value of the difficulty index ranges from 0 (no student answered

2

Page 16: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

correctly) to 1 (all students answered the question correctly). In other words, the higherthe difficulty index, the easier the problem. Perhaps it should be renamed the easinessindex. Experts generally recommend that the average difficulty of a 4 part multiple choicetest be between 60% and 80%. Questions on the BPST ranged from 4 to 8 parts.

The discrimination index of a question measures an item’s ability to distinguish be-tween low-achieving, and high-achieving students. It is calculated as the difference inthe proportion of the top quartile of students who answer a question correctly and theproportion of the bottom quartile of students who answer a question correctly, althoughsome authors use the top 27% and bottom 27%. The quartiles can be determined in otherways, as well. For example, they could be determined by the test itself (internal criterion),or an external criterion can be used, such as the students over all score in a class, or GPA,[1]. Here we will use an internal criterion. Discrimination values for questions below 0.25are considered poor. Questions with scores below 0.25 in all three groups will be replacedor revised in the next iteration of the BPST.

The point-biserial correlation (PBC) is defined the correlation of each item with theoverall scores on the test. It tells us how much each question correlates with the students’overall scores, and is thought to indicate the reliability of individual test items. Higherscores are better, and a cutoff of 0.2 is often used.

A summary of these measures for the pre-test, post-test, and capstone class can befound in Table 2.

3.1 Discussion of Item Analysis

To determine which questions should be replaced, we are using a discrimination cut-offof 0.25 and a PBC cut-off 0.2. Questions 2 and 15 have low discriminating power for allthree groups, and should therefore be replaced. Additionally, question 14 and 16 have lowdiscriminatory power on both the pretest and capstone test, and low PBC on the post test,so should be considered for replacement. Together, questions 13 and 14 test a student’sability to understand the difference in order of universal and existential quantifiers, animportant skill taught in MATH 210. As such, replacing this question should be done sowith great care. Additionally, questions 3 and 8 have low scores for both the posttest andcapstone groups.

4 Whole Test Measures

In addition to the item analysis, there are several standard whole test measures that willbe calculated.

Cronbach’s Alpha is a measure of a test’s internal consistency, and is given by

α =k

k − 1

(1 −

∑ki=1 σ

2Yi

σ2X

),

3

Page 17: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Tab

le2:

Diffi

cult

yIn

dex

,D

iscr

imin

atio

nIn

dex

,an

dP

oint-

Bis

eria

lC

orre

lati

onP

re-t

est

Post

-tes

tC

ap

ston

eD

ifficu

lty

Dis

crim

inati

on

Poin

t-b

iser

ial

Diffi

cult

yD

iscr

imin

ati

on

Poin

t-b

iser

ial

Diffi

cult

yD

iscr

imin

ati

on

Poin

t-b

iser

ial

Qu

esti

on

Ind

exIn

dex

Corr

elati

on

Ind

exIn

dex

Corr

elati

on

Ind

exIn

dex

Corr

elati

on

10.1

90.4

30.4

30.4

2-0

.33

-0.1

30.2

50.5

00.5

52

0.8

30.0

20.2

10.8

80.0

60.4

40.8

80.1

30.1

43

0.2

70.2

90.2

50.0

80.1

70.2

60.3

80.0

0-0

.11

40.6

90.2

90.3

70.7

10.5

00.4

30.8

80.5

00.3

05

0.6

20.5

70.3

30.5

80.6

70.3

80.7

50.0

00.2

16

0.4

20.4

30.3

20.6

30.8

30.6

00.7

50.0

00.2

17

0.1

90.4

30.3

50.4

60.6

70.4

40.1

30.0

0-0

.28

80.8

80.2

90.5

41.0

00.0

0N

A1.0

00.0

0N

A9

0.5

00.5

70.3

20.8

30.3

30.4

40.5

00.5

00.4

610

0.3

80.4

30.2

90.5

00.0

00.0

70.6

30.5

00.5

711

0.3

10.0

00.0

70.3

30.8

30.7

30.6

31.0

00.7

412

0.1

50.0

0-0

.09

0.4

60.6

70.4

60.2

50.5

00.4

613

0.8

50.4

30.5

81.0

00.0

0N

A0.8

80.5

00.5

014

0.2

70.1

40.2

20.5

00.3

30.1

90.2

50.0

00.2

515

0.8

00.0

90.2

40.9

10.0

20.0

00.9

7-0

.13

-0.3

916

0.7

80.0

90.1

80.9

50.3

00.1

50.9

10.1

30.2

1A

ver

age

0.5

10.2

80.2

90.6

40.3

00.3

20.6

30.2

60.2

5

4

Page 18: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Figure 1: Item Measure Relationships

0.0

0.5

1.0

0.25 0.50 0.75 1.00Difficulty Index

Dis

cri

min

ation Index

Test

Capstone

Posttest

Pretest

−0.25

0.00

0.25

0.50

0.75

0.25 0.50 0.75 1.00Difficulty Index

Poin

t−bis

eri

al C

orr

ela

tion

Test

Capstone

Posttest

Pretest

−0.25

0.00

0.25

0.50

0.75

0.0 0.5 1.0Discrimination Index

Poin

t−bis

eri

al C

orr

ela

tion

Test

Capstone

Posttest

Pretest

5

Page 19: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Table 3: Cronbach’s Alpha, Ferguson’s Delta, and Average Point-biserial CorrelationPre-test Post-test Capstone

(n1 = 26) (n2 = 24) (n3 = 8)Cronbach’s Alpha 0.23 0.29 0.27Ferguson’s Delta 1.01 0.99 0.93Average Point-biserial Correlation 0.29 0.32 0.25Average Point-biserial Correlation withquestions 2, 3, 8, 14, 15, 16 removed 0.30 0.38 0.37

where k is the number of test items, σ2Yi

is the variance of the individual test items, andσ2X is the variance of the observed total test scores. This measure is often used to evaluate

a scale reliability. That is, is assumes that the test is using a Likert scale, such as 1-5 with1 being strongly disagree and 5 being strongly agree. Since this test does not use a Likertscale, the usefulness of this parameter to measure reliability is limited. Furthermore, it isassumed that the questions have uni-dimensionality, that is, they are trying to measurethe same thing. This is not the case with the BPST, as it is trying to measure manydifferent aspects material taught in MATH 210.

Ferguson’s Delta is a single value that gives us an idea how discriminatory the entiretest is. Broadly, it measures how well student scores are distributed over the entire rangeof possible scores. If δ > .9, a test is considered discriminating. It is calculated as

δ =(m+ 1)(n2 −

∑f 2i )

mn2,

where m is the number of test items, n is the number of test takers, and the fi’s representthe frequency of each score.

A summary of these measures for the pretest, posttest and capstone groups can befound in Table 3.

5 Course Assessment

Knowing the strengths and weaknesses of the BPST itself, it is now possible to examineits implications on student learning in MATH 210. In this section, we compare the pre-and post-test scores for the fall MATH 210 class. We also compare the post-test resultsfrom MATH 210 with the results from the capstone class (MATH 400) to assess retentionand improvement of these skills.

5.1 Normalized Average Learning Gains

Normalized average learning gains (NALG)

NALG =< post-test > − < pretest >

1− < pretest >,

6

Page 20: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Figure 2: Itemized Results

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16

Prop

or%o

nCo

rrect

BPSTFall2015Results

Pre-test

Post-test

are calculated for the entire BPST, the pre-calculus portion of the exam, the MATH 210portion of the exam, and for the exam with question 2, 3, 8, 14, 15, and 16 removed.These results are summarized in Table 4.

Hypothesis tests were performed for comparing two independent means. The NALGfor the entire test is 26.65%. However, we can see from the p-values for the pre-calculuscontent that majority of learning gains are in areas specifically addressed in MATH 210.For material specifically covered in the course, the NALG is 34.16%.

The mean BPST score for the capstone class was 10, which means that the normalizedaverage learning gains from the end of MATH 210 to the beginning of the senior year whenthey take MATH 400 are −2.5%. Removing the questionable items from the BPST doesnot help. In fact, when these question are removed, the normalized average learning gainsdrop to −5.9%. When a t-test is performed comparing the two means at the α = 0.05significance level, we fail to reject the null hypothesis, meaning that there is not enoughevidence to reject the hypothesis that the two populations means are the same. In essence,this means that from the time our students take MATH 210 until they graduate, theyretain the skills developed in this course. We might hypothesize that they students wouldimprove upon these skills between the two exams since they are required to use them inother courses. However, this hypothesis is not supported by this analysis. If this resultis borne out in future evaluations, the Mathematics Department may want to considercurriculum changes that support the continued development of these skills.

5.2 Correlation with Final Course Grades

Interestingly, when the pre- and post-test are correlated with final grade, we find thatpre-test scores are not linearly correlated with final grade. The correlation coefficient for

7

Page 21: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

Table 4: Mean BPST scores and Normalized Average Learning GainsPre-test Post-test NALG t-test

(n1 = 26) (n2 = 24) (df = 23) p-valueFull Test (out of 16) 8.14 10.23 26.65% 0.0011Precalculus component (out of 4) 1.99 2.08 4.85% 0.3312Proof skills component (out of 12) 6.15 8.15 34.16% 0.000045Test with Questions2, 3, 8, 14, 15, 16 removed (out of 10) 4.31 5.92 28.27% 0.00049

Figure 3: Posttest Correlation with Final Course Grades

y=2.6514x+50.876R²=0.30044

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00

CourseGrade

%

Post-testScore(outof16)

pretest with final grade is r1 = .2809, but the critical value for the Pearson correlationcoefficient with n = 25 is .396 (α = .05). On the other hand, post-test scores on theBPST and final grades are linearly correlated, with a correlation coefficient of r2 = .548,which is significant even at the α = .01 level. This result is encouraging, as it suggeststhat it is not what a student knows when they begin the class, but what they know whenthe class ends that is most highly corollated final grade, see Figure 3.

6 Discussion

We now turn our attention to what the BPST can help us learn about the course as awhole. What information, if any, can be garnered about the strengths and weaknesses ofMATH 210 from this assessment tool? In the following examination, questions 2, 3, 8,15, and 16 are disregarded since they will be replaced in the next iteration of the BPST.

8

Page 22: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

The rest of this section will examine the most troublesome and most encouraging results,in that order.

The most troublesome results stem from questions 5, and 11, which we review next.Question 5 has to do with the implication

“If P , then Q.”

and its converse

“If Q, then P .”

which are not logically equivalent statements. This concept is covered early in thesemester, and is a fundamental to material covered later in the course. Question 5 readsas follows:

Do the following two statements mean the same thing?

“If I am healthy, then I will come to class.”“If I come to class, then I am am healthy.”

The average for this question on the pretest was .62 and on the posttest it was .58,meaning that there was no improvement on this question over the course of the semester.

Question 11 address a student’s ability to recognize a correct proof. The average forthis question on the pretest was .31 and on the posttest it was .33. Perhaps even moreconcerning is that on both the pre- and post-test, the most popular answer to this questionwas option A, in which the proof begins by assuming the conclusion.

The results of the two questions above suggest more time should be spent early in thesemester making sure logical foundations are strong, and less time on definitions that areparticular to individual subject areas.

The largest gains from pre- to post-test, were made on questions 9, 7, and 12. Question9 is an algebraic expression question in which students are asked to determine if theexpression is always true, sometimes true, or never true. This is a useful skill to havewhen proofs are done by cases. The absolute increase in this question was +0.38 andthe NALG for this item was 76%. Question 7 covers the logic of the implication “If P ,then Q.” The NALG for question 7 were 35%. Finally, question 12, like question 11, alsotests the ability to recognize a correct proof. Here, the NALG were 34%, a result that isencouraging, considering the outcome of question 11.

Conclusions

This is only a preliminary report. As such, no concrete recommendations are being madein terms of changes to the course or curriculum. The author advises a revision of theBPST in accordance with the findings above, and re-administering it next year before itis adopted as an official departmental assessment tool.

9

Page 23: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

References

[1] Lin Ding and Robert Beichner. Approaches to data analysis of multiple-choice ques-tions. Physical Review Special Topics - Physics Education Research, (5), 2009.

[2] Sandra Merchant and Andrew Rechnitzer. Development and analysis of a basic proofskills test.

10

Page 24: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

CALCULUS MASTERY EXAM

1. What is f ′(x) if f(x) = x5 + sin(x) + ex + 7?

A. 16x

6 − cos(x) + ex + 7x

B. 5x4 + cos(x) + ex

C. 5x4 + cos(x) + ex + 7

D. 0

E. 5x4 + cos(x) + xex−1

2. Find the equation of the line tangent to the graph y = x2 at x = 3.

A. x = 3

B. y = 6x

C. y = 6x + 3

D. y = 6x− 9

E. Not enough information.

3. Determine limx→∞x2+7x+6

4x2+11x−15 .

A. −25B. ∞C. 0

D. 14

E. 1

4. Determine∫ 2

0(3x2 + 6x + 2)dx.

A. 0

B. 8

C. 24

D. 20

E. 42

5. Integrate

∫10x

(5x2 − 3

)2/3dx.

A.2

3

(5

3x3 − 3x

)−1/3+ c

B.2

3

(5x2 − 3

)−1/3(10x) + c

C.3

5

(5x2 − 3

)5/3 (5x2)

+ c

D.3

5

(5x2 − 3

)5/3+ c

E. 3503 x

43 − 30 + c

Page 1 of 3

Page 25: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

CALCULUS MASTERY EXAM

6. Determine the convergence of the infinite series∑∞

n=11n .

A. The series converges absolutely.

B. The series diverges to infinity.

C. The series converges conditionally.

D. The series diverges to negative infinity.

E. Not enough information.

7. Create a vector that starts at the point (4,−1, 2) and ends at the point (3, 1,−5).

A. −1i + 2j +−7k

B. 1

C. 7i + 3j +− 252 k

D. 1i− 2j + 7k

E. 7i− 3k

8. Let f(x, y) = (xy − 1)2. Compute ∂f∂y .

A. 2(xy − 1)

B. 2(xy − 1)y

C. 2(xy − 1)x

D. (xy − 1)2x

E. 1

9. Evaluate the integral∫ 1

0

∫ 1

0x2y dx dy.

A. 14

B. 16

C. 2

D. 12

E. 1

10. A total of L feet of fencing is to form three sides of a level rectangular yard. What is the maximumpossible area of the yard, in terms of L?

A. L2

9

B. L2

8

C. L2

4

D. L2

E. 2L2

Page 2 of 3

Page 26: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

CALCULUS MASTERY EXAM

11. Given f(x) =√x +√x, find f′(1).

A.√

2

B. 34√2

C. 1√2

D. 3√2

E. 14√2

12. Determine the value of∑∞

n=12n+1

3n .

A. 3

B. 2

C. 32

D. 4

E. ∞

13. Find an equation of the plane that passes through the (noncollinear) points P (2,−1, 3), Q(1, 4, 0) andR(0,−1, 5).

A. 5x + 4y + 5z = 20

B. −12x + 3y + 9z = 0

C. −12x− 3y + 9z = −24

D. 10x + 8y + 10z = 42

E. 10x− 8y + 10z = 42

14. Compute the following indefinite integral:∫x ln(x)dx

A. 12x

2 1ln(x) + C

B. 12x

2 ln(x)− 14x

2 + C

C. ln(x) + 1 + C

D. 12x

2 ln(x) + C

E. 12x

2 ln(x)− 16x

2 + C

15. Solve the following differential equation: y dydx = x(y2 + 1).

A. y2 = cex2 − 1

B. 12x

2 = 12y

2 + ln(y) + c

C. ln(x) = 12y

2 + ln(y) + c

D. 12y

2 = 12x

2(y2 + 1)

E. 12y

2 = x(y3

3 + y)

Page 3 of 3

Page 27: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 1

NAME:

1. Find the set of all values of x for which

|2− x2| < 2

(a) (0,√

2) (b) (0, 2) (c) (−2, 0) (d) (−√

2, 0)

(e) (−√

2, 0) ∪ (0,√

2) (f) (−2, 0) ∪ (0, 2) (g) (−2, 2) (h) (−√

2,√

2)

2. For all real numbers, a, b, c, and d, we have the following expression

a− b

c− d

Select all expressions that are equivalent.

(a) b−ac−d

(b) b−ad−c

(c) a−bd−c

(d) − b−ac−d

(e) −(b−a)−(d−c)

(f) a−b−(c−d)

(g) −b+a−d+c

(h) a−b−(c−d)

3. Which of the following is a sketch of the function g(x) = |x2 − 4|?

(a)

(b)

(c)

(d)

(e)

(f)

4. If, in a class of 30 students, 8 own a cat, 15 a dog, and 5 own both, how many own only a cat?

(a) 8 (b) 3 (c) 12 (d) 0

Page 28: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 2

5. Do the following two statements mean the same thing?

“If I am healthy, then I will come to class.”“If I come to class, then I am am healthy.”

(a) Yes

(b) No

6. Do the following two statements mean the same thing?

“If it is Wednesday, then I lift weights.”“If I do not lift weights, then it is not Wednesday.”

(a) Yes

(b) No

7. Consider the statement:

“If I do not walk my dog today, I will walk my dog tomorrow.”

If the previous statement is true, determine if the following statement is true or false.

“I walk my dog on both days.”

(a) True

(b) False

For each of the statements below, indicate whether each statement is

(a) always true: true for any choice of the variables

(b) sometimes true: true for some variable choices, but not for all choices, or

(c) never true: not true for any choices of the variables.

8. For real numbers u and v,(u− v)2 ≥ 0

(a) always true

(b) sometimes true

(c) never true

Page 29: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 3

9. For real numbers a and b,(a + b)2 = a2 + b2

(a) always true

(b) sometimes true

(c) never true

10. For real numbers x and y, √x2 + y2 < x

(a) always true

(b) sometimes true

(c) never true

Page 30: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 4

11. Below is a statement and three proofs. Select the proof of the statement that is correct andcomplete.

“For any positive numbers a and b, a+b2≥√ab.”

Proof A: Assuming that a+b2≥√ab,

Multiply both sides by 2 a + b ≥ 2√ab

Squaring (a + b)2 ≥ 4ab

a2 + b2 + 2ab ≥ 4ab

a2 + b2 − 2ab ≥ 0

(a− b)2 ≥ 0

Which is true for positive numbers. So the assumption was true.

Proof B: For all positive numbers

(√a−√b)2 ≥ 0

a− 2√a√b + b ≥ 0

a + b ≥ 2√a√b

a + b

2≥√ab

So the result is true.

Proof C: Look at the diagram. The area of the large square is equal to (a+b)2. The unshadedarea is equal to 4ab. Since the area of the whole square is larger than the unshaded area, wehave

(a + b)2 > 4ab

a + b > 2√ab

a + b

2>√ab

So the result is true.

a b

a

b

Page 31: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 5

12. Below is a statement and three proofs. Select the proof of the statement that is correct andcomplete.

“If n2 is an odd number, then n is an odd number.”

Proof A: If n was an even number, then n = 2m for some m. Then n2 = 4m2 = 2(2m2) willalso be an even number. So, as n2 is an odd number, then n must be an odd number.

Proof B: If n was an odd number, then since odd × odd = odd, n2 is also odd. Then n2 + nwould be the sums of two odd integers and would therefore be even. Since n2 +n = n(n+ 1) itis also the product of two consecutive numbers and so it certainly is even. Therefore n is odd.

Proof C: Since n2 is odd, we have n2 = 4m2 + 4m+ 1 for some m. This means n2 = (2m+ 1)2

for some m. So n = 2m + 1, which is odd.

Page 32: Yearly Assessment Report – Mathematics 2015-16 · PDF file · 2016-07-01Yearly Assessment Report – Mathematics 2015-16 ... the program has a three-pronged plan to improve

MATH 210–Transition to Abstract Mathematics | BASIC PROOF SKILLS TEST 6

13. True or False: For every real number a, there exists a real number b such that a− b = 4.

(a) True

(b) False

14. True or False: There exists an integer x such that for every integer y, x + y = 3.

(a) True

(b) False

15. For a pair of integers (a, b) we have the following definition (for this test only, this is not astandard definition):

When a is even or b is odd then the pair (a, b) is called happy.

Select all pairs below that are happy.

(a) (1, 0) (b) (−2, 3) (c) (3, 0) (d) (−1, 1)

(e) (2, 0) (f) (5, 1) (g) (3,−3) (h) (1, 4)

16. For two integers a and b with a 6= 0, we say that a divides b if there is an integer c such thatb = ac. In this case we write a|b. Select all statements below that are true.

(a) 8|64 (b) 3|20 (c) 2|0 (d) 3|57

(e) 1|2015 (f) 4|46 (g) 2|35 (h) 6|42