mth 100 sloat assessment study report (spring...

MTH 100 SLOAT ASSESSMENT STUDY REPORT (Fall 2011)

prepared by Soraida Romero and Carlos Castillo, the Course Coordinators for MTH 100

I. Introduction:

An assessment study on MTH 100 at Essex County College was conducted in Fall 2011 as part of SLOAT (Student Learning Outcomes Assessment Team). The principal coordinators for this study were the following three full-time faculty members of the Math and Physics Division (MAP): Carlos Castillo, Instructor of Mathematics, Ron Bannon, Associate Professor of Mathematics, and Soraida Romero, Professor of Mathematics.

Purpose: The purpose of this study was three-fold. First, the study attempted to determine whether MTH 100 students are achieving some of the Measurable Performance Objectives (MPOs) that are related to the four (4) Course Goals (CGs), one of which is also a General Education goal, that have been established by the Math Department for this course. These four MTH 100 course goals are as follows:

CG #1: Demonstrate knowledge of the fundamental concepts and theories from algebra and geometry.

CG #2: Utilize various problem-solving and critical-thinking techniques together with algebra to set up and solve application problems taken from a variety of disciplines. This course goal also satisfies Gen Ed goal #2 which states that students will use appropriate mathematical and statistical concepts and operations to interpret data and to solve problems.

CG #3: Communicate accurate mathematical terminology and notation in written and/or oral form in order to explain strategies to solve problems as well as to interpret found solutions.

CG #4: Use calculators effectively as a tool to solve such problems as those described above.

The second purpose of this study was to ascertain whether two particular factors or

variables have a correlation with student success rates in MTH 100. The two factors that were looked at were as follows: 1) giving students access to class notes (either on-line or hard copies)

MTH 100 – 1

provided by their instructors; and 2) requiring students to use WebAssign, the on-line homework software, and making it count toward their final course grade.

The third purpose of this study was to follow-up on three of the recommendations made as a result of the MTH 100 Fall 2010 SLOAT study. Specifically, it was recommended that workshops should be offered to students on how to solve word problems algebraically. In addition, it was recommended that particular changes be made to the course to help students perform better; as a result, in Fall 2011, one change that was made to MTH 100 was to add one more Departmental exam near the end of the semester, which tested students’ knowledge of the last topics covered in the course; and another change was to include a review session before each exam in the course.

II. Methodology:

a. Population sample used

It was decided to use all students enrolled in 5 sections of MTH 100 as the cohort sample for this study. The following five sections and instructors of MTH 100 were selected to participate in this study conducted in Fall 2011, with a total student population of 167 students. These five sections, which were day sections taught by three full-time math instructors, are as follows:

MTH 100 – 004 MWF 10:00 – 11:20 (full-time instructor Carlos Castillo)MTH 100 – 013 TRF 11:30 – 12:50 (full-time instructor Ron Bannon)MTH 100 – 015 MWF 10:00 – 11:20 (full-time instructor Ron Bannon)MTH 100 – 016 TRF 7:00 – 8:20 (full-time instructor Soraida Romero)MTH 100 – 025 TRF 1:00 – 2:20 (full-time instructor Soraida Romero)

The students in Professor Castillo’s section were given class notes, which were prepared by the instructor, allowing them to follow the daily lectures using these notes; the students in his class were also required to use the WebAssign on-line homework software, which constituted 15% of their final grade for the course. Students in both sections of MTH 100 taught by Professor Bannon were provided access to on-line class notes prepared by the instructor, and they also were required to use WebAssign, which made up 20% of their final grade for the course. The students in Professor Romero’s two sections of MTH 100 were not provided with class notes to supplement the class lectures, and WebAssign use was encouraged but not made mandatory. Instead, homework made up 10% of the students’ final course grade for Professor Romero’s students, but this was determined by the grade students obtained in homework

MTH 100 – 2

projects that were assigned to students five times during the semester and graded by the instructor.

b. Instrumentation

There were six (6) different methods used to gather data for this study. These were as follows:

i. Multiple-choice questions on Departmental exams blueprinted to MPOs;ii. A brief student questionnaire at the end of the offered workshop;iii. Student data relating to final course grade and homework grade earned, which was

obtained from each instructor at the end of the semester;iv. Student questionnaire filled out by students who used WebAssign and utilized

instructor-prepared class notes;v. A questionnaire disseminated to all MTH 100 instructors in Fall 2011 asking for their

opinion on the changes that were made to the course in Fall 2011 as well as their opinion on what can be done to help increase the student success rate; and

vi. Statistical analysis to determine correlations between success rates and the two variables: use of class notes and on-line homework.

i. Multiple-choice questions blueprinted to MPOs

As a means of determining whether students taking MTH 100 are meeting the Measureable Performance Objectives (MPOs) for this course, Professors Castillo and Romero decided to include eight multiple-choice questions blueprinted to eight (8) MTH 100 MPOs as part of the MTH 100 Fall 2011 Final Exam. Scan Tron sheets (with a section to place the answer for each of the 8 multiple-choice questions) were given to the three instructors of this study to be used by the students in the 5 sections of this study to facilitate the easy grading of and Student Learning Outcomes (SLO) data collection on these multiple-choice questions. The collected SLO was analyzed to determine if the 8 MPOs were met by the students in the 5 sections of MTH 100. (See Appendix A of this report for a sample of the Scan Tron sheet used in this study.)

ii. Student questionnaire distributed at the end of the offered workshop

There was one workshop offered to all MTH 100 students in Fall 2011 . This topic of this workshop, which was facilitated by Professor Castillo, was how to solve word problems algebraically. A brief assessment instrument was given out at the end of the workshop to determine if the students liked the workshop and if they found it useful. (See Appendix B of this report for workshop handouts including the workshop flyer used to advertise the event, the

MTH 100 – 3

workshop handout, and the brief workshop questionnaire filled out by the participants at the end of the workshop.)

iii. Data obtained from each instructor

At the end of the semester, each instructor was asked to submit to Professor Romero a sheet indicating the following data for each student participating in the study: name, ID #, final course grade (i.e., both numerical course average and letter grade), and – if the student participated with on-line homework – the overall grade each student earned from these homework assignments. Professor Romero organized the data and created an Excel spreadsheet that was used for further analysis of the data. (See Appendix C of this report for a copy of the Excel spreadsheet prepared by Professor Romero and used for this part of the study.)

iv. Student questionnaire on use of WebAssign and class notes

Toward the end of the semester, a questionnaire was distributed to students in the 4 classes participating in this study in which students were provided with instructor-prepared class notes and were required to use WebAssign. This questionnaire asked them if they used the class notes and WebAssign and if they liked using them. It also asked them if they would recommend that all math classes use these items. (See Appendix D of this report for a copy of the student questionnaire used in this study.)

v. Instructors’ questionnaire

A questionnaire was distributed to all MTH 100 Fall 2011 instructors. The first part of the questionnaire asked their opinion on how many Departmental exams should be given in MTH 100; how many multiple-choice questions should be included per exam as well as how many questions overall; if a review session should be conducted before each exam; and, if they made WebAssign mandatory for students, what percent the WebAssign score counts toward the final course grade. The second part of the questionnaire was open-ended and asked the instructors’ opinions on ways to improve the student course success rate, innovative teaching strategies that they found successful, and their suggestions on improving the course outline. (See Appendix E of this report for a copy of this instructor questionnaire.)

MTH 100 – 4

vi. Statistical analysis of the student data

With the assistance of a graduate student who took an educational research course at The College of New Jersey, Excel was used to conduct the statistical analysis of the data received from instructors. This graduate student, Danielle Romero, is the daughter of one of the study coordinators. There were three different types of analyses done – descriptive, correlations, and T-tests. The results of this analysis are shown in Appendix F of this report.

III. Results obtained from the six different methods used to gather data for this study

(a) Results from multiple-choice questions blueprinted to MPOs

Several MTH 100 final exam questions were blueprinted to particular MPOs for the course in Spring 2011 as well as in Fall 2011. Although SLO data was collected for Spring 2011, this data was not analyzed since the Scan Tron machines to be used for scanning student responses for the questions on the final exam were not yet available at that time. Since then, we have secured the technology to scan these responses, which is now available for all faculty to use in the new Faculty Assessment Center. We will therefore present the results for the acquisition of MPOs by students, which were obtained from the multiple-choice questions included in the final exams for Spring 2011 (previously collected but unreported) as well as Fall 2011.

In Spring 2011 , out of the original 247 students chosen to participate in the MTH 100 SLOAT study, 206 students took the final exam. Each student took one of four possible versions of the final exam. This exam consisted of 23 questions including 8 multiple-choice questions used to determine the acquisition of 8 distinct course MPOs. It was found that 5 out of the 8 MPOs (62.5%) assessed were achieved by 70% or more of the students who participated in the study. This means that 5 out of the 8 MPOs examined were achieved by the MTH 100 cohort sample. A condensed item analysis was done for each of the four versions of the final exam and this analysis can be found in Appendix G of this report. The acquired MPOs, along with the percent of students who achieved each MPO, are as follows:

(MPO 1.4) Solve radical equations. (77.67 %) (MPO 1.13) Perform basic operations on complex numbers. (72.33 %)(MPO 1.16) Graph a parabola in the Rectangular Coordinate System. (76.21 %)(MPO 1.17) Graph a circle in the Rectangular Coordinate System. (79.13%)

MTH 100 – 5

(MPO 2.1) Apply algebraic methods to solve varied real-world applications (such as, consecutive integer problems, coin/stamp problems, distance problems, investment problems, area problems, and work problems) that can be modeled by a linear equation, quadratic equation, rational equation or system of equations. (77.18 %)

3 out of the 8 MPOs (37.5%) assessed were not achieved by students who participated in the Spring 2011 SLOAT assessment study. These MPOs are listed below followed by a description of the reason why the MPO was probably not met by the students in the sample.

(MPO 1.5) Solve quadratic equations. (61.65 %)

In the Fall 2010 MTH 100 study, it was found that 63.11 % of the participating students taking the final exam achieved this MPO. Unfortunately, this percentage decreased slightly to 61.65 % in the Spring 2011 MTH 100 study, when students had to solve a quadratic equation where the solutions were irrational. As was stated in the Fall 2010 MTH 100 SLOAT study final report, students may have had a difficult time solving the quadratic equation on the final exam since it required the use of the quadratic formula, a formula which many students often forget, or the use of completing the square technique, a more complicated method which students usually avoid, in order to solve it.

(MPO 1.12) Perform basic operations on radical expressions. (67.96 %)

Even though this MPO was not achieved by students participating in the Spring 2011 SLOAT assessment study, this percentage (67.96%) was slightly higher than the 61.86 % of the Fall 2010 study. In general, students continue to struggle to understand and master this topic. Since radicals are covered quickly at the end of the semester and instructors generally do not have enough time to reinforce this content before the final exam, it may be that students need more independent practice on working with radical expressions to help them answer these questions correctly on the final exam.

(MPO 4.1) Write and explain solutions to application problems related to the course material using appropriate mathematical terminology and notation. (50.49 %)

It was found that 58.69 % and 50.49 % of the students failed to achieve this MPO in the Fall 2010 and Spring 2011 SLOAT studies, respectively. The same type of question was tested in each study. The question asked the students to determine whether and explain why two given lines were parallel, perpendicular, or neither. Even if the students knew that the lines were perpendicular, they still had to explain why. There were three multiple-choice answers that all

MTH 100 – 6

indicated that the lines were perpendicular but each gave different explanations why. On a timed and high-pressure test like the final exam, students may not know how to explain their answers appropriately and may simply just have guessed one out of the three choices presented.

Out of the original 167 students chosen to participate in the Fall 2011 MTH 100 SLOAT study, 97 students took the final exam. Each student took one of two possible versions of the final exam. This exam consisted of 22 questions including 8 multiple-choice questions used to determine the acquisition of 8 distinct MPOs. A condensed item analysis was done for both versions of the final exam and this analysis can be found in Appendix G of this report. It was found that of the 8 MPOs that were tested, only 3 out of the 8 MPOs (37.5%) were achieved by 70% of the students in the study. This indicates a much lower performance than that found in Spring 2011.

The acquired MPOs, along with the percent of students who achieved each MPO, are as follows:

(MPO 1.2) Solve literal equations. (83.51%)(MPO 1.7) Solve systems of equations. (80.41%)(MPO 1.18) Determine whether a given relation is a function, find its domain, and use

function notation (80.41%)

5 out of the 8 MPOs (62.5%) assessed were not achieved by students who participated in the Fall 2011 SLOAT assessment study. These MPOs are listed below followed by a description of the reason why the MPO was probably not met by the students in the sample.

(MPO 1.5) Solve quadratic equations. (67.01%)

Alarmingly, SLOAT-study participating students have failed to achieve this MPO for three consecutive semesters. Specifically, it was found that 63.11%, 61.65%, and 67.01% of these students failed to achieve this MPO in the Fall 2010, Spring 2011, and Fall 2011 studies respectively. It is clear that students struggle to solve quadratic questions without the quadratic formula being provided on the exam.

(MPO 1.12) Perform basic operations on radical expressions. (63.92%)

Again, SLOAT-study participating students have failed to achieve this MPO for three semesters in a row. It was found that 61.86%, 67.96%, and 63.92% of these students failed to meet this MPO in the Fall 2010, Spring 2011, and Fall 2011 studies, respectively. Despite scheduling a review session before the exam on radicals, students still failed to achieve this

MTH 100 – 7

MPO. Since it may be that many students are learning the topic of radicals for the first time, they may need much more time or practice to grasp this more difficult algebraic topic.

(MPO 2.1) Apply algebraic methods to solve varied read-world applications that can be modeled by a linear equation, quadratic equation, rational equation or system of equations. (63.92%)

Even though this MPO was achieved by students in the Spring 2011 study (77.18%), it was not achieved in either of the Fall semester studies (68.00% in 2010 and 63.92% in 2011). The question used to test this MPO in the Fall 2011 semester was more challenging than the one given in the Spring 2011 semester. In the Fall 2011 study, students were expected to set up an appropriate equation, then solve it. However, in the Spring 2011 study, a student only had to set up the appropriate equation (and not solve it). This may indicate that students are able to set up but not solve application problems.

(MPO 3.1) Use a calculator to perform basic arithmetic operations, evaluate powers and find the square root of a number. (52.58%)

It was found that 68.33% and 52.58% of the students did not achieve this MPO in the Fall 2010 study, as well as in the Fall 2011 study. While the distance formula was provided in the 2010 study, it was not provided in the 2011 study. Thus, it could be that students in Fall 2011 got this question wrong because they forgot the formula or did not know how to use it. Therefore, this result does not necessarily mean that students do not know how to use a calculator to find the square root of a number. To examine this issue further, it appears that a better question should be used to test the acquisition of this MPO.

(MPO 4.1) Write and explain solutions to application problems related to the course material using appropriate mathematical terminology and notation. (47.42%)

Unfortunately, students also failed to achieve this MPO for three semesters in a row with the following percentages of students answering blueprinted questions correctly: 58.69% (Fall 2010), 50.49% (Spring 2011), and 47.42% (Fall 2011). Even though a handout with similar types of problems was designed and e-mailed to all MTH 100 instructors asking that they review these types of questions in class or assign them to students for extra credit, a larger percentage of students failed to acquire this objective this semester than in previous semesters! Again, the same type of question was tested in each study. The question asked the students to determine whether two given lines were parallel, perpendicular, or neither. Even if the students knew that the lines were perpendicular, they still had to explain why, and it is clear from these findings that students continue to have difficulty explaining the reason for their answers.

MTH 100 – 8

(b) Results from the workshop questionnaires

A workshop entitled "Solving Word Problems Algebraically" was conducted on Thursday, November 15, 2011 by Professor Castillo. While over 600 flyers were prepared and disseminated the week before the workshop to all students enrolled in 22 daytime sections taught at the Main Campus, it was disappointing to note that only 8 students showed up! Professor Castillo conducted the workshop the same way he did in Spring 2011. He first explained to students that anyone can solve a word problem using algebra; then he shared a well-prepared, informative handout, which included a general strategy for using algebra to solve word problems; and finally, he showed them how to use this strategy on a few selected word problems that are covered in MTH 100.

The workshop evaluation forms filled out by the participants of this second workshop were extremely positive. Eighty-eight percent (88%) of the participants rated the workshop as excellent, and the other twelve percent (12%) rated it as good. All (100%) indicated that they enjoyed the workshop, learned something new, and would recommend this type of workshop to other students. Student comments listed at the end of the questionnaire indicated that this type of workshop should be conducted more often per semester, especially before exams. Another student even requested that a workshop on the “mathematical theory” pertaining to algebraic topics covered in MTH 100 be offered in the future.

(c) Results from student data obtained from each instructor

Using the Student Data Spreadsheet for the 167 students involved in this study (see Appendix C), collected data was analyzed to determine success rates based upon the following two variables:

Final course grades earned in MTH 100 and student access to instructor-prepared class notes – that is, if there is a difference in success rates in MTH 100 between those students who had access to class notes prepared by the instructor to supplement class lectures as part of their course and those who did not have access to these class notes

Final course grades earned in MTH 100 and required usage of the WebAssign on-line homework system by the students – that is, if there is a difference in MTH 100 success rates between those students who were required to do on-line homework as part of the course and those who were not

The following results from this analysis were determined for the 167 students in this study:

(1) Success rate in MTH 100 and access to class notes for lectures:

MTH 100 – 9

Of the five (5) sections participating in this study, four sections containing a total of 98 students (59% of the students in this study) had access to instructor-prepared class notes to supplement the lectures in their MTH 100 course and 69 students (41%) did not. Comparing the success rates of students who had access to class notes and those who did not, the following was noted: 12 of the 30 students (40%) who were provided with hard copies of instructor-prepared class notes passed the course; 20 of the 68 students (29%) who were provided with access to electronic copies of instructor-prepared class notes passed the course; and 32 of the 69 students (46%) who were not provided with any type of class notes passed the course. In general then, comparing the success rates of those who used some form of instructor-prepared class notes with those who were not granted this access, it was surprisingly determined that 32 of the 98 students (33%) who had access to class notes passed the course whereas 32 of the 69 students (46%) who did not have access to class notes passed the course. A comparison of course success rates of students who had access to class notes (electronic or hard copy) and those who did not is shown in the graph below.

Success Rate and Usage of Class Notes

Success Rate0%5%

10%15%20%25%30%35%40%45%50%

On-line Class Notes

Hard-copy Class Notes

No Class Notes

Discussion: In this study it appears that students did not benefit from being given instructor-prepared class notes since students who did not have this access had a higher success rate in the course than those who were given these notes. It actually appears that being given this access to class notes actually lowered students’ success rates. A closer look will be given to the relationship between class notes and success rates in the statistical portion of this study to determine if there is a statistical correlation between success rate in the course and access to instructor-prepared class notes.

(2) Success rate in MTH 100 and required usage of WebAssign:

87 of the 167 students (52%) were required to complete WebAssign homework, which was worth a relatively small percent of their final course grade for the course (15% in one class and

MTH 100 – 10

20% in two other classes). 80 of the 167 students (48%) were not required to use WebAssign, and WebAssign usage did not count at all toward the students’ final course grades. Comparing the success rates in the course of those who used WebAssign with those who did not, it was found that 32 of the 87 students (37%) who used WebAssign passed the course, whereas 32 of the 80 students (40%) who did not use WebAssign passed the course. A closer breakdown of this data showed an interesting result. Of the 87 students who used WebAssign, 41 (47%) scored 60% or higher as their WebAssign average while 46 (53%) scored poorly in WebAssign achieving an average score of less than 60%. Comparing the success rate of three groups of students – those who used WebAssign and scored well in it, those who used WebAssign and scored poorly in it, and those who did not use WebAssign at all, it was found that 28 of the 41 students (68%) who used WebAssign and scored well in it passed the course; 4 of the 46 students (9%) of those who used WebAssign and scored poorly in it passed the course; and 32 of the 80 students (40%) of those who did not use WebAssign at all passed the course. The course success rates of these three groups of students in this study are illustrated in the graph below.

Success Rate and Web Assign Usage

Success Rate0%

10%

20%

30%

40%

50%

60%

70%

Used WebAssign and earned a score of 60% or higher

Used WebAssign and earned a score of less than 60%

Did not Use WebAssign at all

Discussion: While the overall course success rates in MTH 100 were lower for those students in this study who used WebAssign than those that did not, it may appear at first that WebAssign usage did not give these students any advantage. However, a closer breakdown of the data reveals that perhaps it is not necessarily important for a student to simply use WebAssign but rather do well in it. That is, if a student uses WebAssign and does well in it, this can substantially help increase the student's chance of passing the course. A closer look will be given to the relationship between WebAssign usage and course successs rate in the statistical portion of this study to determine if there is a statistical correlation between success rate and WebAssign usage in the course.

MTH 100 – 11

(3) Other findings:

Grades obtained by students in this study: Of the 167 students, 64 students (38%) passed the course, which is much lower that the success rate of students taking MTH 100 in general (listed as 47% in the 2011 Mathematics Program Assessment Report.) The breakdown for success rates for the three types of classes – the two sections that used on-line class notes and on-line homework (29%), the one section that used hard copy class notes and WebAssign on-line homework (40%), and the two sections that did not use any class notes and were not required to use any on-line homework (46%) is shown in the following graph.

Success Rate and WebAssign/Class Notes Usage

Success Rate0%5%

10%15%20%25%30%35%40%45%50%

On-Line Class Notes & We-bAssign

Hard-Copy Class Notes & We-bAssign

No Class Notes & No WebAssign

Discussion: Since the highest course success rate was noted for the two classes that did not use class notes or WebAssign, it may appear that this is method of instruction was the most successful for the students in this study. It is notable that this 46% course success rate for the group that used no WebAssign or class notes is almost the same as the 47% course success rate listed for MTH 100 in the 2011 Mathematics Program Assessment Report. However, the correlation between using WebAssign, using class notes, and success rates in the course will be looked at and discussed later in the statistical section of this report.

(d) Student questionnaire used in this study

The student questionnaire used in this study and included as Appendix D was disseminated to the three classes that used class notes and WebAssign and was filled out by a total of fifty-eight (58) students. The results of this questionnaire show that 100% of the students who used the class notes found them useful, and 86% recommended that all math classes use instructor-prepared class notes. When asked if they liked using WebAssign, 81% of the students responded that they did like it, 79% indicated that they prefer WebAssign over homework from the text, and 78% recommended that all math classes should use WebAssign.

MTH 100 – 12

The results of this questionnaire suggest that although students in this study who did use class notes and WebAssign homework did not necessarily do better in the course than those who did not, these students enjoyed using these tools to learn and would like to continue their use in the math classroom.

(e) Instructor questionnaire used in this study

The instructor questionnaire used in this study and included as Appendix E was disseminated to 19 MTH 100 instructors who were teaching MTH 100 during the day at the Main Campus. Of these 19 instructors, 11 were full-time faculty members and 8 were adjuncts. The questionnaire was distributed the week before the end of the semester and was anonymously filled out and returned by 11 of these 19 instructors (58%). Even though during the Fall 2011 semester the Mathematics Chairperson instituted a change in the testing policy of the course by requiring all classes to take 5 Departmental exams instead of the 4 that were normally given, most respondents (55%) indicated that they would prefer going back to the usual 4 exams. Also, most respondents (56%) indicated that they would prefer 20 to 25 questions per exam, and most respondents (60%) prefer increasing the number of multiple-choice questions from the current 8 to 10 questions. Another policy change made this Fall 2011 semester was conducting a review session before each exam, and 64% of the instructors responded that they had complied and agreed with this new policy. In terms of WebAssign usage, it was found that 45% of these instructors do require students to use WebAssign and most made WebAssign count as 10% of their final grade. The open-ended portion of the questionnaire did not produce any new information than what is already known, however some of the comments given by questionnaire respondents are given below.

Question: Why is the success rate of students in MTH 100 so low?

Students are doing poorly because they lack basic algebra skills; if they took MTH 086 or MTH 092, it appears they did not do well in these pre-requisite courses.

Students don’t do homework. Some students don’t try to learn until half-way through the course when it’s too late. There is too much material to cover in a short amount of time. Students delay taking MTH 100 right after MTH 092 and then forget pre-requisite

material. Student lack time to practice problems due to job/home demands.

Question: Do you have any suggestions to improve the success rate of students in MTH 100?

MTH 100 – 13

Advise students to learn how to balance work and studies to achieve realistic goals. Lessen the number of topics covered in the course and ensure less overlap with those

topics covered in MTH 092. Continue the review sessions before each exam. Perhaps utilize recitation sessions or some other way to increase group practice. Provide seminars on test-taking strategies and time management skills. There should be tutors that only help MTH 100 students since most students who go to

get help at the tutoring area have to wait too long and give up.

Question: Have you employed any innovative teaching strategy in your class that you found successful and that you recommend others use?

Give them handouts on concepts for additional review. Use class notes and give them to students. Give quizzes after every chapter covered. Ask the better students to mentor struggling students for extra credit.

Question: How can the syllabus/course outline for MTH 100 be improved?

Set aside two days for review at the end of the course before the final exam. Spend more time on important topics like factoring and fractional expressions. Use uniform WebAssign. Put everything on-line so students can access it.

Question: Other Comments?

Limit class size. Reduce the number of topics covered in the course. Need more time or less material to cover. Don’t give an exam so close to the final.

(f) Statistical analysis of the results found in this study

Excel was used to conduct a statistical analysis of the data obtained in this study. Specifically, a descriptive analysis, correlations, and T-tests were all used and the detailed statistical results are included as Appendix F.

SECTION 1: Descriptive analysis

MTH 100 – 14

A descriptive analysis was conducted, which involved a sample comprised of 167 participants, 87 of which took advantage of the online homework tool WebAssign and 98 of whom received class notes from their professors. Out of all students who took advantage of the WebAssign online homework, the average WebAssign grade was 48%.

The average final course grade for the full sample of 167 students was 49.5%; the average final course grade for students who were given class notes was 46.9%; the average final course grade for students who were not given class notes was 53.2%; the average final course grade for students who utilized the WebAssign online homework tool was 50.6%; and the average final course grade for students who did not use WebAssign was 48.3%.

SECTION 2: Correlations

A correlation analysis was conducted in order to see if there were any correlations between WebAssign usage, the use of provided class notes, and final course grade. A statistically

significant correlation occurs when the correlation coefficient value is greater than . It was found that there was a statistically significant positive correlation (.871) between the final grade average in the course and WebAssign grade, meaning that when one increases, so does the other. This implies that students who have higher WebAssign scores have a better chance of passing the course. It was also found that there was a negative correlation between the final course grade and the use of instructor-prepared class notes. However, this was not a

statistically significant correlation since the correlation coefficient was , which is less than

. Both of these correlations confirm the results of the data collected from the instructors and discussed in section (c) of the Results section of this report.

SECTION 3: Comparison of final course grades and WebAssign means (T-Tests)

A T-Test comparison of means was conducted on the data to see if there is a difference in final course grade for the treatment group (students who used WebAssign) and the control group (students who did not use WebAssign). The data is summarized in the table below. A statistically significant difference occurs if the p-value is less than .05.

Treatment Final Course Grade

MTH 100 – 15

Used WebAssign on-line homework 50.64Did not use WebAssign 48.32

P-Value 0.00

This test found that there was a statistically significant difference between the treatment group (WebAssign) and the control group (no WebAssign) in their final course grades since the p-value is less than .05. As the table above shows, the final course grade for the treatment group was higher (50.64%) than that of the control group (48.32%). Although the difference in scores is statistically significant, both the control and treatment groups still had failing final course grades. However, this analysis does show that students who used WebAssign scored almost two percentage points higher in their final course grade average than those who did not.

SECTION 4: Comparison of final course grades and class notes means (T-Tests)

Collected data was also analyzed to see if there is a difference in the final course grade for the treatment group (students who were provided with class notes) and control group (students who were not given class notes). The data is summarized in the table below. A statistically significant difference occurs if the p-value is less than .05.

Treatment Final Course GradeProvided with class notes 48.87

Not provided with class notes 53.23P-Value 0.00

This T-Test comparison of means determined that there was a statistically significant difference between the treatment group (class notes) and the control group (no class notes) in their final course grades since the p-value is less than .05. It was found that the treatment group final course grade (48.87%) was slightly lower than that of the control group (53.23%). Again, although the difference in scores is statistically significant, both the control and treatment groups still had failing average course grades. While it was found that there is correlation between using class notes and the final course grade, it pointed out an unusual result – that the correlation was a negative correlation. Specifically, it was found that students who did not use instructor-prepared class notes for the course had a final course grade over four percentage points higher than those who did use the class notes!

MTH 100 – 16

Summary, Conclusion and Recommendations:

Summary

This study of 167 students taking MTH 100 in Fall 2011 was designed to assess the following:

(1) If the 8 MPOs for MTH 100 that were included in the final exam were achieved by the students in the sample;

(2) If students who used instructor-prepared class notes to supplement the class lectures as part of their MTH 100 course were more successful in the course than those who did not;

(3) If students who used the WebAssign on-line homework package in MTH 100 were more successful than those who did not;

(4) If students liked using WebAssign homework and found it useful;(5) If students liked using instructor-prepared class notes and found it useful;(6) If instructors liked the changes recently made in MTH 100 (i.e., raising the number of

exams from four to five and conducting a review session before each exam);(7) If MTH 100 instructors are using any successful innovative teaching methods that they

would recommend others use and have additional suggestions on how to improve the course; and

(8) If students liked attending workshops on math related topics and found them useful.

Conclusions

The results of this study confirmed some of the findings of earlier MTH 100 SLOAT studies as well as uncovering a new unexpected one. These are as follows:

(1) Students continue to have difficulty achieving particular MPOs of MTH 100, specifically solving quadratic equations, performing operations on radical expressions, explaining the reason for their solutions, solving written problems using algebra, and using a calculator.

(2) Comparing the course success rates of students who used WebAssign with those who did not, it was found from the instructor-provided data that those who did not use the on-line homework package had a higher course success rate (40%) as compared to those who did use it (37%). However, further analysis of this data showed that when the WebAssign group was divided up into two components – students who earned a WebAssign average of 60% or higher and those who scored less than 60% in WebAssign – then the highest course success rate (68%) was found for students who used

MTH 100 – 17

WebAssign and earned a WebAssign average of over 60%, the second highest course success rate (40%) was found for those who did not use WebAssign at all (40%), and the lowest course success rate (9%) was found for those who used WebAssign but earned a WebAssign average of less than 60%. Thus it appears that using WebAssing and scoring well in it does increase a student's chance of passing the course.

(3) A positive correlation between WebAssign usage and final course grade was confirmed by the statistical analysis which found that the students who used WebAssign had a final course grade of 50.64%, which is over 2% higher than the group who did not use WebAssign (48.32%). Thus, this data analysis found that WebAssign usage did indeed statistically increase final course grade.

(4) From the data submitted by instructors, lower course success rates (33%) were found for those students who used instructor-prepared class notes than those who did not (46%). However, when the class notes group was divided up into two groups, those who used the on-line instructor-provided class notes and those who were given hard-copy class notes by the instructor, it was found that the highest course success rate (46%) was still for those who were not provided any notes, followed by those who were given the hard-copy class notes (40%). The lowest course success rate (29%) was attained by students who were given access to on-line provided class notes.

(5) A negative correlation between class notes and final course grade was confirmed by the statistical analysis, which found that the group who did not use class notes had a final course grade of 53.23%, which is over 4% higher than the group who did use class notes (48.87%). Thus, this data analysis found that using instructor-prepared class notes statistically decreased final course grade. While one would tend to think that providing students with class notes would be beneficial to them, there is no way of knowing that they even use these notes. In particular, since some of these notes were simply available on-line, perhaps many students did not even access or take advantage of these notes.

(6) All (100%) students who used instructor-prepared class notes found them useful, and most (81%) liked using WebAssign. The majority of the students wanted to continue using these learning tools.

(7) While the Mathematics Department instituted an increase to the number of MTH 100 Departmental exams from 4 to 5, most instructors (55%) prefer keeping it at 4 exams.

(8) Most instructors (64%) agreed with the newly instituted Department policy of conducting a review session before each Departmental exam in Fall 2011 by indicating that they found the review helpful to students and thought it could help them perform better on exams.

(9) Students were very receptive to workshops on math-related topics and want to see more of these workshops given on a regular basis each semester.

MTH 100 – 18

Recommendations

Due to the findings provided by this MTH 100 SLOAT study, the coordinators of this study

make the following recommendations:

Despite asking the Math Department to consider including a formula sheet consisting of the distance formula, the quadratic formula, the area of a rectangle, and the Pythagorean Theorem on the MTH 100 final exam in order to ensure that student errors are not based on forgetting formulas, this has not become a standard policy. The Math Department should be asked to reconsider the previous suggestion after analyzing the appropriate data in this report, which showed that the students are not achieving the MPOs that require the use of these formulas.

Even though a suggestion was made to the Math Department Curriculum Committee to include the chapter on radicals at a much earlier time, this has not been followed through on and instead an additional exam and several review sessions have been added to the course outline. The MTH 100 coordinators have decided to conduct a new follow-up SLO assessment study in Spring 2012 to see if covering the chapter on radicals earlier in the course makes a difference in the acquisition of this MPO as tested on the MTH 100 final exam. If it is determined that covering radicals earlier in the semester is beneficial, then the course coordinators will meet in the summer months to rearrange the topics of MTH 100 so that this change may be instituted in Fall 2012. In the meantime, a supplemental worksheet containing problems involving radicals will be made available to any instructor who would like to use it.

In order to facilitate the process of learning how to solve word problems, seminars should be conducted by the Math and Physics Department on a more regular basis than was done in the past, and these should be open to all instructors (part-time and full-time) to discuss strategies and best-practices for teaching how to solve word problems.

While the number of students who participated in the math-related workshop offered as part of this study was small, the positive feedback received from these students indicates that these types of workshops should be expanded and offered more often in the future. The Department should look for ways of increasing the attendance at these workshops such as offering incentives to students to attend them.

A supplemental worksheet containing problems requiring a verbal explanation will again be made available for use by all interested instructors (see Appendix H for a copy of one that has been distributed in the past).

MTH 100 – 19

All MTH 100 instructors should make a concerted effort to have their students use the WebAssign on-line homework software since the students who participated with and did well in WebAssign had a higher overall final course average than those who did not.

Since those students who used WebAssign and scored a WebAssign average of 60% or higher had a success rate of close to eight times higher than those who scored less than 60% in WebAssign, it appears that simply using WebAssign alone is not sufficient for passing the course. Rather it seems that doing well on this on-line homework and earning a grade of 60% or higher makes it more likely that one can succeed in MTH 100.

While this study found a negative correlation between using instructor-prepared class notes and success in the course, since 100% of the students who used these class notes found them useful, it is recommended that more studies on the use of this learning tool be conducted in the future and preferably with a larger student sample.

Since most MTH 100 instructors prefer keeping the number of exams in MTH 100 at 4 rather than 5, the Department should give this new policy further thought. Due to the fact that 5 exams takes away more time from instruction, lowering the number of exams to 4 seems much more practical and beneficial to students.

While most MTH 100 instructors found the use of review sessions before each exam beneficial to students, it is probably advisable that instructors conduct these review sessions only if time permits since they also indicated that there is not enough time to cover all of the course material.

MTH 100 – 20

Appendix A – MTH 100 Sample Scan Tron Answer Sheet

MTH 100 – Appendix A – 1

Appendix B – Word Problem Workshop Flyer, Handouts & Feedback Survey

MTH 100 – Appendix B – 1

Word Problems Strategies and Common Word Problems in MTH 100

Read and reread the problem until you UNDERSTAND it.

Use the variable x to represent the unknown quantity. Technically, you can use any variable to represent the unknown.

(a) If there is more than one unknown quantity, represent the other unknown quantity in terms of x as well.(b) If two scenarios are described in the problem such as there is a plane flying with the wind and later on it flying against the wind, use x to represent one unknown and y to represent the other unknown.

(a) Build an equation relating the unknown quantities. (b) If you use two variables to represent the two unknowns, then you will need to build two equations with these two variables (a system of two linear equations).

(a) Solve the equation. (b) If you use two variables, solve the system of equations.

Check your solution to see if it makes sense in the context of the problem.

1. What is the total value of 4 dimes and 2 nickels?

2. A drawer contains some dimes and quarters. The number of dimes is two less than three times the number of quarters. The total value of the coins is $4.20. Find the number of each type of coin in the drawer.

3. A chemist mixes a 50% acid solution with a 25% acid solution. How many milliliters of each solution should the chemist use to a 100 ml solution that is 30% acid?

4. Flying with the wind, a pilot flew 600 miles between two cities in 4 hours. The return trip against the wind took 5 hours. Find the rate of the plane in calm air and the rate of the wind.

5. Pump A can fill a pool in 6 hours and Pump B can fill the same pool in 3 hours. How long will it take to fill the pool if both pumps are used?


Workshop Assessment Form

Workshop Name: Solving Word Problems AlgebraicallyGiven by: Carlos CastilloDate: November 15, 2011

1. Did you enjoy the workshop?

Yes No

2. Did you learn something useful from this workshop?

Yes No

3. Should this type of workshop continue to be offered by the Math Department?

Yes No

4. Would you recommend attendance at this workshop to other students?

Yes No

5. Overall, how would you rate this workshop?

Excellent Good Fair Poor

6. Comments:


Appendix C – Excel Student Data Spreadsheet

MTH 100 SLOAT STUDENT DATA (Fall 2011)Legend: WebAssign? 0 = not used; 1 = used

Class Notes? 0 = not used; 1 = used

Last First ID # Section Web Assign?

WebAssignGrade

Class Notes

?

Final Grade

for course

Final Letter Grade

XXXXXX XXXXXX XXXXXX 4 1 71 1 29.65 WXXXXXX XXXXXX XXXXXX 4 1 100 1 99.75 AXXXXXX XXXXXX XXXXXX 4 0 0 1 43.65 FXXXXXX XXXXXX XXXXXX 4 1 46 1 31.7 WXXXXXX XXXXXX XXXXXX 4 1 83 1 59.85 DXXXXXX XXXXXX XXXXXX 4 1 23 1 16.4 WXXXXXX XXXXXX XXXXXX 4 1 100 1 89.5 AXXXXXX XXXXXX XXXXXX 4 1 99 1 61 IXXXXXX XXXXXX XXXXXX 4 1 100 1 94.15 AXXXXXX XXXXXX XXXXXX 4 1 8 1 8.2 WXXXXXX XXXXXX XXXXXX 4 1 67 1 62.55 DXXXXXX XXXXXX XXXXXX 4 1 27 1 73.72 CXXXXXX XXXXXX XXXXXX 4 1 18 1 70.3 CXXXXXX XXXXXX XXXXXX 4 1 100 1 96.9 AXXXXXX XXXXXX XXXXXX 4 1 13 1 7.45 FXXXXXX XXXXXX XXXXXX 4 1 96 1 67.8 CXXXXXX XXXXXX XXXXXX 4 1 100 1 91.6 AXXXXXX XXXXXX XXXXXX 4 1 100 1 89.5 AXXXXXX XXXXXX XXXXXX 4 1 94 1 70.65 CXXXXXX XXXXXX XXXXXX 4 1 100 1 93 AXXXXXX XXXXXX XXXXXX 4 1 26 1 12.1 WXXXXXX XXXXXX XXXXXX 4 0 0 1 24.25 WXXXXXX XXXXXX XXXXXX 4 1 95 1 62.4 DXXXXXX XXXXXX XXXXXX 4 1 10 1 1.5 NSXXXXXX XXXXXX XXXXXX 4 1 41 1 57.4 DXXXXXX XXXXXX XXXXXX 4 1 89 1 65.45 DXXXXXX XXXXXX XXXXXX 4 1 10 1 9.6 FXXXXXX XXXXXX XXXXXX 4 1 60 1 15.7 FXXXXXX XXXXXX XXXXXX 4 1 27 1 41.9 IXXXXXX XXXXXX XXXXXX 4 1 100 1 98.7 AXXXXXX XXXXXX XXXXXX 16 0 N/A 0 89.1 AXXXXXX XXXXXX XXXXXX 16 0 N/A 0 0 NG

Appendix D – Student Questionnaire

MTH 100 – Appendix C – 1

MTH 100 SLOAT Study Student Assessment Form (Fall 2011 )

Name:______________________________ ID Number:________________________________

Directions: The Math Department is participating in a MTH 100 student outcomes study in Fall 2011 to determine how well students learn by using online class notes and the WebAssign software package. Kindly fill out this brief questionnaire to help us assess the success rate of those students who do use these learning tools in comparison to those who don’t. Please be honest in your assessment since your responses to these questions will only be used by the coordinators of this study and not made available to your instructor in any way.

1. Did you use the course class notes made available to you by your instructor (either on-line or in paper form)?

Yes No

2. Did you find the notes useful?

Yes No

3. Did you use WebAssign to do your course homework?

Yes No

4. Did you like using this WebAssign homework software?

Yes No

5. Which method do you prefer to use for homework problems?

Online (like WebAssign) Problems from the textbook

6. Overall, would you recommend all math courses to use on-line course class notes?

Yes No

7. Overall, would you recommend all math courses to use on-line homework software like WebAssign?

Yes No

Thank you for your assistance completing this questionnaire. We wish you success in the course!

Appendix E – Instructor Questionnaire


MTH 100 Instructor Questionnaire (Fall 2011)Instructions: In order maintain the quality of the MTH 100 course and improve the success rate of its students, we are asking your assistance by filling out this questionnaire. Kindly fill out the questionnaire with your frank and confidential responses regarding MTH 100 and return it to the mailbox of Professor Soraida Romero on or before Friday, December 16, 2011. The course coordinators for MTH 100 will tally the responses and inform you of the questionnaire results in early 2012. Thanks so much for your cooperation.

1. Departmental Exams :

(a) How many exams would you recommend per semester? Less than 4 4 5

(b) How many questions would you recommend per exam? 15 – 19 Exactly 20 20 – 25

(c) How many multiple-choice questions should be on each exam? 8 10 All

(d) Should there be a review session in class before each exam? Yes No Only if time permits

(e) If you did conduct a review session before each exam this semester, did you find it useful? Yes No Not applicable

2. Homework :

(a) Do you require your students to do WebAssign homework? Yes No

(b) Is the final course grade for your MTH 100 students dependent on homework done (either WebAssign or homework from the text)?

Yes No

(c) If you do use homework to determine grades for MTH 100, what percent of the final course grade does homework count?

5% 10% 15% 20% Higher (specify %):____ 3. Why do you think the student success rate in MTH 100 is so low?

____________________________________________________________________________


____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

4. Do you have any suggestions or ideas on how to improve the success rate of student in MTH 100? ____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

5. Have you employed any innovative teaching strategy in your MTH 100 class that you found very successful and would recommend others try? If so, please describe it.

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

6. Do you have any suggestions or ideas on how to improve the course outline for MTH 100?____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

7. Other Comments:________________________________________________________

______________________________________________________________________

Thanks so much for your input!!


Appendix F – Results of Statistical Analyses

Appendix F APPENDIX F

MTH 100 – Appendix F – 1

- APPENDIX F (continued) -


APPENDIX F (continued)


Appendix G – Condensed Item Analysis of Student Achievement of MPOs

MTH 100 – Appendix G – 1

Appendix H – Supplemental Handout on Communications Skills

MTH 100 Communication Skills Handout

Part A: Open-ended questions.

Directions: Solve each problem and explain your answer.

(1) Is a function? Answer yes or no and explain why.______________________________________________________________________

(2) What value(s) must be excluded from the Domain of the function: ? Answer and explain why.______________________________________________________________________

(3) Are the lines given by parallel, perpendicular or neither? Explain your answer.

______________________________________________________________________

(4) Is a solution of the system ? Explain your answer.______________________________________________________________________

(5) Are the lines perpendicular? Explain your answer. ______________________________________________________________________

Part B: Multiple-choice questions.

Directions: Circle the appropriate letter for each question.

(6) Is a solution of ?(a) No, because when you substitute –1 for y and 5 for x, the equation is met.

(b) Yes, because the equation has an infinite number of solutions.

MTH 100 – Appendix H – 1

(c) Yes, because when you substitute –1 for x and 5 for y, the equation is met.

(d) No, because the equation has no solutions.(7) How can we determine if two lines are parallel?

(a) You have to graph the lines to make sure they don’t meet.(b) You must find the slope of each line and if these slopes are equal, they are parallel.(c) You must make sure that the slopes of the two lines are “negative reciprocals”.(d) You have to be told that the lines are parallel.

(8) Is x = 5 in the domain of the function ?(a) No, because when you substitute 5 for x, you would get a zero in the denominator

and this makes the function undefined at that value.(b) No, because this value does not make the numerator equal to zero.(c) Yes, because x is in the numerator and the denominator.(d) Yes, because the function is undefined at x = –6.

(9) Is a function?(a) No, because the number 2 is repeated.(b) Yes, because each x value has a y value.(c) No, because the number 0 is repeated.(d) Yes, because there are no repeats of the x values.

(10) Are these lines parallel, perpendicular or neither? (a) They are “neither” since they have different slopes. (b) They are parallel since they both have slopes of –5. (c) They are perpendicular since their slopes are negative reciprocals of each other. (d) You can’t tell by the equation alone and need to see the graph of each.

MTH 100 – Appendix H – 2

mth 100 sloat assessment study report (spring...

Documents