fig reading plus report - hartnell college€studentsrespondedeitherstronglydisagree,disagr...
TRANSCRIPT
INTEGRATING WEB-‐BASED LITERACY INSTRUCTION IN ENGLISH BASIC SKILLS: A PILOT REPORT
Angelo Bummer, Jaclyn Hatch, Dr. Sunita Lanka, Dr. Lee Ritscher, and Paul Yun
Overview
Reading Plus is a web-‐based literacy program designed to improve students’ reading rate, comprehension, and vocabulary. To do so, the program guides students through three main exercises called iBalance— eye focus exercises—ReadAround—vocabulary exercises— and SeeReader—comprehension exercises. Students who consistently engage with the program exercises on a weekly basis show significant reading improvement; those who inconsistently engage with the program show some improvement; and those who minimally engage do not show improvement in reading ability as assessed by the program. Engagement with the program is an issue that significantly affects student success, and it is recommended that required lab support should accompany a required Reading Plus component of a basic skills course.
Methodology
Reading Plus was integrated into the curriculum of three sections of English 253, an integrated reading and writing course two levels below transfer, during the fall semester of 2014 at Hartnell College. One section of English 253 was at the King City Center, and two sections were at the main campus. Overall, 72 students completed the pilot program. The program data included in this report measures student success with students’ progression through content levels throughout the semester and student results of beginning semester, mid-‐semester, and end-‐of-‐semester benchmark assessments, which assesses proficiency, vocabulary, and comprehension levels as well as students’ reading rate. Student success and program efficiency is also measured through student survey responses.
I. READING IMPROVEMENT
A. Content Level Assessment
One way students demonstrate improvement with reading is through mastering content levels; each content level corresponds to a grade level; for example, content level 1 correspond to reading material appropriate to the first grade. The chart below illustrates the number of content (or grade) levels students were able to progress through during the semester:
43% of students were unable to move on to more complex content
24% of students mastered one level
17% mastered two levels
15% mastered three levels
1% mastered four levels
57% of students mastered one or more content levels during the course of the semester.
33% of students mastered two or more content levels.
B. Benchmark Comparison
Reading improvement is also assessed through a comparison of benchmark results. Students completed an initial assessment called a benchmark, a similar benchmark half way through the semester, and then still another at the end of the semester—through comparison of these benchmark results at different points of the semester, we are allowed to measure success with the program.
31
17
12
11
1
0 levels
1 level
2 levels
3 levels
4 levels
0 20 40
Content Levels Mastered
Content Levels
1. The charts below illustrate the benchmark results from one section of English 253 at the King City Center:
Overall, students made a 0.7 gain in proficiency, a 0.2 gain in comprehension, and a 0.4 gain in vocabulary.
On average, students increased their reading rate by 32.3 words per minute during the course of the semester.
ProZiciency Level Comprehension Level Vocabulary Level
Benchmark 1 8.5 10.1 9 Benchmark 2 8.9 9.7 10 Benchmark 3 9.2 10.3 9.4
0 2 4 6 8 10 12
Axis Title
Benchmark Comparison
140.8
171.4 173.1
Reading Rate
Benchmark Comparison: Reading
Rate Benchmark 1 Benchmark 2 Benchmark 3
2. The charts below illustrate the benchmark results for two sections of English 253 at the main campus:
Overall, students made a 0.9 gain in proficiency, a 0.8 gain in comprehension, and a 0.4 gain in vocabulary.
On average, students increased their reading rate by 21.1 words per minute during the course of the semester.
ProZiciency Level Comprehension Level Vocabulary Level Benchmark 1 6.2 7.4 7.4 Benchmark 2 7 8 8.5 Benchmark 3 7.1 8.2 7
0 1 2 3 4 5 6 7 8 9
Axis Title
Benchmark Comparison
140.8
171.4 173.1
Reading Rate
Benchmark Comparison: Reading
Rate Benchmark 1 Benchmark 2 Benchmark 3
C. Student Survey Results
Reading improvement is also measured through student perception of learning. Four statements in our student survey (numbers 7,8,9, and 15) were designed to measure how students felt Reading Plus contributed to their success. The statements were “I could tell the Reading Plus exercises helped improve my reading skills;” “After using Reading Plus, I felt more confident in my reading skills than I did before;” “Using Reading Plus makes me want to read more;” and “ Overall, Reading Plus is very practical and helps me with my work in other classes.” Students responded either strongly disagree, disagree, neutral, agree, or strongly agree.
The graphs below illustrate student responses to the questions listed above:
65% of students agreed or strongly agreed
30% remained neutral
5% disagreed
58% of students agreed or strongly agreed
37% remained neutral
5% disagreed or strongly disagreed
0 4
22 29
18
Strongly Disagree Disagree Neutral Agree
Strongly Agree
Q 7
0 10 20 30 40
7. Could tell exercises in Reading Plus helped improve reading skills
Series1
1 3
27 24
18
Strongly Disagree Disagree Neutral Agree
Strongly Agree
Q 8
0 5 10 15 20 25 30
8. After Using Reading Plus I felt more conHident about my reading skills than I did
before
Series1
42% of students agreed or strongly agreed
40% remained neutral
18% disagreed or strongly disagreed
56% of students agreed or strongly agreed
38% remained neutral
6% disagreed or strongly disagreed
2
11
29
22
9
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
Q 9
0 10 20 30 40
9. Using Reading Plus Makes me want to read
more
Series1
1 4
28 28
14
Strongly Disagree Disagree Neutral Agree
Strongly Agree
Q15
0 5 10 15 20 25 30
15. Overall, the Reading Plus program is very
practical and helps me with my work in other classes.
Series1
II. STUDENT ENGAGEMENT
Program data shows that only 11 of the 72 students who completed English 253 also completed all of their Reading Plus assignments, and that an additional 12 completed at least 80% of their assignments. However, 49 students completed less than 80% of their assignments, with 16 of them completing less than 20% of their assignments.
32% of students completed all of their assignments or at least 80% of them
68% of students completed less than 80% of their assignments
Questions 16 of the student survey was designed to assess student perceptions of completing their assignments. They responded all (A), most (B), half (C), less than half (D), or some (E).
50% of students felt that they completed most or all of their assignments
50% of students felt that they completed half of their assignments or less
11 12 14
19 16
100% 80%-‐99% 50-‐79%
20%-‐49% Less than 20%
0 5 10 15 20
Assignment Completion
Assignment Completion
9
32
18
6
6
0 5 10 15 20 25 30 35
A (All)
B (Most)
C (Half)
D (Less Than Half)
E (Some)
Q16
How many assignments did you complete?
Series1
III. NUMBER OF ASSIGNMENTS
We used two different approaches to assigning Reading Plus exercises. The first approach was to assign a consistent number of exercises each week—four SeeReader, two ReadAround, and two iBalance exercises. The second approach was to vary the number of exercises each week at the discretion of the instruction—for example, students may be assigned two SeeReader and two ReadAround exercises one week, and the be assigned two iBalance and two See Reader exercises the next week.
Student survey questions 11, 12, 13, and 14 allowed students offer input on the number and mode of exercises assigned.
Below are student responses to our first approach—a consistent number assigned each week.
37% agreed or strongly agreed; 4% agreed or strongly agreed
41% remained neutral 25% remained neutral
22% disagreed or strongly disagreed 71% disagreed or strongly disagreed
4
10
26
12
11
Strongly
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30
11. The number of weekly
assignments was good
11
14
24
13
1
1
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30
12. There should be more weekly exercises
12
34% agreed or strongly agreed 11% agreed or strongly agreed 42% remained neutral 19% remained neutral 24% disagreed or strongly disagreed 70% disagreed or strongly disagreed Below are student responses to our second approach—varying the number of assignments from week to week.
23% agreed or strongly agreed 80% agreed or strongly agreed 11% remained neutral 20% remained nuetral 66% disagreed or strongly disagreed
5
8
22
10
8
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30
13. Too many weekly exercises;
I was overwhelmed
13
16
21
10
5
1
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30
14. Assign more weekly exercises; wrap up the program mid-‐semester
14
6
6
2
3
1
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 2 4 6 8
11. The number of weekly assignments
was good
11
0
0
4
9
7
Strongly
Disagree
Neutral
Agree
Strongly Agree
0 5 10
12. There should be more weekly
exercises
12
20% agreed 16% agreed 25% remained neutral 26% remained neutral 55% disagreed or strongly disagreed 58% disagreed or strongly disagreed
4
7
5
4
0
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 2 4 6 8
13. Too many weekly exercises; I was overwhelmed
13
7
4
5
3
0
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 2 4 6 8
14. Assign more weekly
exercises;wrap up the program mid-‐
semester
14
IV. STUDENT ACCESS AND SUPPORT
Questions 5, 6 and 10 in our survey were designed to assess student access and support.
Question 5 addresses student access to the internet, which they need in order to complete their assignments.
64% of students either agreed or strongly agreed
25% of students remained neutral
11% either disagreed or strongly disagreed
Students were given in-‐class support from their instructor; additional students were able to contact the reading plus coordinator electronically for additional support. Question 6 addresses the extent to which students received adequate support.
73% of students either agreed or strongly agreed
19% of students remained neutral
8% of students either disagreed or strongly disagreed
Question 10 assessed whether or not students would make use of a lab to help them complete their assignments.
1 7
18 23 24
0 5 10 15 20 25 30 Strongly Disagree
Neutral Strongly Agree
Q 5
5. Except On Occassion, I was able to use a computer with
Internet Access
Series1
1 5
14 30
22
0 5 10 15 20 25 30 35
Strongly Disagree Disagree Neutral Agree
Strongly Agree
Q 6
6. I was able to get support quickly if and when I had
problems
Series1
36% of students agreed or strongly agreed
46% of students remained neutral
18% of students disagreed or strongly disagreed
2
11
35
18
9
0 10 20 30 40
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
Q10
10. A tutoring lab would have been a great support
Series1
V. PROGRAM PERFORMANCE
Student survey questions 2,3, and 4 addressed the functionality of the program itself.
84% agreed or strongly agreed
14% remained neutral
2% disagreed
67% agreed or strongly agreed
28% remained neutral
5% disagreed or strongly disagreed
0
1
9
36
17
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30 40
2. The placement test gave clear instructions
2
1
3
20
38
11
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 10 20 30 40
3. The program placed me at the right reading level
3
62% agreed or strongly agreed
23% remained neutral
15% disagreed or strongly disagreed
6
5
17
27
18
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0 5 10 15 20 25 30
4. After the Hirst two weeks, I did not encounter any technical problems
4
RECOMMENDATIONS
• Required lab support should be integrated into the curriculum to increase student engagement and overall student success with the program.
• Instructors should assign a consistent number of weekly exercises.
• Instructors should assign benchmark assessments during the beginning of the semester and before finals week.
• Instructors should collect program data at the end of the semester by saving the following reports: Class Instructional Summary Report; Class Progress Report; Class Benchmark Comparison Report; Student Instructional Summary Report