Enhancing the student learning environment

Robert Garrick, Ph.D., P.E. – Rochester Institute of TechnologyElizabeth Dell - Rochester Institute of TechnologyLarry Villasmil, Ph.D. – Rochester Institute of TechnologyRobert P. Lillis – Evalumetrics Research

TABLET PCS - DIGITAL PEN-BASED TECHNOLOGY USE AND THE IMPACT ON

EDUCATING ENGINEERS

1

Contents• Problem Statement• Education Research – What has been Done• Instructional and Technology Domain• Methodology• Blended Experiment Design, Why

• Examples• Findings• Next Steps

2

Problem Statement• Although a significant number of programs exist to attract and retain

students into engineering/engineering technology programs, five year graduation rates only range from 10% to 40% of the initial entering freshman engineering cohort [1].

• Compared to others, engineering programs have the longest time to completion [2].

• Students that leave an engineering program are seldom replaced [3].

• The majority of the attrition occurs during the first three years [4].

• A primary cause of the high attrition rates in engineering/engineering technology programs is the perception that the learning environment is often un-motivating and unwelcoming. [5]

3

1. (Maura J. Borrego, Miguel A. Padilla, Guili Zhang, Consortium for Student Data Exchange). 2. (Fortenberry, et. al., 2007; Ohland, et.al.,2008; Seymour and Hewitt, 1997) 3. (Ohland et. al., 2008). 4. (Borrego, Padilla). 5. (Bergval, Sorby, and Worthen, 1994; Busch-Vishniac and Jarosz, 2004; Harris, et. al., 2004; Salter and Persaud,

2003; Sax, 1994; Vogt, Hocevar, and Hagedorn, 2007).

Solutions – What has been Done

1. (Karl A. Smith & Goldstein, 1982; Karl A. Smith, Johnson, & Johnson, 1981; Felder, 1995; Johnson, Johnson, & Smith, 1991, 1998, 2007; MacGregor, Cooper, Smith, & Robinson, 2000; Millis & Cottell, 1997; Prince, 2004; K.A. Smith, Cox, & Douglas, 2009; Karl A. Smith, Sheppard, Johnson, & Johnson, 2005; Terenzini, Cabrera, Colbeck, Parente, & Bjorklund, 2001)

2. (Hake, 1998).3. (Crouch, 2001)

4

• Creating an engaging and cooperative learning environment has been a key issue in engineering education for many decades [1].

• A seminal and comprehensive study [2] found a substantial improvement in conceptual understanding of physics principles through interactive engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback.

• Key elements of interactive engagement include [3]: a) cooperative learning activities b) group problem solving.

A Local Pilot Study (Pilot #1 study)

5

• The Pilot #1 study established [1]:

a) students who had a lower GPA entering the class (GPA >2.0, but <3.0) experienced the most significant increase in test scores in the class.

b) The class involved in Pilot #1 study was a late freshman/early sophomore engineering introductory class that had traditionally a high (>15%) rate of withdrawals, and D or F grades. Over the three year pilot #1 study, the withdrawal, D and F grades rate fell below 10%. (Parthum, 2009).

Solutions – What has been Done

1. (Parthum, 2009)

A Typical Introductory Eng/ET Course • The Pneumatic and Hydraulic Systems course is offered at Rochester

Institute of Technology (RIT) to second year students. • The course has had a high rate of low grades and withdrawals

averaging 22.8% over the last ten times it was taught

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

1 2 3 4 5 6 7 8 9 10

% D, W, or F grade

6

Solutions – A Diagnostic

Solutions

• A previous study (Pilot study #1) within the RIT department of Mechanical and Manufacturing Engineering Technology (MMET) demonstrated that Tablet PC based lectures helped students who had lower GPAs [1].

• A Tablet PC lecture environment has also been reported to increase student interest and involvement [2].

7

A Technology Rich Learning environment

1. (Parthum, 2009)2. (Berque, Johnson, & Jovanovic, 2001; Birmingham, DiStasi, & Welton, 2009; Chidanandan et al., 2007;

Chidanandan et al., 2008; Johri & Lohani, 2008; Lohani, Castles, Johri, Spangler, & Kibler, 2008; Sneller, 2007; Stanton, 2008).

Conceptual Framework of selecting the technology tools to enhance the learning environment

Benlloch-Dualde, J.V. ; Buendía, F; Cano J.C; Supporting instructors in designing Tablet PC based courses

8

Instruction/Technology Domain

Benlloch-Dualde, J.V. ; Buendía, F; Cano J.C; Supporting instructors in designing Tablet PC based courses

9

What and Why actions of the InstructorInstruction/Technology Domain

Benlloch-Dualde, J.V. ; Buendía, F; Cano J.C; Supporting instructors in designing Tablet PC based courses

10

Tablet PC specificInstruction/Technology Domain

• Activities to Engage Students– Delivering Course Content … Multiple screens and display hardware. – Gathering Content feedback… Polls, group/individual practice.– Course Laboratory Practice… Using Tablet/Collaborative software

environment. – Course Review Session Competition… Team environment

11

Blended Experiment DesignMethodology

PREVIOUS SLIDECURRENT SLIDE

AUXILIARY SCREENVIDEOS / SUPPORT MATERIAL

12

Delivering Course Content – Multi-display FeaturesMethodology – Examples

13

Gathering Content Feedback – Polls, group/individual practice

Methodology – Examples

How do I connect this circuit? (use a different color for each line)

Course Laboratory Practice – Using Tablet/Collaborative software environment

Methodology – Examples

Competition Structure:• Two teams of students (Red team – Blue team)• Three subgroups per team (six total groups)• Six “rounds” of competition (timed)• Three problems per round (one problem per group)

Results:• Students exposed to 18 different problems for review in one class period• Students electronic notebooks have their solutions and those of other groups to review

different approaches along with instructor solutions • Students receive problem solving suggestions from team members• Instructor able to monitor progress of teams with the collaborative software monitoring

feature and personalize instruction.

Opportunity:• Utilize Tablet / Collaborative Software environment to create a competitive team based review

sessionAdvantages:• Students involved (engaged) in completing problems rather than “traditional” instructor solved

problems, timely feedback on errors, peer to peer mentoring with group work, ability to solve multiple problems simultaneously

Blue Team Red Team

Groups 1 & 4

Groups 2 & 5

Groups 3 & 6

15

Course Review Session Competition– Team environment

Methodology – Examples

Study Period Class # of Students Experimental design

Evaluation Method

Fall 2009 Pneumatics and Hydraulics

25 Computer lab vs. TLT studio

Post Survey and focus group

Fall 2010 Pneumatics and Hydraulics

75 Computer lab vs. TLT studio

Post Survey and focus group

Fall 2010 Dynamics Recitation

80 TLT studio (2 classes) – Standard room 2 classes)

Post Survey and focus group

Fall 2010 Thermo-Fluids 25 TLT Studio Post Survey and focus group

Spring 2011 Applied Fluid Dynamics

25 TLT Studio Post Survey and focus group 16

Pilot Study #2 - ~ 230 studentsFindings - Scope

17

• ~ Even distribution among 2nd, 3rd and 4th/5th year students• ~ Normal distribution of entering GPAs• All full time students.• Most students between 18 and 22 years old with 3.7% above 23.During Lectures:• 83% reported usually taking notes using a pen/pencil and notebook. • 8% reported usually taking notes using a laptop /desktop computer.• 14% reported usually not taking notes during lectures

1st year 0%2nd year 38%3rd year 31%4th year 10%5th year 21%

Academic Year Distribution

2.0 to 2.5 11%2.5 to 3.0 36%3.0 to 3.5 36%3.5 to 4.0 17%

Entering Class GPA Distribution

Student DemographicsFindings - Scope

18

• 14% of the students preferred the traditional/standard lecture learning environment.

• Students preferred the following alternative lecture environment features: ( % of preferred/strongly preferred)• Instructor’s notes directly over the presented PowerPoint during lecture (81%)• Animations or videos incorporated into the PowerPoint lecture (78%)• Group problem solving work (74%)• Example problems completed by the instructor (72%)• Tablets, collaboration software and multi-screen projection (71%)• Real time integration of lecture and student’s personal notes (68%)

• These preferences for the lecture environment features were consistent over the academic year but such preferences increased with student academic year.

Pilot #2 ResultsFindings

19

• Students responded that they were more likely to take notes in the technology rich lecture environment particularly third, fourth or fifth year students including students with the lowest GPA entering the class.

• Students reported that they were more likely to use these notes for both homework and pre-test reviews primarily the students with the lowest GPA who reported improved comprehension of the material.

Pilot #2 ResultsFindings

• Students preferred solving problems in class using the technology rich environment.

• Students overwhelmingly agreed that working virtually in groups was an effective method to do in-class problem solving.

• The ability to make corrections seeing them immediately and the ability to watch the process were the most common positive comments.

• Recalling Hake, “interactive engagement” is “designed in part to promote conceptual understanding through engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors” (p 65) (Hake, 1998).

• The focus group facilitator asked questions about the use of the tablet PCs, group work, note taking, preparation for tests and overall learning.

• An independent evaluator reviewed video recordings of the focus groups and scored each student remark as positive or negative in one of several categories.

20

Focus Group Comments Positive

Comments Total

Comments Proportion

Positive Active Learning 11 11 100.0% Learning 4 4 100.0% Engagement 14 15 93.3% Prepare for Tests 4 5 80.0% Take Notes 20 28 71.4% Group Work 21 30 70.0% Involved in process 5 10 50.0%

Pilot #2 Results – Focus GroupFindings

• Overall, the students recommended using the technology rich lecture environment.

• Students with a lower entering GPA perceived a greater benefit from this learning environment.

• Results appear in agreement with those found in the seminal work of Hake and others that an interactive and engaging learning environment can result in improved student learning of the material.

• A technology rich environment allows the instructor to implement an interactive and engaging learning environment using digital media, Tablet PCs and collaborative software.

• A technology rich environment also increases student likelihood of note taking and using these notes (especially for the attrition vulnerable population with lower GPAs).

21

Main ConclusionsFindings

Next Steps• Continue our studies of this environment with focus on:• Students with a lower entering GPA• Rochester Institute of Technology’s Deaf and hard of hearing students

• Ability to see note-taker’s notes live• 3 screen display • Ability to play-back pen-stroke by pen-stroke each slide

• Underrepresented groups• Anonymity of technology rich environment

• Work with others external to Rochester Institute of Technology to support rigorous engineering/STEM education research grant opportunities

22