[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

2009 International Conference on Engineering Education (lCEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

Measuring Critical Thinking Skills in Engineering

Mathematics Using Online F arums

Seibu Mary Jacob, Betsy Lee and Gillian Rosemary Lueckenhausen Swinburne University of Technology (Sarawak Campus)

Kuching, Sarawak, Malaysia {sjacob, blee, glue}@swinburne.edu.my

Abstract-The development of critical thinking skills is vital in the teaching and learning of Engineering Mathematics. Forty students of the subject were exposed to two ill structured problem solving sessions over the Blackboard discussion forums. They also attempted the CCTST (California Critical Thinking Skills Test) test to measure their general critical thinking skills prior to and after the forum sessions. The critical thinking is measured using an adapted model and the progress in the level of critical thinking is checked from first forum to the second. Statistical tests are performed to check if there is a significant progress from the preCCTST to the post-CCTST scores. Results indicate from the forum postings that critical thinking has progressed from first forum to the second, and results from the CCTST scores confirms significant differences.

Keywords- critical thinking skills, online discussion forums, CCTST, engineering mathematics

I. INTRODUCTION

Critical thinking is not a luxury but a requirement that should not be neglected [1] in education. One of the greatest experiences for students in higher education is to have the opportunity to think freely and challenge other students' ideas with their own. Online discussion forums serve as powerful mechanism for exploring and communicating mathematical concepts and fostering academic community in any mathematics course [2]. Research has revealed that structured asynchronous discussion forums or online discussion forums (ODFs) have tremendous potential to foster "communities of inquiry" and trigger a critical spirit [3]. Therefore this study was designed to investigate into the effectiveness of using ODFs to encourage and develop critical thinking skills (CTS) in engineering mathematics. The objectives were: (1) to measure critical thinking skills exhibited during the problem solving sessions over ODFs, using a suggested model; (2) to check whether CTS improves after students participate in the problem solving sessions over the ODFs, using CCTST as an outcome measure.

II. LITERATURE REVIEW

A. Engineering mathematics/education and critical thinking Critical thinking is the art of analyzing and evaluating

thinking with a view to improving it [4]. ABET Engineering Criteria 2000 (EC 2000) program outcomes require that graduating engineers are able to communicate effectively, understand the global and societal context of their work, are aware of contemporary issues, and are able to engage in lifelong

978-1-4244-4844-9/09/$25.00 ©2009 IEEE 225

learning [5]. The development of critical thinking skills is essential if a student is to achieve these outcomes.

B. Rl structured problem solving and critical thinking "Teachers are problem posers and problem-solvers; they are researchers, and they are intellectuals engaged in unraveling the process both for themselves and for their students" -p 112 [6]. Jonassen [7] has emphasized in the engineering context the fact that ill structured problem solving knowledge is distributed among team members, since knowledge exists not only in the heads of learners, but also in the conversations and social relations among collaborators. Cognitive abilities may be distributed across time and minds. Individual students cannot learn to fulfill all problem solving roles. Rather, students could learn how to interact with others. Problems to encourage critical thinking and dialogue should: (1) be not so difficult to get started; (2) require more than a straightforward algebraic attack to engage in sophisticated problem solving; (3) give students some practice to develop mathematical notation; (4) be open ended so that students could be encouraged to formulate questions and start exploring their own.

C. Discussion Fornms and critical thinking Research studies [8], [9], [10] have been done to investigate if

technology contributes to critical thinking. Proponents of collaborative learning claim that the active exchange of ideas within small groups not only increase interest among the participants, but also promotes critical thinking", according to Gokhole [11]. Intellectual online discussions should be encouraged [3] in large class environments, so that students enjoy high quality discussions and voice their thoughts and where an instructor can guide, model, and develop students' CTS. Unless the instructor (moderator) play a pedagogical role in teaching , modeling and prompting Socratic dialogues, an interactive session alone cannot effectively help students become independent critical thinkers over a short period of time.

D. CCTST and critical thinking The CCTST (Form 2000) is a standardized tool of critical

thinking skills and reasoning problems [9]. The instrument is discipline neutral, and aimed at undergraduates, graduates and adult professionals. The 34 multiple choice items in the CCTST cover the domain of the five cognitive skills - analysis, evaluation, inference, deductive reasoning and inductive reasoning. One point is given for each correct answer, and the

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

maximum score is 34. The test reliability (calculated using the Kuder Richardson-20 (KR-20) internal consistency method) of CCTST Form 2000 is 0.78 - 0.84 [10].

III. RESEARCH STUDY

The study assumes that [9]: (1) Critical thinking skills need to be made explicit;(2) Critical thinking skills need to be practiced and reinforced;(3) Critical thinking is a reflective, nonlinear process that involves thought and action;(4) Critical thinking is both a process and an outcome that can be taught and practiced;(5) Critical thinking can be structured and supported through online discussion forums, which provide an opportunity for interactivity, reflection, and critical inquiry among participants

The study involved 40 students of a fIrst year university Engineering Mathematics unit spanning 14 weeks, in a university in Malaysia. The students were divided into 13 collaborative groups of 3 or 4 for the purposes of tutorial work and online discussion forums. The discussion forums are made available to the collaborative groups under the Blackboard Learning System (http://www.blackboard.com) [8]. These sessions were planned to facilitate student engagement and interaction outside of the classroom [2]. These forums formed part of the 10% of the internal assessment for the unit. A problem solving task was assigned to the students over the Discussion Forum during Week 5 (Forum 1) and Week II(Forum 2) of the semester and given a deadline of seven days.

The topic of the problem was aligned with the course content. But it was different from the standard tutorial problems encountered in the unit. Instead, they were application problems in the engineering context. Figure 1 shows the question posted under Forum l.The students, in each group, were asked to discuss the scope of the problem and to collaborate together towards the solution. The instructor encouraged these collaborative sessions by moderating (scaffolding) the discussion thread in order to stimulate both mathematical learning and mathematical understanding. Socratic questioning techniques [3], [12] were used by the moderator to lead students into critical thinking, which in tum, should help them to interpret and manipulate aspects of the problems and the solutions. These questions include, but are not limited to: questions for clarifIcation, probing questions, and questions that direct students to extend, generalize, or make connections between the mathematics in different postings. The moderator also contributed by pointing out questions or issues that have not been resolved, as well as summarizing, synthesizing, and generalizing observations that have similar themes.

A sample posting: "That was great job, guys! Good team work ... Your equation T3z=T3*3/5- do you think you can bring in W using this, because of the vertical equilibrium at C? So, will the components of T3 have W as a constant? Then considering the equilibrium at P for the x and the y components, could you frame equations?"

226

The discussion forum postings were downloaded and subjected to qualitative and quantitative analysis. One posting (message) was considered as the unit of analysis.

II •• ICI CI.II - ,.,..." ...... " ....

A ;r

A simple derrick crane is constructed as indicated. The wires AP and BP are in tension, and the arm of the derrick, PC is loaded with a weight W at C. The x and y components of the forces at P are always in equilibrium. Determine the range of the angle theta that will ensure that the tensions Tl and T2 are always positive and hence the wires will not slacken.

Figure I. The question for Forum 1

IV. RESULTS

A. Descriptive Statistics for the Discussion fornm postings The model for the analysis of the discussion forum postings

in the current study was adapted from the model proposed and tested by Perkins and Murphy [13]. The category names from the model of Perkins and Murphy and the concepts of Paul and Elder have been used to come up with the model which identifIes critical thinking in problem solving as shown in Table 1[14]. The indicators have been modifIed, in comparison to the model by Perkins and Murphy to suit the identifIcation and measurement of the critical thinking during the stages of problem solving in Mathematics. The model includes the indicators and description to each category. The higher levels of critical thinking are associated with the stages of Inference and Strategies. Sample postings are shown in Figures 2 and 3.

Hypothesis 1: The CTS shows an improvement from Forum 1 to Forum 2; Or ODFs can encourage CTS of students. The postings were coded into the four categories, using the indicators in the model as guides. In cases where more than one critical thinking process appeared within a posting, only one code is associated, which seemed to be the most important in the context. Some postings were coded as 'unclassifIed', if they were personal or social in nature, and not part of the discussion and analysis of the problem. Table 2 shows the descriptive statistics of the postings in Forum 1 and Forum 2. Cl, AI, 11, SI and U1


represent the categories Classification, Assessment, Inference, Strategies and Unclassified of Forum 1. C2, A2, 12, S2 and U2 represent those of Forum 2. Clearly the means of C2, A2, 12 and S2 are higher than those of Cl, AI, 11, and SI. There is seen an increase in the statistics of the higher categories of critical thinking, namely inference and strategies. The hypothesis is thus proved true. The external prompts, stimulus, guidance of the moderator (instructor) played an important role in encouraging group discussion and CTS during the online discussion forum sessions

INFERENCE

Forum: Forumi for Critical Thinking Date: 25 Aug 2008 Author: Student 1 Subject: Re: Question

Times Read: 10

Since the x and y components at P are at equilibrium the resultant force due to Tl and T2 is equal to T3.

Tl(X) + T2(X) + T3(X)

Tl(y) + T2(y) + T3(y)

Tl(z) + T2(z) + T3(z). The components of the resultant force is Sqrt of 52; therefore this should have an equal magnitude to T3. But the relation between the angle and the tensions is what i do not understand.

Figure 2. Sample Posting-inference

STRATEGIES

Forum: Forum 1 for Critical Thinking Date: 28 Aug 2008 Author: Student 2 Subject: Re: The Question

Times Read: 8

I try to convert all of the forces into xy plane first, then I get these equations ... Tlx= -Tl(4/-v6 I) Tly= -Tl(3/-v6 I) T2x= -T2(4/-V6 I) T2y= T2(3/-V6 I) T3x= T3(4/5)cosO T3y= T3(4/5)sinO then, I try to apply SUM of Fx=O T3x - Tlx - T2x = 0 Tlx + T2x = T3x -Tl(4/-v6 I) - T2(4/-v6I) = T3(4/5)cosO ............... ( I) next is SUM of Fy=O T3y+T2y - Tly = O Tly - T2y = T3y -Tl(3/-v6 I) - T2(3/-v6 I) = T3(4/5)sinO .................. (2) finally maybe we can compare (2) n (1), then maybe we can get the answer",but I' m not sure also, hehehehe . . .

Figure 3 . Sample Postings- strategies

B. CCTST results

Hypothesis 2: There is no significant difference in the CTS of students before and after participation in the ODFs.

227

TABLE I. MODEL FOR IDENTIFYING ENGAGEMENT IN CRITICAL THINKING DURING PROBLEM SOLVING

Clarification

Formulates the problem precisely and clearly. Analyses, Identifies one or Identifies Defines or negotiates or more underlying relationships criticizes the discusses the assumptions in among the different definition of scope of the the parts of the parts of the relevant terms problem problem problem

Assessment

Raises vital questions and problems within the problem. Gathers and Provides or asks Make value assesses relevant for reasons that judgment on the information. proffered assessment criteria

evidence is valid or argument or or relevant. situation.

Inference

Reasons out based on relevant criteria and standards

Makes Arrives at well Makes Frames appropriate thought out generalizations relationships deductions from conclusions from relevant among the discussed results. results. different parts

of the problem.

Strategies

Thinks and suggests open mindedly within alternative systems of thought.

Propose specific Discuss possible Evaluate possible Predicts steps to lead to steps. steps. outcomes of the solution. proposed steps.

TABLE II. DESCRIPTIVE STATISTICS OF FORUM 1 AND FORUM 2

Category Minimum Maximum Mean S.D

C I 0 4 1.17 1.093

Al 0 5 1.24 1.220

I I 0 3 0.54 0.897

S I 0 3 0.49 0.810

V I 0 9 0.83 1.935

C2 0 5 1.46 1.343

A2 0 7 2.76 2.095

12 0 7 1.05 1.396

S2 0 4 1.27 1.265

U2 0 8 1.32 1.955

In the California Critical Thinking Skills Test (CCTST), critical thinking skills are identified as Analysis, Inference, Evaluation, Induction and Deduction [15].The students were required to take the 45 minute CCTST in class under the supervision of the researcher, at the beginning and the end of the 14-week unit, during which the online discussion forums took place. The CCTST was administered before and after the discussion forums, as an outcome measure. The CCTST instrument was chosen over others because of its validity and reliability for measuring critical thinking and interdisciplinary aptitude [8]. Figure 4 shows the mean CCTST scores - pre and

2009 International Conference on Engineering Education (ICEED 2009), December 7-8, 2009, Kuala Lumpur, Malaysia

post for the total, analysis, inference, evaluation, induction and deduction.

The post scores are generally higher than the pre scores in all the categories of critical thinking. But the analysis and deduction skills do not show very marked differences. The total mean score for the pre-CCTST was 13.31 and the total mean score for the post CCTST was. 15.97. Paired samples t-test performed on the pre and post CCTST scores for the 40 students was significant (p=0.00) at 5% significance level. Thus hypothesis 2 was rejected at 5% significance level. The results confirm those from the studies of [3] and [12] that learning is accomplished as students interact with their peers or instructor and benefit from combining their levels of expertise, offering support, distributing the thinking load and confronting alternative points of view.

Hypothesis 3: There is no variation in CCTST results among the 13 groups of students.

-< -< l> l> " " sa sa " " 0> 0> 0> 0> 0. 0.

tg t;;- � � c: c:

!l. !l. 0 w· w· ,;- o· " � c- O> 0>

" " " '" c- O>

0> g- '" c- O>

� !!i " '" '" � !!i " '" 0' !!i !!i 0' !!i " n: " n:

Figure 4. Box plot showing the pre and post mean CCTST scores

20

� 15 co. .. I CUj; 10

:::!:+' o I- 5

20

o

r-r-

r- __

r- -r-r-- -

In 1 2 3 4 5 6 7 8 9 11 12 13

Group

r- ----

r-_r- r- r-r--

1 2 3 4 5 6 7 8 9 11 12 13

Group

Figure 5. Graphs showing the pre and post total CCTST scores among groups

228

There was found variation in the pre- total CCTST among the groups (p=0.02) at 5% significance level. But no variations were significant for the post total CCTST among the groups (p=0.595) at 5% significance level. Figure 5 shows that though there were variations in the pre CCTST scores, the post CCTST scores have slightly improved are reflected almost uniformly among the 13 groups.

V. CONCLUSION

The study was an attempt to measure CTS in an engineering mathematics unit. The participation in the ODFs varied in the different groups and it was a challenge for the moderator to monitor the postings in the different groups on a daily basis during the one week period. It was clear that student-teacher interactions communicate high standards, provide students with regular timely feedback, and facilitate student-student interactions.

The quality of the online discussions have improved in critical thinking skills from Forum I, in the first half of the semester to Forum 2, in the second half of the semester. The qualitative analysis using an adapted model showed how the discussions of the students moved from the lower to the higher phases of critical thinking. The CCTST results indicated that students have progressed in their levels of CTS from the beginning to the end of the 14-week long semester. There was variation in the general CTS, as was measured by the CCTST in the beginning of the unit. But towards the end there is seen a progress in all the groups and no variation was significant in the total post CCTST scores.

The study was an initial phase of the measurement of CTS using the CCTST to confirm the progression of the CTS measured by the adapted model in the ODF postings. The study could be extended to include attitude to mathematics (measured pre and post), and also the unit final exam results.

ACKNOWLEDGMENT

The study has been funded under the Seed Grant scheme of the Swinburne University of Technology (Sarawak Campus), Malaysia. We extend our thanks to the institution for supporting us so as to make the study a success.

REFERENCES

[ I] H. I. AkyOza and S. Samsa, "The effects of blended learning environment on the critical thinking skills of students," ,Procedia Social and Behavioral Sciences I (2009), pp. 1744-1748,2009.

(2) E. L. Johnson and K. H. Green, "Promoting Mathematical Communication and Community via Blackboard," PRIMUS, vol. 17(4), pp.325-337, 2007.

[3] Y. C. Yang, "A catalyst for teaching critical thinking in a large university class in Taiwan: asynchronous online discussions with the facilitation of teaching assistants," Education Tech Research Dev, vol. 56, pp. 241-264, 2008.

[4] R. Paul, and L. Elder, The Miniature Guide to Critical Thinking: Concepts & Tools. Foundation of Critical thinking., 2006. Accessed at http://www.criticalthinking.org/concepts_tools.pdf


[5] B. Gunnink and K. L. S. Bernhardt, Writing, critical thinking, and engineering curricula, 32nd ASEEIIEEE Frontiers in Education Conference, F3H2-7, 2002 IEEE

[6] A. Lieberman, A., & L. Miller , "Teacher development in professional practice schools," Teachers College Record, vol. 92(1), pp. 105-122 in T.P. Carpenter, M. L. Blanton, P. Cobb, M. L. Franke, 1. Kaput, and K. McCain, Scaling Up Innovative Practices in Mathematics and Sciences. 2004. Available online at www.wcer.wisc.eduJ ncisla. Accessed July 14,2009.

[7] D. Jonassen, "Everyday Problem Solving in Engineering: Lessons for Engineering Educators," Journal of Engineering Education, 2006. Retrieved from http://findarticles.com/p/articles/mi_qa3886/is_200604/ ai_n I7186571/print in June 2008

[8] 1. F. Copolla , BA Thomas, and J. D. E. Thomas, The Effect of Technology Integration on Critical Thinking Skills in a Graduate Introductory Information Systems Course. Proceedings of the Information Systems Education Conference (ISECON) 2001 v 18 (Cincinnati): §17a. Retrieved from: http://proc.isecon.org/2001l17a1index.htrnl

[9] P.L. Duphorne, and C.N. Gunawardena, "The Effect of Three Computer Conferencing Designs on Critical Thinking Skills of Nursing Students," American Journal of Distance Education, 19:1,37 - 50, 2005.

[10] J. Wessel, and R. Williams, "Critical thinking and learning styles of students in a problem-based, master's entry-level physical therapy program," Physiotherapy Theory and Practice, vol. 20(2), pp.79 - 89, 2004.

[11] A. A. Gokhale, "Collaborative Learning Enhances Critical Thinking. " Journal of Technology Education, Fall, 1995, http://scholar.lib.vteduJejournals/JTE/v7nllpdf/gokhale.pdf in J.F .Copolla , B.A. Thomas, and J. D. E. Thomas, The Effect of Technology Integration on Critical Thinking Skills in a Graduate Introductory Information Systems Course. Proceedings of the Information Systems Education Conference (ISECON 2001), Retrieved from: http://proc.isecon.org/2001l17a1index.htrnl

[12] Y-T.C. Yang" TJ. Newby and R. L. Bill,. Using Socratic Questioning to promote critical thinking skills in asynchronous discussion forums in distance learning environments. American Journal of Distance Education, 19(3), pp. 163-181,2005.

[13] C. Perkins, and E. Murphy, Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory study. Educational Technology & Society, 9(1), 298-307, 2006.

[14] S. M. Jacob, and H. K. Sam., Measuring Critical Thinking in Problem Solving through Online Discussion Forums in First Year University Mathematics. Proceedings of the International MultiConference of Engineers and Computer Scientists 2008 - IAENG International Conference on Internet Computing and Web Services (ICICWS 2008), Hong Kong, ISBN 978-988-98671-8-8, pp.816-821, 2008.

[15] P. A. Facione, N. C. Facione, M.A Blohm, K. Howard and CA Giancarlo, Test Manual: California Critical Thinking Skills Test (CCTST). California Academic Press, Millbrae, CA, 1998.

229

[ieee 2009 international conference on engineering education (iceed) - kuala lumpur, malaysia...

Documents