FINAL PROJECT

REPORT

Master on Problem Based Learning

University of Aalborg

Denmark

STUDENTS

Hannes du Toit

Javier Blasco

Nestor Arana

SUPERVISOR Verner Larsen

DATE

15th February 2012

1

Abstract

Can students achieve deep learning with PBL-style teaching? What PBL principles and theories are needed for students to really reach the intentional outcomes of the module?

This is an investigation of three universities that participate in this project: North-West University (South Africa), Universidad de Zaragoza (Spain) and Mondragon University (Basque Country). Based on their experiences, several problems in the learning process were identified. Differences and similarities between the teaching experiences at these three universities were investigated Surface learning allows the student to pass the exam, do some exercises but the students do not really understand the concept they are applying, and they forget quickly. When you achieve deep learning you understand the concepts, you are able to relate them and know where and when to apply them. In order to obtain a better understanding of deep learning, the following processes were done. Different learning theories were investigated to have a better understanding of learning approaches and learning performances. Then PBL principles were reviewed in relation with deep learning. With a PBL experiment, the investigators tried to identify deep learning at the three universities to try to measure the impact of deep learning at each university. The applications of Andersen’s PBL principles were tested at each university to try to understand how the students perceive PBL. The level of deep learning at each university was tested with Dolman’s questionnaire. The implementation contexts were also described at each university. It helped to understand better the conditions of the experiments. The time frame difference (academic calendar) between Europe and South Africa made the investigation more complex. The different levels of PBL implementation also add an interesting twist to the results. The results of the average for the three universities are quite similar. Despite the cultural/logistical differences between the universities (Spain vs South Africa vs The Netherlands, number of students, engineering vs medicine, different tutors styles, etc.), the underlying response of all set of students are identical. Small adjustments on teaching to fit a specific scenario will be needed, but the core issues how to reach deep learning are applicable independent of culture and context.

2

TABLE OF CONTENTS 1. Introduction2. Problem analysis

2.1 Universities experiences2.2 Conclusion

3. Problem statement4. Theoretical framework

4.1 Learning4.2 PBL4.3 Conclusion

5. Empirical design5.1 Research questions5.2 Implementation contexts5.3 PBL implementation questionnaire and interviews5.4 Dolmans’ PBL-R-SPQ test5.5 SOLO taxonomy5.6 Conclusion

6. Analysis and results6.1 Dolmans’ test6.2 PBL implementation

7. Conclusions8. References

Appendix I - Description of PBL contextsAppendix II - PBL implementation questionnaireAppendix III: PBL Implementation questionnaire MUAppendix IV: PBL Implementation questionnaire NWUAppendix V - Dolmans questionnaire results

3

1 Introduction

Teaching engineering in a fast changing world to students from a new generation calls for new teaching styles, designed to meet the challenges lecturers face to deliver competent graduates for the industry.

This study was conducted as part of the requirements for Module 2 of the

MPBL program by three enthusiastic lecturers in different parts of the world, facing the same problems, but with different symptoms, (in different cultures) to change surface learning into deep learning.

We studied different learning and teaching strategies to gain background to

develop a better understanding what active learning styles such as PBL entails. This helped us to be able to identify where PBL fits in the map of teaching and learning theories. This background cleared our helicopter view to be able to have a better comprehension where we are at the moment and what we need to do to deliver better quality engineers to industry.

The team, in true PBL style, works together to share and compare what each

other learn and understand from the theory and how their training program currently fits in the bigger picture. With the identification of similarities and differences between the training at the three universities, we are developing a bigger vision of the problems in our own institution and will be better equipped to solve it.

Like in all other study programs, we realize that we will not be able to solve

all the problems we are facing after completing this study. This is only the start of the journey. Our expectations for this effort are only to develop a sense of what to look for and where to explore to find solutions to the problems we face. Thanks to the MPBL program at Aalborg, we will be better equipped to be change agents with global vision.

4

2 Problem analysis

PBL methodology is worldwide used in schools, high schools, universities and other educational institutions. The literature shows that there are multiple ways to implement the PBL. Among the most well-known ones are Aalborg University (Denmark), Maastricht University (The Netherlands), McMaster University (Canada) and Sheerbrook (Canada), each of which has a different PBL model. If we look more carefully at each university we can also see differences among the disciplines or diplomas (Guerra 2011).

This is the case of the three universities that participate in this project:

North-West University (South Africa), Universidad de Zaragoza (Spain) and Mondragon University (Basque Country). Those three universities, based on their experiences, have identified several problems in the learning process. Mondragon University (MU) claims that in the implementation of PBL process the students show a weak relation between theory and practice, North-West University (NWU) explains that students tend to reproduce or copy the previous solutions and Universidad de Zaragoza (UZ) stress that the students have a lack of transferring or extrapolating knowledge and that the knowledge is evaporated from one year to the next. In all cases, it seems that the students don’t really understand, they do things without thinking, they reproduce without understanding and they are unable to extrapolate their knowledge.

There are also some differences between the three universities. In MU,

it seems that the students do the project work without making an effort to research the needed theory and construct the relation between theory and practice. One reason can be that it is not asked in the assessment. Another reason might be that they do not have help (from the supervisor,...) constructing this relation,... In NWU, the problem can be related with what is asked in the assessment. If you can get good marks only practise and memorizing old exam papers, it is not needed to study in depth and put in much more effort. In UZ it seems that students need to relearn some topics from previous years.

2.1 UNIVERSITIES EXPERIENCES

In this section, we are going to describe deeper each universities’ problematic.

UZ

5

I give my students some assignments in two contexts: as a classroom or in pairs in the laboratory. I have noticed quite different “depths of learning” in the outcome. Below are my reflections about this.

In the laboratory, we follow this procedure:

1.For safety reasons, the students follow a step-by-step procedure. E.g. “(1) Turn on this device. (2) Write down the temperature. (3) Increase the voltage. etc”

2. They collect some data.3. So, while they are in the lab, there is no room for their learning. They

are just robots doing movements. They can chat about the last movie they have watched. Moreover, we don’t have time to include teacher-student interaction during the lab session.

4.At home, they plot the data, do some calculations, answer some questions (not open ones) and write out a report.

5. This report is a poor-quality, “copy & paste” document the students pass between them from year to year. It is very frustrating for the lecturers to grade.

In an exercise classroom:

1. I give the students the “recipes” to classify and analyze the problems.2. Once they learn the trick, they are very efficient at solving the problems.3. However, they are incapable of extrapolating their “knowledge” to similar

problems, mainly because they have not actually learnt.

One trick I have used with very nice results in the laboratory is the following:

1. They do the same except for the report.2. Instead, I hand them a large list of questions where I make them explain

the meaning, significance of the measurements or the procedure they have followed.

3. At the end of the lab session, I will ask one question to each member and they will have to reply individually but the grade will be for the group.

With this, I have forced the bright students to explain the questions to the passive or less-gifted ones. I have noticed the most important point to get good learning is the way you assess them. That forces them to address the problem in a different way, making themselves some questions and looking for answers.

NWUMost of the modules at NWU are still teaching the traditional way. The

lecturer is using a handbook, and followed the prescribed book for a semester, dividing the contents into the amount of weeks available. Some practical experiments are done in the laboratory, but the experiments stayed the same for a long time. Students built a history of results, and senior students share their results from the previous years to the next generation. So, students republish the same results year after year. Post graduate students are running the practicals, therefore the trainers change every year. Nobody re-design the

6

experiments and limited funds are allocated to purchase new equipment to upgrade the experiments.

Some of the lecturers developed a pattern of setting up exam papers, and the students can therefore predict what to expect in each exam. If the lecturer does change his pattern, the class average drop visibly. This put the lecturer in the spotlight and he actually gets in trouble because of a lower pass rate. So, they don’t want to change their recipe, because it keeps them out of trouble.

Since the start of Professional Practice as a first year module, where we try to teach them a helicopter view of engineering PBL style, things slowly start to change in the faculty. We try to help the students to develop critical thinking; questioning the quality of training they get. Some of the senior students, who act as tutors, start to realize that it is not so nice to have good marks for modules, and don’t have skills intended to develop with that module. Some of the students start to put pressure on the lecturers to change their style of teaching, but the lazy students are not happy with the changes. So, the students are clearly divided and some want change but others don’t.

There is a growing interest in engineering education strategies amongst faculty staff. Lecturers start to realize that there is a need for change in their teaching styles. Some support PBL style teaching but others are still not convinced.

Practices that still promotes surface learning. Examples below are not applicable to all modules, but it does occur in the faculty:

1.The assessment strategy to have a few class tests, some lab experiments and one or two big semester tests.

2. Exam questions re-appearing - even with the same values (lecturer can use the same memorandum to mark the papers).

3. Teach first principles and then ask integrated questions combining the principles in the exam.

4. Scare tactics to have impossible difficult first class tests to motivate with fear.

5.Unannounced class tests to force students to attend class instead of making it worthwhile to be in class.

6. In contact sessions just read from the prescribed book, or just repeat the worked out examples of the handbook on the board.

7. Sarcastic reaction to student’s questions in class. Students are looking for other resources if they do not understand the contents, and do not pay attention in class.

8. Penalty class test to punish a noisy class.9.Group in design projects are not allowed to visit the lecturer while

working on the project for facilitation. The end results are evaluated and then some of the groups discovered that they were not on standard.

10. No feedback on exam and test results, explaining what was wrong and why they got a certain mark.

One of the biggest problems at this university is the high student/lecturer ratio. The faculty is handled as a money making industrial factory. High number of students per lecturer and additional pressure to do research as well

7

adds a lot of pressure on the staff. Promotion criteria are measured in number of publications, and results in teaching moving to the lower end of the priority list. With Professional Practice, I try to find solutions for teaching in the African contest. Traditional African communities are operating in family groups and the well being of the group is more important than the individual needs. Using that concept in teaching strategies, empower senior students to get involved in training, not only to help others, but to develop their own skills, without realising it. This added manpower enables me to implement PBL-style teaching strategies that match also the local cultural needs. I see possibilities to change the productivity of a 3rd world country with PBL if correctly implemented.

MU

Between 2002 and 2009, a teaching group at MU, which includes Prof. Nestor Arana, has used project-based learning. In 2008 we modified the method making more emphasis in problem analysis to introduce project oriented problem-based learning. The first method is focused mainly on doing and not so much on thinking. This is why the student sometimes feels lost. In a daily basis project-work they apply lots of trial and error without reflection. Project-based learning applied in MGEP is more focused on skills and not so much on theoretical knowledge, in consequence the theoretical learning is superficial; they have difficulties to relate the experience with the theoretical framework.

The aim to introduce the POPBL was to make students more conscious about where they would like to go, thinking before doing and making more emphasis in the theory. The experience was successful; On the one hand, the student started doing more complex thing. And on the other hand, they understand better what they are doing. Now they are more able to relate theory and practice. Anyway, the relation between theories and practices, and also the relation among different theories can be increased.

Nowadays, on the one hand, the type of project is subject centered and interdisciplinary; some of several (normally three) subjects learning outcomes are worked by project work. This interdisciplinary type of projects can redirect the project solution too much making it sometimes artificial. Interdisciplinary and also make artificial the project and this is a “negative” aspect.

In many cases, the lecturer shows no confidence in the student; it is why he (the lecturer) tries to make the project himself before. The students notice quickly that the lecturer knows the solutions and try to identify it, mainly because this can give him a better mark at the end. In this context, a great amount of energy is focused on solution search. It is difficult to concentrate in the learning process, create a solution free real discussion with students. Only if you tell them that, honestly you have not solution, that the evaluation criteria are clear and available for them and that you role is to help them in the learning process, things start changing.

2.2 Conclusion

8

Once the experience of each university is described, we are going to analyze them:

● The previous experience in PBL is different in each university: no experience, small and 9 years.

● There are various implementation levels: single-course, various-courses and institution level.

● The number of students per classroom also changes from 20-40 students to several hundred students.

● All the universities claim the students achieve surface learning, because they are focused on passing the exam.

● Using the PBL some students also achieve surface learning. They make an application but they don’t really understand the theory beyond their application.

● The process also seems to affect the learning performance. For example, project based learning is more focused on doing and use a trial-and-error strategy. POPBL, with a more open problem, make more emphasis in the problem analysis part. This seems to allow think/reflect more and using less trial-and-error.

● One interesting point that arises from the UZ is that changing the assessing way you can affect the student learning results.

All the universities agree that PBL can be a good method, but we need to take into account that the starting point and the context of each one is different. Not only that, we need to be careful about how we implement it and if we would like to achieve good results or learning performance from the student. In all the cases we would like to avoid the surface learning.

We understand that surface learning allows the student to pass the exam, do some exercises but the students do not really understand the concept they are applying, and they forget quickly. Deep learning is the opposite of surface learning. When you achieve deep learning you understand the concepts, you are able to relate them and know where and when to apply them.

9

3 Problem statement

The challenge for every training program will be to ensure that the students, who enrolled for the program will not only pass the modules to have a qualification, but also, as described by (Illeris 2007): “have detailed knowledge” and activate students to change their “attitudes, beliefs and understandings” about the topic.

John Biggs (Biggs 2007) explained that the scenario changed in the higher education structure. Ten years ago only 15% of the school leavers went to higher education, where in many countries more than 40% of them are enrolled in different programs. “One of the important characteristics of the university, the pursuit of excellence, is endangered”.

But what is the excellence? For example, from the content point of view Bloom (Bloom 1956) and SOLO taxonomies (Biggs 1982) illustrate already different levels of learning, starting from the recall of data up to the ability to make judgments about the value of ideas. The responsibilities for teachers are to design and assess modules in such a way that, if alignment between the teachers and the students takes place, the student has the skills needed. With Kolb (Kolb 1984) and Cowan (Cowan 2006) contributions, reflection on action add skills to the student to be in touch what was learnt and how to apply that in new scenarios.

As it was shown in the problem analysis chapter new curriculums are based on competences where some are technical and others generics. To achieve those competences, active learning methods, such as PBL, are needed.

Some preliminary PBL implementation in MU shows that the students depth of learning can be modified depending on the PBL model employed. The assessment method/style can also be a factor (Biggs 2007). Research work carried out at Sherbrooke University by Prof. Denis Bérnard (Bernard 2008) shows that learning centeredness can be one factor that affects the depth of learning. But, what are the variables that affect to depth of learning? What are the guidelines that need to be followed to achieve deep learning when we use the PBL methodology? In order to obtain the answer of this question we propose the next process:

1. Look into learning theories which try to explain the different learning

approaches and learning performances. 2. Find a scientific definition of deep learning (DL). This should be

completely independent of PBL.3. Review the PBL principles and relate them with DL.4.Design and/or implement a PBL experiment focusing in DL in an

engineering diploma of three different universities.5.Measure the impact of the PBL experiment in DL from teachers,

students and achievement point of view.

10

4 Theoretical framework

Illeris can be a good starting point for the theoretical framework. His triangle (Illeris 2007) describes the three elements (Content, Incentive and Intention) that we need to take into account in learning process. One of the advantages of the PBL is that it takes into account all those tree elements. Deep learning can be seen from different perspective defined by Illeris’ triangle.

Another discussion could be about the learning process in general and DL in particular. From the content point of view, Kolb (Kolb 1984), for example, proposes a learning cycle, based in Piaget (assimilation and accommodation). Vigotsky proposes the term Zone of proximal development (ZDP); "the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers" (Vygotsky 1978). Both authors, among others, help to understand active methodologies like PBL. But what more we need to take into account in a PBL learning process?

Biggs (Biggs 2007) proposes a constructive alignment. This concept stresses that Intended Learning Outcomes (ILO), Assessment Task (AT) and Teaching and Learning Activities (TLA) need to be aligned. Is it the case with the PBL?

Another concept that Biggs proposes (Biggs 2007) is a Student Approach of Learning (SAL). It classifies the students in two types, students with deep learning and surface learning approach. He stress that the Structure of Observed Learning Outcome (SOLO) can be an interesting taxonomy to classify the Intended Learning Outcomes (ILO). This can help the students of both learning approach to work together and increase the overall performance. His SAL questionnaire can be used to identify each type of student. Can we use it in the PBL framework?

Finally, it could be interesting to review the PBL principles described in (Kolmos 2009) and (Andersen 2002). Those principles can help us to organize all the variables that impact in the implementation and relate them with DL.

4.1 Learning

Illeris (Illeris 2007) suggest that learning has three dimensions, as it is drawn in the next figure:

● Content: getting knowledge and developing skills.● Incentive: motivation, emotional part.● Interaction: with the learning context and objects (books, people,

room, artifacts …).

11

Figure 1 -Illeris triangle

First, from an overall point of view, the term “deep learning” can refer to the depth of the content students learn. You can simply have more or less detailed knowledge about a certain topic. This approach is orientated at the content/cognitive dimension of learning.

Secondly, deep learning can also refer to the extent to which the self is reorganized – to which extent student’s attitudes, beliefs and understandings are being changed. Such approaches are more orientated at the incentive dimension of learning (emotional and motivational).

Thirdly, you might see deep learning improved or constrained by the interaction processes which are part of most learning (acting and communicating with other – being a part of a community etc.).

John Biggs (Biggs 2007) classifies students’ learning approach in two types, students with deep and surface learning approach:

● Deep learning approach: driven by internal motivation (or intrinsic motivation) and curiosity. There is a personal commitment to learning. New mental structures are built. It arises from curiosity and if a student has success in a given task, his intrinsic motivation increases.

● Surface learning approach: the motivation is just to carry out the task. The student tries to identify important items to pass the exam and just memorizes them.

Other type of motivation that does not attain deep learning approach, is extrinsic motivation. The students are rewarded/punished if they do/do not achieve the outcome. In social motivation, the students are eager to please

12

their families. The motivation comes from the competition. You have to “kill” your opponents. Here the student is missing the collaborative work. This puts off many students.

It is important to say that the student approach of learning can be influenced not only by the student background but also by the teacher style (e.g. pressing the student, stress for exams, standardized tests, etc), how they perceive the environment characteristics, as classroom climate, how they understand the educative context is going to ask him and the learning experience they have.

For students that do not have an intrinsic motivation, Feather (Wigfield 1994) proposes an “expectancy-value” theory (see figure 2). To engage the student they need to know that the topic has to be worthwhile and the student has to expect success when engaging the learning task. To improve expectations of success, the student must know where he is going and get feedback telling how well he is progressing. The student is the owner of the learning process. Deep engagement requires time and thus an overloaded course won’t allow this.

Figure 2 - Expectancy-value paradigm

In order to measure the learning outcomes and the degree they have achieved, we propose to use two taxonomies: SOLO and Bloom’s. Both taxonomies point to the content dimension of learning.

On the one hand, SOLO taxonomy was proposed by J. Biggs and K. Collis in 1982. It stands for “Structure of Observed Learning Outcomes” (SOLO). This theory analyzes how far the student has been able to grasp

13

the information in terms of identification of relevant aspects and their relationship/integration. It is a way of measuring the level of abstraction and generalization. This taxonomy suggests five levels of understanding, with increasing level of performance (2-3 are quantitative, 4-5 are qualitative):

1. Pre-structural (bits of unconnected and nonsense information)2. Uni-structural (identify, recite, follow a recipe)3. Multi-structural (classify, combine, enumerate)4. Relational (relate, compare, analyze)5. Extended-abstract (generalize, hypothesize, theorize) On the other hand, Bloom taxonomy was proposed by B. Bloom in 1956

and reviewed by (Anderson 2001). It consists of a set of learning objectives or activities that educators set for students. Bloom, together with Krathwohl and Simpson (Anderson 2001) talks about three domains that should be mastered by students:

● Cognitive: knowledge, comprehension and critical thinking.● Affective: other skills such as attitudes, emotions, and feelings● Psychomotor: ability to physically manipulate a tool.

In the cognitive domain, the author distinguishes six categories and presents them in increasing complexity order:

1. Knowledge: Recall data or information

2. Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words.

3. Application: Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the work place.

4. Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.

5. Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.

6. Evaluation: Make judgments about the value of ideas or materials. There are several differences between those two taxonomies. Firstly,

14

working with competences (see figure 3) we understand that there are three component that need to be taken into account; knowledge, attitude and skills. Reviewed Bloom taxonomy focuses on all of them and SOLO taxonomy in knowledge (Declarative and Functioning).

Figure 3 - Competence’s elements

Secondly, Bloom taxonomy was developed from a teacher perspective, and SOLO from both students and teachers perspective. SOLO takes into account how the students learn.

Finally, Bloom taxonomy is not hierarchical, for example in Bloom “Understanding” level you can find “identify”, “discuss”, and “explain” that are three different learning level from SOLO’s point of view. Bloom is focus on the outcomes and SOLO focus on the quality of learning.

From our point of view SOLO seems best suited to align the students and teacher perspective. The students can understand better what the teacher/institution is going to ask him/her. This is an interesting advantage to apply the expectancy-value theory and motivate those students that do not have an intrinsic motivation.

Because SOLO gives a hierarchical structure of the students learning, that is the quality of learning, it seems best suited to define their learning performance or depth of their learning. We can say that a student achieves deep learning if he achieves a qualitative phase (levels 4 & 5) of SOLO taxonomy. When the students are able to relate a given concept or when they extend the relation with other concepts.

The previous theories allow us to put in place the basis of what is deep learning based on SOLO taxonomy and how to take the students motivation to

15

achieve deep learning. Now we need to understand better the learning process.

Jean Piaget’s concept (Piaget 1980), inspired in the biological phenomenon of accommodative learning is to be seen as a reconstruction of “mental schemes” and can therefore be considered as transcendent learning – a “qualitative going beyond”. Piaget is orientated to the content cognitive dimension. Kolb (Kolb 1984) uses Piaget’s assimilative and accommodative learning in his learning cycle as we can show in the next figure.

Figure 4 - Kolb’s learning cycle

Carl Rogers’ (Rogers 1961) concept of significant learning is something that involves the “whole person” and his existence. It makes a difference in the student’s behaviour and courses of action he chooses in the future. It is orientated at the incentive dimension of learning. Jack Mezirow’s (Mezirow 2000) concept of transformative learning is similar to Rogers but may be a little more orientated to the content/cognitive side.

Illeris (Illeris 2007) stresses there could be four levels of learning ordered from low to high level:

1. Cumulative learning. It happens when we learn by heart. E.g. a list of telephone numbers.

2. Assimilative learning. It happens when the impressions from surroundings are incorporated and linked with the previous knowledge.

3. Accommodative learning. In this case there is partial or full restructuring of mental schemes.

4. Transformative (or significant) learning. This learning is like a

16

catharsis, bigger than the previous accommodative learning.

As we can see, each type of learning affects, in a different degree, the metal schemes configuration. Those four degrees or levels can be used to help students in the learning process and propose them a different type of learning in order to achieve deep learning.

Donald Schön and Chris Argyris (3 dimensions but mostly content) are known from the concept of “reflection in action” which has to do with the thinking taking place while people act. Their orientation is to all three dimensions (yet mostly to the content side) - also the interaction dimension, because of the focus on reflection in action.

John Cowan in his book “On becoming an innovative university teacher: Reflection in action” (Cowan 2006) relates deep learning with meta-learning. He stresses that it can be achieved by means of reflection. Cowan, based on Schön’s reflection model and Kolb’s learning cycle, defines three types of reflection: “reflection for”, “reflection in” and “refection on”.

● Reflection for action is a reflection about what is to be done in the learning process. For example: What can happen if we implement Schön’s theory in the classroom?

● Reflection in action is a reflection about the action that is happening in this moment. For example: What is happening right now in our MPBL project?

● Reflection on action is a reflection about the actions made before. For example: How well did we do the project?

Schön’s and Cowan’s reflection helps students to be more aware of what will be going on, what is going on and what happened before. It develops a meta-cognition and helps in deep learning approach.

In conclusion, first the students need to use not only an assimilative learning but an accommodative learning and if possible a transformative learning. The more mental schemes are restructured, the more relations are made between the concepts and, in consequence, the higher SOLO’s performance level and deep learning.

Second, Kolb (Kolb 1984) proposes an interesting learning cycle that integrates both assimilative and accommodative learning in a learning cycle. Cowan adds to Kolb learning cycle, using a Schön’s reflection model, a higher level of learning or meta-learning.

17

4.2 PBL

Once we have defined what deep learning is, the conditions to achieve it and we have understood what learning is and the kind of learning that allows deep learning, we need to use a teaching learning activity.

PBL is a well suited for competences based curriculums. Biggs (Biggs 2007) stresses that the Intended Learning Outcomes (ILO), Assessment Task (AT) and Teaching and Learning Activities (TLA) need to be aligned. As we can see in the figure 5, throughout the semester, each PBL team can follow its own path. Each path is equivalent to a sequence of smaller TLAs. Because all paths can be different from each other, we think that in a PBL environment, the alignment at “micro” level cannot be guaranteed. At a “macro” level, it is aligned because in competences-based curriculum there are not only technical competences but also generic competences. Those “uncontrolled” situations generate spaces to learn generic competences such as process competences, project management, project organization, etc. The “uncontrolled” path also means that the process is student directed; in others words, the students are the owners of the learning process. As we said before, this is an important factor to create motivation and, in consequence, to achieve deep learning. To control those uncontrolled situations there is also a need of iteration with a group members (discussion), with oneself (reflection), with the supervisor, with the objects, etc. Those discussions generate more connections between the concepts which is one of the goals of deep learning (4 & 5 of SOLO taxonomy).

Figure 5 - Different group process in a PBL

18

Some authors share the view (Kolmos 2009) and (Andersen 2002) that the PBL has learning principles. On the one hand, Kolmos (Kolmos 2009) proposes that those principles can be grouped in three dimensions: Contents, cognitive learning and collaborative learning (see figure 6).

Figure 6 - The PBL learning principles (Kolmos 2009)

On the other hand, Andersen (Andersen 2002) suggests that PBL has four principles:

1. Problem orientation: open vs closed cases/problems.

2. Student direction: The learning centeredsness

3. Exemplarity: representative of what the student will find in his profession. However, it is difficult for a teacher who has never worked in the industry to propose a real-life problem.

4. Inter or Trans-disciplinarily: one/various subjects involved. It is part of the requirements of a good problem. It happens in real life but it is very difficult for a facilitator to be expert in various fields. Besides, in current programs, the subjects are segregated, so you cannot teach somebody else’s subject.

Each of these principles could have different strength, at least on a practical level. Based on such PBL-practices (Andersen 2002) drew a diagram representing the two first principles, problem orientation on one axis and student direction on the other:

19

Figure 7 - The degree of problem orientation and student direction

As you can see, the two dimensions (axis) open a space for various PBL-courses where the “strength” of the problem orientation or the student direction differs.

Some PBL-courses can be very open ended (Problem oriented) and student directed to a large extent (“star”). But you may find PBL-variations where more emphasis is put on the problem orientation than on the student direction and vice versa. (Circle and square).

Case-based PBL-courses can have very complexed intertwined problems (ill structured cases see (Jonassen 1997) but the process by which the students work their way through the case can be controlled by the teacher to a considerably extent (circle). Or you can have a more closed case (well-structured), which may still involve many elements which should be brought together, perhaps over a longer period and therefore require quite a lot of control and cooperation in the group of students.

The diagram (See Figure 7) is of course not mathematical. It is relative and no two PBL models are the same. But the diagram may open for a discussion of what principles should be given high priority and also what specifically makes the difference in the problem orientation and the student direction from one PBL model to another. Furthermore, it may become clearer what studies falls outside the category of PBL if they come close to the bottom left hand corner. Again it is not possible to determine mathematically when that border is crossed. Didactical arguments must be made in each case.

What about the last two dimensions? We think they can also be evaluated according to “strength” but it would be too confusing to include those in the same diagram. “Exemplarity” has to do with the content of the PBL-course and

20

to what extent it is representative for a larger area of the profession that the students will enter into. Inter- or trans-disciplinarity (Savin-Baden 2007) may be more complicated to go into, but to keep it simple you can start by asking whether there are one or more subjects involved. Another aspect is how the subjects are organized and by whom, and also how they are contextualized, but that is a more complicated issue which I cannot go further into here.

As we have described in the problem analysis chapter in MU there where two PBL implementation: narrow and open problem. Both point of view are close to Illeris’ learning dimensions (content, incentive and interaction) and quite similar. They have the same concepts distributed in different way. In this project work we selected the four principle proposed by Andersen. The diagram proposed in the figure 7 helps to place the previous PBL applied in MU. Also it helps to understand what other different possibilities can exist if we introduce the student direction principle.

There are several curriculum types. Pozuelos (Pozuelos 2007) defines four: Traditional, technological, spontaneous and investigative.

A traditional curriculum responses to “what” and it is mainly focused on the content. Technological responses to “for what” and the objectives are the observable actions. Spontaneous responses to “how”, there are no specific objectives. Finally, investigative is more systemic and it is focused on the “process”.

Depending of the curriculum type, the evaluation needs to be different. For example, traditional and technological evaluations are focused on the product and spontaneous and investigative are more focused on product and process.

Other researchers such as Taba (Taba 1962) classify the curriculum design in three types: subject-centred (emphasis the subject), learner-centred (emphasis individual) and problem-centred (emphasis areas of living, personal and social).

The project works in technological and investigative curriculum are quite different. In technological curriculum we focus more on the artefacts and the context where it will be applied. In the investigative curriculum we focus on the research and on the process; we make more than in the artefact itself. If we make the same analysis from Taba’s point of view, a subject-centred curriculum forces the project work to be a subject-based project. But if it is a problem-centred, the project can response to a more open problem area and the subject can be more heterogeneous.

The curriculum type can also affect the type of PBL we intend to implement. In this project work we didn’t deal with this aspect.

21

4.3 Conclusion

We have started describing Illeris’ three dimensions of learning: content, incentive and interaction. All those learning were discussed each one from a different perspective.

John Bigg’s classifies students’ learning approach in two types, students with deep and surface learning approach. For those that have not intrinsic motivation (Wigfield 1994) proposes an “expectancy-value” theory where there are two conditions: the student needs to understand what is expected from him and it has to be valuable for him. SOLO taxonomy is interesting because it takes into account the student and teacher perspective. The students can understand better what the teacher/institution is going to ask them, from the expectancy-value theory. On the other hand because it is hierarchical from the learning point of view it is best suited to define their learning performance or depth of their learning. We can say that the students achieve deep learning if they achieve a qualitative phase (levels 4 & 5) of SOLO taxonomy.

Illeris also describes four learning levels: cumulative, assimilative, accommodative and transformative. Each level represents a different mental-schemes rearrangement. The more mental schemes are restructured the more relations are made among the concepts and in consequence the higher SOLO-s performance level and deep learning.

Kolb (Kolb 1984) proposes an interesting learning process or cycle that integrates both an assimilative and an accommodative learning in a learning cycle. Cowan add to Kolb learning cycle using a Schön’s reflection model a higher level of learning or meta-learning that use a transformative learning.

PBL is the teaching learning activity we are going to use. Kolmos’s three principles show that PBL uses all three learning dimensions (Content, Incentive and Interaction). Andersen (Andersen 2002)’s four principles give us a more clear idea of how a PBL can be adapted in two of those four dimensions: Problem orientation and student direction. The problem orientation is related with the exemplarity (worthy task) of the problem. Those elements affect the students’ motivation and in consequence the necessary (but not sufficient) condition to achieve the deep learning. The curriculum type of each university is a factor that can constraint for example the exemplarity of the task and the students’ direction. Those constraints are not clear limits, there is some flexibility the teacher can play with. Some research (Bédard 2008) shows that more problem-orientated and student-centred frameworks ask for deeper learning approach.

22

5 Empirical design

5.1 Research questionsIn order to analyse the problem stated in this project, we propose the

following research questions and certain ways to assess them:

Research question Where this is answered

1.- Which PBL principles (Andersen2002) have been put in the PBL design in each practice (university)?

See implementation contexts

2.- How have the students perceived the PBL? What is the motivation/level of satisfaction of the students with the PBL implementation?

See results of PBL implementation questionnaire and interviews.

3.-What kind of approach is adopted by the student in the PBL part: deep or surface learning? (Dolmans 2010)

See Dolmans PBL-R-SPQ questionnaire.

4.-Has deep learning been achieved? See analysis on SOLO taxonomy level.

We are interested in looking at different aspects of PBL and the quality of the learning achieved by the students. For each one, we are going to devise an ad-hoc test or use an existing one.

The qualitative questionnaires help us to measure in a large amount of students certain aspect defined in the questionnaire and aligned with our research questions. In order to have a good measurement tool we are going to use reliable questionnaires, those that have Cronbach alphas greater than 0.70. To analyse the result obtained from this type of questionnaires, we only need to compute the mean and the standard deviation of each item.

The problem of the qualitative questionnaire is that, on the one hand, there can exist unexpected aspects (Cowan 1999) that they were not taken into account in the questionnaire and on the other hand, the results of qualitative questionnaire need explanation in order to understand, it is why we will use the open-ended questionnaire.

5.2 Implementation contexts

23

The present experiment is being carried out simultaneously at three universities:

● University of Zaragoza (UZ)● North-West University (NWU)● Mondragon University (MU)

Each university has its own implementation context (E.g. institution- or programme-level constrains, number of students, teacher’s experience on PBL, timetable, etc). Thus there is little room for changes in the PBL implementation. As a result, each university will perform its own execution of PBL. This fact will pose some difficulties for the proper comparison of the three experiments. However, the authors of the report believe this can be overcome by carrying out a detailed analysis of each PBL experiment outcome.

In the next subsections, we are going to explain each universities context. Anyway in Appendix I you will find more detailed information.

UZThe lecturer, Javier Blasco, has been teaching since 1998 but has never used

PBL. He teaches Fluid Mechanics to Chemical Engineers. He is also head of an advanced, online piping course for graduate engineers. He attends teaching courses regularly at his university. All his teaching is on the spring semester, so he will conduct his experiment later than MU and NWU.

Each group has to look into a chemical engineering problem and analyse it within the fluid mechanics framework. E.g. A group may choose to study hydrocyclone design for the chemical industry. Therefore, the problems are open and several disciplines are involved in its solution. The problem is chosen by each group, which increases motivation, and later approved by the tutor. The students make decisions about their learning on their own; that is, they decide how to approach the problem, they produce their theories, they follow their path. The tutor is just there to warn them if they are failing in some points.

The expected exemplarity of the problem is large since the students have to end up providing a design to be applied in the chemical industry. However, since the tutor has never worked in a chemical industry, we might not always achieve a high exemplarity.

The lecturer acts as the tutor of the group and meets them weekly in the classroom.

NWUHannes du Toit has been teaching since 1994 and applying PBL for 4 years.

He teaches Professional Practice I & II for engineering students. To cope with the large amount of students in each group (around 380 in the first year and 320 in the second year), he employs senior students as tutors. He is highly involved in engineering education and participates actively in workshops on the subject.

Multi-disciplinary groups of 6 were formed, using also the MBTI-personality

24

test (Myers-Briggs Type Indicator) to ensure diversity in each group. Each group works on a real life problem to develop concept designs to satisfy the needs of a real client. Then three groups merged to form an engineering company to develop the best idea into a demonstrated prototype, using engineering principles in Project Management to solve the problem.

Tutors are not allowed to participate in the team finding solutions, but only observe and assess the group as they progress. They also ensure that peer group assessment is fare and accurate.

Using Action Lists in each team ensures that each individual in the group knows what was expected from him, and there is also a trace of achievement of all the team members, because each member are peer assessed on his contribution as indicated on the action list.

With PBL style teaching I hoped to develop a new attitude with the students move away from being happy to just prepare and passed the exams, memorizing previous exam papers. I hope to convince the students how important it is to actually being able to apply knowledge in practice and to develop skills to apply in the workplace.

Figure8 and Figure9 below, shows the construction, assembly and evaluation of one of the projects done by first year students. This prototype is the start of a real international rock climbing wall.

Figure 8 - Assemble and construction

Figure 9 - Test

25

MU The lecturer, Nestor Arana, has been teaching since 2002 and applying PBL

for 8 years. He teaches Computer Science on the 3rd and 4th semester with a 50-50 mixture of classical lectures and PBL. He is also in charge of the teacher training of MU instructors.

Each student group (4-5 member per group) has a different problem. They are open in the sense that there exists a variety of solutions and the student can propose/develop/find its own solution. Normally, all of them has a known solution. Because several subject need to be included in the project, the interdisciplinarity is the main PBL principle. We try to do an exemplary problem, related with the type of problem they will find in their work. They are close to real work but it is not always possible because we need to guarantee that first the involved subject can be learned. The project is directed by the students supported by two types of supervisor (the tutor and the expert).

In the 3rd semester, there are 8 experts and 6 tutors. All the 6 tutors are also experts, that means that there are 8 teacher involved.

Figure 10 - Student working in their meeting room (MU)

You can find a video of a developed robot shown by student in the exam day (2012/01/26) in the next web page: http://www.youtube.com/watch?v=vahkU0_8iDw&feature=player_embedded

The project is presented via Mudle platforme with the next structure: participating subject, problem description (one for each group, there are six teams), the PBL methodology description, student/teacher roles description, the learning goals and their criteria and the main deadlines.

Participating subject were:○ Computer Science: Digital systems, Network, Object Oriented

Language, and Computer Systems.○ Telecommunications: Digital systems, Network, Object Oriented

Language, and Circuits theories.

The problem title were:○ In Computer Science diploma

26

● Betting machines (Team 1)● Video games (Team 2)● Room management (Team 3)● Political election system (Team 4)

○ In Telecommunication diploma● Automated cleaning of commercial areas(Team 5)● Automated baggage distribution and storage (Team 6)

5.3 PBL Implementation questionnaire and interviewThis test (see Appendix II) has been developed ad hoc by the authors of this

report. Its aim is to check whether the students have perceived the PBL as the lecturer expected. E.g. the lecturer’s intention might have been to make the students the owners of the judgement but the students may have felt they were forced to follow a given argumentation.

Also we will conduct interviews. Its aim is to gather qualitative information which cannot be grasped from multiple-choice tests. Thus, the interviews will have an open structure, although they will have some underlying research questions (see below). The lecturer will let the group talk freely. This way, the discussion will reveal deeper thoughts (Cowan 1999).

The interviews will be carried out at the end of the PBL course. They may be recorded to make note taking easier (otherwise, two instructors are needed).

The size of the interview will be 5-8 students, preferably one from each group so that different inputs can be collected at the same time. We believe that number of students is the appropriate to promote open discussion. A smaller group would make the students feel “interrogated”, whereas a larger group would put off many students.

The interview will be used to collect information about:

● To understand what means the quantitative questionnaire.● Unexpected results obtained in the questionnaires.

5.4 Dolmans’ PBL-R-SPQ test

These authors reformulate in 2010 Biggs R-SPQ questionnaire (1985) in order to better fit with a PBL environment. They claim that the questionnaire provides a valid and reliable tool to measure students’ learning approach in PBL.

5.5 SOLO Taxonomy

As we have argued in the theoretical framework chapter we decided to use if possible the SOLO taxonomy. In NWU and MU the PBL implementation was

27

already planed, it is way a minor o no change was made. In the case of UZ a new PBL implementation allows a greater freedom to introduce major changes.

UZ

ILO Solo level Degree of achievement

To integrate different disciplines of the diploma to analyse an industrial problem

4 Relational Not available

To presents experiment analyses and results in the proper format

not applicable Not available

To employ project management and project organization techniques

not applicable Not available

To reflect and theorize 5 Extended abstract

Not available

Since the PBL experiment at UZ has just started, I cannot give information

about the degree of achievement.

NWU

ILO Solo level

Degree of achievement

Orientation and Group division with MBTI personality test

1 done

Meeting procedures, documentation templates, poster guidelines, financial procedures

2 done

Literature research, interview with client, tender for the project, concept designs

3 done

Determine the best concept design, Marketing assignment

4 done

Detail design, consultation with experts, industry visit, construction of a prototype, test, evaluate and improve prototype,

5 done

28

It is clear from the outcomes published in the student’s study guide that the aim of this module is to equip students with the skills to meet the higher order levels of SOLO’s Taxonomy.

MU

In the 3rd semester project there is a rubric where ILO and the criteria or achievement degree of each ILO are defined (see the next table). The semester was already planned, it way in this table a minor changes where introduced. For example, we described a generic ILO in terms of SOLO taxonomy. In the project there are 11 ILOs (Computer Science) and 12 ILOs (Telecomunication).

ILO Solo level

(1-2) (3-4) (5-7) (8-10)

Different sub-networks that communicate using dynamic routing, but the best protocol is used.

There is only a single sub-network.

Different sub-networks are linked, but not using dynamic routing .

Different sub-networks that communicate using dynamic routing, but the best protocol is not used.

Different sub-networks that communicate using a dynamic routing and the best protocol is used.

Write the project report justifying how and why.

The student name the components and procedures used in the digital circuits design.

The student describe the components and procedures used in the digital circuits design.

The student argue the components and procedures used in the digital circuits design.

The student theorize the components and procedures used in the digital circuits design.

5.6 Conclusion

In this chapter we have defined the specific research question we will like to deal with in this project work. Which PBL principles have been put in the PBL design in each practice (university)? How have the students perceived the PBL? What is the motivation/level of satisfaction of the students with the PBL implementation? What kind of approach is adopted by the student in the PBL part: deep or surface learning? And, Has deep learning been achieved?

Second, we have described the implementation contexts of each university. This section helps us to understand better in with conditions we are going to carry out our experiments. In UZ for example they can implement a new PBL experiment, but only in the spring semester of 2011-2012. In NWU and MU they have implemented with few or no modification in the autumn semester of 2011-2012.

Third, also the PBL implementation questionnaire and the associated interview that are going to help to understand how was the student perception

29

of PBL implementation were explained. We also introduced Dolmans PBL-R-SPQ questionnaire. This questionnaire helps to identify the student learning approach during the experiment.

Finally, we describe the ILOs of each university.

6 Analysis and resultsIn the following two sections, the results obtained at MU and NWU will be presented. The results concerning UZ cannot be shown because the experiment has just started this semester.

6.1 Dolmans’ PBL-R-SPQ testThis section explains the data analysis of the PBL-R-SPQ questionnaire (Dolmans 2010) and the main conclusions drawn from the analysis. Data analysisThe number of questionnaires filled by the students is 17 out of 28 MU and 13 out of 360 NWU. The above sample number is smaller than the actual number of students. The reason for this is that we have had difficulties getting the students to fill the tests because of two reasons:

● MU: the students where busy working in the project work and preparing their project exams.

● NWU: the students were on holiday by the time the questionnaires were available.

After analyzing the answers to the other questionnaire (PBL implementation), we have arrived at the conclusion that the students are being honest when filling the questionnaires. Thus, we believe the results of the data analysis can be representative of the classroom regardless of the reduced sample size. The questionnaire is made up of 17 questions which can be grouped in two categories:

● Deep approach● Surface approach

Each categories can be broken down into two sub-categories:● Deep/surface motivation● Deep/surface strategy

The question are replied using the following wording and numerical scale:

30

Never or rarely (1), sometimes (2), half of the time (3), frequently (4), always or almost always (4) For each question, the answers are averaged across the students. The standard deviation is also calculated for each question. The results are summarized in the following table and compared to (Dolmans 2007): Table 1. Comparison of Dolmans test in the three universities

NWU MU DOLMANS

AVER SD AVER SD AVER SD

DEEP 3.43 1.18 3.16 1.16 3.50 0,48

SURFACE 2.26 1.19 2.25 1.1 2,35 0,5

From the above tables, it can be gathered that:

● The results of the average for the three universities are quite similar.● Some discrepancy can be found on the standard deviation. It is smaller

in the case of Dolmans. This might be attributed to the size of the sample (262 in Dolmans compared to 13 and 17 in NWU and MU, respectively).

● In general, it could be said that students are slightly proned to be deep learners (values greater than 3) and less likely to adopt a surface approach (values smaller than 3).

● Despite the cultural/logistical differences between the universities (Spain vs South Africa vs The Netherlands, number of students, engineering vs medicine, different tutors styles, etc.), the underlying response of all set of students is identical.

The information for each question can be found in Appendix V. The first half of the table is marked in blue and deals with deep approach, whereas the second half (shown in red) looks into surface approach. Some comments about the referred table:

● Deep approach:● Largest differences between universities (>1 in mean): question 2,

NWU are deeper.● Less deep approach (mean <3): questions 9 and 14.

● Surface approach:

31

● Largest differences between universities (>1 in mean): question 12, NWU are less surface.

● More surface approach (mean >3): question 16

6.2 PBL Implementation

MUIn the next section we make a summary of the results of the PBL

implementation questionnaire (see Appendix III) and the open discussion. The summary is divided in four PBL principles (Andersen 2002): Problem orientation, Exemplarity, interdisciplinarity and Student direction.

Problem orientationEverything has turned around the problem (but constrained by the subjects)

they have selected. The problem openness of each project was different from one to another. Anyway the openness was bigger than the previous two semesters. The student feel that the problem was affordable and they liked.

ExemplarityThe problem has involved real life situations. In the open discussion the

students says that the project allows them to link the subject with the real wold. Other added that the project help them to be more ready with for the engineering work... that it is not important to have a good mark in a classical exam, but how you can use this knowledge in a real problem. It seems that the students identify the problem with real-life situation.

InterdisciplinarityThe project has involved several disciplines

Student directionThe student didn’t use strategies like internal rules, calendar/planning

(because in the previous semester project they didn’t have to do that) and meeting with supervisor-tutor. They didn't either use brainstorming technique but instead they used open discussion to define the project at the very beginning. It seems that the process competence was in general not taken into account. For example, as they said in the open discussion the student roles were divided regarding the technical competences and process competences like project management roles (project leader, secretary, coordinator, …) are not taken into account.

The supervisor was not another member and he did not always give them a direct solution. The process was guided by the supervisor-expert (and not the tutor) using open questions and not giving the direct solution forcing them to learn new thing to solve the problem. But when they were completely lost they can turned to the expert. It seems that the process main guide are the supervisor-expert. As they says in the open discussion, for them it was very

32

important to make work their solution.

The course was not full of rules. They have adopted the conclusion and leading their project in a freedom environment. They have developed new theories from their discussions and they have also used previous knowledge. Some group have used collaborative tools. Anyway they says that this environment didn’t led them to cheat.

NWUIn this section we comment the results of the PBL implementation

questionnaire at NWU (see Appendix IV for more information). The following alignments were done to meet Andersen’s PBL principles:

Problem Orientation

Each team received a different assignment to solve (open problems). There is no worked out solution to these real problems. The students have to find the best solution to this problem to meet the needs of the real client in the community.

Student DirectionThe lecturer does not have to be an expert in solving the problems. Students have to find experts in industry to consult and to assist them with possible solutions. There is no prescribed technical manual to supply them with the technical know-how to solve the problem. The only prescribed notes are the process to solve real problems, which they have to apply to their own project.

Exemplarity

These real projects to real clients are as close as possible what a student can experience at university level. Even before they start with the hard core theory from their theoretical modules, they have the opportunity to set their frame of mind how engineers operate. This gives them the tools they need, not only in industry, but also for the rest of their studies to know where to fit all new knowledge. This change in mindset prepares them to see new knowledge and skills in perspective and how to apply it.

Inter or Trans-disciplinarily

To solve real life problems, and constructing prototypes, mechanical systems need electronic control and sometimes chemical processing. Adding these different fields produce a machine. With safety guidelines, financial planning and marketing skills to be developed, a useful product (sponsored by industry) is produced to be used in the community.

33

SOLO

Qualitative questions in the questionnaire were used to confirm if the students really understood the purpose of this module and if they really added some deep learning to their system. The students strongly agreed with statements in the questionnaire that proofs higher levels of SOLO were reached. The detail results and discussions are available in this report below.

The biggest challenge still is to activate students that are really focussed on surface learning, and really try to pass the module with as little effort as possible. More rules and regulations will restrict PBL development and freedom, but negative students always find loop holes in the system to take shortcut.

For 2012, I am following a new strategy to give teams even more freedom. If they are on track, they may continue without any tutor present at meetings, but the teams now are much more powerful to expose and punish team members not contributing to the project. Better and clearer documentation systems empower the facilitators to monitor team progress on the electronic platform to identify problems and cover-ups and deal with it. This means that the tutors are monitoring and focus on rehabilitation, to give the students the opportunity to really move to higher levels of SOLO.

The students strongly agreed with statements in the questionnaire that proofs higher levels of SOLO were reached.

● Able to make my own decisions● Generate new theories● Applying proper management techniques● Used milestones to monitor project executions● Problem reflected a real life situation● Forced to learn new things to solve the problem

With the interview, it was confirmed that the students really understood

the concept of having a helicopter view first before you start any assignment in life. In South African culture it is not common to question structures and authority. To me it was clear that we are breaking up this barrier where students finally shows signs of asking why they need to do or to know things, and they asking more application questions in other modules. They also are getting frustrated with teachers not linking the contents of the module to real life context.

My module was already finished at the start of the project, and therefore we could only reflect what happened. No experiment was possible to improve the teaching style to promote deep learning. The value of this study for me is to confirm that the programme I implemented is in line with what the experts

34

suggest in the literature. It also supplies me with new strategies and a clear view how to improve my strategy, which I implement already in 2012.

Preliminary conclusionThe students learning approach questionnaire (see Table 1) is less deep

3.43 if we compare with Dolmans (Dolmans 2007) 3.50 and more superficial comparing 2.26 (MU) with 2.35 (Dolmans).

If we analyse from the PBL implementation point of view, the process competences was one of the weakness identified point. The student didn’t apply strategies/tools to guide their own process. The interaction between supervisor-tutor and the student group was small and very high with supervisor-expert.

If we identify the learning outcomes rubric almost all the learning outcomes criteria are product oriented like the next examples “Different sub-networks that communicate using dynamic routing, but the best protocol is used”, “Users and groups have been created, these are grouped in organizational units with coherent with well defined roles and functions...”. Few criteria were defined using SOLO taxonomy and the process. For example “The student argue the components and procedures used in the digital circuits design.” This can be the only learning criteria that promote deep learning. This can be also the reason why the students were so focus in functioning their product and no sow much in their process.

35

7 Conclusions

All three universities involved in this study have identified several problems in the learning process. Although each university has its own context and style of teaching in different parts of the world, all have agreed that most students are focused on passing the modules with surface learning. Even students following PBL modules complete the assignments without really understanding the theory beyond their application, and still are unable to relate to new knowledge and where and when to apply them.

In our attempt to solve the problem we tried to find a scientific definition of Deep Learning looking into learning theories, review the PBL principles and choose the ones we feel have the largest impact on Deep Learning.

A PBL experiment was conducted including all three universities. Solo Taxonomy’s qualitative phase (level 4&5) seems to address the depth of learning. Illeris describes four learning levels and concludes that the more mental schemes are re-constructed, the closer students get to deep learning. Kolb’s learning cycle also promotes higher level of learning. Kolmos’ principles identify problem orientation and student direction to play an important role achieving deep learning.

It is clear that there is not a single recipe to follow to achieve deep learning and rely on the teacher’s ability to analyze the situation to design the correct context for a specific group of students. Culture and social background plays a role, but the training level (year of study) will also dictate what process will be suitable for a specific group of students.

The experimental results of the three universities are quite similar. Despite

the cultural/logistical differences between the universities, the underlying response of all set of students are identical.

36

8 References

Andersen, O.D. and V. Larsen. 2002. Problembaseret Læring, FoU-projekt nr. 2000-2734-91, DEL.

Anderson, L. W., and Krathwohl, D. R. (Eds.). 2001. A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational objectives: Complete edition, New York : Longman.

Arana, N., G. Aldekoa and U. Markiegi. 2011. Development of feedback press for compentence based program. 3rd International Research Symposium on Problem-Based Learning. 28-29 November 2011, Coventry University

Bédard, D. 2008. Comparing PBL Models: The need and potential of research. Recorded conference (37 min). UNESCO Chair - Problem Based Learning, Aalborg University. URL: http://www.hum.aau.dk/video/2008/pbl/denisbedard/

Biggs J. and K. Collis 1982. Evaluating the Quality of Learning: the SOLO taxonomy New York: Academic Press

Biggs, J. and C. Tang. 2007. Teaching for quality learning at university (3rd ed.) Open University Press.

Bloom, B. S. 1956. Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.

Cowan, J. 1999. A handbook of techniques for formative evaluation. Routledge

Cowan, J. 2006. On becoming and innovative university teacher. Reflection in action. Buckingham: Open University Press.

Dolmans, D., I. Wolfhagen and P. Ginns. 2010. Measuring approaches to learning in a problem based learning context. International Journal of Medical Education. 2010; 1:55-60

García Martín, M.T., I. García Fernandez and J.C. Torre Puente. 2011. Enfoques de aprendizaje y otras variables cognitivo-motivacionales en alumnos universitarios de primer curso, Mondragon Unibertsitatea. Humanitate eta Hezkuntza Zientzien Fakultatea, Eskoriatza.

George J.W. and J. Cowan. 1999. Handbook of Techniques for Formative Evaluation. New York: Routledge

Graaff, E. de and A. Kolmos. 2003. Characteristics of Problem-based Learning, International Journal of Engineering Education, Vol. 19, No. 5, p657-662.

Guerra, A & Kolmos, A 2011, 'Comparing problem based learning models:

37

suggestions for their implementation', in J Davies, E de Graaff & A Kolmos (eds), PBL across the diciplines: Research into best practiceAalborg Universitetsforlag, Aalborg, pp. 3-17.

Illeris, K. 2007. How we learn. Learning and non-learning in school and beyond. 2nd ed. New York: Routledge

Jonassen, D. 1997. Instructional design models for well-structured and ill-structure problem-solving learning outcomes. Educational Technology Research and Development, 45, 1, 65-94, Springer: Boston

Kolb, D. 1984. Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. ISBN 0132952610.

Kolmos, A., E. de Graaff and X. Du. 2009. Diversity of PBL - PBL learning principles and models. Research on PBL. Practice in Engineering Education. Sense Publishers.

Mezirow, J. (2000). Learning to think like an adult: Core concepts of transformation theory. In J. Mezirow (Ed.) & Associates, Learning as Transformation (pp. 3- 34). San Francisco: Jossey-Bass.

Piaget, J. (1980). Recent studies in genetic epistemology. Cahiers de IaFondation Archives Jean Piaget

Pozuelos, F. 2007. Diseño, desarrollo e innovación del currículum. Materiales para la Docencia, 65. Servicio de Publicaciones Universidad de Huelva. (Ejemplo de Programa organizado y presentado en base a la enseñanza por proyectos)

Rogers, C. (1961) On Becoming A Person: A Therapist's View of Psychotheraphy Boston: Houghton Mifflin.

Savin-Baden, M. 2007. Challenging Models and Perspectives of Problem-Based Learning in Management of Change edited by Erik de Graaff and Anette Kolmos. SENSE publisher, pp 9-30

Taba, H. 1962. Curriculum Development: Theory and practice, New York: Harcourt Brace and World.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. (M. Cole, V. John-Steiner, S. Scribner, & E. Souberman, Eds.). Cambridge, Massachusetts: Harvard University Press.

Wigfield, A. 1994. Expectancy-Value theory of achievement motivation: A developmental perspective. Education Psychology Review, 6(1), 49-78.

38