Investigating

Student

Perspectives on

School Wide

Research Courses

Kenneth Fordyce, Rory Ewins, Pete Allison,Ioanna Papageorgiou

4 June 2015

Research Courses

Sources of knowledge: Understanding and

analysing research literature Oct-Nov

Conceptualising research: Foundations,

assumptions and praxis Jan-Feb

Research methods: Planning research

Mar-Apr

Blended delivery to 300-500 MSc students per

year from 2010/11 onwards

Research Objectives / Assumptions

Insights into student perspectives (going beyond

SSLC consultations and course feedback) can

feed into the ongoing development of the

research courses.

These insights can also inform a wider

understanding of ‘current factors’ (international

students, online learning, research courses)

which play a key role in Masters level study in UK

universities.

Research Questions

1.Does the course content and format represent specific

challenges for international students? Ken – Part 1: Criticality

2.How is the blended learning format perceived by students in

relation to other course formats? Rory – Part 2

Research Methods

Baseline questionnaire to collect demographic data and

information on prior experience of studying research

methods, and of studying in an online environment.

Focus groups conducted in early February and early July

of the Masters academic year. 19 students took part in

February, and 8 of these students also took part in July.

Reflective diaries aimed at capturing gradual change

over time in relation to the challenges of the research

courses and the way they did or did not connect with

other subject courses and the dissertation.

Data: Questionnaire

239 responses

28 nationalities; 170 responses (71.1%) from

Chinese participants

Gender ratio: female 89.5%: male 10.5%

81.5% had not lived in an English-speaking

environment previously

85.2% had not previously studied at a university in

an ‘English-speaking’ country

Previous experience of studying online: Yes 43.5%:

No 56.5%

Data: Questionnaire

Statements (percentage who ‘agree’ or ‘strongly agree’):

I use the Internet regularly: 97.9%

I am confident about using the Internet: 76.8%

I am comfortable with learning online as part of my studies:

52.9%

I am comfortable with the idea of doing all my learning online:23.1%

Undergraduate Dissertation: YES, 84.5%: NO, 15.5%

Previous Study of Research Methods: YES, 67.4%: NO, 32.6%

Focus Groups

2 time points in PGT year: February and July

Focus on: (1) blended learning aspects of the

Research courses; (2) critical thinking as a key

feature of ‘Research and Inquiry’ in a UK PGT

context

Time 1 (n = 19): 3 groups of Chinese L1 students; 1

group consisting of 2 Greek and 1 Turkish

students

Time 2 (n = 8): 2 groups of Chinese L1 students

(from original 3 groups); same group of Greek

and Turkish students

Initial Conclusions

The majority of students studying on the

research courses are dealing with a high

degree of ‘newness’: language,

academic context and expectations,

research orientation, online learning, and

so on.

Critical Thinking

Why focus on critical thinking?

One of the main attributes distinguishing SCQF Level 11 from

Level 10, e.g., Masters study from UG study.

Sources of Knowledge (1st generic research course) – first

two learning outcomes:

(1) demonstrate critical awareness of current debates

concerning the purposes and interpretation of research;

(2) evaluate strengths and weaknesses of different research

paradigms and philosophies…

Critical thinking often an ‘assumed’ generic skill for PG

students but may not always be a component of UG study

International students required to attain academic and

linguistic levels to enter Masters programmes but no guarantee

these encompass skills of critical analysis and thinking.

Why focus on critical thinking?

…because everyone have their

definition about critical thinking so I

think because it’s different from

person to person so it’s really hard

for me to say ah I really understand

what critical thinking is [C5P9]

What is critical thinking?

“Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing,

synthesizing, and/or evaluating information gathered from,

or generated by, observation, experience, reflection,

reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal

intellectual values that transcend subject matter divisions:

clarity, accuracy, precision, consistency, relevance, sound

evidence, good reasons, depth, breadth, and fairness.”

(as defined by the National Council for Excellence in

Critical Thinking, 1987; emphasis added)

What is critical thinking?

Moore, T. (2013). Critical thinking: seven definitions in search

of a concept. Studies in Higher Education, 38, 506-522.

Identified 7 main strands through interviews with academics

in history, philosophy and cultural studies:

i. Judgement

ii. Scepticism

iii. Simple originality

iv. Sensitive readings

v. Rationality

vi. An activist engagement with knowledge

vii. Self-reflexivity

International Student Entry Requirements:

Example of MSc TESOL

Normal entry requirements

All applicants must possess a first degree in a language-related

subject area e.g. Linguistics. Applicants must receive a 2:1/1st

Honours degree or an equivalent overseas qualification.

Information about English language requirements can be found

through the following link: English language requirements

Please note: Students studying at Chinese 211 institutions require

a minimum overall average of 80% for courses and at least 80%

on the dissertation; for non-211 institutions, an average of 85% is

required for courses with 80% on the dissertation.

There is no requirement in relation to being able

to apply criticality in academic contexts.

Scottish credit and qualifications

framework (SCQF): Level 11

http://scqf.org.uk/the-framework/scqf-levels/

Most frequent adjectives in the Level 11 descriptors:

(1) critical (x 7)

(=2) professional, significant, specialised (each x 4)

• critical understanding (x2)

• critical knowledge

• critical awareness

• critical analysis

• critical evaluation

• critical reflection

Moray House School of Education:

Taught Masters Generic Handbook

“The university has general requirements for work at Masters level

that are aligned to the Scottish Credit and Qualifications

Framework at level 11.” (p.13)

Most frequent adjectives in the grade descriptors for the six

categories on the Common Marking Scheme:

1.coherent x 8

2.critical x 7

3.academic x 6

4.clear x 6

coherent framework (x 4), coherent and analytical framework (x 2),

coherent understanding

critical discussion (x 4); critical reflection; critical understanding; critical

evaluation

Perceptions of critical thinking expressed

in Focus Group 1

Multiple perspectives 10

Weighing up pros and cons 10

NOT passive acceptance of ideas 4

Arguments require supporting evidence 4

Objectivity/avoiding bias 2

Expressing own opinions 2

Think deeply about reasons behind opinions 2

Criticality applied to everyday life 2

Refer to multiple resources 2

Ask myself am I convinced? Why? Why not? 2

[a further 23 aspects mentioned by 1 participant each]

no right or wrong answers

C2P5: I think it refers to thinking in different perspectives and

there is not right or wrong answer.

C2P9: I think when we read all the articles we cannot get a

specific conclusion you know always the paper will say further

study needs to be conducted to see more deeply to the issue

yeah I think because the authors of different papers they view

an issue from different angles so of course they will have

different answers so yeah I think critical thinking is important for

us to know that there is no right or wrong answer just based on

the perspective you choose of course you need to use different

methods to approach your questions.

NOT passive acceptance of ideas

Ok so when someone says something or writes something

you need to be critical … and you can’t take it for

granted that ok if he says so it’s true you have to find to

search for what lies behind his views and if there are any interests hidden or any interests that he has so he says that

or he writes that and the reasons why he or she would

support or not support what he says or would contradict

another’s opinion and also look on, if we talk about

research, look on the means that he has used to prove

what he says or to support what he says because maybe they are not so reliable or there is no validity and also look

for strengths and weaknesses. [GTP20]

3 statements about critical thinking

[1] Critical thinking is a key component for Masters level

study.

Very near complete agreement with this statement.

[2] Critical thinking is a new approach to learning for me

7 participants agreed; 4 suggested some degree of

agreement; 8 disagreed

A mixed picture but generally clear that it is a new

approach/concept to many, which suggests that

awareness of the concept should not be assumed.

[3] Critical thinking makes me feel uncomfortable

6 participants suggest some degree of concern/anxiety in

relation to critical thinking while the remainder disagreed.

anxiety mostly in relation to assessment

When I am working on my assignment because sometimes I

think it’s very difficult to refute a certain theory, I can even

though I disagree with this theory personally I can’t find the

references to support my idea so sometimes I feel stressful but of

course I know that it’s a good approach for our development,

yes but sometimes I just feel stressful [C1P2]

when I was approaching to the day of assignment and I am

always uncertain about whether I was being critical or not in my assignment and it sometimes feel insecure because you don’t

know I am not sure whether I was being critical and which

opinion … agree with [C2P8]

uncomfortable being in grey areas

when you need to shape your own opinion especially on

matters that are really complicated political for example or I

think it is really difficult because there is not one truth there

are many truths so I think it’s not the easiest thing to do to shape a firm and strict opinion on something I mean I don’t

because there is not black and white so many times it’s

difficult to really explore and see if someone has, why

someone says that or what are the interests because it’s not

always so obvious even if you read from different sources I

don’t think it’s always that easy I mean ok for now I mean in real life in matters like more political [GTP20]

2nd set of focus groups

Students asked which of the identified features

(16 taken from 1st of focus groups) they most

and least associated with critical thinking.

Findings slightly more limited due to the lower

number of participants.

There was a general sense of increased

positivity and confidence in relation to being

critical.

Anxiety about critical thinking only identified in

relation to assessment.

Personal strength; finding own voice

critical thinking means you have a complete view of

something and therefore you have a strong basis you

have a strong basis on what you say or what you express

so there would be no need in feeling anxious or

uncomfortable [GT2P20]

Yeah because as my friend said I basically don’t I have to

rely on a wide range of references and build an argument using them so also well basically I was reading the article

before the master level study I was basically reading the

article and taking many things for granted but now

actually I leant to compare with other definition of the

arguments on the same topic and sort of yeah and trying

to find my own voice [GT2P18]

A generic cognitive skill

yeah I think of course I think for me critical

thinking is different from other you know what

we gained from all the courses it’s just special

skill I think it’s for example maybe when we graduate from here we will forgot what we

learnt in the specific courses I mean not

knowledge part but critical thinking is a skill that

we can take away and then yeah for all the

things so I think of course it’s helped me to study here for the master level [C5P9]

Implications: Teaching

Avoid the ‘non-pedagogy, of osmosis’

(Turner, 2011, p.21)

Make the implicit explicit.

assessment validity?: endeavour to

uncover and clarify implicit assumptions

lurking in assessment criteria

Implications: Research

When assessors require students ‘to be more critical’,

what are they actually expecting?

‘The evident importance of critical thinking in higher

education, as well as the seeming pedagogical

uncertainty surrounding the concept, suggests there is a need to find out more about how the ideal is

actually understood and used by academics…

Take categorizations identified from focus groups for

larger scale empirical work across different subject

disciplines, students from different educational

backgrounds, English proficiency levels, etc.

Blended Learning

Blended learning

exposes students to a greater variety of learning tools,

enhancing the overall learning experience (Eugenia 2008)

enhances students’ reflective skills and their ability to

reframe and reinterpret existing knowledge, values and

beliefs (Cooner 2010)

allows students the opportunity to coordinate with their

classmates and consider material in considerably more

depth (Cooner 2010)

provides learners with greater control over the pace of

learning, instructional flow, selection of resources and time

management (Chung and Davis 1995)

Blended learning

Many students view online tasks as additional or extra-

curricular work, choosing not to participate or engage

(Orton-Johnson 2009)

Those who did engage eventually abandoned online

learning tasks, perceiving traditional texts as more

‘authentic’ sources of academic knowledge (Forsyth &

Archer 1997; Johnson & Kiviniemi 2009)

Students often experience frustration, difficulty and

confusion utilising web-based study materials (Hara & Kling

2000, 2002; Parkinson, Greene, Kim & Marioni, 2003)

1st focus groups, February

Participants were asked to rank by usefulness:

Watching videos

Reading video transcripts

Reading books/articles

Using the discussion board

Having tutor input

Participating in group work/discussion

then discuss their usefulness for different purposes.

1st focus groups

What was considered most useful?

Books/articles, teacher input and group discussion featured, but 9

nominated the videos as most or equal-most useful.

Watching videos. It’s more systematic … and if I can’t catch up with

some ideas I can repeat it back again. [C1P1]

Some of the videos are very conducive to your understanding of

the abstract concept … also because we can improve our listening

ability. [C1P2]

The videos. I like them very much. I think that when I watch them

with the transcripts together I am getting more information than the

books. [C4P19]

1st focus groups

What was considered most useful?

Some emphasized the transcripts over the videos.

Some jokes I can’t understand, so I will read the transcript, why the

jokes make others laugh. [C3P17]

When you watch the video at the same time, you can grasp what

the lecturer is talking about. [C2P7]

In most videos the lecturer, they might have accent, sometimes

their tone make me feel sleepy, to be honest. ... And also sometimes,

like, they speak too fast, and English is not my first language, so it

may be hard to understand what they are talking about, so I highly

depend on the transcripts. [C3P11]

1st focus groups

What was considered least useful?

A few nominated the video transcripts or the videos themselves.

Sometimes the pace in the video is so slow, and I can read the

transcript in a short time to get the information, and also if I listen to

something it’s easy for me to get distracted. [C2P5]

Sometimes on the transcripts you can’t find the charts or the graphs

… and when you watch the slides in the video it will help you better

understand what the lecturer is saying. [C2P7]

I can learn from different ways, like reading articles I learn from

reading, and for me, video [helps] train my listening … just train my

learning strategies. … Video is good, but video transcript, I mean for

me it’s the least useful. [C2P9]

1st focus groups

What was considered least useful?

Most nominated the discussion board.

We hardly ever use it. We are not initiated to use it. Maybe if

someone throws a question then other people will start to use it, but

it’s just, like, nobody. [C1P4]

We will put our homework on the discussion board and we will reply

to others, but we won’t read all of the students’ work. It’s a little bit

[boring]. We will read one or two, or some longer ones, just ignore

others. [C3P17]

Group discussion is face-to-face and is more intense than the

discussion board. [C2P8]

1st focus groups

What was considered least useful?

Same gave cultural reasons for not using the discussion board.

Most of the students are from East and we don’t [use it] to

communicate. If we have questions we prefer to contact with the

tutor immediately, like email them or ask them questions, or we

prefer to talk with our peers … face-to-face. [C3P11]

The tutor said we can just use the discussion board instead of email

her directly [so that she] can solve the questions for all of us, but for a

lot of Asian students … we are worried we are asking silly questions

[that] may bother others, so we are used to just email the tutor

directly. [C3P17]

1st focus groups

What was considered least useful?

Others gave practical reasons for not using the discussion board.

[We were asked] to comment one of the article your classmate

have written, and I noticed that not everyone finished this task. Even

if someone have written a comment on, it is just very general ideas,

so I don’t think I can learn something from the discussion board.

[C3P14]

It might be not very user-friendly. If you, like, make the discussion

board … like most of us are used to … we might engage in this

discussion board more often. [C3P12]

2nd focus groups, July

Have you revisited any of the materials from the three courses

(the readings, the videos, the video transcripts, or the posts on

the discussion board) to help with your assignments, other

courses, or your dissertation? If so, which ones and why?

Little of the uniquely online material was revisited. Most

nominated the readings.

2 of 8 participants mentioned revisiting videos and

transcripts.

2nd focus groups

Now that you have done all three courses, what is your

opinion about the value of the discussion boards?

Some participants still said they didn’t use the discussion

boards, and saw little value in them.

I think it’s not very useful or helpful because in my class ... few

students use it. Maybe the students are busy with their

assignments or dissertation so nobody uses it. [C5P16]

It just never occurred to me to use it. [C7P12]

2nd focus groups

But others now saw value, even if they didn’t use them.

I mostly used them in the last course which was for the poster

proposal, but I never wrote. … I mostly opened it to see the notes of

the tutor and maybe if there were some questions of others. … I

can’t say I was part of it—I never posted something—but I think it’s

very good that they exist. [C6P20]

I didn’t use it very often [but] it’s good for I think for the peers to

communicate with each other and also for the teacher-peer

communication. … I was checking it regularly. … My opinion is

positive actually because these people asking their questions and …

they were sharing their writing for formative assessments, so people

were commenting on each other’s work, so in that times it was okay.

[C6P18]

2nd focus groups

For research methods number 1 and 3 … it was just like blank.

People say you can use it, but nobody posts, and the teacher

doesn’t help [by] initiating some posts. But the second one ... [the

teacher] initiated the conversation and students asked follow-up

questions, which makes it all kind of natural. … So I think it

depends on the teacher. … [The teacher in the second course]

specifically said, ‘If I have mentioned this question in the board

don’t ask me in person,’ so it’s sort of, like, intimidating. You want

to check the board, you know, you don’t want to make [a] fool of

yourself, so I think for that course specifically was fully used by ... a

lot of people. When it’s used it could be very helpful. When it’s not

used I mean how can it be helpful? [C5P4]

2nd focus groups

Would regular tutor input improve their usefulness? Yes.

If the tutor is going to post things there then I will make sure to check

it up, if it’s the teacher. [C7P12]

I can’t remember something important that I read there … from the

tutor. I remember only that she put the presentations from the

course, and maybe she replied to some questions, but I can’t

remember something valuable. Yeah, it would be more beneficial.

[C6P19]

Why would we choose to use the discussion board instead of other

communication … we have QQ, we have WeChat, we can even

chat on Facebook if we want to, so it’s all about the teacher putting

input … otherwise I don’t see why we would prioritise the discussion

board [over] other ways to communicate. [C5P4]

2nd focus groups

Would the possibility of anonymous posting improve their usefulness?

6 out of 8 thought so, 2 were unsure.

I think that when you know that there will be anonymity you are

more free to ask questions that maybe may seem otherwise stupid,

or you would be maybe shy to ask, so yes, why not? [C6P20]

I don’t know. Actually, for me, it didn’t make very much difference,

anonymity, but maybe it could be beneficial for some, like,

inconfident people to raise their questions, yeah. [C6P18]

I don’t know, because I can’t remember others’ name. [C7P8]

2nd focus groups

I was in a group project, … five people, and two of us were just …

not working with the rest of the three girls, and we were really

having trouble. … We wanted to talk to the teacher, but it was

sort of, you feel embarrassed … you don’t want them to have a

negative impression. … If I could … put it on the discussion board

… then maybe the teacher as a figure can talk to us as a group

instead of knowing who said that … that’s something that would

make you feel uncomfortable when you talk to the teacher if he

recognise your face. [C5P4]

2nd focus groups

Can you identify ways that the online materials enabled you to be

more reflective? Generally, the videos helped here.

I think so … especially in video, because they, usually they make

examples in their own research. [C7P8]

The videos, [where] those people who talked were referring to

specific examples of research, so it was … easier to understand

practically what has to be done. So yes, they helped me. Not all of

the videos, but I think most of them were very helpful. [C6P20]

Yeah … for example, if they were talking [in the videos] about

ethical concerns and they were, like, sort of matching their own

ethical concerns in the research with the material … it just made me

a bit more reflective toward my own research. [C6P18]

I don’t think the videos helped me to be reflective, because I could

barely finish watching the video. [C5P4]

Some blended conclusions

Videos were valued by participants for extending the readings,

and for their repeatability.

Transcripts enhanced the value of the videos, and were preferred

by some, but sometimes missed key information.

Discussion boards went largely unused, with exceptions.

Stronger tutor input and direction could turn this around, as most

participants see this as adding value.

Anonymity could also remove inhibitors to posting.

Some blended conclusions

We observed some of the problems with blended learning

reported in the literature, and not all of the supposed benefits.

But there were indications of practical steps that can address

perceived problems.

If students don’t see the value in an online resource, they won’t

use it.

Improved direction from course developers, course organisers and

tutors adds value…

And so does greater engagement.

Most of the perceived problems are solvable.