rsd framework recent progress in the electrical ...€¦ · rsd symposium, melbourne, nov 27 2012...

RSD Symposium 2012

Nov 27, Melbourne

Monash University, Caulfield Campus

Brian Ng

Said Al-Sarawi

RSD Framework – Recent Progress in the Electrical & Electronic Engineering Context

Life Impact The University of Adelaide

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Outline of Presentation

• History of RSD Framework in EEE

• Why RSD made sense for EEE

• A Way Forward

• Wrap up and discussions

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

History of RSD Framework in EEE

• Timeline

• 2006: Applied to supervision and assessment of research project

component inside Masters coursework

• 2007: Second cohort of Masters coursework student; assessment

framework for Defence Masters thesis

• 2009: Trial of using RSD for Honours capstone project assessment

• 2010-current: School-wide use of RSD for Honours project assessment

• Findings:

• Moderately positive outcomes identified, e.g. reduced plagiarism

• School wide adoption is difficult, and required (significant) annual

tweaking

• Enjoys a good level of backing/support

• Various initiatives to “upscale downwards”

• RSD workshops, pilots

• School level engagement

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Why RSD made* sense for EEE?

• Provides clear and explicit assessment criteria

• Highly transparent assessment process

• Verbose rubrics allows for communication to a range of assessors

• Assessment process emphasises development and assessment of

research skills

• Valuable information for aspiring students

• Descriptors for skills which need be developed (and assessed

against!)

• BUT each of these also had its corresponding downsides

*and still makes!

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Brave New World – 2014 and Beyond

• Student interest EEE has declined over the past 10 years

• In favour of other disciplines such as Mechanical

• New educational environment

• Teaching modes and emphases change with technology

• Employment landscape also much different

• Need to produce newer breed of graduates

• External drivers loom large on the horizon, e.g. AQF8, international

market

• Curriculum refresh cycle is an opportunity to address these

• Much soul searching as a School?

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

EEE Workshop 2012

• School of E&E Engineering convened for a series of 4 workshops in

semester 2

• Key stakeholders were in attendance

• Aim for some “grand design” curriculum to address raised issues

• Discussions

• High level graduate attributes

• Pathway to developing them over 4 years

• Practical constraints

• Pedagogical framework

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Characteristics of Our B.E.

• Multiple facets: core science, technologies & applications,

professional practice, research

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Pedagogical Model

• Emphases on Engineering Science and the System Engineering

methodology

• Some subtle difference to the scientific method

• Latter is highly relevant to workplace practices for

contemporary engineering in Australia

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Consequences and Challenges

• Fundamental shift in thinking for our School’s teaching

• Current Levels 1 and 4 can remain with minor tweaking

• Revamp Levels 2 and 3

• “construct a bridge to connect the two banks”

• Significant investments in the coming years

• Delivery and assessments will need to be revised

• Material may not need much modification beyond repackaging

• No longer be obsessed with coverage

• Pace of change will need careful management

• Where to find time to do all this?

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Role of RSD in All This …

• Critical to get the “language” right – RSD is ready to inform this

process

• Much discussions on how students evolve over 4 years

• or

• Development framework

• Scaffolding for group work is novel (for us!)

• Assessment framework – rubrics

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Wrapping Up and Discussions

• Work in progress at Program level • Many factors driving this process: high complexiy

• Process needs completion by 2013, deployment 2014

• Contact us! brian.ng@.adelaide.edu.au said.alsarawi@adelaide.edu.au

Many thanks to generous support from library services at

Adelaide, and John, Lyn, Leanne, Cindy, Irene and all

other organisers of this Symposium

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Research Skills Development Framework

• Framework developed by Willison and O’Regan (2007)

– “Commonly known, commonly not known, totally unknown: a framework for

students becoming researchers”

• Makes explicit and coherent the incremental attainment of research

skills

– Emphasis on degree of autonomy in inquiry

– Context sensitive

– Useful for generating marking rubrics to drive learning activities towards

objectives

• RSD framework used in various disciplines

– Diverse range of disciplines at University of Adelaide

– Participation at other institutions, e.g. Monash, Macquarie, Melbourne, South

Australia

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Details: Use in Honours Project

• Traditional “cap-stone” project level 4 (Honours)

• 6 units; multiple objectives

– Professional practice skills, project management

– Technical outcomes are vital

– Research skills development embedded

• Major revamp in 2010

– RSD is identified as a suitable framework for research facet (guidelines)

– New assessment process for all project components (policy)

• Assessment matrices were developed using the RSD framework

– Make the once implicit explicit

– Serve to communicate feedback to students

– Also excellent documents to base conversations on

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

2010/11 Honours project assessment components

Activity Type Group / Individual Weight Due Assessor(s)

Proposal seminar Formative Group 5% Sem A week 3 Supervisor, co-supervisor

Stage 1 progress report

Formative Group 10% Sem A week 5 Supervisor, co-supervisor

Stage 2 progress report

Formative Individual 10% Sem A week 12 Supervisor, co-supervisor

Semester A performance

Summative Individual 10% Sem A week 12 Supervisor, co-supervisor

Final seminar Summative Group 10% Sem B break Independent assessor

Final report Summative Individual 20% Sem B week 11 Supervisor, independent

assessor

Project exhibition Summative Group 5% Sem B week 12 Independent assessor

Final performance Summative Individual 20% Sem B week 12 Supervisor, independent

assessor

Outcomes Summative Group 10% Sem B week 12 Independent assessor

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

2012 Honours project assessment components

Activity Type Group / Individual Weight Due Assessor(s)

Proposal seminar Formative Individual 10% Sem A week 3 Supervisor, co-supervisor

Stage A progress report & performance

Summative Individual 25% Sem A week 12 Supervisor, co-supervisor

Final seminar Summative Individual 15% Sem B break Panel of > 3 assessors

Final report Summative Individual 20% Sem B week 11 Supervisor, co-supervisor

Project exhibition Summative Group 10% Sem B week 12 Panel of > 3 assessors

Final performance Summative Individual 20% Sem B week 12 Supervisor, co-supervisor

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

RSD for Honours: Issues and Lessons

• Significant investment in effort for School staff

– Especially “core” personnel e.g. course coordinators, champion

• Transition arrangements must be communicated to students and

staff members alike

– Staff training on assessment principles would particularly be useful

• New for many students

– Increases quantity, and subsequently, the demand for feedback

– Interpretation issues for descriptions

• Acceptance issues can arise

– Among staff members: considered a waste of time

– Inhomogeneous application across the course

– Impact with regards to workload, supervision, feedback, … etc

• Evaluation should be integrated

– Embedded throughout the year

– Student interviews? Other reflections?

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Details: RSD for Masters (Defence) Minor Thesis

• Duration of the project – minor thesis : – 0.5 full time equivalent workload (12 units)

• Aims: – To train students in research and development methodology and

techniques

– Develop critical evaluation skills appropriate to their project topic

– Develop project management experience

• Outcomes: students are expected to gain skills in

– Engineering design, research and verification

– Critical evaluation of findings

– Project and time management

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Past Minor Thesis Assessments

• Students shall be assessed on: – the performance of research, planning, execution and management of

the project

– the quality of the project deliverables

– Written thesis as a standalone document

• Shortcomings of existing approach: – Clear assessment criteria do not exist

– It was hard to describe the assessment to non-academics

– It does not allow for direct assessment of research skill development

– Requirements to achieve various grades are not clear

– No explicit information on which skills need to be developed

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Minor Thesis Assessment Example

• Overall comments on the thesis and research outcome

• General feedback

• Technical feedback

• Pedagogical Assessment of the thesis from RSD

Framework point of view

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Pedagogical Assessment of Minor Thesis using RSD Framework

A. Candidate embarks on an inquiry and so determine a need for

knowledge/understanding – demonstrated level is 4 – 5/10

B. Candidate finds/generates needed information/data using

appropriate methodology – demonstrated level is 4 – 7/10

C. Candidate critically evaluate information data and the process of

find/generate this information data – demonstrated level 4 – 7/10.

D. Candidate organizes information collected/generated –

demonstrated level is 4 – 7/10.

E. Candidate synthesis, analyse and apply new knowledge –

demonstrated level 4 – 7/10.

F. Candidate communicates knowledge and the processes used to

generate it, with an awareness of ethical, social and cultural issues

– demonstrated level 4 – 5/10.

Recent Progress in EEE

RSD Symposium, Melbourne, Nov 27 2012

Final Assessment Matrix