JudgingLethbridge

Tuesday 14 May 2013

2

3

4

5

Thank You All

Roy GolsteynCWSF Chief Judge

Marc RousselCWSF Deputy Chief Judge

6

Exhibits – 1st Choice Savings Centre

Location

7

National Judging Committee

Judith SoonChair

Jeff HoyleVice-Chair

CarolineWhippey

PatrickWhippey

8

National Judging Committee

• Responsible for judging at CWSF• Responsible for supporting judging process at Regional

Science Fairs• Ensures integrity and consistency in judging• Educates about research ethics & academic integrity • Assesses compliance with YSC research policies

9

Canada Wide Judging Advisory Panel

Don ThomasEdwin Tam

Plus the members of the National Judging Committee

Roy Golsteyn Ben Newling

CWSF 2014 CWSF 2015CWSF 2013

Marc Roussel

10

Dianne FraserQ.O.P.

James GrantI.T.

Mark DzurkoCWSF 2010

Canada Wide Judging Advisory Panel

Plus the members of the National Judging Committee

11

Judging at CWSF

• CWSF is for and about the finalists

• The judging experience is the raison d’être

• The goal of the CJAP is to run a superb judging operation, and thus guarantee a successful CWSF.

12

Numbering the Projects 02 03 16

Challenge Category Counter01 Discovery 01 Junior 7 - 8 0102 Energy 02 Intermediate 9 -10 0203 Environment 03 Senior 11 - 12 0304 Health 0405 Information 0506 Innovation 0607 Resources 07

Energy - Senior - Project 16

13

Ordering the Projects

Projects ordered by Challenge Awards 01 Discovery 02 Energy

03 Environment 04 Health 05 Information 06 Innovation 07 Resources

14

Judging Task

• To be fair• To be sensitive• To be comprehensive• To be a positive role model

15

Preparation

• Check your Registration information is complete• Visit http://judging.youthscience.ca/• Review all the pages on this site• Review the Project Judging Form• Read the Project Reports, available 1 week prior• Prepare questions

16

Judges Orientation

From To Event

3:30 pm 3:50 pm Registration for all Team Captains

4:00 pm 5:00 Orientation for all Team Captains, morning & afternoon

4:00 pm 6:00 Registration

5:00 6:30 Supper. Sit at Morning Judging Team TablesReview morning judging process

6:30 7:30 Judging Workshop in PE 250

7:30 8:00 Sit at Afternoon Judging Team Tables. Review afternoon judging process

8:00 10:00 View projects without the finalistsReview log books and displayPrepare questions for tomorrowView extra projects in addition to your own.

Monday 13 May 2013

17

Judging Timetable - 1

Start End Event7:00 am 8:30 am Continental breakfast and orientation in teams8:20 8:50 Orientation in Teams. Attendance is mandatory,

even if your first judging slot is empty9:00 12:30 Excellence Award and International judging12:15 12:30 Lunch for judges without a 12:00 appointment.12:30 1:45 Lunch and discussion in judging teams1:45 2:00 Deadline for entry of results into data base2:00 2:15 Interdisciplinary Award Judges meet in teams2:15 5:30 Interdisciplinary Award Judging2:00 2:15 Special Awards judges meet in teams2:15 3:30 Special Awards judging

Tuesday 14 May 2013

18

Judging Timetable - 2

Start End Event2:15 2:30 Cusp Judges meet in teams2:30 5:30 Cusp Judging3:30 5:30 Celebration Judging5:30 6:00 Upon leaving, hand in all paper work in the boxes

provided.5:30 5:40 Finalists leave the judging hall

Tuesday 14 May 2013

19

Participants compete against all others in their grade category

• 10 Gold - $700• 20 Silver - $300• 40 Bronze - $100

Awarded in each of• Junior – grades 7-8 • Intermediate – grades 9-10 • Senior - grades 11 - 12

Excellence Awards

20

Judging Criteria

• Scientific thought (50%)• Originality & Creativity (33%)• Communication (17%)

• Visual display

• Oral presentation

• Project report

• Logbook

Evaluation of Excellence Awards

21

Judging Excellence Awards - 1• All interviews are scheduled, 9:00am – 12:30pm• Teams of 4 judges assess 7 projects each• Judging periods of 30 minutes: 20 minute

interview with finalists; 10 minute write-up• Each finalist is judged four times• Every team has a captain• If there is a fifth judge, pair up with another judge

but evaluate each finalist separately

22

Judging Excellence Awards - 2• 12:30 pm – 1:45 pm over Lunch• Teams of 4+ judges discuss and rank projects• CONSENSUS - complete forms• Each team member has an equal voice • Each project receives an appropriate score, composed

of Level (1 – 4) and Rating (0 – 9)• Enter results into Database using the Playbooks• Pass in all paperwork to Admin

23

Project Judging Form - 1Part A Scientific Thought 50%

Experiment Innovation Study

Level 1 - LowReplicate a known experiment to confirm previous findings .

Build a model or device to duplicate existing technology or to demonstrate a well-known physical theory or social/behavioural intervention.

Existing published material is presented, unaccompanied by any analysis.

Level 2 - FairExtend a known experiment with modest improvements to the procedures, data gathering and possible applications.

Improve or demonstrate new applications for existing technological systems, social or behavioural interventions, existing physical theories or equipment, and justify them.

Existing published material is presented, accompanied by some modest analysis and/or a rudimentary study is undertaken that yields limited data that cannot support an analysis leading to meaningful results.

Level 3 - GoodDevise and carry out an original experiment. Identify the significant variables and attempt to control them. Analyze the results using appropriate arithmetic, graphical or statistical methods.

Design and build innovative technology; or provide adaptations to existing technology or to social or behavioural interventions; extend or create new physical theory. Human benefit, advancement of knowledge, and/or economic applications should be evident.

The study is based on systematic observations and a literature search. Appropriate analysis of some significant variable(s) is included, using arithmetic, statistical, or graphical methods. Qualitative and/or mixed methods study should include a detailed description of the procedures and/or techniques applied to gather and/or analyse the data (e.g. interviewing, observational fieldwork, constant comparative method, content analysis).

Level 4 - ExcellentDevise and carry out original experimental research in which most significant variables are identified and controlled. The data analysis is thorough and complete.

Integrate several technologies, inventions, social/behavioural interventions or design and construct an innovative application that will have human and/or commercial benefit.

The study correlates information from a variety of peer-reviewed publications and from systematic observations, and reveals significant new information, or original solutions to problems. Same criteria for analysis of significant variables and/or description of procedures/techniques as for Level 3.

24

Part B: Originality and Creativity 33%Level 1 Level 2 Level 3 Level 4The project design is simple with little evidence of student imagination. It can be found in books or magazines

The project design is simple with evidence of student imagination. It uses common resources or equipment. The topic is a current or common one.

This imaginative project makes creative use of the available resources. It is well thought out, and some aspects are above average.

This highly original project demonstrates a novel approach. It shows resourcefulness and creativity in the design, use of equipment, construction and/or the analysis.

Project Judging Form - 2

25

Part C: Communication 17%Communication is based on four elements: visual display, oral presentation, project report with background research, and logbookLevel 1 Level 2 Level 3 Level 4Most or all of the four elements are simple, unsubstantial or incomplete. There is little evidence of attention to effective communication. In a pair project, one member may have dominated the discussion.

Some of the four elements are simple, unsubstantiated or incomplete, but there is evidence of student attention to communication. In a pair project, one member may have made a stronger contribution to the project.

All four elements are complete and demonstrate attention to detail and substance. The communication components are each well thought out and executed. In a pair project both members made an equitable contribution to the presentation.

All four elements are complete and exceed reasonable expectations of a student at this grade. The visual display is logical and self-explanatory, and the exhibit is attractive and well presented. The project report and logbook are informative, clearly written and the bibliography extends beyond web-based articles. The oral presentation is clear, logical and enthusiastic. In a group project, both members contributed equitably and effectively to the presentation

Project Judging Form - 3

26

Project Judging Form - 4

Use the rubric to assign a level to Parts A, B and C for the project. In addition to the Level, please assign a single letter rating: H (high), M (medium) or L (low) that reflects the quality of the project and its strength relative to the other projects you have assigned the same level. Note: Finalists will not see this sheet.

Part A: Scientific Thought

Level 1 - 4 Rating (HML)

Part B: Originality & Creativity

Level 1 - 4 Rating (HML)

Part C: Communication

Level 1 - 4 Rating (HML)

Judging Notes

Notes on your verbal feedback

3 H

2 M

4 M

Graphing is weak. Spelling errors on board. Weak lab notebook

I enjoyed your explanation of kinetic energy. You should work to strengthen your understanding of your graph, and learn about error bars. Explained Electric Current

27

Judging Team WorksheetConsensus Scores – Scientific ThoughtAfter filling in the judges’ names and project numbers, enter each judge’s level and rating (H,M or L) for each project.Following discussion of each project’s scoring by all team members, enter a consensus level (1 - 4) and rating (1 – 9) in the right hand column .Note: Consensus values are determined through team discussion, not by mathematical calculation (e.g. mean, median, mode)Use the Blackberry Playbook to enter the consensus values for each project.

Judge ConsensusLevel Rating

Project Abbott Baker Combes Dawkins Elm

010204

010205

010206

010209

010211

010214

010220

L3 M 2 H 2 L 3 M 3 3 2

3 M 2 L 2 M 2 L 2 L 2 3

Repeat for: (b) Originality and Creativity (c) Communication

Enter into Playbook

28

Entering Team Results

Show the Playbook screen here

29

Feedback During Judging

Give feedback during judging at the end of each interview.

Feedback is very important to the finalists!

Remember: Encourage, encourage, encourage!

Be constructive in your comments• Balance a thing to improve with two positives about the

project.

*New this Year*

30

Feedback During Judging - 2

Make a note of the feedback you provided in the Judging Notes section of the Judging Form. e.g.

• suggested how to interpret the data better; • suggested a book or article to be read;• explained a concept poorly understood e.g. kinetic

energy

As long as any feedback is noted on the judging form, it can be included in the discussion prior to ranking. It should not have a substantial impact on the final results.

31

Please Sign your Name

Be sure to sign your name on the finalist’s timetable before you leave each project.

32

Lunch

We need to maximize the time spent in discussion.

We will call your table number for lunch.

We will ensure you spend only ten minutes in the line up.

Discussions must be complete by 1:45 pm.

33

Afternoon JudgingFive Judging Activities

• Cusp Judging: Review projects close to boundaries• Top Gold• Gold – Silver• Silver – Bronze• Bronze – no award

• Interdisciplinary Awards• Special Awards• Challenge Awards• Celebration Judging

34

Excellence Award Cusp Judging

Time: 2:30 – 5:30 pmTeam Captains and Category Leaders meet at your tables for instructions.Interview projects on the Cusps:

• Top 6 Gold• Gold – Silver boundary• Silver – Bronze boundary• Bronze – no award boundary

35

Excellence Award Cusp Judging 1

Working with the Team Captain, enter the project numbers to be judged,as assigned by the Category Leader

Project Number Rank

Comment

Project Number Rank

Comment

Project Number Rank

Comment

5 more projects

010205 8

010220 5

030209 1

Individual Judge

36

Excellence Awards Cusp Judging 2Team Consensus

Enter the Ranking (e.g. 1-8) of each project by each judge on your team. Through reasoned discussion, determine a consensus rank for each project highest (1) to lowest (8)

Project Number

Judge’s Name 010304 010316 020305 020314 040306 050308 070304

Henry Higgins

Ian Ibsen

Janet James

Kelly Kaczka

Lorna Lewis

Mandy Maclin

Consensus Rank

8 7 3 1 2 4 58 6 4 3 2 1 7

6 8 5 2 1 3 7

7 8 5 2 3 4 6

7 6 8 1 2 4 3

6 8 4 2 1 7 5

7 8 4 1 2 3 5

1morecol

37

Excellence Awards – Final Cusp Ranking

Rank Project # Title (abbreviated)

1

2

3

4

5

6

7

8

010103

020106

030119

010115

070108

060111

040102

050109

Wind Turbines Noise Stress

Sleep on This

Heavy Metal Mitigation

Can Tires Replace Furnace Oil?

Seed preconditioning to increase crop yield

Can Your Diet Prevent Alzheimers?

Distraction

Does An Electric Field Affect Plant Growth?

38

Each Finalist can self-nominate for up to three Interdisciplinary Awards

Interdisciplinary Awards - 1

Examples• The Manning Innovation Achievement Awards• Renewable Energy Award• Canadian Stockholm Junior Water Prize

39

Interdisciplinary Awards - 2

Some projects will not be well matched to the criteria.

Judge them with enthusiasm.

Our emphasis is on celebrating the finalist’s achievement, not just on selecting the winner.

40

Interdisciplinary Awards - 3

Time: 2:00 – 3:30First round interview is scheduled. First Interview starts at 2:00 pmTen minutes per interview6 interviews per judge maximumEach project is judged twiceEliminate the bottom 80% in round one

Round One

41

Interdisciplinary Awards - 4

Repeat the Round 1 process on the remaining 20%

A third round may be required for a few awards

Final result is by consensus

Complete paperwork and hand it in

42

Project Results

Interdisciplinary Award - 5

Renewable Energy Award - JuniorAn outstanding project related to both energy and air quality with a demonstrated interest in environmental stewardship.Yes = Top 20%; No = Bottom 80% or the project does not meet the award criteria.

Top 20%

Round 1 Yes No

Comments

060102Frost BusterMadalon Burnett

Top 20% go on to Round 2

Project on melting ice. Does not know what Latent Heat means

43

Interdisciplinary Award Final ResultRenewable Energy AwardJuniorAn outstanding project related to both energy and air quality with a demonstrated interest in environmental stewardship.Please give the winner and one alternate

Project # Name(s) Project Title1

Alt010112 Albert Atkinson A Better Air Filter

060105 Barbara Bull

Team Leader Signature

Yardlee Yates

Using Microbes to Remove Metals

44

Special Awards

Judging based on the Excellence AwardsSelf-nomination not required

Examples• Canadian Association of Physicists Prize• Award for Excellence in Astronomy

45

Special Award Judging - 1

• 2:00 pm Meet at your tables• Review the list of highest ranked projects who

are eligible• Interview the highest ranking candidates

46

Special Award Judging - 2

Project Number Meets Criteria Yes No Rank

Comment

Project Number Meets Criteria Yes No Rank

Comment

Project Number Meets Criteria Yes No Rank

Comment

4 more projects

010205 3

010220 2

030209 1

Individual Judge

47

Special Award Final ResultCAP Physics PrizeSeniorCanadian Association of PhysicistsAn outstanding project in the Physical and Mathematical Sciences related to Physics

Project # Name(s) Project Title

1

Alt010306

010315

Gryb Carbon Nanotubes

Hammond Luminescence in Rare Earths

48

Challenge Awards• Challenge Awards recognize the top project in

each of the seven Canada Wide Youth Science Challenges in each Grade Category.

– Junior - $500 and certificate– Intermediate - $750 and certificate– Senior - $1000 and certificate

49

Challenge Award Judging - 1

• 2:00 pm Meet at your tables• Review the list of highest ranked projects who

are eligible• Interview the highest ranking candidates

50

Challenge Award Judging 2

Project Number Meets Criteria Yes No Rank

Comment

Project Number Meets Criteria Yes No Rank

Comment

Project Number Meets Criteria Yes No Rank

Comment

4 more projects

020205 3

020320 2

020109 1

Individual Judge02 Energy

51

Challenge Award Final Result02 Energy

Project # Name(s) Project Title

1

Alt020109

020205

Ireland Liquid Solar Cells

Jones Wind Turbines – Friend or Foe?

53

Celebration Judging 1Typically about 80 finalists will not get judged at all during the afternoon for:• Excellence Award• Interdisciplinary Award• Special AwardMost will spend 2 hours without an interview We will give them two Celebration Judgings

54

Celebration Judging - 2Many judges will finish judging by 4:00 pm• Go to the Celebration Judging Table• Select two projects from the list• Visit those projects for 10 - 15 minutes• Celebrate the work done• Give as much feedback as you can – be

constructive• Encourage! Encourage! Encourage!• Goal: Every finalist has two afternoon interviews

55

Ambassadors

• Dressed in UV shirts• All are previous winners at the CWSF• Support finalists and resolve any issues

– My Judge has not shown up– My computer just died– I am not feeling well

56

Mentorship - 1Level Description

0 I did not receive any mentoring.

1 I exchanged a few emails or phone calls, and/or met with my mentor once or twice to discuss my ideas.

2 I had occasional contact with my mentor by email or phone, and/or met occasionally with my mentor who provided some advice or materials.

3 I had regular contact with my mentor by email or phone, and/or met regularly with my mentor who provided advice, materials, assistance with design/testing, or data analysis.

4 I had regular face-to-face contact with my mentor and regular access to advice, materials, space, equipment, design/testing, or other personnel in a specialized facility.

5 I worked closely with my mentor over an extended period of time to develop the project idea, plan and conduct the research/development, and analyze the results or test the innovation.

57

Mentorship - 2

• All professional scientists receive extensive mentoring. • Read the section Projects – Mentorship here:

http://cwsf.youthscience.ca/fairs/current?tid=163• Does the finalist have a good grasp of the project, and did

he/she do the work?• Do not discount a project just because it was mentored.

58

Non-Disclosure Agreement• Judging information is confidential and is

not discussed outside the judging hall.• Intellectual property belongs to finalists• All digital notes and 5 page reports are to

be deleted after judging is over• Do NOT discuss judging matters on social

media eg Twitter, Facebook.

59

Conflict of Interest

• are related to the finalist• have judged the project before• have mentored the project• have other potential conflicts of interest

THEN

IF YOU

You must consult the Chief Judge

60

Keep All Paper

PLEASEDO NOT

TAKE ANY PAPER AWAY

All paper is sorted and filed for a year

61

Judging 101

A Dramatic Presentation

Patrick & Caroline Whippey

62

The Judge as seen by…

62

Fellow Judges Finalists

63

Oops !

63

The following are based on real events, and they have all happened.

Viewer Discretion is advised.

64

Over-enthusiastic Judge

64

• do not give finalists false hope• “I enjoyed meeting you.”• “I particularly liked….”

65

Sarcastic Judge

65

• this is not a Msc/PhD examination• do not belittle - be joyful, not judgmental• Every project is to be enjoyed and valued

66

Insensitive Judge

66

• Never discuss the projects in the exhibit hall where finalists are present

67

“Helpful” Feedback

67

• Give constructive feedback: balance positive and negative

• Encourage, encourage, encourage

68

In Summary

68

We want every finalist to finish judging and say, “Wow, that was a fantastic experience”

Please help us make that happen!

69

Questions?

70

Thank You

Again !

Roy GolsteynCWSF Chief Judge

Marc RousselCWSF Deputy Chief Judge