baseline data evaluation report

DER-NSW 2010 reporting: Implications from the 2010 data collection

Dr Sarah K. Howard [email protected]

Thursday, May 5th 2011

Evaluation RQs

•  How does the DER-NSW program influence teacher pedagogy?

•  What is the influence of the DER-NSW program on students’ understanding, skills, and attitudes?

•  What are the consequences of the DER-NSW program on students’ educational outcomes?

Response rate

•  Student survey A: Total started survey, 17,079 Total completed survey: 13,760 (80.6%)

•  Student survey B: Total started survey, 18,659 Total completed survey: 14,692 (78.7%)

•  Teacher survey: Total started survey, 4,627 Total completed survey: 3,637 (78.6%)

Distribution across subject areas

Other 16%

English 13%

Mathematics 11%

Science 15%

HSIE 13%

PDHPE 6%

Creative Arts 7%

TAS 17%

Languages 2%

Student gender distribution

Male 47% Female

53%

Aboriginal or Torres Straight Islander origin

25183

1698 400 382

0

5000

10000

15000

20000

25000

30000

No Yes, Aboriginal Yes, Torres Strait Islander

Yes, both Aboriginal and Torres Strait

Islander

.8%

Years in an Australian school

371 689 1373

15436

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

less than 1 year 1 to less than 3 years

3 to 7 years more than 7 years

3.3%

Student access to computers at home

No 5%

Yes 95%

Do you have access to a computer at home?

5%

92%

3%

Is this home computer connected to the internet?

Not connected

Yes connected

I don't know if it is connected

RQ #1, Pedagogy

•  Teachers who felt using ICT was ‘fun’ rather than ‘important’ showed: –  Positive conceptions of students’ creativity & organization –  Beliefs that students showed more understanding when using ICT in the

classroom –  They were more likely to use email

•  We do see a significant relationship between teachers’ primary subject area and their use of ICT in the classroom –  Mathematics is consistently lower on shifts towards student-centred

practices and ICT use –  Other subject areas were less consistent

RQ #1, Pedagogy

Note. ‘0’ = Never, ‘4’ = 1-3 times a week, ‘8’ = Many times a day

5.68 6.02 6.15 6.36 6.53 6.67 6.83 6.95

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

How often do you use a computer when teaching?

Mathematics Languages PDHPE English Creative Arts Science HSIE TAS

Frequency of ICT used in teaching, by subject area

Belief that working with ICT is fun and confidence using laptops

2.94 3.13 3.06 3.23 3.14 3.14 3.17 3.21 2.67 2.77 2.79 2.83 2.87 2.87 2.89 3.07

0 0.5

1 1.5

2 2.5

3 3.5

I think working and teaching with computers is fun I feel confident that I can effectively use the laptop in my classes.

Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree

RQ #1, Pedagogy

Teachers' beliefs of the importance of working with ICT

3.47

3.56

3.59

3.6

3.6

3.62

3.63

3.65

3.22

3.38

3.37

3.35

3.49

3.43

3.49

3.43

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7

Mathematics

English

PDHPE

Languages

TAS

Science

Creative Arts

HSIE

It is important for students to work with computers It is important to me to work with a computer


RQ #1, Pedagogy

2.26 2.36 2.41 2.45 2.53 2.61 2.64 2.71

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Mathematics

Creative Arts

Science

English

PDHPE

HSIE

Languages

TAS

Teaching is becoming more student-centred with use of the laptops


RQ #1, Pedagogy

•  “I’m a bit of a believer of you still need to write your maths solutions in your book because you can’t solve an equation, you can’t write your solution to an equation on the laptop unless you’re really familiar with the equation editor in the program, in the software, but that takes time…I see other subjects using that as a great tool for their learning for English or history and so on. So I think for Maths, yeah, we’re a little bit lower on the scale of usefulness.” (School 1, Teacher 1)

•  “I think what I like about the laptops is that gives you a different medium for production in that you do your writing exercises but I know a lot of teachers up in the faculty have been doing things like creating your own posters and TV type advertisements, book trailers, things like that and so it’s getting them to demonstrate their knowledge in different ways and I think that that’s a good thing.” (School 3, Teacher 1)

RQ #1, Pedagogy

RQ #1 & 2, Engagement

2.58

2.65

2.31

2.37

2.62

3.05

3.23

2.94

3.00

3.29

0.00 1.00 2.00 3.00 4.00

Students'work is more creative when they use computers

Using a computer helps students to be more organised

Students' writing seems to improve when they use computers

I feel students understand what they are doing in class better when they

use computers

The more we use computers in class, the more it seems students

enjoy school

Student Teacher


Teacher and students' beliefs about ICT and learning outcomes

•  Students’ perceptions of the importance of ICT in certain subject areas reflected teachers’ perceptions – e.g. importance in Mathematics

•  Students reported more positive perceptions of ICT in learning than teachers: –  They reported more use of ICTs in school (could be personal

or in combined classes) –  They showed a more positive conception of engagement in

learning when using ICT •  Students without or with limited access to computers

at home showed consistently less confidence performing ICT tasks and lower engagement with ICT

RQ #2, Engagement

RQ #2, Engagement

2.50 2.54 2.76 2.88

3.03 3.04 3.07 3.12

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Students' perceptions of the importance of ICT in the subject areas


•  “It really depends what I get but I’d say I’d use it at least maybe three times, four times a week…Yeah but it really depends what lessons I have because you know in maths I can’t use it at all because she won’t let us…she says it won’t help us at all…write it down like usual…[So what might be a typical assignment you’d have that you might do at home on your, on your laptops?] Research assignments…I’ve normally got them for history…We’re, we’re doing one on Romans so basically things on Romans, things on like fantasy or myths as well.” (School 4, Student 2)

RQ #1 & 2, Pedagogy

Students' confidence performing ICT practices*

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100% D

ownl

oad

imag

es

Cre

ate

Pow

erP

oint

R

esea

rch

info

rmat

ion

Em

ail

Edi

t writ

ten

wor

k W

rite

first

dra

ft O

rgan

ize

wor

k Ta

ke n

otes

D

ownl

oad

pict

ures

C

reat

e m

ultim

edia

S

ocia

l net

wor

k P

ost c

omm

ents

to

Ent

er d

ata

to

Plo

t a g

raph

C

reat

e sp

read

shee

t B

log

Cre

ate

mov

ie

Cre

ate

web

page

P

odca

st

Cre

ate

com

pute

r

*The figure is calculated on how many students felt they could perform this task by themselves or with help

RQ #2, Engagement

RQ #2, Engagement

3.01

3.22

3.40

3.40

3.41

3.43

3.45

3.49

3.54

3.44

3.68

3.79

3.83

3.80

3.85

3.79

3.86

3.87

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50

Create webpage

Write first draft

Create PowerPoint

Download images

Edit written work

Enter data to spreadsheet

Research information

Email

Download pictures

Access to a computer at home No computer access at home

Students' confidence performing ICT practices & home access

RQ #2, Engagement

3.00

3.23

3.03

3.11

3.14

3.37

3.13

3.23

2.80 2.90 3.00 3.10 3.20 3.30 3.40 3.50

It is very important to me to work with a computer

I think working with a computer is fun

I use a computer because I am very interested

Time goes by quickly when I am working with a computer

Access to a computer at home No computer access at home

Students' engagement with technology & home access


•  “But I think it comes from the fact that they may have been exposed to computers but they just use them for games and things… they have no access or they don’t access the other programs…Because they’re not familiar with some of the programs and they’re asking you how to – something basic that most kids would know how to do ... And half of them say at the start, ‘Well how do I do that’…Or even they just sort of go, ‘I’d rather not, can I just write it’… And even adding attachments to an email, like a lot of kids will say, ‘How do I attach the file ...’ and I’m always surprised by that.” (School 3, Teacher Focus Group)

RQ #2, Engagement

•  “I do believe they’re good because like their – instead of going to computer labs we can stay in class and they’re, they’re just – they’re good to have for research and so anything that you need for like, preparation for school… or something like that and that’s why they’re good but, and it’s just that we were flattered to have them because they’re our own laptops and I guess they’re rather expensive so they’re fun to have…Well they can help with like our assignments because I’ve got like three assignments on here that I would have to do at school and send home for my computer which doesn’t work so it helps me.” (School 4, Student Focus Group)

RQ #2, Engagement

•  Case study teachers felt it was too soon to predict if students’ performance on tests would improve

•  Some students felt their grades had improved with use of the laptop in class – They felt more engaged in learning – Enjoyed exploration tasks

•  Issues seem to exist around having students perform tasks on the laptops, when testing will not be on the laptops

RQ #3, Assessment

For 2011 •  RQ#1: Further investigation into differences

between teachers’ personal & pedagogical conceptions of ICT practices in the subject areas (impact on learning & students’ perceptions of ICT)

•  RQ#2: Deeper examination of the relationship between students’ home access & engagement with ICT practices, e.g. updated questionnaire items on access

•  Investigate the interaction between leadership and ICT up-take, e.g. email communication at the school level

baseline data evaluation report

Education