american federation of teacherstimss 1 the american federation of teachers looks at timss

American Federation of Teachers TIMSS 1

The American Federation of Teachers

looks at

TIMSS


International Association for the Evaluation of Education Achievement (IEA)

National Center for Education Statistics (NCES)

National Science Foundation (NSF) 41 participating nations-Grade 8 26 nations-Grade 4 21 nations-Grade 12

Third International Mathematics and Science

Study (TIMSS)


What TIMSS looked at

Structure of education systems expectations, supports, decision makers,

stakes Curriculum

rigor, focus, specificity, authority, expectation Delivery of instruction

goals, structure, and content of lessons Student Achievement

9-year-olds 13-year-olds end of secondary school


Sampling/validity More than $2 million to validate sampling Samples included:

representative regions metropolitan and non-metropolitan sites socioeconomic variety demographic variety different German states and types of schools different size communities and schools in Japan

Achievement of video students is very similar to country as a whole.


TIMSS: Grade 4

When comparing achievement in 26 TIMSS countries, U.S. fourth grade students are: above the international average in mathematics. above the international average in science.

NCES, 1997. Third International Mathematics and Science Study. Pursuing Excellence


Who do U.S. fourth graders outperform?

MATHEMATICS Latvia Greece Scotland Iran Thailand Cyprus England

Iceland Portugal Norway New Zealand Kuwait

NCES, 1997. TIMSS. Pursuing Excellence

SCIENCE England Latvia Canada Israel Singapore Iceland Slovenia Greece Ireland Portugal Scotland Cyprus

Hong Kong Thailand Hungary Iran New Zealand Kuwait Norway


Whose 4th graders do significantly better than U.S. students? NCES, 1997. TIMSS. Pursuing Excellence

MATHEMATICS Singapore

Korea Japan

Hong Kong Netherlands

Czech Republic Austria

SCIENCE Korea


TIMSS: Grade 8

When comparing students in 41 countries, U.S. eighth grade students are:

below the average in mathematics above the average in science

SOURCE: National Center for Education Statistics. Third International Mathematics and Science Study. Pursuing Excellence


Whose 8th grade students scored significantly better than those of the U.S.?

MATHEMATICS

Singapore France

Korea Hungary

Japan Russia

Hong Kong Australia

Belgium Ireland

Czech Republic Canada

Slovak Republic

Switzerland

Netherlands

Slovenia

Bulgaria

SCIENCE

Singapore

Czech Republic

Japan

Korea

Bulgaria

Netherlands

Slovenia

Austria

Hungary

SOURCE: National Center for Education Statistics. Pursuing Excellence


TIMSS: Population 3

Students at the end of secondary school

U.S. grade 12

Ranges from grade 9 to grade 14

Four reports General knowledge of mathematics General knowledge of science Advanced mathematics Physics


TIMSS: Population 3General Knowledge of Mathematics

Significantly better Netherlands

Sweden Denmark

Switzerland Iceland Norway France

New Zealand Australia Canada Austria

Slovenia Germany Hungary

Same as U.S. Italy

Russian Federation Lithuania

Czech Republic U.S.

Significantly lower Cyprus

South Africa

American Federation of Teachers

SOURCE: NCES,1998. Pursuing Excellence. TIMSS


Scale score comparisonGeneral Mathematics Knowledge

Netherlands 560 TIMSS average 500

U.S. 461 South Africa 356



TIMSS Population 3General Knowledge of Science

Significantly better Sweden Netherlands Iceland Norway Canada New Zealand Australia Switzerland Austria Slovenia Denmark

Same as the U.S. Germany France Czech Republic Russian Federation United States Italy Hungary Lithuania

Significantly lower Cyprus South AfricaAmerican Federation of Teachers SOURCE:

NCES,1998. Pursuing Excellence. TIMSS


Scale Score ComparisonGeneral Knowledge of Science

Sweden 559 TIMSS average 500 United States 480 South Africa 349



How do our best 4th grade students do?The TIMSS top 10 percent in math

05

10152025303540

Sng Kor Jpn HK Cz Irl U.S.

Mullis et al, 1997. Mathematics Achievement in the Primary School Years

Percent of Students

Singapore

Korea

Japan

Hong Kong

Czech

Ireland

U.S.


How do our best 4th grade students do?The TIMSS top 10% in science

1716

1311 11 11

9

02468

1012141618

Kor US Eng Jpn Sing Cz Ca

Martin et al, 1997. Science Achievement in the Primary School Years

Percent of Students

Korea US England Japan Singapore Czech Canada


How do our best 8th grade students do?The TIMSS top 10 percent

5

13

32

18

45

31

1114

0

5

10

15

20

25

30

35

40

45

Math ScienceSOURCE: National Center for Education Statistics, 1996. Third International Mathematics and Science Study

U.S. Japan

Singapore Hungary


How do our other 4th grade students do?The TIMSS top quartile/top half in math

0102030405060708090

Top quartile Top halfMullis et al, 1997. Mathematics Achievement in the Primary School Years

Percent of Students

Singapore Korea Japan Hong Kong

Czech Ireland U.S.


Percentage of 8th grade students in the international top quartile and half in math

18

45

58

8374

94

26

63

0102030405060708090

100

Top 25% Top halfSOURCE: Beaton et al. 1996. Mathematics Achievement in the Middle School Years

U.S

Japan

Singapore

France


Variation in Performance: Grades 7/8The international average in 8th grade mathematics is 30 points higher than at 7th grade. If 30 points is roughly a year’s difference:

the average U.S. 8th grader is more than a year behind Russian, Hungarian and French students in math;

more than two years behind Czech and Belgian students;

more than three years behind Japanese, Korean and Singaporean students.

The scores of students at the U.S. median are similar to those of Japan’s lower 25%.

SOURCE: Beaton et al, 1996, Mathematics Achievement in the Middle School Years


Advanced math: Top 5 and 10 percent

Top 10% Mean

Slovenia* 629 France 612 Canada 567 TIMSS 554 U.S. 485 Czech Rep. 485

*Sampling problem

Top 5% Mean

Slovenia* 664 France 645 Canada 620 TIMSS 601 U.S. 543

SOURCE:Mullis et al, 1998. Mathematics and Science Achievement in the Final Year of Secondary School. IEA


Physics: Top 5 and 10 percent

Top 10%

Mean

Slovenia* 652 Sweden 630 TIMSS 533 Canada 522 U.S. 451

Top 5% Mean

Slovenia* 698 Sweden 678 TIMSS 583 Canada 574 U.S. 485

SOURCE: Mullis et al, 1998. Mathematics and Science Achievement in the Final Year of Secondary School. IEA


U.S. Fourth Grade Performance Strengths/ Weaknesses

Mathematics Above the international

average

Whole numbers

Fractions/proportionality

Data and probability

Geometry

Patterns, relations, functions

Below the international average

Measurement, estima-tion, and number sense

Science Above the international

average

Earth science

Life science

Physical science

Environmental issues and the nature of science

SOURCE: NCES, 1997. Pursuing Excellence.


U.S. Grade 8 Performance: Strengths/ Weaknesses

SOURCE: National Center for Education Statistics, 1996. Pursuing Excellence. Math

U.S. students better in:Fractions and number sense Data representation

U.S. students weaker in:

MeasurementGeometry

Comparing only topics they taught does not relative standings.

Science U.S. students better in:

Earth science Life scienceEnvironmental

U.S. students weaker in:

ChemistryPhysics

U.S. teachers say significantly change the


Martin et al, 1997.

Science Achieve-

ment in the Primary School Years 46

35

28

33

27 26 25

0

5

10

15

20

25

30

35

40

45

50 Top Quartile

Percent students

Korea US England Japan Singapore Czech Canada

How do other U.S. 4th graders do?The TIMSS top quartile in science


Comparative Performance at Grades 4/8 Science

Mullis et al. Mathematics Achievement in the Primary School Years. IEA NCES. TIMSS. Pursuing Excellence

FOURTH GRADE Korea*** Japan United States Austria Australia Netherlands Czech Republic England Canada Singapore Slovenia Ireland u Scotland***Statistically above all other countries

EIGHTH GRADE Singapore Czech Republic Japan Korea Netherlands Slovenia Austria Hungary

England Australia Ireland United States


U.S. Performance: Grade 4 and Grade 8

Mullis et a.l, 1997. Mathematics Achievement in the Primary School Years

The U.S. is the only country participating in the 4th grade TIMSS whose math standing falls from above average at fourth grade to below average at eighth grade.


TIMSS: Population 3Advanced Math

Compared to All

U.S. Calculus students

Significantly better France Russian Federation Switzerland Denmark Cyprus Lithuania

Significantly lower Germany Austria

Compared to Only U.S.

AP Calculus students

Significantly better France

Significantly lower Slovenia Italy Czech Republic Germany Austria

SOURCE: NCES,1998. Pursuing Excellence.


Scale score comparisonAdvanced math

France 557 U.S. AP 513 TIMSS 501 U.S. Calculus 492 Pre-calculus 442 Austria 436



The crème de la crème: math The average score of U.S. students is lower than

that of their peers at both ends of the scale even though the difference between top and bottom is similar to that of most countries.

The most advanced 5% of U.S. math students score similarly to 10 to 20% of the age cohort in most of the other countries.

SOURCE: NCES,1998. Pursuing Excellence


TIMSS: Population 3Advanced science: physics

Compared to all physics students

Significantly better Norway Sweden Russian Federation Denmark

Significantly lower Austria

SOURCE: NCES,1998. Pursuing ExcellenceTIMSS

Same as U.S. Slovenia Germany Australia Cyprus Latvia Switzerland Greece Canada U.S. France Czech Republic


Scale score comparisonPhysics

Norway 581 TIMSS 504 U.S. AP 474 Austria 435 U.S. Non-AP 423

American Federation of Teachers



WARNING! No single finding of the TIMSS study by itself

can explain why a country’s students do or do not do well. It is essential to see all the pieces of the puzzle and how they fit together--to figure out why, when some things look the same on the surface, there are huge differences in results;

or why, when thingsseem quite different,

results are quite similar.


The lesson profiles of France and Spain are similar. Yet, French students score significantly better than Spanish students.

Swiss lessons emphasize students’ responsibility for their own learning in contrast to very teacher-directed instruction in France. Both do well.

Canada’s math curriculum has more topics than the U.S. curriculum. Her students score significantly better in math. Schmidt et al, 1996. Characterizing Pedagogical Flow.

For example...


We must compare, conjecture, and ask what made the difference. Was it:

the subject matter knowledge of teachers? the amount of time students worked alone? the quality of textbooks and resources? the presence of a common core curriculum? having stakes for student achievement? training?

What patterns emerge from data for all the student populations and all the reports?


What sets us apart from countries whose students scored

significantly better? no nationally defined curriculum less advanced mathematical content disruption of the educational process no rewards or sanctions for academic

performance weak induction process for teachers little opportunity for professional interaction with

colleagues


Teaching out of field

The U.S. Commission on Teaching and America’s Future reports:

nearly one-fourth of all secondary teachers do not even have a college minor in their main teaching field, including;

30% of mathematics teachers 56% of physical science

teachers. These proportions are much higher

in high-poverty schools and in lower track classes.

What Matters Most: Teaching for America’s Future. 1996.


Teaching and America’s Future recommendations

Get serious about standards, for both students and teachers.

Reinvent teacher preparation and professional development.

Overhaul teacher recruitment and put qualified teachers in every classroom.

Encourage and reward teaching knowledge and skill.

Create schools that are organized for student and teacher success.

Source: National Commission on Teaching & America’s Future, 1996. What Matters Most: Teaching for America’s Future


Possible Reasons for Our Downward Slide from 4th to 8th grade

Less rigorous content Unfocused curriculum Out-of field teaching Lack of a common curriculum Little incentive for high achievement


Why is the U.S. relatively worse at 12th grade than 8th?

Could it be:

amount of instruction? course-taking in final year? curriculum? outside of school activity? existence of consequences?


How did course-taking in the final year affect end-of-secondary TIMSS

results? U.S. students not taking math in grade

12: 34% Canadian students not taking math in

final year: 46% At least one third of students are no

longer taking math in Iceland, Netherlands, Switzerland.


Other Findings from end-of-secondary TIMSS

Strong relationship between calculator use and achievement.

Computer/achievement relationship is different. Unlike other countries, U.S. advanced math and

science students did not report having more than 5 hours of instruction a week.

Virtually no single-step or single-stage problems on TIMSS or NAEP.



Employment and School Climate Hours per day working at a paid job. (3 or more hours)

U.S. 55%

TIMSS 18% Had something stolen at school in the month prior to taking

TIMSS

U.S. 24%

TIMSS 13% Was threatened by another student in the month preceding

TIMSS

U.S. 11%

TIMSS 7%



What do we make of it?

Countries having higher achievement at the end of school than at 8th grade tended to have older students at the end of school than countries whose standing was relatively worse at the end of school.

Countries where more students were taking math in their final year of secondary school were not more likely to have a better position relative to other countries than they did in 8th grade.

NCES, 1998. Pursuing Excellence. TIMSS.


First in the World Consortium Strong foundation in the middle grades Challenging curriculum and higher expectations in

high school Better prepared math and science teachers (all have

majors or minors in what they teach) Significant professional development opportunities

for all. Learning networks to share best practices and teaching techniques. Incentives to work with experts and take significant course work.

Source: First in the World Consortium


Curriculum

The United States is one of a small number of countries whose curriculum is not determined nationally or at least “strongly advised” nationally.

Most countries can identify a common core of knowledge that all students are expected to learn.

Schmidt, 1996. A Splintered Vision.


Curriculum in the early years

More topics in both U.S. math and science than others

Content similar to the rest of the world in math

Science is more focused in 4th grade than in 8th. We do less well in physical science even at this level.

SOURCE: William Schmidt. TIMSS Curriculum Study

and NCES, 1997. Pursuing Excellence.


Between fourth and eighth grades, other countries introduce six to eight math and eight to eleven science topics in a focused way.

Most of the U.S. curriculum is still unfocused and splintered. Only one of the topics introduced in U.S. math is done so with focused attention.

SOURCE: William SchmidtTIMSS Curriculum Study

Curriculum in later years


Grade 8 curriculum: more findings

U.S. curriculum is less rigorous than in other countries;

covers more topics per year than most countries;

is not as focused as in Germany and Japan in math.

In science, topic coverage is similar to other countries. Schmidt, 1996. A Splintered Vision.


Level of 8th grade content by average international placement

8th grade

9th grade

7th grade

Germany

Japan

United States NCES, 1996. TIMSS Videotape Study web

site: http:\\ed.gov/NCES/TIMSS/html


What makes strong mathematics lessons?The Videotape Classroom Study analyzed:

teachers’ goals for lessons treatment of concepts and applications the presence of alternative solution

methods how mathematical principles, properties

and definitions were used whether proofs were included whether concepts were connected the kinds of tasks assigned


Four Kinds of Goals

Mathematical Skills how to solve specific kinds of problems, use standard formulas...

Mathematical Thinkingexploration, development, comprehension of mathematical concepts; multiple solutions

Social/Motivationalnon-mathematical

Test Preparation


Teachers’ goals: skills/solving specific problems vs. thinking and understanding

mathematics

55

31

0

25

73

0

61

22

6

0

10

20

30

40

50

60

70

80

Skills Thinking Test prpSOURCE: National Center for Education Statistics. Pursuing Excellence, 1996.

Germany

Japan

U.S.


Structure and delivery of lessons

Characteristic lesson scripts U.S. and German teachers present a procedure or

skill and then have students apply it. Japanese teachers have students solve a problem

first, then draw on student thinking to develop understanding of a concept.

Coherence and U.S. lessons: switched topics more often, did not link the changes, had more “irrelevant diversions,” and more interruptions from outside.

SOURCE: NCES, 1996. Pursuing Excellence


What could 4th grade students in the top half in math answer correctly?

Here is a number sentence.

2000 + + 30 + 9 = 2739

What number goes in the to make this sentence true?

Answer: ______________

SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School Years


What could fourth grade students in the top quartile in math answer correctly?

25 x 18 is more than 24 x 18 How much more?

A. 1

B. 18

C. 24

D. 25

SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School

Years. IEA


What could 9-year-olds in the top 10 percent in math answer correctly?

There are 10 girls and 20 boys in Juanita’s class. Juanita said that there is one girl for every two boys. Her friend Amanda said that means 1/2 of all the students in the class are girls.

How many students are there in Juanita’s class? Is Juanita right? Answer___ Use words or pictures to explain why. Is Amanda right? ___ Use words or pictures to explain why.

SOURCE: Mullis et al, 1997. Mathematics Achievement in the Primary School Years. IEA


Questions that differentiated 8th grade students at various marker levels SOURCE: Beaton et al, 1996. Mathematics Achievement in the Middle School Years

Top 10% Top half Lower half

Percent Distance on map (a)Subtraction

If the price of a Using the scale 6 0 0 0can of beans is on a map - 2 3 6 9raised from 60 ( 1cm = 1km)cents to 75 cents find the distance (b) Fractionswhat is the per- between two cent increase given cities. Write a fraction that the price? is larger than

2/7.


Grade 8 Item comparisons SOURCE: Beaton et al., 1996. Mathematics Achievement in the Middle School Years

Lower Half Top Half Top 10%

OOO OOO If m represents Juan has 5 fewer

Which set of shapes a positive num- hats than Maria, and

is arranged in the ber, which of Clarissa has 3 times

same pattern? these is as many hats as Juan.

equivalent to If Maria has n hats,

m+m+m+m? which of these repre-

m+4 sents the number of

4m hats that Clarissa has?

m4 5-3n 3n-5

4(m+1) 3n 3(n-5)

n - 5


End of Secondary Math Literacy

From a batch of 3000 light bulbs, 100 were selected at random and tested. If 4 of the light bulbs in the sample were found to be defective, how -many defective light bulbs would be expected in the entire batch?


End of secondaryMath Literacy

Using the set of axes below, sketch a graph which shoes the relationship between the height of a person and his/her age from birth to 30 years. Be sure to label your graph, and include a realistic scale on each axis.

(Picture of 11 x 16 grid)


End of secondary Advanced Students

The vertices of the triangle PQR are the points p(1,2), Q(4,5) and R(-4,12). Which one of the following statements about triangle PQR is true?

A. PQR is a right triangle with the right angle P.

B. PQR is a right triangle with the right angle Q.

C. PQR is a right triangle with the right angle R.

D. PQR is not a right triangle.


End of SecondaryAdvanced math students

See p. 157 and scan in question box


Write down two different things that people can do to help reduce air pollution.

SOURCE: Martin et al, 1997. Science Achievement in the Primary School Years. IEA

In science, the upper half of fourth graders could generally answer this correctly


In science, upper quartile fourth graders could generally answer this correctly


The picture shows two forms of sugar--solid cubes and packets of loose sugar. One cube has the same mass of sugar as one packet.

Which of the two forms of sugar will dissolve faster in water?__________

Give a reason for your answer.


Item 4th graders in the top 10 percent in science were likely to answer correctly

To find out whether seeds grow better in the light or dark, you could put some seeds on pieces of damp paper and

A. keep them in a warm, dark place

B.keep one group in a light place and another in a dark place

C.keep them in a warm, light place

D. put them in a light or dark place that is cool



Treatment of mathematical conceptsIn U.S. 8th grade math, concepts were developed much

less frequently than in Germany and Japan.

2317

787783

22

0102030405060708090

Germany Japan U.S.TIMSS Videotape Study web site: http:\\www.ed.gov/NCES/TIMSS/hmtl

Stated

Developed


Structure and delivery of lessons

Characteristic national lessons U.S. and German teachers present a procedure or

skill; then students apply it. Japanese teachers have students solve a problem

first, then draw on student thinking to develop understanding of a concept.

Coherence and U.S. lessons switched topics more often did not link parts of the lesson had more “irrelevant diversions” more interruptions from outside

SOURCE: NCES, 1996. Pursuing Excellence


Classwork and Seatwork

Japan switches back and forth between the two and has shorter segments. (Stevenson &

Stigler, The Learning Gap)

U.S. and German seatwork focuses on routine procedures; Japan’s seatwork is thinking-oriented.

89

4

41 44

96

10

10

20

30

40

50

60

70

80

90

100 NCES, 1996. Pursuing Excellence

G erm anyJapanU .S .

Routine Thinking


Frequency of Alternative Solution MethodsGrade 8 math

12

7

19

14

42

8

0

5

10

15

20

25

30

35

40

45

Germany Japan U.S.NCES,1997. TIMSS Videotape web site http:\\www.ed.gov/NCES/TIMSS/hmtl

Teacher

Student


Homework

Comparing Germany, Japan, and the U.S.:

The U.S. has about as much homework as others.

The U.S. and Germany spend more class time sharing homework than Japan.

Only the U.S. works on homework during class and counts it in grades.

8

2

11

0

2

4

6

8

10

12

Classtim e

SOURCE: NCES, 1996. TIMSSweb site: http:\\www.ed.gov/NCES/TIMSS/hm tl

Germ any

Japan

U.S.


Linking was important for lesson coherence

There are times when teachers want students to understand ideas in relation to each other. If a teacher in the videotape study made a specific connection to tie ideas together during a lesson, researchers called it a “link.”

Remember when Marie said...


Percent of grade 8 math lessons that include explicit linking by the teacher

40

96

40

0

20

40

60

80

100

NCES, 1996. Pursuing Excellence

Germany

Japan

U.S.

Ger. Jpn. U.S.


Expectations of classroom talkfor students

U.S. Japanmore yes/no or much more likely to short answer questions include explana-

tions

define terms and state clarify and rules elaborate

Stevenson & Stigler, 1992. The Learning Gap


Lesson characteristics that contribute to mathematical quality

Coherence of sequencing Way in which examples contribute

to lesson’s central concept Type of reasoning required of

students Increase in complexity of problems



Mathematical quality of lessonsWhen lessons were rated for the quality of math content, 87

percent of U.S. lessons were given the lowest rating.

40

13

87

37

57

13

2330

00

10

20

30

40

50

60

70

80

90

Germany Japan U.S.NCES, 1996. Puruing Excellence

Low

Medium

High


Proofs and Deductive Reasoning

10

21

53

62

0 00

10

20

30

40

50

60

70

Proofs Deduct.Reason.

SOURCE: NCES, 1996.TIMSS Videotape Study web site

http://www.ed.gov/NCES/TIMSS/hmtl

Germany

Japan

U.S.


Other TIMSS findings What is written during a lesson:

In Japan, 80% of what is written is still visible at the end of class, compared to about 20% of what has been written in American classrooms.

Class time:

U.S. students spend more time in math class than their peers.

Homework:

Only U.S. students spend time working on the next day’s homework in class.

TV habits: Students in other countries report watching as much TV as their

U.S. peers.

TIMSS Videotape Study web site: http:\\www.ed.gov/NCES/TIMSS/html NCES, 1996. Pursuing Excellence


Grouping

In the U.S., 8th grade students are generally put into different math classes on the basis of ability. The content differs for different classes.

In Germany, although tracked to different schools, students study the same content, but the rigor varies.

In Japan, classes are heterogeneous through eighth grade; everyone studies the same thing.



REMEMBER!

No single finding of the TIMSS study

by itself can explain the achievement

level of a country’s students. It takes a whole coherent system.

american federation of teacherstimss 1 the american federation of teachers looks at timss

Documents