assessment, accountability and adaptation april 13, 2006
DESCRIPTION
Assessment, Accountability and Adaptation April 13, 2006 copies of this presentation can be found at www.business.duq.edu/faculty/davies. Education is a product. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/1.jpg)
1
Assessment, Accountability and Adaptation
April 13, 2006
copies of this presentation can be found atwww.business.duq.edu/faculty/davies
![Page 2: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/2.jpg)
2
Education is a product.
Economics provides tools for adequately addressing assessment, accountability, and adaptation, but to use the tools, we must first admit that education, while occupying a place of great importance, is fundamentally a product.
![Page 3: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/3.jpg)
3
How does the market judge products, hold producers accountable, and adapt to consumers’ needs?
Assessment
AccountabilityAdaptation
![Page 4: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/4.jpg)
4
The cycle begins with the consumer who develops a mental list of important attributes and a acceptable tradeoffs. Together, these form the basis of the consumer’s assessment of the product.
Assessment
AccountabilityAdaptation
•Attributes
•Tradeoffs
![Page 5: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/5.jpg)
5
Consider grocery stores…
The Consumer Has a List of Important Attributes
Store is located in a safe and convenient location.
The produce is fresh.
There is a large selection of brands.
The store has a deli.
Prices are reasonable.
The Consumer Is Aware of Acceptable Tradeoffs
Convenient location is more important than fresh produce.
Lower price is more important than having a deli.
![Page 6: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/6.jpg)
6
The consumer assesses the performance of the store by comparing the store’s attributes to the consumer’s list of important attributes and acceptable tradeoffs.
Assessment
•Attributes
•Tradeoffs
![Page 7: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/7.jpg)
7
The consumer holds the store accountable via repeat purchase. If the store’s attributes are acceptable to consumers, the store’s sales rise. If not, the store’s sales fall.
Assessment
Accountability
•Attributes
•Tradeoffs
![Page 8: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/8.jpg)
8
If sales fall, the store will mimic and/or improve upon attributes of stores with rising sales. If the store’s sales rise, competing stores will mimic and/or improve upon this store’s sales. This is adaptation.
Assessment
AccountabilityAdaptation
•Attributes
•Tradeoffs
![Page 9: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/9.jpg)
9
The consumer now re-assesses the store in light of the store’s new attributes and any changes in the consumer’s attributes / tradeoffs. This is outcomes assessment.
Assessment
AccountabilityAdaptation
•Attributes
•Tradeoffs
![Page 10: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/10.jpg)
10
In the market place, what might cause the AAA cycle to break down?
•Attributes
•Tradeoffs Assessment
Adaptation Accountability
![Page 11: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/11.jpg)
11
Break in the Cycle #1: Consumer is unidentified
•Attributes
•Tradeoffs
Assessment
AccountabilityAdaptation
If the consumer is unidentified, questions arise:
1. What attributes are important?
2. What tradeoffs are acceptable?
With an unidentified consumer:
Appropriate assessment becomes unclear.
Source of accountability becomes unclear.
Appropriate adaptation becomes unclear.
![Page 12: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/12.jpg)
12
Assessment
AccountabilityAdaptation
Break in the Cycle #2: Accountability not Possible
1. Repeat purchase – the consumer must have a need to purchase again so as to signal pleasure;
2. Mobility – the consumer must be able to purchase elsewhere so as to signal displeasure;
3. Short-run assessment – the consumer must be able to assess product quality prior to the repeat purchase decision.
If accountability is not possible
Appropriate adaptation becomes unclear.
Incentive for adaptation disappears.
Possibility of accountability requires:
•Attributes
•Tradeoffs
![Page 13: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/13.jpg)
13
How does the AAA cycle play out for primary and secondary education?
Assessment
Adaptation Accountability
•Attributes
•Tradeoffs
First: Can we identify the consumer? Second: On what is assessment
based?
Third: Is accountability possible?
Fourth: What adaptations result?
![Page 14: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/14.jpg)
14
First: Can We Identify the Consumer?
• Student?Attributes: friends, sports, extracurricular activities.
• Parents?Attributes: safety, education, sports, future opportunities.
• Colleges?Attributes: critical thinking skills, math skills, communication skills, personality, drive.
• Employers?Attributes: work ethic, communication skills, ability to function as part of a team.
• Society?Attributes: likelihood of incarceration, likelihood of becoming a welfare recipient.
![Page 15: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/15.jpg)
15
Second: On What is Assessment Based?
• If Consumer = StudentAssessment based on the student’s social environment.
• If Consumer = ParentsAssessment based on parents’ satisfaction with perceived educational environment.
• If Consumer = CollegesAssessment based on students’ college preparation.
• If Consumer = EmployersAssessment based on students’ job skills.
• If Consumer = SocietyAssessment based on crime and poverty measures.
![Page 16: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/16.jpg)
16
Consumer = Student
(assessment = social envrnmnt)
Consumer = Parents
(assessment = educational envrnmnt)
Consumer = Colleges
(assessment = college preparation)
Consumer = Employers
(assessment = job skills)
Consumer = Society
(assessment = crime/poverty rates)
Consumer Can Assess Quality in Short Run?
Consumer Mobility?Consumer Requires Repeat Purchase?
Yes No Yes
YesMaybeYes
Yes Yes Yes
YesMaybeYes
Yes No No
Only three definitions of “consumer” permit accountability.
Third: Is Accountability Possible?
![Page 17: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/17.jpg)
17
How we define “the consumer” ultimately determines how education will adapt.
Definition of “consumer” determines
attributes and tradeoffs, which determine
assessment, which determines
accountability, which determines
adaptation.
Fourth: What Adaptations Result?
![Page 18: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/18.jpg)
18
Assessment
AccountabilityAdaptation
Assessment is college preparation.
Accountability is imposed via acceptance rates.
Schools adapt by making students more attractive to colleges so as to boost acceptance rate.
Fourth: What Adaptations Result when Consumer = Colleges?
![Page 19: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/19.jpg)
19
Assessment
AccountabilityAdaptation
Assessment is college preparation.
Accountability is imposed via acceptance rates.
Schools adapt by making students more attractive to colleges so as to boost acceptance rate.
Unintended Consequence
Schools will encourage students who are technically, rather than academically, talented to go to college rather than to prepare for the job market.
Fourth: What Adaptations Result when Consumer = Colleges?
![Page 20: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/20.jpg)
20
Assessment
AccountabilityAdaptation
Assessment is job skills.
Accountability is imposed via employment and wage rates.
Schools adapt by making students more attractive to employers so as to boost employment and wage rates.
Fourth: What Adaptations Result when Consumer = Employers?
![Page 21: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/21.jpg)
21
Assessment
AccountabilityAdaptation
Assessment is job skills.
Accountability is imposed via employment and wage rates.
Schools adapt by making students more attractive to employers so as to boost employment and wage rates.
Unintended Consequence
Schools will encourage students who are academically, rather than technically, talented to prepare for the job market rather than to go to college.
Fourth: What Adaptations Result when Consumer = Employers?
![Page 22: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/22.jpg)
22
Assessment
AccountabilityAdaptation
Assessment is educational environment.
Accountability is imposed via changing schools.
Schools adapt by making themselves more attractive to parents so as to boost enrollment and retention.
Fourth: What Adaptations Result when Consumer = Parents?
![Page 23: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/23.jpg)
23
Assessment
AccountabilityAdaptation
Assessment is educational environment.
Accountability is imposed via changing schools.
Schools adapt by making themselves more attractive to parents so as to boost enrollment and retention.
Unintended Consequence
Schools will develop educational environments that conform to parents’ perceptions of quality education.
Fourth: What Adaptations Result when Consumer = Parents?
![Page 24: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/24.jpg)
24
Let’s assume that:
1. We can skip the step of identifying the consumer, and
2. Standardized Testing is the appropriate assessment tool,
What is the impact of assessment on adaptation?
Can We Assess Education Without Identifying the Consumer?
![Page 25: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/25.jpg)
25
Assessment
AccountabilityAdaptation
Assessment is Standardized Testing.
Accountability is imposed via funding (“high-stakes testing”).
Schools adapt by making students better able to pass the test so as to boost funding.
Fourth: What Adaptations Result when Consumer = Undefined?
![Page 26: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/26.jpg)
26
Assessment
AccountabilityAdaptation
Assessment is Standardized Testing.
Schools adapt by making students better able to pass the test so as to boost funding.
Unintended Consequence
Schools will prepare students to pass the Standardized Test (“teaching to the test”).
Accountability is imposed via funding (“high-stakes testing”).
Fourth: What Adaptations Result when Consumer = Undefined?
![Page 27: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/27.jpg)
27
Let’s assume that:
1. We can skip the step of identifying the consumer, and
2. Standardized Testing is the appropriate assessment tool,
What attributes do school districts control?
Fourth: What Adaptations Result when Consumer = Undefined?
![Page 28: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/28.jpg)
28
An oft-cited important attribute is the Student-Teacher ratioThis strategic plan stresses the need for strategies aimed at keeping more teachers in the classroom.
– Strategic Plan, Texas Higher Education Coordinating Board, 2002.
Teacher recruitment is the key to the drive to reduce class size.
– Congresswoman Eddie Bernice Johnson, (D-TX), 2005.
Utah will undertake several initiatives aimed at keeping more teachers in the classroom.
– Utah Governor John Huntsman (R), 2005.
Growing numbers of students in our nation’s schools…mean that our need for teachers increases each year.
– Meeting the Challenges of Recruitment and Retention, NEA, 2005.
![Page 29: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/29.jpg)
29
400
420
440
460
480
500
520
540
560
580
10 12 14 16 18 20 22 24
Student/Teacher Ratio, 2003
NA
EP
Sco
res,
200
3
8th Grade 4th Grade
Student-Teacher Ratio has no apparent impact on NAEP scores.
Source: National Center for Education Statistics, U.S. Department of Education
Each dot represents one state.
Reported scores are sums of state averages for math and reading.
![Page 30: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/30.jpg)
30
400
420
440
460
480
500
520
540
560
580
100 200 300 400 500 600 700 800
Student/School Ratio, 2003
NA
EP
Sco
res,
200
3
8th Grade 4th Grade
School size has no apparent impact on NAEP scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 31: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/31.jpg)
31
400
420
440
460
480
500
520
540
560
580
$2,500 $3,000 $3,500 $4,000 $4,500 $5,000 $5,500 $6,000 $6,500 $7,000 $7,500
Instructional Spending per Pupil (2000$, State Cost of Living Adjusted), 2003
NA
EP
Sco
res,
200
3
8th Grade 4th Grade
Spending per Pupil has no apparent impact on NAEP scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 32: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/32.jpg)
32
400
420
440
460
480
500
520
540
560
580
$2,500 $3,000 $3,500 $4,000 $4,500 $5,000 $5,500 $6,000 $6,500 $7,000 $7,500
Instructional Spending per Pupil (2000$, State Cost of Living Adjusted), 2003
NA
EP
Sco
res,
200
3
8th Grade 4th Grade 8th Grade (1998) 4th Grade (1998)
Spending per Pupil has no apparent impact on NAEP scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 33: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/33.jpg)
33
400
420
440
460
480
500
520
540
560
580
$3,000 $3,500 $4,000 $4,500 $5,000 $5,500 $6,000 $6,500
Non-Instructor Spending per Pupil (2000$, State Cost of Living Adj.), 2003
NA
EP
Sco
res,
200
3
8th Grade 4th Grade
Source: National Center for Education Statistics, U.S. Department of Education
Spending per Pupil has no apparent impact on NAEP scores.
![Page 34: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/34.jpg)
34
400
420
440
460
480
500
520
540
560
580
$40,000 $45,000 $50,000 $55,000 $60,000 $65,000
Median Family Income (2000$, State Cost of Living Adjusted), 2000
NA
EP
Sco
res,
200
3
8th Grade 4th Grade
Median Family Income has no apparent impact on NAEP scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 35: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/35.jpg)
35
Perhaps standardized tests are inadequate for assessing the quality of education.
What about dropout rate?
This measure comes closer to reflecting parent/student choice-to-purchase.
![Page 36: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/36.jpg)
36
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
10 12 14 16 18 20 22 24
Student/Teacher Ratio, 2000
Dro
po
ut
Rat
e (G
rad
es 9
-12)
, 20
00Student-Teacher Ratio has no apparent impact on
dropout rate.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 37: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/37.jpg)
37
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
100 200 300 400 500 600 700 800
Student/School Ratio, 2000
Dro
po
ut
Rat
e (G
rad
es 9
-12)
, 20
00School Size has no apparent impact on dropout
rate.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 38: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/38.jpg)
38
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
$2,500 $3,000 $3,500 $4,000 $4,500 $5,000 $5,500 $6,000 $6,500
Instructional Spending per Pupil (2000$, State Cost of Living Adjusted), 2000
Dro
po
ut
Rat
e (G
rad
es 9
-12)
, 20
00Spending per Pupil has no apparent impact on dropout
rate.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 39: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/39.jpg)
39
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
$3,000 $3,500 $4,000 $4,500 $5,000 $5,500 $6,000
Non-Instructor Spending per Pupil (2000$, State Cost of Living Adjusted), 2000
Dro
po
ut
Rat
e (G
rad
es 9
-12)
, 20
00
Source: National Center for Education Statistics, U.S. Department of Education
Spending per Pupil has no apparent impact on dropout rate.
![Page 40: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/40.jpg)
40
1.0%
2.0%
3.0%
4.0%
5.0%
6.0%
7.0%
8.0%
9.0%
40,000 45,000 50,000 55,000 60,000 65,000
Median Family Income (2000$, State Cost of Living Adjusted), 2000
Dro
po
ut
Rat
e (G
rad
es 9
-12)
, 20
00Median Family Income has no apparent impact on
dropout rate.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 41: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/41.jpg)
41
Does anything predict Standardized Test performance or Dropout Rate?
![Page 42: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/42.jpg)
42
500
510
520
530
540
550
560
500 510 520 530 540 550 560
8th Grade NAEP Scores, 1998
8th
Gra
de
NA
EP
Sco
res,
200
3For 8th grade, past NAEP scores predict future NAEP
scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 43: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/43.jpg)
43
390
400
410
420
430
440
450
460
470
480
360 380 400 420 440 460 480
4th Grade NAEP Scores, 1998
4th
Gra
de
NA
EP
Sco
res,
200
3
Source: National Center for Education Statistics, U.S. Department of Education
For 4th grade, past NAEP scores predict future NAEP scores.
![Page 44: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/44.jpg)
44
510
515
520
525
530
535
540
545
550
555
560
420 430 440 450 460 470 480
4th Grade NAEP Scores, 2003
8th
Gra
de
NA
EP
Sco
res,
200
34th grade NAEP scores predict 8th grade NAEP
scores.
Source: National Center for Education Statistics, U.S. Department of Education
![Page 45: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/45.jpg)
45
Conclusion
Attributes that conventional wisdom suggests are appropriate have little or no impact on Standardized Test Results or Dropout Rates.
But, school performance is not random – states that perform well in the future are those that have performed well in the past.
Supposition
As most of the attributes employed by school districts are systemic, evidence suggests that educational quality may be better influenced via individual attributes.
![Page 46: Assessment, Accountability and Adaptation April 13, 2006](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813733550346895d9ebf7c/html5/thumbnails/46.jpg)
46
Assessment, Accountability and Adaptation
April 13, 2006
copies of this presentation can be found atwww.business.duq.edu/faculty/davies