predicting outcomes of systemic change in education

October 12, 2006 Predicting Outcomes of Systemic Change in Education

1

Predicting Outcomes of Systemic Change in Education

Theodore FrickKenneth R. ThompsonJoyce Koh


2

The Problem: No Adequate Theory

K-12 schools under increasing pressure with NCLB (No Child Left Behind)

Failing schools will have real incentive to change

Questions are: Change what? Change how? Attempts to change will be trial and error if no

valid theory What we need is good educational systems

theory


3

Bridge Analogy

Consider an old bridge that is failingStructurally weakToo few lanes for trafficTraffic increasing

If not fixed, will collapse: vehicles plunge into river

Would we build a new bridge by trial and error?


4

Bridge Analogy Cont’d

No, we would not build a bridge by trial and error!

Modern engineers use proven scientific theories: Newtonian physics, classical mechanics, statics, structural engineering

But in education, we are attempting to change systems by trial and error


5

We Need Educational Systems Theory

Lewin: “There is nothing so practical as good theory.”

We have theories of: InstructionLearningPedagogy, e.g., Montessori method

We have had no educational systems theory


6

ATIS: Axiomatic Theories of Intentional Systems

Until now!ATIS provides a robust, complex theory

that can be applied to educational systems

See Thompson (2006) seminal articles in Scientific Inquiry Journal: http://www.iigss.net/Scientific-Inquiry/table.htm Foundations and definitions Methodology of theory construction


7

NCLB Example

To make this more concrete, consider the following scenario:

Smithtown School #9 failed to achieve state standards for No Child Left Behind (NCLB)


8

SMITHTOWN SCHOOL #9

Parents start transferring children to other schools

Scenario


9

ATIS Prediction – Axiom 13

Then filtration increases

NCLB rating deters enrollmentEnrollment falls

If input decreases

Year 1 Year 2 Year 3

SMITHTOWN SCHOOL #9

This is a FAILING school. Tommy shouldn’t enroll here!


10


Then storeput decreases

Fewer students attending classesEnrollment falls

If input decreases



11


Then fromput decreases

Fewer students to graduate

ADMINISTRATIONOFFICE

Hmm…there aren’t as many diplomas to print this year!

Enrollment falls

If input decreases



12


Then feedout decreases

Fewer graduatesEnrollment falls

If input decreases



13

SMITHTOWN SCHOOL #9 BOARD MEETING AGENDA:

How to improve achievement scores?


If filtration increases Then adaptability increases

Smithtown adapts tomaintain system stability

SMITHTOWN SCHOOL #9

NCLB rating deters enrollment

This is a FAILING school. Tommy shouldn’t enroll here!


14

Using ATIS with Smithtown’s adaptation strategies

How can Smithtown adapt? Change the structure (cf. Senge, 2006; Thompson, 2006)

What if Smithtown changes its structure by increasing STRONGNESS of affect relations that are of type: guidance of learning?


15

Smithtown’s proposed strategy

Structural change: Increase avenues of instruction through:Teaching aidesPeer tutoring Instructional technology e.g. using e-

Learning software

Increase strongness


16


If strongness increases Then hierarchical order decreases

After: Less focus on teacher as guide of learning.

GUIDE

GU

IDE

GU

IDE

Teaching aides

Teachers

E-learningsoftware

Peer tutoring

More ‘guidance of learning’ connections for students

GUIDE

GUIDEGUIDE

Before: Teacher is main guide.


17


Then flexibility increases

More different ways for guidinglearning of students

Peer tutoring

Teaching aides

E-learningsoftware

Teachers

If strongness increases

Teaching aides

Teachers

E-learningsoftware

Peer tutoring



18

SMITHTOWN SCHOOL #9


Then filtration decreases

Smithtown #9 makes NCLB rating. This encourages enrollment.

They’ve made AYP. Tommy can enroll here!

FAILURESUCCESS


Teaching aides

Teachers

E-learningsoftware

Peer tutoring


AYP = Annual Yearly Progress (part of NCLB law)


19


Then isomorphism increases

Smithtown replicates successstrategy for more schools

SMITHTOWN SCHOOL #9

SMITHTOWN SCHOOL #1

SMITHTOWN SCHOOL #12SMITHTOWN

SCHOOL #25

SMITHTOWN SCHOOL #5

SUCCESS

Smithtown #9 makes NCLB rating.This raises enrollment.

They’ve improvedachievement scores and made AYP. Tommy canenroll here!

SMITHTOWN SCHOOL #9


Increasestrongness

Increasestrongness

Increasestrongness

Increasestrongness

AYP = Annual Yearly Progress (part of NCLB law)


20

Summary

If we have a valid educational systems theory,

Based on predictable temporal patterns and configurations,

Then we can change an education system with a reasonable expectation that it will actually be improved.


21

ATIS: Axiomatic Theories of Intentional Systems

Basic strategy (Thompson, 2005) Affect relations determine system properties. System properties determine system state. System state determines the system-descriptive axiom set. The axiom set provides logic-based predictive outcomes. Affect relations also determine system topological structure in that

every affect relation defines a topology. Topological structure provides dynamic, real-time predictive

outcomes. System-descriptive axiom set and system topological structure

together determine total system structure and system-predictive outcomes.

The logical analysis as dependent upon the system axioms comes first, followed by a topological analysis that establishes in fact the vectored system outcome; that is, that the system is actually taking the path indicated by the logical analysis.


22

Axiomatic Theories of Intentional Systems – Thompson (2006)

Examples of axioms and theorems: If system input decreases, then filtration increases. If system filtration increases, then adaptability increases. If system strongness increases, then hierarchical order

decreases. If system strongness increases, then flexibility increases. If system strongness increases, then input increases. If system strongness increases, then filtration decreases.

See full theory (over 200 axioms/theorems) and reports at: http://www.indiana.edu/~aptfrick/reports/


23

Applying ATIS to a specific system

Only some of the axioms/theorems in ATIS will apply to a given system.

PESO (Predicting Education System Outcomes) is the software tool that makes predictions, based on specific conditions in a particular system.

In other words, PESO uses ATIS as an expert system.


24

PESO Demo

Link to PESO Prototype (restricted access)

1. Enter system condition(s)2. PESO applies ATIS and makes

predictions. PESO developed in such a way that

additional axioms and theorems can be easily added (Flash ActionScript objects)


25

APC Demo

Analysis of Patterns in Configuration (Frick & Thompson, 2006)Specify components and affect relationsAPC software calculates values of structural

propertiesLink to prototype

Uses measures defined in ATIS Graph Theory:

http://www.indiana.edu/~aptfrick/reports/11ATISgraphtheory.pdf


26

Verifying Systems Theory

The systems theory consists of axioms and theorems for making predictions

Axioms and theorems consist of dynamic and structural properties

APT&C can be used as a verification methodology


27

For ATIS Theory Validation

We need ways of measuring system: Dynamics (temporal change) Structures (affect relations)

This will be done by: Analysis of Patterns in Time Analysis of Patterns in Configuration In short: APT&C, see Frick (2005) grant proposal:

http://education.indiana.edu/~frick/proposals/apt&c.pdf This will be covered in the APT&C presentation on Friday at

AECT in Dallas.


28

Using Theoretical Predictions

We can use theoretical predictions to make practical decisions, e.g., Not smoke, to reduce chances of lung

cancer later in life. Take umbrella if rain is predicted to be

highly likely.


29

Imagine for the moment that…

We have a valid educational systems theory that:Can predict education systems outcomes

based on current conditions (ATIS), and Is based on empirically verified temporal

patterns and configurations in systems


30

Systemic Change Model

This leads to an inquiry-based systems change strategy:

Get Ready >> Set >> Go!


31

Change Strategy: Get Ready >> Set >> Go!

Phase 1: Get Ready Identify the specific current education system to be improved. Over some interval of time, measure system properties (e.g.,

input, regulation, complexity, strongness) with Analysis of Patterns in Time and Configuration (APT&C)

Use Predicting Educational Systems Outcomes (PESO) software to predict outcomes based on observed system properties under existing conditions (e.g., complexity increases, decreases, or remains constant). These predictions are based on how the system is currently designed and operates under existing conditions, before any new design is implemented.

If these outcomes are what are wanted, then do not modify the system. Otherwise, proceed to Phase 2.


32


Phase 2: Set Use PESO software to model newly

envisioned educational system designs – i.e., the changes desired which are feasible.

Run PESO predictions far out enough in time to make sure all the consequences of the newly designed system would be acceptable. Are these the wanted outcomes? If yes, proceed to Phase 3.


33


Phase 3: Go! Implement the new design chosen in Phase 2. Over some interval of time, measure system

properties with APT&C. Verify that predicted system outcomes have

occurred. If not, was something important overlooked in the observation and analysis of this particular system? Proceed to Phase 2.


34

SimEd Technologies

We refer to:

ATIS theory model APT&C software PESO software, and the ‘Get Ready, Set, Go!’ model

as

SimEd Technologies


35

SimEd Technologies

Currently under developmentFurther ATIS theory developmentAPT and APT software developmentPESO development

Need to obtain major funding to support these activities


36

SimEd Technologies

Questions?

For more information on SimEd Technologies:

http://simedtech.com

predicting outcomes of systemic change in education

Documents

strategystructural change

educationatis prediction

good theory

proven scientific theories

old bridge

new bridge

complex theory

educationusing atis