Psychometrics, Dynamics, Psychometrics, Dynamics, and Functional Data and Functional Data

AnalysisAnalysis Jim RamsayJim Ramsay

McGill UniversityMcGill University

“The views

Testing as Input/Output Testing as Input/Output AnalysisAnalysis

A test score is actually a derivative with A test score is actually a derivative with respect to time.respect to time.

Consequently a differential equation Consequently a differential equation model for testing data seems natural.model for testing data seems natural.

Dynamic testing data will be more and Dynamic testing data will be more and more important.more important.

We have some new tools for working with We have some new tools for working with dynamic data.dynamic data.

So let’s consider how to use time as a So let’s consider how to use time as a covariate.covariate.

Learning to Play GolfLearning to Play Golf

We buy some clubs. We play a few games, We buy some clubs. We play a few games, each being an 18 item test. It’s harder each being an 18 item test. It’s harder than it looks.than it looks.

We take a lesson. We play a few games. We take a lesson. We play a few games. Our score improves to a better level.Our score improves to a better level.

We take another lesson, play some games, We take another lesson, play some games, and things improve again. and things improve again.

Key question: How quickly is a lesson Key question: How quickly is a lesson reflected in an improvement in score?reflected in an improvement in score?

BrainergyBrainergy

Energy is defined Energy is defined as “the capacity to as “the capacity to do work.”do work.”

Kinetic energy E = Kinetic energy E = MvMv22/2, and involves /2, and involves mass, distance, mass, distance, and time.and time.

• We are interested in We are interested in “the capacity to “the capacity to solve problems.”solve problems.”

• Problems involve Problems involve difficulties (=mass), difficulties (=mass), number of problems number of problems (=distance) and (=distance) and time.time.

• Let’s call mental Let’s call mental energy energy brainergybrainergy..

Brain PowerBrain Power

What counts is problem solving per What counts is problem solving per unit time.unit time.

Power = energy expended per unit Power = energy expended per unit time.time.

Brain powerBrain power = maximum difficulty of = maximum difficulty of problem solvable per unit time, or problem solvable per unit time, or

number of lighter problems solved per number of lighter problems solved per unit time.unit time.

That is, That is, brain power = d brainergy/dtbrain power = d brainergy/dt..

Brain Power and Time Brain Power and Time ScalesScales

We need the concept of brain power when We need the concept of brain power when we consider intelligence on two time we consider intelligence on two time scales:scales:

1.1. Long termLong term: How much knowledge is : How much knowledge is available over large time intervals, like a available over large time intervals, like a school yearschool year

2.2. Short termShort term: How much new knowledge : How much new knowledge can be acquired over a short time can be acquired over a short time interval, like a single class.interval, like a single class.

Tests Measure Brain PowerTests Measure Brain Power

Mental tests and psychological scales Mental tests and psychological scales are one of the greatest technological are one of the greatest technological achievements of the 20achievements of the 20thth century. century.

Tests work so well because they are Tests work so well because they are time-limited.time-limited.

Test scores reflect brain power rather Test scores reflect brain power rather than brainergy.than brainergy.

Inputs to Brain PowerInputs to Brain Power

Information about the structure of Information about the structure of the problems.the problems.

A set of tools to solve them.A set of tools to solve them. Training in the use of these tools.Training in the use of these tools. All these require time.All these require time. Inputs to acquisition of brain power Inputs to acquisition of brain power

are functions of time.are functions of time.

A Differential Equation in A Differential Equation in TimeTime

Links one or more time-derivatives, Links one or more time-derivatives, dx/dt, ddx/dt, d22x/dtx/dt22,…, to the function ,…, to the function x(t)x(t) itself.itself.

Is a model for system dynamics: Is a model for system dynamics: change over time.change over time.

Can also include one or more input or Can also include one or more input or covariate functions.covariate functions.

x(t)x(t) is a long-term description. is a long-term description. dx/dtdx/dt is a short-term description. is a short-term description.

A Simple ExampleA Simple Example

E(t)E(t) is brainergy, is brainergy, dE/dtdE/dt is brain is brain power.power.

f(t)f(t) is an input function of time, is an input function of time, such as education.such as education.

αα and and ββ are constants, are constants, ββ > 0 > 0..

( ) ( )dE

E t f tdt

Most differential equations don’t have Most differential equations don’t have explicit solutions, but this one does.explicit solutions, but this one does.

Let Let EE00 be brainergy at time be brainergy at time t = 0t = 0, and , and which will often be which will often be 00..

0 0( ) [ ( ) ]

tt tE t e E e f u du

Let’s see what happens when Let’s see what happens when αα=1, =1, ββ varies, and varies, and f(t)f(t) is a step function. is a step function.

0 0

( ) [ ( ) ]tt tE t e E e f u du

The slope of The slope of E(t)E(t) when when f(t)f(t) goes goes positive is positive is ββ..

ββ controls how fast the system controls how fast the system responds to the input responds to the input f(t).f(t).

If the system is a problem solver, If the system is a problem solver, then then ββ indicates how quickly the indicates how quickly the person learns to solve a problem.person learns to solve a problem.

After about 4/After about 4/ββ time units, full time units, full capacity is reached, and the system capacity is reached, and the system is ready for more input.is ready for more input.

Fitting Differential EquationsFitting Differential Equations We have noisy discrete-time data, and We have noisy discrete-time data, and

want to use them to estimate a want to use them to estimate a differential equation.differential equation.

We want a solution We want a solution E(t)E(t) to the to the equation to fit the data as well as equation to fit the data as well as possible.possible.

We need lots of flexibility in choosing We need lots of flexibility in choosing a differential equation, and we can’t a differential equation, and we can’t assume that there is an explicit assume that there is an explicit solution to the equation.solution to the equation.

Functional Data AnalysisFunctional Data Analysis

A collection of methods for analyzing A collection of methods for analyzing curves or functions as datacurves or functions as data

A common theme is using derivatives A common theme is using derivatives in various waysin various ways

See Ramsay and Silverman (1997) See Ramsay and Silverman (1997) Functional Data AnalysisFunctional Data Analysis. Springer.. Springer.

And Ramsay and Silverman (2002) And Ramsay and Silverman (2002) Applied Applied Functional Data AnalysisFunctional Data Analysis. . Springer.Springer.

Two Functional Data Analysis Two Functional Data Analysis TechniquesTechniques

L-spline SmoothingL-spline Smoothing: given noisy data : given noisy data and a differential equation, find a and a differential equation, find a function function E(t)E(t) that will smooth the data that will smooth the data and at the same time be nearly a and at the same time be nearly a solution to the differential equation. solution to the differential equation.

Principal Differential AnalysisPrincipal Differential Analysis: given a : given a function function E(t),E(t), estimate a linear estimate a linear differential equation for which differential equation for which E(t)E(t) is a is a solution.solution.

Estimating a DIFE from noisy Estimating a DIFE from noisy datadata

We’ve recently combined these two We’ve recently combined these two methods into a technique for estimating a methods into a technique for estimating a differential equation from noisy data.differential equation from noisy data.

In our simple example, this amounts to In our simple example, this amounts to estimating parameters estimating parameters αα and and ββ..

But much more complex DIFE’s can be But much more complex DIFE’s can be estimated as well, including linear or estimated as well, including linear or nonlinear, and single or multiple variable nonlinear, and single or multiple variable systems.systems.

An Oil RefineryAn Oil Refinery

Here are some data from an oil Here are some data from an oil refinery in Corpus Christi.refinery in Corpus Christi.

The input The input f(t)f(t) (reflux flow) is (reflux flow) is negatively coupled to the output negatively coupled to the output E(t)E(t) (tray 47 level).(tray 47 level).

The smooth curve is a solution to the The smooth curve is a solution to the differential equation that best differential equation that best represents this relationship.represents this relationship.

Perhaps this oil refinery is not too smart!

Many situations will call for multiple Many situations will call for multiple outputs: Performance with a putter, a outputs: Performance with a putter, a driver, and an iron, for example. Or in driver, and an iron, for example. Or in algebra and geometry.algebra and geometry.

And many situations will involve And many situations will involve multiple inputs: Regular classes, multiple inputs: Regular classes, tutoring sessions, labs and etc.tutoring sessions, labs and etc.

The technology used in these The technology used in these illustrations can handle these illustrations can handle these situations, at least for linear situations, at least for linear differential equations. Nonlinear differential equations. Nonlinear equations don’t pose any problem in equations don’t pose any problem in principle.principle.

Some Simulated DataSome Simulated Data

Imagine that the data are golf scores Imagine that the data are golf scores over successive games, and that the over successive games, and that the input is a set of three equally-spaced input is a set of three equally-spaced lessons from a golf pro. lessons from a golf pro.

The following slides show three The following slides show three golfers. Which is a future Tiger golfers. Which is a future Tiger Woods? Woods?

These lessons are nicely timed.

This person needs to find another sport!

This person should get lessons more often!

Is this Model Good Enough?Is this Model Good Enough?

Specifying Specifying ββ to be constant is too simple. to be constant is too simple. Allowing for fatigue, boredom, and other Allowing for fatigue, boredom, and other things requires a function things requires a function ββ(t).(t).

A first order equation can’t allow for A first order equation can’t allow for sudden transient effects like insight. We sudden transient effects like insight. We may need a differential equation may need a differential equation involving higher derivatives.involving higher derivatives.

We may need nonlinear equations as We may need nonlinear equations as well.well.

A Nonlinear Differential A Nonlinear Differential EquationEquation

21 2

1 22

22 1

2 21

( ( ))[ ( ) ] ( )

1 ( ( ))

( ( ))[ ( ) ] ( )

1 ( ( ))

dE A BE tE t f t

dt C BE t

dE A BE tE t f t

dt C BE t

•The summed output from these two equations will exhibit both the rapid learning and long-term retention required of human learners.•See H. R. Wilson (1999) Spikes, Decisions and Actions, Oxford, for many more examples of differential equation models in neuroscience.

Control TheoryControl Theory

Engineers who work with Engineers who work with input/output systems have input/output systems have developed ways of designing developed ways of designing feedback loopsfeedback loops to optimize outputs. to optimize outputs.

We’re working with a team of We’re working with a team of chemical engineers at Queen’s chemical engineers at Queen’s University.University.

Where Would the Data Come Where Would the Data Come From?From?

Can we design customized learning Can we design customized learning situations, like golf, and track how a situations, like golf, and track how a learner makes progress as a function learner makes progress as a function of time and inputs?of time and inputs?

Perhaps video and computer games Perhaps video and computer games are nearly what we need.are nearly what we need.

We already know that people will pay We already know that people will pay big money to have these big money to have these experiences.experiences.

Would corporations with deep Would corporations with deep pockets pay for this kind of testing? pockets pay for this kind of testing?

ConclusionsConclusions

Dynamic testing would generate Dynamic testing would generate performance data over time that depend performance data over time that depend on one or more functional covariates.on one or more functional covariates.

New tools are available for these data New tools are available for these data that fit them with a differential equation.that fit them with a differential equation.

Dynamic psychometricsDynamic psychometrics looks promising! looks promising!