# etc5410: nonparametric smoothing methods etc5410: nonparametric smoothing methods smoothing splines

Post on 31-Mar-2020

0 views

Embed Size (px)

TRANSCRIPT

ETC5410: Nonparametric smoothing methods 1

ETC5410: Nonparametric smoothing methods

July 2008

Rob J Hyndman http://www.robjhyndman.com/

ETC5410: Nonparametric smoothing methods 2

Outline

1 Density estimation

2 Kernel regression

3 Splines

4 Additive models

5 Functional data analysis

ETC5410: Nonparametric smoothing methods 3

ETC5410: Nonparametric smoothing methods

3. Splines

1 Interpolating splines

2 Smoothing splines

3 Regression splines

4 Penalized regression splines

5 Other bases

ETC5410: Nonparametric smoothing methods Interpolating splines 4

Outline

1 Interpolating splines

2 Smoothing splines

3 Regression splines

4 Penalized regression splines

5 Other bases

ETC5410: Nonparametric smoothing methods Interpolating splines 5

Interpolating splines

ETC5410: Nonparametric smoothing methods Interpolating splines 6

Interpolating splines

Observations: (xi , yi) for i = 1, . . . , n.

A spline is a continuous function f (x) interpolating all points and consisting of polynomials between each consecutive pair of ‘knots’ xi and xi+1.

Parameters constrained so that f (x) is continuous.

Further constraints imposed to give continuous derivatives.

ETC5410: Nonparametric smoothing methods Interpolating splines 6

Interpolating splines

Observations: (xi , yi) for i = 1, . . . , n.

A spline is a continuous function f (x) interpolating all points and consisting of polynomials between each consecutive pair of ‘knots’ xi and xi+1.

Parameters constrained so that f (x) is continuous.

Further constraints imposed to give continuous derivatives.

ETC5410: Nonparametric smoothing methods Interpolating splines 6

Interpolating splines

Observations: (xi , yi) for i = 1, . . . , n.

A spline is a continuous function f (x) interpolating all points and consisting of polynomials between each consecutive pair of ‘knots’ xi and xi+1.

Parameters constrained so that f (x) is continuous.

Further constraints imposed to give continuous derivatives.

ETC5410: Nonparametric smoothing methods Interpolating splines 6

Interpolating splines

Observations: (xi , yi) for i = 1, . . . , n.

Parameters constrained so that f (x) is continuous.

Further constraints imposed to give continuous derivatives.

ETC5410: Nonparametric smoothing methods Interpolating splines 7

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Linear interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 7

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Linear interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 8

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Quadratic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 8

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Quadratic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 8

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Quadratic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 8

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Quadratic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 8

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Quadratic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 9

Interpolating splines

●

●

●

●

●

●

●

●

0 20 40 60 80 100

0 5

10 15

Cubic interpolating spline

Age

C um

ul at

iv e

de at

hs (

'0 00

)

ETC5410: Nonparametric smoothing methods Interpolating splines 10

Interpolating splines

To define a cubic spline uniquely, it is necessary to specify two additional constraints.

Usually, this is done by specifying the form of the function at the two extremes, x1 and xn.

For example, a “natural” spline is obtained by requiring f ′′(x1) = f

′′(xn) = 0, thus making the curve linear at the extremes.

Implementation in R

plot(x,y)

lines(spline(x,y))

ETC5410: Nonparametric smoothing methods Interpolating splines 10

Interpolating splines

To define a cubic spline uniquely, it is necessary to specify two additional constraints.

Usually, this is done by specifying the form of the function at the two extremes, x1 and xn.

For example, a “natural” spline is obtained by requiring f ′′(x1) = f

′′(xn) = 0, thus making the curve linear at the extremes.

Implementation in R

plot(x,y)

lines(spline(x,y))

ETC5410: Nonparametric smoothing methods Interpolating splines 10

Interpolating splines

To define a cubic spline uniquely, it is necessary to specify two additional constraints.

Usually, this is done by specifying the form of the function at the two extremes, x1 and xn.

For example, a “natural” spline is obtained by requiring f ′′(x1) = f

′′(xn) = 0, thus making the curve linear at the extremes.

Implementation in R

plot(x,y)

lines(spline(x,y))

ETC5410: Nonparametric smoothing methods Interpolating splines 10

Interpolating splines

To define a cubic spline uniquely, it is necessary to specify two additional constraints.

Usually, this is done by specifying the form of the function at the two extremes, x1 and xn.

For example, a “natural” spline is obtained by requiring f ′′(x1) = f

′′(xn) = 0, thus making the curve linear at the extremes.

Implementation in R

plot(x,y)

lines(spline(x,y))

ETC5410: Nonparametric smoothing methods Interpolating splines 10

Interpolating splines

To define a cubic spline uniquely, it is necessary to specify two additional constraints.

Usually, this is done by specifying the form of the function at the two extremes, x1 and xn.

For example, a “natural” spline is obtained by requiring f ′′(x1) = f

′′(xn) = 0, thus making the curve linear at the extremes.

Implementation in R

plot(x,y)

lines(spline(x,y))

ETC5410: Nonparametric smoothing methods Interpolating splines 11

Monotonic interpolation

‘Hyman filter’ ensures spline is monotonic by constraining derivatives while (possibly) sacrificing smoothness.

Second derivative may no longer be continuous at all knots.

Implementation in R

require(demography)

plot(x,y)

lines(cm.spline(x,y), col=4)

Reference: Smith, Hyndman and Wood (JPR, 2001)

ETC5410: Nonparametric smoothing methods Interpolating splines 11

Monotonic interpolation

‘Hyman filter’ ensures spline is monotonic by constraining derivatives while (possibly) sacrificing smoothness.

Second derivative may no longer be continuous at all knots.

Implementation in R

require(demography)

plot(x,y)

lines(cm.spline(x,y), col=4)

Reference: Smith, Hyndman and Wood (JPR, 2001)

ETC5410: Nonparametric smoothing methods Interpolati

Recommended