time series analysis topics in machine learning fall 2011 school of electrical engineering and...
TRANSCRIPT
![Page 1: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/1.jpg)
Time Series Analysis
Topics in Machine Learning
Fall 2011
School of Electrical Engineering
and Computer Science
![Page 2: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/2.jpg)
Time Series Discussions
• Overview• Basic definitions• Time domain• Forecasting• Frequency domain• State space
![Page 3: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/3.jpg)
Why Time Series Analysis?
• Sometimes the concept we want to learn is the relationship between points in time
![Page 4: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/4.jpg)
Time series: a sequence of
measurements over time
A sequence of random variablesx1, x2, x3, …
What is a time series?
![Page 5: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/5.jpg)
Time Series ExamplesDefinition: A sequence of measurements over timeDefinition: A sequence of measurements over time
Finance
Social science
Epidemiology
Medicine
Meterology
Speech
Geophysics
Seismology
Robotics
![Page 6: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/6.jpg)
Three Approaches
• Time domain approach– Analyze dependence of current value on past values
• Frequency domain approach– Analyze periodic sinusoidal variation
• State space models– Represent state as collection of variable values– Model transition between states
![Page 7: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/7.jpg)
Sample Time Series Data
Johnson & Johnson quarterly earnings/share, 1960-1980
![Page 8: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/8.jpg)
Sample Time Series Data
Yearly average global temperature deviations
![Page 9: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/9.jpg)
Sample Time Series Data
Speech recording of “aaa…hhh”, 10k pps
![Page 10: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/10.jpg)
Sample Time Series Data
NYSE daily weighted market returns
![Page 11: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/11.jpg)
Not all time data will exhibit strong patterns…
LA annual rainfall
![Page 12: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/12.jpg)
…and others will be apparent
Canadian Hare counts
![Page 13: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/13.jpg)
Time Series Discussions
• Overview• Basic definitions• Time domain• Forecasting• Frequency domain• State space
![Page 14: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/14.jpg)
Definitions
• Mean
• Variance
mean
variance
![Page 15: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/15.jpg)
Definitions
• Covariance
• Correlation
N
i
yixi
N
yxYXCov
1
))((),(
YX
YXCovrYXCor
),(
),(
![Page 16: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/16.jpg)
Y
X
Y
X
Y
Xr = -1 r = -.6 r = 0
Y
X
Y
Xr = +.3r = +1
Y
Xr = 0
Correlation
![Page 17: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/17.jpg)
Redefined for Time
•
•
•
,...2,1,0)()( tforXEt tXErgodic?
Mean function
),()0(
)()( tht
X
XX XXCor
hh
Autocorrelation
),()( thtX XXCovh lag
Autocovariance
![Page 18: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/18.jpg)
Autocorrelation Examples
Positive
lag
Negative
lag
![Page 19: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/19.jpg)
Stationarity – When there is no relationship
• {Xt} is stationary if– X(t) is independent of t
– X(t+h,t) is independent of t for each h
• In other words, properties of each section are the same
• Special case: white noise
![Page 20: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/20.jpg)
Time Series Discussions
• Overview• Basic definitions• Time domain• Forecasting• Frequency domain• State space
![Page 21: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/21.jpg)
Linear Regression
• Fit a line to the data• Ordinary least squares
– Minimize sum of squared
distances between points
and line
• Try this out at http://hspm.sph.sc.edu/courses/J716/demos/LeastSquares/LeastSquaresDemo.html
y = x +
![Page 22: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/22.jpg)
R2: Evaluating Goodness of Fit
• Least squares minimizes
the combined residual
• Explained sum of squares
is difference between line
and mean
• Total sum of squares is the total of these two
y = x +
i iYYRSS 2)(
i
YYESS 2)(
ii i YYYYRSSESSTSS 22 )()(
![Page 23: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/23.jpg)
R2: Evaluating Goodness of Fit
• R2, the coefficient
of determination
• 0 R2 1• Regression minimizes RSS and so
maximizes R2
y = x + TSS
RSS
TSS
ESSR 12
![Page 24: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/24.jpg)
R2: Evaluating Goodness of Fit
TSS
RSS
TSS
ESSR 12
![Page 25: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/25.jpg)
R2: Evaluating Goodness of Fit
TSS
RSS
TSS
ESSR 12
![Page 26: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/26.jpg)
R2: Evaluating Goodness of Fit
TSS
RSS
TSS
ESSR 12
![Page 27: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/27.jpg)
Linear Regression
• Can report:– Direction of trend (>0, <0, 0)– Steepness of trend (slope)– Goodness of fit to trend (R2)
![Page 28: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/28.jpg)
Examples
![Page 29: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/29.jpg)
What if a linear trend does not fit my data well?
• Could be no relationship• Could be too much local variation
– Want to look at longer-term trend– Smooth the data
• Could have periodic or seasonality effects– Add seasonal components
• Could be a nonlinear relationship453423121 QbQbQbQbtbaX t
![Page 30: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/30.jpg)
Moving Average
• Compute an average of the last m consecutive data points
• 4-point moving average is
• Smooths white noise
4
)( 321)4(
tttt
MA
xxxxx
k
kjjtjt xam
• Can apply higher-order MA
• Exponential smoothing
• Kernel smoothing
![Page 31: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/31.jpg)
Power Load Data
5 week
53 week
![Page 32: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/32.jpg)
Piecewise Aggregate Approximation
• Segment the data into linear pieces
Interesting paper
![Page 33: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/33.jpg)
Nonlinear Trend Examples
![Page 34: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/34.jpg)
Nonlinear Regression
![Page 35: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/35.jpg)
Fit Known Distributions
![Page 36: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/36.jpg)
ARIMA: Putting the pieces together
• Autoregressive model of order p: AR(p)• Moving average model of order q: MA(q)• ARMA(p,q)
![Page 37: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/37.jpg)
ARIMA: Putting the pieces together
• Autoregressive model of order p: AR(p)
• Moving average model of order q: MA(q)• ARMA(p,q)
tptpttt wxxxx ..2211
![Page 38: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/38.jpg)
AR(1),
0 20 40 60 80 100
-20
24
9.0
![Page 39: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/39.jpg)
AR(1),
0 20 40 60 80 100
-4-2
02
49.0
![Page 40: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/40.jpg)
ARIMA: Putting the pieces together
• Autoregressive model of order p: AR(p)
• Moving average model of order q: MA(q)
• ARMA(p,q)
tptpttt wxxxx ..2211
tqtqttt wwwwx ..2211
![Page 41: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/41.jpg)
ARIMA: Putting the pieces together
• Autoregressive model of order p: AR(p)
• Moving average model of order q: MA(q)
• ARMA(p,q)– A time series is ARMA(p,q) if it is stationary and
tptpttt wxxxx ..2211
tqtqttt wwwwx ..2211
qtqtt
tptpttt
www
wxxxx
..
..
2211
2211
![Page 42: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/42.jpg)
ARIMA (AutoRegressive Integrated Moving Average)
• ARMA only applies to stationary process• Apply differencing to obtain stationarity
– Replace its value by incremental change from last value
• A process xt is ARIMA(p,d,q) if– AR(p)– MA(q)– Differenced d times
• Also known as Box Jenkins
Differenced x1 x2 x3 x4
1 time x2-x1’ x3’-x2’ x4’-x3’
2 times x3’-2x2’+x1’ x4’-2x3’+x2’
![Page 43: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/43.jpg)
Time Series Discussions
• Overview• Basic definitions• Time domain• Forecasting• Frequency domain• State space
![Page 44: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/44.jpg)
Express Data as Fourier Frequencies
• Time domain– Express present as function of the past
• Frequency domain– Express present as function of oscillations, or
sinusoids
![Page 45: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/45.jpg)
Time Series Definitions
• Frequency, , measured at cycles per time point• J&J data
– 1 cycle each year– 4 data points (time points) each cycle– 0.25 cycles per data point
• Period of a time series, T = 1/– J&J, T = 1/.25 = 4– 4 data points per cycle– Note: Need at least 2
![Page 46: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/46.jpg)
Fourier Series
• Time series is a mixture of oscillations– Can describe each by amplitude, frequency and
phase– Can also describe as a sum of amplitudes at all
time points– (or magnitudes at all frequencies)
– If we allow for mixtures of periodic series then
)2sin()2cos( ttxt
Take a look
q
iiiiit ttx
1
)]2sin()2cos([
![Page 47: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/47.jpg)
Example
)100/62sin(3
)100/62cos(21
t
txt
)100/402sin(7
)100/402cos(63
t
txt
)100/102sin(5
)100/102cos(42
t
txt
3214 tttt xxxx
![Page 48: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/48.jpg)
How Compute Parameters?
• Regression• Discrete Fourier Transform
• DFTs represent amplitude and phase of series components
• Can use redundancies to speed it up (FFT)
2/
1
)]/2sin()/()/2cos()/([n
jiit ntjnjntjnjx
n
t
ntjitexnnjdd
1
/221
)/()(
![Page 49: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/49.jpg)
Breaking down a DFT
• Amplitude
• Phase
22 ))/(())/((|)/(|)/( njdInjdRnjdnjA
)))/((/))/(((tan)/( 1 njdRnjdInj
![Page 50: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/50.jpg)
Example
-1
0
1
2
GB
P
-1
0
1
2
GB
P
-1
0
1
2
GB
P
-1
0
1
2
GB
P
-1
0
1
2
GB
P
-1
0
1
2
GB
P
1 frequency
2 frequencies
3 frequencies
5 frequencies
10 frequencies
20 frequencies
![Page 51: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/51.jpg)
Periodogram
• Measure of squared correlation between– Data and– Sinusoids oscillating at frequency of j/n
– Compute quickly using FFT
n
tt
n
tt ntjx
nntjx
nnjP
1
2
1
2 ))/2sin(2
())/2cos(2
()/(
![Page 52: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/52.jpg)
Example
P(6/100) = 13, P(10/100) = 41, P(40/100) = 85
![Page 53: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/53.jpg)
Wavelets
• Can break series up into segments– Called wavelets– Analyze a window of time separately– Variable-sized windows
![Page 54: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/54.jpg)
Time Series Discussions
• Overview• Basic definitions• Time domain• Forecasting• Frequency domain• State space
![Page 55: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/55.jpg)
State Space Models
• Current situation represented as a state– Estimate state variables from noisy observations
over time– Estimate transitions between states
• Kalman Filters– Similar to HMMs
• HMM models discrete variables• Kalman filters models continuous variables
![Page 56: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/56.jpg)
Conceptual Overview
• Lost on a 1-dimensional line• Receive occasional sextant position readings
– Possibly incorrect
• Position x(t), Velocity x’(t)
x
![Page 57: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/57.jpg)
Conceptual Overview
0 10 20 30 40 50 60 70 80 90 1000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
• Current location distribution is Gaussian• Transition model is linear Gaussian• The sensor model is linear Gaussian• Sextant Measurement at ti: Mean = i and Variance = 2
i
• Measured Velocity at ti: Mean = ’i and Variance = ’2i
Noisy information
![Page 58: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/58.jpg)
Kalman Filter Algorithm
• Start with current location (Gaussian)• Predict next location
– Use current location– Use transition function (linear Gaussian)– Result is Gaussian
• Get next sensor measurement (Gaussian)• Correct prediction
– Weighted mean of previous prediction and measurement
![Page 59: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/59.jpg)
0 10 20 30 40 50 60 70 80 90 1000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Conceptual Overview
• We generate the prediction for time i+, prediction is Gaussian• GPS Measurement: Mean = i+ and Variance = 2
i +
• They do not match
prediction Measurement at i
![Page 60: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/60.jpg)
0 10 20 30 40 50 60 70 80 90 1000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Conceptual Overview
• Corrected mean is the new optimal estimate of position• New variance is smaller than either of the previous two variances
measurement at i+
corrected estimate
prediction
![Page 61: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/61.jpg)
Updating Gaussian Distributions
• One-step predicted distribution is Gaussian
• After new (linear Gaussian) evidence, updated distribution is Gaussian
tx ttttttt dxexPxXPeXP )|()|()|( :11:11
)|()|()|( :11111:11 tttttt eXPXePeXP
PriorTransition
Previous step
New measurement
![Page 62: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/62.jpg)
Why Is Kalman Great?
• The method, that is…• Representation of state-based series with
general continuous variables grows without bound
![Page 63: Time Series Analysis Topics in Machine Learning Fall 2011 School of Electrical Engineering and Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022062417/551772ac5503463e368b4cf2/html5/thumbnails/63.jpg)
Why Is Time Series Important?
• Time is an important component of many processes
• Do not ignore time in learning problems• ML can benefit from, and in turn benefit,
these techniques– Dimensionality reduction of series– Rule discovery– Cluster series– Classify series– Forecast data points– Anomaly detection