fastdtw: toward accurate dynamic time warping in linear time and space
DESCRIPTION
FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space. Department of Computer Sciences Florida Institute of Technology Stan Salvador and Philip Chan. Outline. Dynamic Time Warping (DTW) Problem Statement Related Work for Speeding up DTW FastDTW Algorithm - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/1.jpg)
FastDTW: Toward Accurate Dynamic Time Warping in
Linear Time and Space
Department of Computer SciencesFlorida Institute of Technology
Stan Salvador and Philip Chan
![Page 2: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/2.jpg)
Outline
• Dynamic Time Warping (DTW)
• Problem Statement
• Related Work for Speeding up DTW
• FastDTW Algorithm
• Evaluation of FastDTW
• Contributions
• Limitations and Future Work
![Page 3: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/3.jpg)
Dynamic Time Warping (DTW)
• Aligns two time series by warping the time dimension
• Warping - expanding/contracting the time dimension
Time
![Page 4: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/4.jpg)
The Dynamic Time Warping Algorithm
• A dynamic programming approach
• Solutions to slightly smaller problems used to find larger solutions
![Page 5: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/5.jpg)
The DTW Cost Matrix
1
1
|X|
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
![Page 6: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/6.jpg)
Distance of Min-Cost Warp Path
1
1
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
|X|
![Page 7: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/7.jpg)
Finding Min-Cost Warp Path
)]1,1(),1,(),,1(min[ jiDjiDjiD
1
1
|X|
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
![Page 8: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/8.jpg)
Advantages of DTW
• DTW is optimal
• An intuitive distance measurement
• Local variation in the time axis is common– Handwriting– Speech– “Events” that start after varying delays
![Page 9: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/9.jpg)
Disadvantages of DTW
• O(N2) time and space complexity
• Only practical for small data sets (<3,000)
• Time series are often very long
• Data mining requires a scalable DTW algorithm
![Page 10: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/10.jpg)
Problem Statement
• We desire an efficient Dynamic Time Warping algorithm– Linear time complexity– Linear space complexity– Warp path is needed in addition to warp
distance– Warp path must be nearly optimal
![Page 11: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/11.jpg)
Does DTW Need to be Faster?
“Myth 3: There is a need (and room) for improvements in the speed of DTW for data
mining applications.”
(Keogh today-9:45am)
• Keogh: many time series
• FastDTW: Long time series
![Page 12: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/12.jpg)
Existing Methods to Speed Up DTW
1. Constraints – only fill in part of the cost matrix
2. Abstraction – sample the data before time warping
![Page 13: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/13.jpg)
Constraints
• Still O(N2) if the window width is a function of input size (linear if the width is constant)
• Assumes a near-optimal warp path stays near the i=j axis
• Accuracy depends on the domain
Sakoe-Chiba Band (Sakoe & Chiba 1978)
Itakura Parallelogram (Itakura 1975)
![Page 14: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/14.jpg)
Abstraction1/51/1 1/1
• O(N) if N pts are sampled down to ≤
• Assumptions– Sampling preserves time series structure– Small deviations from the optimal path cause
little increase in warp-path distance
(Keogh & Pazzani 2000), (Chu et al. 2002)
N
![Page 15: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/15.jpg)
Our FastDTW Algorithm
• A multi-resolution approach inspired by a multi-level graph bisection algorithm (Karypis 1997)
• 3 key operations1. Coarsening – reduce
the resolution of a time series
2. Projection – use a low-res warp path as an initial solution at a higher resolution
3. Refinement – Refine a projected warp path locally adjusting the path
![Page 16: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/16.jpg)
Sample Run of FastDTW
1/8 1/4 1/2 1/1
![Page 17: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/17.jpg)
FastDTW Algorithm
1. Set the resolution to be the coarsest2. Find the initial path using regular DTW3. Repeat
a. Double the resolutionb. Project the path onto the finer resolutionc. Find a path through the projected area (plus a
small radius around the projected area)
4. Until the original resolution is reached
![Page 18: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/18.jpg)
Complexity
• O(N) time
• O(N) space
• Details in the paper
![Page 19: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/19.jpg)
Evaluation Criteria
• AccuracyThe error of an approximate Time Warping algorithm:
% error = where:
approxDist – the warp path distance of the approximate algorithm
optimalDist – the warp path distance of the DTW algorithm
• EfficiencyRuntime (measured in seconds)
100
toptimalDis
toptimalDisapproxDist
![Page 20: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/20.jpg)
Evaluation Procedure (Accuracy)
• Data Sets – UCR Time Series Data Mining Archive (Keogh & Folias 2002), 3 groups used:
1. Random – 45 unrelated time series (earthquakes, random walk, eeg, speech, etc.)
2. Trace – 200 time series simulating nuclear power plant failure (4 classes)
3. Gun – 200 time series of a gun being drawn and pointed (2 classes)
• Procedure1. Run FastDTW, Constraints (Sakoe-Chiba Band), and Data
Abstraction on all pairs within a data set group, also vary the radius
2. Record the average error of all three methods for a group of data and a radius
![Page 21: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/21.jpg)
Radius0 1 10 20 30
FastDTW 19.2% 8.6% 1.5% 0.8% 0.6%
Abstraction 983.3% 547.9% 6.5% 2.8% 1.8%
Band 2749.2% 2385.7% 794.1% 136.8% 9.3%
Average % Error (Accuracy)
![Page 22: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/22.jpg)
Error in Different Data Sets
Accuracy of FastDTW, Bands, and Data Abstraction
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 5 10 15 20 25 30radius
Err
or
FastDTW-Random FastDTW-Trace FastDTW - GunAbstraction-Random Abstraction-Trace Abstraction-GunBand-Random Band-Trace Band-Gun
![Page 23: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/23.jpg)
Evaluation Procedure (Execution-time)
• Data Sets– Synthetic sine waves with Gaussian noise– 10 to 180,000 data points
• Procedure1. Run FastDTW and DTW on each data set,
vary the radius for FastDTW
2. Compare the Execution times
![Page 24: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/24.jpg)
Execution TimeExecution Time of FastDTW on Large Time Series
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 160,000 180,000
Length of Time Series
Tim
e (s
eco
nd
s)
DTW
FastDTW (radius=100)
FastDTW (radius=20)
FastDTW (radius=0)
![Page 25: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/25.jpg)
Summary of Contributions
• FastDTW – an approximation of DTW– O(N) time and space complexity– Scales well to long time series– Accurate, 8.6% error if radius=1, 0.8% error if
radius=20
![Page 26: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/26.jpg)
Limitations and Future Work
• Limitations– FastDTW does not always find an optimal solution
• Future Work– Examine using different step sizes between resolutions
– Investigate search algorithms to help improve refinement
– Examine # of cells evaluated vs. accuracy between the FastDTW, Abstraction, and Band algorithms.
![Page 27: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/27.jpg)
Questions?
Thanks to those who helped with this research:
Matt Mahoney (Florida Institute of Technology),
Brian Buckley, Walter Schiefele (Interface & Control Systems)
This research is partially supported by NASA
![Page 28: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space](https://reader035.vdocuments.site/reader035/viewer/2022062410/56815848550346895dc59e35/html5/thumbnails/28.jpg)
FastDTW PseudocodeInput: X, Y, radius Output: 1) A minimum distance warp path between X and Y 2) The warped path distance between X and Y
1| // The min size of the coarsest resolution. 2| Integer minTSsize = radius+2 3| 4| IF (|X|≤ minTSsize OR |Y|≤ minTSsize) 5| { 6| // Base Case: for a very small time series run the full DTW algorithm 7| RETURN DTW(X, Y) 8| } 9| ELSE10| {11| // Recursive Case: Project the warp path from a coarser resolution onto the current current resolution. 12| // Run DTW only along the projected path (and also radius cells from the projected path).13| TimeSeries shrunkX = X.reduceByHalf() // Coarsening14| TimeSeries shrunkY = Y.reduceByHalf() // Coarsening15| 16| WarpPath lowResPath = FastDTW(shrunkX, shrunkY, radius)17|18| SearchWindow window = ExpandedResWindow(lowResPath, X, Y, radius) //
Projection19| 20| RETURN DTW(X, Y, window) // Refinement21| }