fastdtw: toward accurate dynamic time warping in linear time and space department of computer...
TRANSCRIPT
![Page 1: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/1.jpg)
FastDTW: Toward Accurate Dynamic Time Warping in
Linear Time and Space
Department of Computer SciencesFlorida Institute of Technology
Stan Salvador and Philip Chan
![Page 2: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/2.jpg)
Outline
• Dynamic Time Warping (DTW)
• Problem Statement
• Related Work for Speeding up DTW
• FastDTW Algorithm
• Evaluation of FastDTW
• Contributions
• Limitations and Future Work
![Page 3: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/3.jpg)
Dynamic Time Warping (DTW)
• Aligns two time series by warping the time dimension
• Warping - expanding/contracting the time dimension
Time
![Page 4: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/4.jpg)
The Dynamic Time Warping Algorithm
• A dynamic programming approach
• Solutions to slightly smaller problems used to find larger solutions
![Page 5: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/5.jpg)
The DTW Cost Matrix
1
1
|X|
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
![Page 6: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/6.jpg)
Distance of Min-Cost Warp Path
1
1
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
|X|
![Page 7: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/7.jpg)
Finding Min-Cost Warp Path
)]1,1(),1,(),,1(min[ jiDjiDjiD
1
1
|X|
|Y|
i
j
Time Series X
Tim
e S
eri
es
Y
Tim
e
Time Series X
Time
![Page 8: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/8.jpg)
Advantages of DTW
• DTW is optimal
• An intuitive distance measurement
• Local variation in the time axis is common– Handwriting– Speech– “Events” that start after varying delays
![Page 9: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/9.jpg)
Disadvantages of DTW
• O(N2) time and space complexity
• Only practical for small data sets (<3,000)
• Time series are often very long
• Data mining requires a scalable DTW algorithm
![Page 10: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/10.jpg)
Problem Statement
• We desire an efficient Dynamic Time Warping algorithm– Linear time complexity– Linear space complexity– Warp path is needed in addition to warp
distance– Warp path must be nearly optimal
![Page 11: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/11.jpg)
Does DTW Need to be Faster?
“Myth 3: There is a need (and room) for improvements in the speed of DTW for data
mining applications.”
(Keogh today-9:45am)
• Keogh: many time series
• FastDTW: Long time series
![Page 12: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/12.jpg)
Existing Methods to Speed Up DTW
1. Constraints – only fill in part of the cost matrix
2. Abstraction – sample the data before time warping
![Page 13: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/13.jpg)
Constraints
• Still O(N2) if the window width is a function of input size (linear if the width is constant)
• Assumes a near-optimal warp path stays near the i=j axis
• Accuracy depends on the domain
Sakoe-Chiba Band (Sakoe & Chiba 1978)
Itakura Parallelogram (Itakura 1975)
![Page 14: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/14.jpg)
Abstraction1/51/1 1/1
• O(N) if N pts are sampled down to ≤
• Assumptions– Sampling preserves time series structure– Small deviations from the optimal path cause
little increase in warp-path distance
(Keogh & Pazzani 2000), (Chu et al. 2002)
N
![Page 15: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/15.jpg)
Our FastDTW Algorithm
• A multi-resolution approach inspired by a multi-level graph bisection algorithm (Karypis 1997)
• 3 key operations1. Coarsening – reduce
the resolution of a time series
2. Projection – use a low-res warp path as an initial solution at a higher resolution
3. Refinement – Refine a projected warp path locally adjusting the path
![Page 16: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/16.jpg)
Sample Run of FastDTW
1/8 1/4 1/2 1/1
![Page 17: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/17.jpg)
FastDTW Algorithm
1. Set the resolution to be the coarsest2. Find the initial path using regular DTW3. Repeat
a. Double the resolutionb. Project the path onto the finer resolutionc. Find a path through the projected area (plus a
small radius around the projected area)
4. Until the original resolution is reached
![Page 18: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/18.jpg)
Complexity
• O(N) time
• O(N) space
• Details in the paper
![Page 19: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/19.jpg)
Evaluation Criteria
• AccuracyThe error of an approximate Time Warping algorithm:
% error = where:
approxDist – the warp path distance of the approximate algorithm
optimalDist – the warp path distance of the DTW algorithm
• EfficiencyRuntime (measured in seconds)
100
toptimalDis
toptimalDisapproxDist
![Page 20: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/20.jpg)
Evaluation Procedure (Accuracy)
• Data Sets – UCR Time Series Data Mining Archive (Keogh & Folias 2002), 3 groups used:
1. Random – 45 unrelated time series (earthquakes, random walk, eeg, speech, etc.)
2. Trace – 200 time series simulating nuclear power plant failure (4 classes)
3. Gun – 200 time series of a gun being drawn and pointed (2 classes)
• Procedure1. Run FastDTW, Constraints (Sakoe-Chiba Band), and Data
Abstraction on all pairs within a data set group, also vary the radius
2. Record the average error of all three methods for a group of data and a radius
![Page 21: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/21.jpg)
Radius0 1 10 20 30
FastDTW 19.2% 8.6% 1.5% 0.8% 0.6%
Abstraction 983.3% 547.9% 6.5% 2.8% 1.8%
Band 2749.2% 2385.7% 794.1% 136.8% 9.3%
Average % Error (Accuracy)
![Page 22: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/22.jpg)
Error in Different Data Sets
Accuracy of FastDTW, Bands, and Data Abstraction
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 5 10 15 20 25 30radius
Err
or
FastDTW-Random FastDTW-Trace FastDTW - GunAbstraction-Random Abstraction-Trace Abstraction-GunBand-Random Band-Trace Band-Gun
![Page 23: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/23.jpg)
Evaluation Procedure (Execution-time)
• Data Sets– Synthetic sine waves with Gaussian noise– 10 to 180,000 data points
• Procedure1. Run FastDTW and DTW on each data set,
vary the radius for FastDTW
2. Compare the Execution times
![Page 24: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/24.jpg)
Execution TimeExecution Time of FastDTW on Large Time Series
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 160,000 180,000
Length of Time Series
Tim
e (s
eco
nd
s)
DTW
FastDTW (radius=100)
FastDTW (radius=20)
FastDTW (radius=0)
![Page 25: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/25.jpg)
Summary of Contributions
• FastDTW – an approximation of DTW– O(N) time and space complexity– Scales well to long time series– Accurate, 8.6% error if radius=1, 0.8% error if
radius=20
![Page 26: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/26.jpg)
Limitations and Future Work
• Limitations– FastDTW does not always find an optimal solution
• Future Work– Examine using different step sizes between resolutions
– Investigate search algorithms to help improve refinement
– Examine # of cells evaluated vs. accuracy between the FastDTW, Abstraction, and Band algorithms.
![Page 27: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/27.jpg)
Questions?
Thanks to those who helped with this research:
Matt Mahoney (Florida Institute of Technology),
Brian Buckley, Walter Schiefele (Interface & Control Systems)
This research is partially supported by NASA
![Page 28: FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space Department of Computer Sciences Florida Institute of Technology Stan Salvador and](https://reader034.vdocuments.site/reader034/viewer/2022050819/56649e195503460f94b06c11/html5/thumbnails/28.jpg)
FastDTW PseudocodeInput: X, Y, radius Output: 1) A minimum distance warp path between X and Y 2) The warped path distance between X and Y
1| // The min size of the coarsest resolution. 2| Integer minTSsize = radius+2 3| 4| IF (|X|≤ minTSsize OR |Y|≤ minTSsize) 5| { 6| // Base Case: for a very small time series run the full DTW algorithm 7| RETURN DTW(X, Y) 8| } 9| ELSE10| {11| // Recursive Case: Project the warp path from a coarser resolution onto the current current resolution. 12| // Run DTW only along the projected path (and also radius cells from the projected path).13| TimeSeries shrunkX = X.reduceByHalf() // Coarsening14| TimeSeries shrunkY = Y.reduceByHalf() // Coarsening15| 16| WarpPath lowResPath = FastDTW(shrunkX, shrunkY, radius)17|18| SearchWindow window = ExpandedResWindow(lowResPath, X, Y, radius) //
Projection19| 20| RETURN DTW(X, Y, window) // Refinement21| }