an algorithm developer’s tool
DESCRIPTION
An algorithm developer’s tool. [email protected] National Severe Storms Laboratory Norman OK, USA http://w ww.wdssii.org/. A developer’s tool. The Warning Decision Support System – Integration Information (WDSS-II) - PowerPoint PPT PresentationTRANSCRIPT
An algorithm developer’s tool
[email protected] Severe Storms LaboratoryNorman OK, USAhttp://www.wdssii.org/
April 22, 2023 [email protected] 2
A developer’s tool
The Warning Decision Support System – Integration Information (WDSS-II) A collection of meteorological algorithms for severe
weather analysis, diagnosis and prediction Hail, tornadoes, wind, lightning
An integrated set of loosely coupled tools for: Severe weather diagnosis Image processing Statistical validation Ground-truth verification
April 22, 2023 [email protected] 3
WDSS-II applications (algorithms and tools) are just executables. launched on the command line. In deployed systems through scripts.
Can easily change input to filtered form, or accumulate a different product (such as rainfall rate or hail size)
Applications exist for many tasks: Image processing (smoothing, dilating, eroding, etc.) Objective analysis (Cressman, Barnes, Gaussian, etc.) Scoring grids (error statistics) Statistical skill based on ground truth
WDSS-II Applications
w2accumulator –i xmllb:/data/realtime/radar/KTLX/code_index.lb \
-I MaxShear_0-3km \
-o /data/realtime/radar/KTLX/ -r -t “30 60 120”
April 22, 2023 [email protected] 4
Creating a new algorithm
An algorithm is essentially a data filter Takes some data as input Produces new data as output
The algorithm developer should be able to specify the scientific processing in the middle Without having to worry about data ingest, data
formats, notification, etc. But provide a library of common computations on
the typical data used.
April 22, 2023 [email protected] 5
w2algcreator
W2algcreator is a WDSS-II tool To write the format-independent code for ingesting
data into your application. The algorithm developer writes an XML file
specifying the inputs and adaptable parameters. The algorithm itself is auto-generated!
With a “fill in the blank” for the scientific computation
WDSS-II class libraries can be used for common computations.
Easy to add new input and output formats.
April 22, 2023 [email protected] 6
Display
The WDSS-II displays are highly configurable to aid trouble-shooting.Display of intermediate outputs is easy and
convenient.
April 22, 2023 [email protected] 7
Example intermediate product
Created with no modification of the display. Just configuration files.
Algorithm developer marks the radar associated with each detection. For easy debugging.
April 22, 2023 [email protected] 8
The end-result
So, what kinds of algorithms have been developed in WDSS-II?
April 22, 2023 [email protected] 9
Single-radar/Multi-sensor algorithms
Some single-radar (multi-sensor) algorithms in WDSS-II
April 22, 2023 [email protected] 10
Multi-radar/multi-sensor algorithms
A typical multi-radar deployment of WDSS-II
April 22, 2023 [email protected] 11
Relevance to Q2
Which of those are relevant to Q2?Some existing severe weather algorithms
may be relevantProbability of hail for identifying radar echoes
with potential hail contamination.More likely:
Developing new algorithmsBuilding algorithms as data filtersExisting lower-level tools.
April 22, 2023 [email protected] 12
Accumulation algorithm
Six Hour Path of Rotational Shear Accumulation of shear to form rotation tracks.
Accumulation could be as: Maximum Rate Total
This tool could be used for rainfall depth from rainrate for example.
April 22, 2023 [email protected] 13
Motion Estimation
Uses K-Means clustering and Kalman filters
Forecast dBZActual dBZ
30 min30 min
April 22, 2023 [email protected] 14
Need for new approach
Traditional centroid tracking Accurate at small scales, but not at large scales Inaccurate when storms merge or split Possible to extract trends from the information
Flow-based tracking Cross-correlation, Lagrangian methods, etc. Are accurate at large scales, but not at small scales Not useful in decision support because trends of
storm properties can not be extracted
April 22, 2023 [email protected] 15
K-Means clustering
K-Means clustering is a hybrid approachCluster the input data to find clusters
Like centroid-based tracking methodsBut at different scales.
Track the clusters using flow-based methods (minimization of cost-functions)
Like flow-based methodsDoes not involve cluster matching (e.g: Titan)
April 22, 2023 [email protected] 16
Example clusters
Two different scales shown
Both scales are tracked
April 22, 2023 [email protected] 17
Extrapolation
Smooth the motion estimates spatially using OBAN
techniques (Gaussian kernel)
temporally using a Kalman filter (assuming constant velocity)
Repeat at different scales and choose scale appropriate to extrapolation time period.
April 22, 2023 [email protected] 18
Trends
The clusters can be used to extract trends of any gridded field. Configurable to extract minimum, maximum, count,
sum, time-delta, etc. of gridded fields within cluster Even fuzzy combination of multiple fields
Extremely useful for research! Statistical properties of storms Changing drop-size distributions with time Which clusters are convective? Trends in rain-rates … Trends in cloud-top temperatures …
April 22, 2023 [email protected] 19
Polygon statistics
Using cluster trends is useful for deriving storm properties.What about extracting statistics around a
fixed location?Maybe around rain gages?
WDSS-II has a tool to provide polygon statistics from any gridded field(s)The polygons can change with time (e.g:
weather service watch areas)
April 22, 2023 [email protected] 20
Quality Control Neural Network (QCNN)
Developed for MDA false alarms in non-storm echo. With QCNN, shows over 90%
reduction in the non-storm MDA false alarms and zero change to detections within storm echo.
The same QC technique would be useful in estimating precipitation as well.
Based on local statistics of reflectivity, velocity and spectrum width fields, vertical statistics and morphological image processing
Handles AP/GC, radar artifacts and some biological signals.
Neural network for optimal combination of inputs.
After QC
Before QC
April 22, 2023 [email protected] 21
Current uses of WDSS-II in the NWS
WDSS-II is a leading edge system Provides capabilities not yet in the “official” National Weather Service
systems. The Storm Prediction Center
defines daily threat areas launch a WDSS-II domain
automatically configures the data ingest and starts the algorithms. NWS forecast offices
WDSS-II products are converted into AWIPS format and piped the AWIPS displays in several NWS forecast offices.
But the AWIPS display is too restrictive. Therefore … The 4D WDSS-II display is to be implemented as a separate app on
AWIPS but controlled from within D2D. Concept of algorithm development capabilities
Being considered for next redesign of AWIPS
April 22, 2023 [email protected] 22
In summary
How can WDSS-II be useful in Q2 As an algorithm development toolkit
Multi-sensor inputs in real-time and for archived cases But limited to user workstations JADE will provide web-based capabilities.
Individual tools Objective Analysis tools and other low-level tools. Image processing filters Quality control of radar data Motion estimation and extrapolation (short-term QPF) Storm statistics Polygon statistics
Please visit this website: http://www.wdssii.org/