Download - Model Flip-Flops and Forecast Opportunities

National Weather Service

Model Flip-Flops and Model Flip-Flops and Forecast OpportunitiesForecast Opportunities

Bernard N. MeisnerScientific Services DivisionNWS Southern RegionFort Worth, Texas


The views expressed herein are those of the author and do not necessarily reflect the position of the National Weather Service.


Model Flip FlopsModel Flip Flops

Primary Motivation: Primary Motivation:

• Introduction of 1200 UTC MEXMOS guidance caused Introduction of 1200 UTC MEXMOS guidance caused concerns among some forecasters. concerns among some forecasters.

• Perception was that guidance from 1200 UTC model Perception was that guidance from 1200 UTC model runs would be substantially different from theruns would be substantially different from the0000 UTC guidance.0000 UTC guidance.(Flips and Flops)(Flips and Flops)

Secondary Motivation: Secondary Motivation:

• Given substantial improvements in MOS guidance, are Given substantial improvements in MOS guidance, are there still opportunities for forecasters to add value, there still opportunities for forecasters to add value, particularly beyond Days 1 and 2?particularly beyond Days 1 and 2?(Forecast Opportunties)(Forecast Opportunties)


Model Flips and FlopsModel Flips and FlopsHow to define them?How to define them?

Current model runlies outside envelopeof previous ensemble.

Run-to-run changein MOS max/mintemperatures >10oF.

Run-to-run change in MOSmax/min temperatures exceedsmonthly Mean Absolute Error.


VerificationVerification

108 Hr108 HrForecastForecast




Model Flip FlopsModel Flip Flops

Data Used:Data Used:MEXMOS Guidance for DFW and PIT Airports.MEXMOS Guidance for DFW and PIT Airports.(Have also looked at a few other sites in NWS Southern Region.)(Have also looked at a few other sites in NWS Southern Region.)

Flip (and Flop) Criterion:Flip (and Flop) Criterion:

Run-to-run change in Max/Min temperature guidance Run-to-run change in Max/Min temperature guidance greater than 10greater than 10ooF.F.


Flip-FlopSpreadsheet

Model“Flips”

Run-to-run changeexceeds this criterion

All these flipswere in the

proper direction


Flip-FlopSpreadsheet

Model“Flip/Flop”

Run to run changeexceeds this criterionfor two consecutive

model runs

This flop was inthe wrong direction.


No discernable differencebetween 0000 and 1200 runs

00 0000 00

00

0000

00

12 12 12

1212

12

12


October 2005 November 2005

December 2005

Mean Absolute Guidance Errors - KDFWMean Absolute Guidance Errors - KDFW

January 2006


DFW Max/Min TemperatureDFW Max/Min Temperature1010ooF ThresholdF Threshold

MonthMonth FlipsFlips

Number Number CorrectCorrect

Flip – Flip – FlopsFlops

Number Number CorrectCorrect

Oct 2005Oct 2005 33 33 -- --

NovNov 77 33 -- --

DecDec 99 44 77 44

Jan 2006Jan 2006 88 55 -- --

FebFeb 55 55 11 --

MarMar 1010 55 33 22

AprApr 22 00 -- --

TOTALTOTAL 4242 25 (60%)25 (60%) 1111 6 (55%)6 (55%)


Model Flips…and FlopsModel Flips…and Flops

Summary:Summary:

• No substantial difference between 0000 and 1200 No substantial difference between 0000 and 1200 UTC model guidance.UTC model guidance.

• Perception of systematic run-to-run changes is not valid.Perception of systematic run-to-run changes is not valid.

• Flips (and flops) occur during the cool season.Flips (and flops) occur during the cool season.

• Typically occur for Typically occur for just one verification timejust one verification time..

• Model flips are rare; flip-flops are very rare.Model flips are rare; flip-flops are very rare.

• 53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts)53 Flips; 11 Flops (out of 400+ model runs/7000+ forecasts)

• Flips are most common for Days 4-6. Flips are most common for Days 4-6.

• Flip-flops are most common for Days 5-6.Flip-flops are most common for Days 5-6.


Forecast OpportunitiesForecast Opportunities




Data Used:Data Used:MEXMOS Guidance for DFW and PIT Airports.MEXMOS Guidance for DFW and PIT Airports.(Have also looked at a few other sites in NWS Southern Region.)(Have also looked at a few other sites in NWS Southern Region.)

Forecast Opportunity Criterion:Forecast Opportunity Criterion:Max/Min temperature guidance error greater than 10Max/Min temperature guidance error greater than 10ooF.F.

Caveat:Caveat:Smaller errors at certain thresholds can be significant!Smaller errors at certain thresholds can be significant!

2828ooF vs 33F vs 33ooF; 99F; 99ooF vs 104F vs 104ooFF


ForecastOpportunitiesSpreadsheet

Observed temperaturecells turn pink whenthey exceed climate

normal by this amount

Performance of National Weather Service Forecasts Compared to Operational, Consensus, and Weighted Model Output Statistics by Jeffrey A. Baars and Clifford F. Mass



These cells turn brownwhen day-to-day change

in observed max/minexceeds this amount



These cells turn bluewhen guidance errorexceeds this amount



Forecast Opportunitiesoccur for calendar days

with large departuresfrom climatology.


Percent of Forecasts:14%

October 20058%

November 20059%

December 200514%

Forecast Opportunities - KDFWForecast Opportunities - KDFW

Threshold 10oF

January 200613%

October 200532%

November 200534%

December 200543%

Threshold 5oF

January 200635%


October 20058%

November 20059%

December 200514%

Threshold 10oF

January 200613%


February 200614%

March 200616%

April 20066%

Threshold 10oF

May 20064%



June 20060%

July 20060%

Threshold 10oF

September 20064%

August 20061%




Forecast Opportunities:Forecast Opportunities:• Typically occur for Typically occur for calendar dayscalendar days and can be and can be

consistent from one run to the next.consistent from one run to the next.

• Are most common for Days 3-7.Are most common for Days 3-7.

• Frequently occur for days on which the observed Frequently occur for days on which the observed max/min temperature departs substantially from the max/min temperature departs substantially from the climatological norm.climatological norm.• MEXMOS guidance typically underestimates the observed MEXMOS guidance typically underestimates the observed

departure from normal.departure from normal.

• Rarely occur for days when the temperature change Rarely occur for days when the temperature change from the previous day is large.from the previous day is large.• GFS seems to handle these events well.GFS seems to handle these events well.


Future WorkFuture Work

Flips and Flops:Flips and Flops:• Can one formulate an objective definition of a model Can one formulate an objective definition of a model

flip (and flop)?flip (and flop)?

• Is it possible to anticipate model flips?Is it possible to anticipate model flips?

• When a flip occurs, can we determine whetherWhen a flip occurs, can we determine whetherthe flip is in the correct direction?the flip is in the correct direction?

Forecast Opportunities:Forecast Opportunities:• How can we identify Forecast Opportunities?How can we identify Forecast Opportunities?


Questions?Questions?

mailto: [email protected]


Download - Model Flip-Flops and Forecast Opportunities

Top Related