FV3 becomes new operational GFS model on 6/12/19
18 responses | 0 likes
Started by WxFollower - June 11, 2019, 11:37 a.m.

 The FV3, which has been running as a parallel test run for at least a year, becomes the new operational GFS on 6/12/19. It is debatable whether or not it will actually perform better than the current version. One problem it has had is that it has had an awful cold bias, especially in winter. The late week 2 portion, the least reliable, will take about 20 minutes longer to complete from what I’ve heard.

 Meanwhile, the GEFS will not change yet. That might be good news, especially if the new GFS performs poorly. 

Comments
By metmike - June 11, 2019, 11:40 a.m.
Like Reply

Larry,

That is huge news for me because I didn't know. Thanks so much!

By WxFollower - June 11, 2019, 12:31 p.m.
Like Reply

 You're welcome, Mike. This will be effective with the 12Z run of tomorrow, not 0Z or 6Z. Unfortunately, I've found it to be quite changeable from run to run, possibly more than the current GFS, but we'll see how it does.

By wxgrant - June 11, 2019, 6:41 p.m.
Like Reply

It does seem to flip flop a lot, especially longer out. It did poorly this year during winter with our snow events. I'll be interested in seeing how it handles the upcoming rains starting Saturday here in the mid Mississippi valley. I remember seeing an article about it. The headline read U.S. forecasts will get more accurate starting June 12th. 

By WxFollower - June 12, 2019, 11:48 a.m.
Like Reply

Reminder: the FV3-GFS test model is becoming the operational GFS with the current (12Z) run. Related to that, the last portion is expected to take longer.


 Another reminder: despite the new operational GFS, the GEFS is not changing at this time and supposedly won’t til at least 2020.

By WxFollower - June 12, 2019, 12:45 p.m.
Like Reply

 Surprisingly, the 12Z GFS ran faster through hour 240 though. So, it may end up not taking longer for the full run. I'm confused.

Edit: So, it appears that the 12Z GFS ran faster through hour 240. I have the 240 hour map coming in 12 minutes faster than the old GFS's 240. But since then (after 240), it slowed down a lot vs the old GFS.


By metmike - June 12, 2019, 12:55 p.m.
Like Reply

The last portion is taking longer than before.........thanks Larry.

By WxFollower - June 12, 2019, 1:36 p.m.
Like Reply

Mike and others,

 Check out what I copied and pasted below this from TrueWx. 

 Note the 6 CDD drop for just days 1-5 vs just the 6Z run, which was based on the old GFS. I've been         following this site for a long time and have NEVER seen that much of a DD change that early in the run from one GFS run to       the next. It is obvious to me that there is a strong cold bias that hasn't been resolved. Not good.

      

 Today is the only day that these easy comparisons can be made since we can see the old and new GFS side by side in terms of DDs.

      

 Compared to the 0Z and 18Z (old GFS), the 12Z GFS has a lower daily CDD vs EVERY day. Vs the 6Z, it is lower vs all but 2 days. This has to be due to strong cold bias. Not good. I think they made a mistake to let this go          operational.


CONUS         


GFS2019-06-12 12z06z00z18z
DateHDDHDD 30Y ClimoHDD 30Y DiffCDDCDD 30Y ClimoCDD 30Y DiffHDDCDDHDDCDDHDDCDD
2019-06-132.240.142.106.548.96-2.422.067.392.087.401.957.86
2019-06-141.190.131.066.769.19-2.431.187.141.207.451.057.77
2019-06-150.530.120.419.759.410.340.3910.670.4310.810.4011.45
2019-06-160.800.110.6911.959.642.310.4913.230.5713.460.5813.55
2019-06-171.000.110.8911.229.841.380.3813.730.4213.740.3714.27
Days 1-55.760.615.1546.2247.04-0.824.5052.164.7052.864.3554.90
2019-06-180.700.100.6010.3310.050.280.5113.790.4414.360.4414.65
2019-06-190.590.090.5010.9310.260.670.6014.090.5415.770.5115.35
2019-06-200.530.090.4412.3110.461.850.4715.240.5116.770.4816.08
2019-06-210.320.080.2413.5810.662.920.3916.330.6216.830.5315.00
2019-06-220.200.070.1315.0710.844.230.2914.070.2915.730.4215.27
Days 6-102.340.431.9162.2252.279.952.2673.522.4079.462.3876.35
2019-06-230.280.070.2115.1911.034.160.2514.440.1915.270.2616.73
2019-06-240.570.060.5113.6811.212.470.3415.430.2715.590.1616.50
2019-06-250.320.060.2612.5211.381.140.3116.000.2116.370.1316.05
2019-06-260.200.060.1413.2011.551.650.1116.050.1615.370.1516.03
Days 11-141.370.251.1254.5945.179.421.0161.920.8362.600.7065.31
Days 1-149.471.298.18163.03144.4818.557.77187.607.93194.927.43196.56
2019-06-270.500.050.4513.3511.701.65
By metmike - June 12, 2019, 3:22 p.m.
Like Reply

I noticed that too Larry, great points but do you think that maybe its because the old GFS had a big warm bias?


If you note, the operational  GFS has had way more CDD's than its ensembles and way, way more than the European model and European Ensembles. This has been going on for a very long time.

This last cooler run of this new operational model was STILL  much warmer than its ensembles, even though that ensemble run was much warmer than any others recently.


The operational GFS has been advertising fairly major heat in week 2 based on its CDD's for the last week., while its ensembles have just been a bit above average and the EE has only had average CDD's.

This has been for every run on every day......not just 1 GFS operational outlier run. 


So I guess we call the ensembles, the FV3 ensembles now. 

By metmike - June 12, 2019, 3:28 p.m.
Like Reply

On a related note and something mentioned last week, the total CDD's have been ramping up this week with 3 contributors..

1. Seasonally they go up

2. Near term cool days(low CDD's) will be replaced with warmer days, even if they are near average

3. The end of the forecast is featuring a warming trend

By WxFollower - June 12, 2019, 3:38 p.m.
Like Reply

Mike said: "I noticed that too Larry, great points but do you think that maybe its because the old GFS had a big warm bias?

If you note, the operational  GFS has had way more CDD's than its ensembles and way, way more than the European model and European Ensembles. This has been going on for a very long time."

-----------------------------------------------------------------------------------------------------

Mike,

 Thanks. Excellent observations. I have noticed that. But no I don't think it is that because the old GFS had had a fairly neutral bias overall recently rather than a big warm bias. This has been shown by Radiant/Maxar many times. It probably had a small warm bias when averaged out, but even then it was quite small vs the recent strong cold biases of the GEFS and EPS.

 So, as I see it, the operational GFS has gone from a small warm bias (at most) to a large cold bias.

Reminder: the GEFS are still based on the old GFS although ironically its cold bias likely means it will now be much closer to the operational GFS than it was before 12Z today!

By wxgrant - June 12, 2019, 4:14 p.m.
Like Reply

There was also a major shift south in rain over the next 10 days with a HUGE increase. It seemed to want to crush us with heavy snow all winter, at least parts of my viewing area. I hope it is right with the shift south. That puts most of the water in the Mississippi River South of Cairo, IL, putting most of the water where the Mississippi can hold it. 

By WxFollower - June 12, 2019, 5:08 p.m.
Like Reply

1. CONUS weighted CDDs for 6/13-26 by run per TrueWx:

18Z (old GFS): 197
0Z (old GFS): 195
6Z (old GFS): 188
12Z (FV3): 163

 Now granted, we all know there was normally much run to run volatility from run to run of the old GFS as well as FV3. So, SOME of the relative coolness of the 12Z could easily be related to that, mainly late in the run. In other words, this may have been a relatively cool FV3 run and the next few runs may end up warmer. However, what makes me quite confident that the FV3 cold bias is a big factor is the CDD total for early days of the run, which hardly ever vary significantly from run to run. So, I'll now break it our for only week one (days 1-7):

CONUS weighted CDDs for 6/13-19 by run:
18Z (old GFS): 85
0Z (old GFS): 83
6Z (old GFS): 80
12Z (FV3): 67

 So, for just days 1-7, the 12Z (FV3) had 13-18 fewer CDDs than the prior 3 runs (all old GFS). That's a lot for week 1. No let's see if the next few FV3 runs are similarly "cool" to the 12Z FV3 for days 1-7.
 
---------------------------------------------------------------------------------------------------------------
 
2. As Mike suggested, some of the contrast from old to new GFS may be enhanced by old GFs having a warm bias. But from what Radiant/Maxar has said, I believe that any old GFS warm bias has been small since last winter compared to overall large cold biases for the GEFS, Euro and EPS. I'm trying to get this confirmed. The FV3 cold bias is well-known and has been admitted by NOAA, especially in winter. But interestingly even that cold bias at least for now appears to be much smaller than the very strong GEFS/EPS/Euro cold biases based on today's 12Z CDDs for days 1-5:

6Z GFS (old): 52
12Z GFS (FV3): 46
12Z GEFS: 39
12Z Euro: 38
12Z EPS: 37

 So, let's suppose the true unbiased CDD were near 50. IF that were true, the old GFS was slightly too warm, the FV3 was too cold, and the GEFS/Euro/EPS were much too cold. I kind of think that is what's happening.

By metmike - June 12, 2019, 5:49 p.m.
Like Reply

Larry,

I have never tracked the forecast CDD or HDD with the actual verification with observations/data.

Are you saying that all the models have a cold bias based on this, which would be greatest with the ECMEN,  with the exception of the old operational GFS?


I always assumed that the ECMEN consistently did the best based on the few times that I look at model verification scores. 

Looking just now at the most recent values, it's easily beating  the other models for every time frame, with the GFS and GFSENS doing the worst........Canadian model in between. 


So the coldest model is doing the best and warmest the worst.


What parameters do they use for determining model verification scores? Temperature must be one of them. 

By WxFollower - June 12, 2019, 6:46 p.m.
Like Reply

Mike said: "Larry,

I have never tracked the forecast CDD or HDD with the actual verification with observations/data.

Are you saying that all the models have a cold bias based on this, which would be greatest with the ECMEN,  with the exception of the old operational GFS?

I always assumed that the ECMEN consistently did the best based on the few times that I look at model verification scores. 

Looking just now at the most recent values, it's easily beating  the other models for every time frame, with the GFS and GFSENS doing the worst........Canadian model in between. 

So the coldest model is doing the best and warmest the worst.

What parameters do they use for determining model verification scores? Temperature must be one of them."

------------------------------------------- 

Mike,

1. Yes. And you can add the CDN and its ensemble to the cold bias group. Per Radiant/Maxar, the CDN is usually stronger than the other two ensembles.

2. Maxar tracks the biases/verification and puts out notes often. Regardless of strong cold bias, EPS has done best overall. So, you're correct.

3. Yes, I think surface temperatures are verified. Also things like 500 mb heights and maybe 850 temps are typically looked at.

 Aside/speculation: Mike, my understanding from Maxar is that snowcover is a significant reason for cold bias. I've seen some operational Euro maps and high and especially lows often do go much too cold over snowcover vs what verifies. However, that isn't a factor this time of year and yet there's still a solid cold bias. So, I was wondering if you think that GW is somehow playing a part in this? Could it be that they're somehow not properly taking into account the warmer oceans and Arctic?


By WxFollower - June 13, 2019, 12:08 p.m.
Like Reply

 Below is a TrueWx CONUS table that has the CDDs for the 1st 3 operational FV3 runs (12Z/18Z of 6/12/19 & 0Z of 6/13/19) vs the very last old GFS run (6Z of 6/12/19, which is on the far right). The rounded totals for days 1-14 for the 1st 3 op. FV3 runs were 166, 168, and 163, respectively. Compare those #s to the last old GFS run's much higher CDDs of 188.  

 For just days 1-5, the 1st 3 FV3 runs had CDDs of 47, 46, and 46, respectively. Compare those #s to the last old GFS run's significantly higher 52. This is all consistent with the idea that the FV3 is significantly cooler than the old GFS.

 

GFS2019-06-13 00z18z12z06z
DateHDDHDD 30Y ClimoHDD 30Y DiffCDDCDD 30Y ClimoCDD 30Y DiffHDDCDDHDDCDDHDDCDD
2019-06-132.250.142.116.608.96-2.362.296.582.246.542.067.39
2019-06-141.230.131.106.739.19-2.461.246.611.196.761.187.14
2019-06-150.500.120.389.999.410.580.639.710.539.750.3910.67
2019-06-160.850.110.7411.769.642.121.2911.740.8011.950.4913.23
2019-06-170.720.110.6111.809.841.961.0910.981.0011.220.3813.73
Days 1-55.550.614.9446.8847.04-0.166.5445.625.7646.224.5052.16
2019-06-180.960.100.8610.8910.050.840.6110.200.7010.330.5113.79
2019-06-190.640.090.5510.4910.260.230.5111.550.5910.930.6014.09
2019-06-200.450.090.3612.4210.461.960.4113.610.5312.310.4715.24
2019-06-210.450.080.3714.0310.663.370.3014.310.3213.580.3916.33
2019-06-220.520.070.4513.2610.842.420.3913.850.2015.070.2914.07
Days 6-103.020.432.5961.0952.278.822.2263.522.3462.222.2673.52
2019-06-230.400.070.3312.9011.031.870.3914.090.2815.190.2514.44
2019-06-240.230.060.1714.7511.213.540.4314.510.5713.680.3415.43
2019-06-250.420.060.3614.9811.383.600.5314.920.3212.520.3116.00
2019-06-260.320.060.2615.7511.554.200.3015.590.2013.200.1116.05
Days 11-141.370.251.1258.3845.1713.211.6559.111.3754.591.0161.92
Days 1-149.941.298.65166.35144.4821.8710.41168.259.47163.037.77187.60
2019-06-270.310.050.2617.1111.705.410.1916.390.5013.35
By metmike - June 13, 2019, 2:53 p.m.
Like Reply

Thanks Larry,

I'm not convinced that the FV3 is better with temperatures and the GFS had a warm bias, maybe a huge warm bias.

The now cooler FV3 is still much warmer than the GFS ensembles(if that's what they are) and much much warmer than the superior Euro ensembles.


For instance, using the same source you have, the last total CDD's for this new model was 174.55 compared to the last for the ensembles of 145.14. That's almost 29.41 more CDD's.   

That difference between the old GFS and the ensembles was greater than that on almost all its runs earlier this week and on its warmest runs 50 CDD's greater!


In contrast, the superior Euro Ensemble from the 0z run had 112.20 CDD's which was a whopping 62.35 CDD's less than this last 12z run(the 12z EE is not updated yet) and 32.94 less than the American model ensembles.

What am I missing because it looks clear to me that the old GFS was much too warm and the GFS ensembles..........which are just variations of the same model were and are too warm and the new FV3, which is still warmer than them, is still too warm but has moved in the right direction.

The EE has been blowing away the other models recently in verification scores but I would like to see actual CDD predictions of all the models vs empirical data/observations for the same day to see which models do best with this parameter. 


By WxFollower - June 13, 2019, 5:03 p.m.
Like Reply

Mike said: "I'm not convinced that the FV3 is better with temperatures and the GFS had a warm bias, maybe a huge warm bias.

The now cooler FV3 is still much warmer than the GFS ensembles(if that's what they are) and much much warmer than the superior Euro ensembles.


For instance, using the same source you have, the last total CDD's for this new model was 174.55 compared to the last for the ensembles of 145.14. That's almost 29.41 more CDD's.   

That difference between the old GFS and the ensembles was greater than that on almost all its runs earlier this week and on its warmest runs 50 CDD's greater!


In contrast, the superior Euro Ensemble from the 0z run had 112.20 CDD's which was a whopping 62.35 CDD's less than this last 12z run(the 12z EE is not updated yet) and 32.94 less than the American model ensembles.

What am I missing because it looks clear to me that the old GFS was much too warm and the GFS ensembles..........which are just variations of the same model were and are too warm and the new FV3, which is still warmer than them, is still too warm but has moved in the right direction.

The EE has been blowing away the other models recently in verification scores but I would like to see actual CDD predictions of all the models vs empirical data/observations for the same day to see which models do best with this parameter. "

-----------------------------------------------------------------------------------------------------------------------------

Hey Mike, Per Maxar mets: at least since last winter, ALL major models and ensembles have been way too cold except for the operational GFS, which apparently has had a pretty small warm bias at most. I'm trying to get clarification from Maxar. By the way, the current GEFS is not at all based on the FV3. That won't occur til at least 2020. The physics are based on an older version of the GFS.

 Yes, the FV3, though cooler than the old GFS, is still markedly warmer than the GEFS and EPS. That's not because the FV3 has a warm bias but rather because the GEFS/EPS both have had a very strong cold bias. Maxar has written about this repeatedly.

By metmike - June 13, 2019, 6:18 p.m.
Like Reply

Thanks Larry!

Sorry I caused you to have to repeat the same response on the cold bias that you stated earlier by stating the same thing as if I didn't read you then.

I guess I should just say that I want to actually look at the data myself because the difference in cooling degrees is incredible and it's hard for me to understand how the best model would by far have the biggest cold bias. How is the EE beating up on the other models if it's getting temperatures too cold on every run and is the worst model of all predicting surface temperatures?

In the real world, like natural gas trading for instance this means everything.

Do you know what parameters that are used for model verification scores?

You asked if this might be related to global warming. Intriguing question. I need to do some homework to possibly answer that.

Weather models and climate models use different equations and are very different but still are based on the physics of the same atmosphere. Climate models probably over amplify the affects of increasing H2O as a greenhouse gas, which is likely one reason for them To be too warm.  If weather models were under amplifying this same thing and causing them to be too cool, you would think the adjustments would be an easy fix. 

Actually, I'm dumbfounded that the best model we have could have such a huge cold bias for so long and they haven't fixed it. No doubt, messing with the equations that would fix it will affect other parameters.

I'm not a modeling expert. Grant may have some insight. In my senior year, we had to create a workable basic  weather model using the right math equations so that it predicted close to the temperatures for a period that had already occurred. 

It was  a massive trial and error exercise, tweaking and changing equations until we got the right results.

Those were NOT the best equations, I'm sure of that but they were the ones that gave the right temperatures for that particular day. It makes me think of a broken clock being right twice a day.