Blizzard of Controversy: Why communicating forecast uncertainty matters
It seems the biggest blizzard to hit New York City, New Jersey and Philly this week is the storm of controversy after what can only be described as a busted forecast on the western edge of the "Blizzard of 2015."
Note: This post adds breaking updates from original post at 9:24 am Tuesday.
Yes, the forecasts for a raging blizzard complete with feet of snow and storm surge driven coastal flooding for big chunk of the east coast including eastern Long Island, Connecticut and eastern Massachusetts were spot on.
In fact there are many forecast success stories with this storm. At first glance the excellent performance of NOAA's upgraded GFS model, which accurately painted a more eastern storm track lower snowfall totals for NYC and points west.
NOAA's upgraded GFS seems to have actually outperformed the European Model in this storm.
Weather forecasting is an inexact science at times. But it also has enough skill to provide tremendous value for weather consumers, business and the US economy.
When forecast snowfall totals of 2 to 3 feet prompt the shut down of the nation's biggest city, then fall short by half or more, questions are (rightfully) being asked of us as professional meteorologists. The answers may have more to do with communicating inherent forecast uncertainty than with the actual forecast itself.
One weather professional I observed on national TV last night did an excellent job of explaining the uncertainty in the forecast for the NYC area was The Weather Channel's Carl Parker.
But not everybody accomplished that task deftly.
Capital Weather Gang's Jason Samenow has some interesting perspective in his posts today from NWS director Louis Ucellini.
National Weather Service director Louis Uccellini, in a press call this afternoon, said his agency did not do enough to communicate the uncertainty in the forecast for New York City and Philadelphia, where snowfall predictions were far too high.
“It is incumbent on us to communicate forecast uncertainty,” Uccellini said. “We need to make the uncertainties clear.”
He added: “We’re going to review this [issue of communicating uncertainty in weather forecasts] very carefully and assess a different approach as we deal with these types of storms.”
However, Uccellini stressed erring on the side of caution and planning for the worst was “the right decision” given the potential for “extraordinary” snow totals.
AP's science writer Seth Borenstein chimes in.
But snowfall in the self-absorbed media capital of New York City, shut down in advance, was under a foot. New Jersey and Philadelphia also were spared.
In a conference call with reporters Tuesday, a defensive Uccellini, who wrote textbooks on winter storms, wouldn't say his agency's forecast was off. Instead, he blamed the way meteorologists communicated and said the weather service needs to do a better job addressing uncertainty.
Private meteorologist Ryan Maue of Weather Bell Analytics slammed the public agency for ratcheting up forecast storm amounts before the system arrived, instead of telling people how uncertain it was.
"The public should be upset that the forecast was blown for NYC and ask for answers," he said in an email.
Uccellini said the agency would review those procedures and consult with social scientists to improve messaging.
As long time weather professionals many of us take great care in trying not to over-hype storms. We are essentially the protectors of credibility for our profession.
Another evolving trend in this era of "social media-rology" is everyone with a laptop, smart device and twitter account thinks they are a meteorologist. It's way to easy too post the latest model output, without having the expertise to know why it may be wrong. The "armchair meteorologist" trend blurs the lines of clear meteorological communications in an era of big weather data. Professionals who hype for profit make it even worse.
The ability of weather consumers to discern between cool professional weather analysis and calculation, and ratings-driven weather hype and weather promotion for the sake of corporate weather profit is becoming increasingly important.
The good news? The laws of choice and competition still apply. Audiences are usually smart enough to pick the most reliable and independent weather providers and place higher degrees of trust in that professionals.
Then again, I may be a naive optimist.
In the era of increasingly extreme weather events, I hope not.
What do you think? Are there any forecasters who you trust and prefer their style of communication? Do you think some forecasters are biased (too low or too high) to certain snowfall outcomes in snowfall events? Is forecast uncertainty properly communicated? How can we serve the public who rely on our forecasts better?
-Original post below-
All weather is local.
That's the tried and true weather wisdom coming out of the big northeast cities today as we add up inches, clock wind gusts and tally storm reports.
Your local weatherperson? In Philly, New Jersey and New York City he's a weather terrorist. In Boston and Nantucket, a local hero.
Forecasts of blizzard conditions, two feet of snow and hurricane force gusts with coastal storm surge flooding verified accurately in eastern Massachusetts.
Severe coastal flooding in reported in eastern Massachusetts.
Hurricane force gusts have raked the Massachusetts coast.
Wide ranging forecasts and eventual snowfall totals are making for some interesting clean up out east today. Some are cleaning up streets. Some are shoveling out from forecasts gone bust.
Gary Szatkowski, is the meteorologist in charge of the National Weather Service's Mt. Holly, New Jersey forecast office. New Jersey, Philly and New York City rode the always dicey western edge of the storm. Give the guy credit, he had the intestinal fortitude to own up to a busted forecast and will probably gain credibility because of that.
Several other meteorologists still have various amounts of explaining to do as predicted snowfall totals didn't come anywhere close to being accurate, and major metro areas were essentially shut down based on inaccurate forecasts of a historic snowfall event.
Let's be honest; I've said many times before forecasting snowfall is one of the toughest forecast a meteorologist has to make. Just like every other forecaster, I've had my share of busts on snowfall so not being judgmental here. That said you're going to hear a lot more in the coming days about the "busted Blizzard of 2015" for areas like Philly and New York City.
Here's a sampling of some of the forecasts and explanations starting to come out of this storm.
From a forecast standpoint, it's been a failure, or better yet, a bust.
Whenever you're forecasting six inches or more of snow in the big city and it fails to happen, you disrupted livelihood and ticked off mom, dad, schools, businesses. To be honest, it was one of the toughest forecasting calls to make in a long time and I never had a high confidence level with this one.
No, this is not even close to worse than Snowmageddon or a top 5 historic snowfall for New York City. Yes, our beloved European Model did let us down a bit on this storm.
The lesson here is a good reminder to all meteorologists about communicating uncertainty in science and weather forecasting. This storm is a perfect example of how some media meteorologists love to over-hype big storms and issue certain sounding forecast based on their latest model runs. It's tempting weather candy for ratings and publicity-driven media forecasters.
The downside? When we're wrong, we damage the credibility of our profession.
Weather forecasting is hard. Forecasting anything in the future is presents difficulty and surprises. Think you can tell us what where the Dow will be in April? What will your neighbor be doing at 7:15 pm tomorrow night?
Our audience is smart enough to know that weather forecasting is an imperfect science. That's why as professionals we need to make great efforts to communicate in a way that presents possibilities, and likelihoods without sounding unequivocally certain when the state of the science is not.
Communicating uncertainty in weather and science is a strength, not a weakness.