New research finds reduced observations from commercial aircraft impact regional weather model’s accuracy

National Weather Service says accuracy of official forecasts has not been affected

Aircraft-based observations are one of the most important meteorological observation sources within the modern global observing system. Flight reductions related to the COVID-19 pandemic raised questions about whether the loss of meteorological observations from aircraft-mounted sensors could affect the accuracy of operational forecast models.

A new study, published in the American Meteorological Society’s Journal of Applied Meteorology and Climatology, has quantified the impact of these missing observations on the forecasts produced by NOAA’s most advanced experimental short-term weather model, the Rapid Refresh (RAP) model. The study was led by the University of Colorado’s Cooperative Institute for Research in Environmental Studies (CIRES) and the NOAA Global Systems Laboratory (GSL).

The controlled experiment used weather observations from surface stations, satellites, radar, wind profilers, and commercial aircraft from discrete 10-day periods in 2018 and 2019. It found reduced skill in several outputs from the most recent version of the RAP - RAPv5 - when the model was denied observations that would be expected from normal air travel patterns. .

NOAA Research continuously seeks to improve weather forecasts by developing new and upgraded models that are rigorously tested to evaluate their performance. Once evaluation criteria are met, upgrades are transitioned to the NWS for use in generating official forecast products. This new version of the RAP is due to transition to the NWS operational suite of models before the end of the year.

This new study had two goals: to summarize the current spatial and temporal distribution of aircraft-based observations globally, with a particular focus over North America, and to quantify the impact of the partial loss of aircraft-based observations to regional model skill during the pandemic.

The study did not evaluate whether degraded model performance translated to reduced accuracy of any National Weather Service (NWS) operational forecast products. “At the National Weather Service, we are not seeing a readily apparent reduction in forecast accuracy resulting from the reduction of aviation observations,” said Curtis Marshall, Ph.D., Program Manager for NWS’ Aircraft-based observations. NWS meteorologists access observations from an extensive surface weather station network, from radar, satellites, weather balloons, and ocean buoys, and incorporate guidance from numerous weather prediction models into their forecasts.

The data denial experiments conducted by GSL researchers with the latest version of the RAP system for recent winter and summer periods first quantified the statistically significant positive forecast impacts of assimilating aircraft observations. A special additional experiment excluding approximately 80 percent of aircraft observations revealed a reduction in forecast skill for both summer and winter amounting to 30-60 percent of the degradation seen when all aircraft observations are excluded. By comparison, during the height of air travel disruption during the pandemic, approximately 75 percent of normal aircraft observations were unavailable. These results represent an approximate quantification of the impact of COVID-19 related commercial flight reductions on numerical weather prediction, demonstrating that regional guidance is likely to show degradation due to the decreased number of aircraft observations.

“This study is the first to look directly at the impact of a partial outage in aircraft observations, rather than trying to extrapolate results from a total data denial experiment,” said GSL/CIRES lead-author Eric James. “This enabled us to get an accurate estimate of the impact of the flight reductions associated with COVID-19, which continue at approximately a 50 percent reduction compared to pre-COVID levels.”

“The only way to carry out a controlled experiment that quantifies the impact of a partial loss of aircraft observations is to examine periods of time where all aircraft observations are available,” said James. “Without a full set of observations to work with during the pandemic, it’s impossible to know how much better model forecasts would have been with the full (normal) amount of aircraft observations.”

Co-author Stan Benjamin, a GSL senior scientist, said that the research project was underway prior to the pandemic’s sudden disruption of passenger aviation.

“We were already working on our new overall aircraft-data coverage and impact study when COVID-19 started to ‘change the world’, so we were able to quickly pivot and design these additional experiments with a ‘partial decrease’ scenario,” said Stan Benjamin, GSL Senior Scientist and co-author of the study. “It’s important for NOAA to be able to rapidly respond, as in this study, to assess the effect of shifts in our observational and forecast capabilities.”

Aircraft observations continue to be an important contributor to forecasts for severe weather, renewable energy generation and transmission, and transportation safety. GSL and CIRES researchers say it is important to identify gaps in aircraft observation coverage and develop strategies to address gaps in a way that provides the most benefit for forecast models. They also noted there are significant opportunities to expand aircraft-observation coverage in the coming decade by taking advantage of data from a new generation of aircraft transponders, from uncrewed aerial vehicles and other new airborne-based-observation technologies.