The data’s in the detail

The word “painstaking” can be overused in science, but it is difficult to choose a more appropriate one for the detailed and diligent work that goes into preparing climate data for use. Jörg Schulz oversees EUMETSAT’s role in this work as Climate Service Product Manager, and he explains some of the problems that are solved every day to help science get a better understanding of climate.

“The basic technology for monitoring weather from space was established 50 years ago, and was very successful in measuring what it set out to measure. But at the time nobody was thinking about using it for climate monitoring.

“We are trying to stretch the satellite data record back as far as we can – back to the 1970s – but over time we are dealing with satellite instruments having similar names, that are actually measuring in slightly different channels which has a large effect on the usability for climate monitoring if not corrected for. We’re also dealing with the effects of the space environment on instruments – the cosmic radiation does a lot of things you can’t plan for,” he says.

Schulz and his team have the goal of creating a long time series of reliable data, but there are no shortcuts. They always question what they are looking at and try to find a second source to validate the measurements.

image of the IASI instrument

IASI before launch

“As an example, we can use the IASI instrument as a reference for the infrared channels on Meteosat. If you see deterioration in a signal, then you can correct for that.

“To build up a reliable time series of Meteosat, we check all possible combinations with other instruments in space measuring at the same wavelength, time and location to make sure to find the best reference for the Meteosat data. The further back in time the more difficult it is because no instrument provides a real reference. It’s like making a path through this forest of instruments, and you have to check at the end that all the different routes reach the same place. “

Like mathematics pupils in school, climate data scientists must always ‘show their work’ so any errors introduced in the interpretation of data can be identified by others using the data. . The differences can be extreme enough for teams to reach opposing conclusions using basically the same data – such as a cooling trend being observed instead of a warming trend in one part of the atmosphere.

“Groups around the world should analyse the same data,” says Schulz, “That is very important because often different but scientifically equally defendable approaches exist that lead to small but relevant differences concerning the analysis of long term change. The application of different approaches may be easy but it is tricky when you have a lot of data and complex instruments."

Keeping a record of exactly how data has been managed and what corrections have been applied means lots of paperwork, but it’s a vital part of the process.

“What we do should be independent of the individuals doing it, so we need a record of exactly what has been done. Our User Guides for products explain how the product has been made, and what its expected limitations are. We also have documents available to show the individual algorithms explaining the maths and physics of what we have done, especially with raw data. These documents might not be the most fun to write and read, but they are absolutely vital to document the process.”

This work facilitating science did not always get recognised, and it was difficult for those involved to publish and gain citations as is the norm for the rest of the scientific community.

“People who do what we do didn’t always get the credit,” says Schulz. “To be published you needed to be addressing a science question – that’s a big hurdle to go off to answer a question, but that is not our task at EUMETSAT.

“But now there are specific journals for data records so when we release a record we can produce a peer-reviewed paper that becomes a resource for others.”

Understanding global processes is a huge challenge, but the data can also be used to solve smaller practical problems in day-to-day life.

“One of our customers is our own SAF network, particularly the CM SAF. They use these data to make products like the solar irradiation map for power plant location planning. If you want to know where to put a power plant, you need to have a nice long-term calibrated dataset and we try to provide this kind of calibration,” he says.

EUMETSAT data records are also successfully used in global reanalysis of the atmosphere based on weather prediction models. Schulz says: “There are a handful centres in the world that use our data for global reanalysis but a reanalysis as produced by ECMWF has more than 10,000 users. This is multiplying the impact of our work on climate science.”

As for the future, Schulz says more data with high quality will be wanted – and quicker.

“There should be a quicker turnover of products for climate services, and we have to do more with higher timeliness maintaining the quality.”

He also believes continued and improved co-operation will be vital.

“We need combinations of data and models to understand the global climate system, and mechanisms to share data that help us understand all of the processes in play. That is how, in the end, all of this work can have a real effect.”