A New Era of Data Compatibility in Astronomy 

17 February, 2025 / Read time: 3 minutes

Groundbreaking data model paves the way for more efficient collaboration among U.S. and international astronomy institutions 

Our partner on the North American side, the U.S. National Science Foundation's (NSF) National Radio Astronomy Observatory (NRAO), is collaborating with U.S. and international astronomical institutions to establish a new standardized, open-source format for processing radio astronomical data, enabling interoperability among scientific institutions around the world.   

When telescopes observe the Universe, they collect large amounts of data over hours, months, and even years. Combining data from different telescopes is especially useful for astronomers, whether to view different parts of the sky, to observe the stars and astronomical objects they are studying in more detail, or at various wavelengths. Each instrument has its strengths, depending on its location and capabilities. ALMA, for example, has unique atmospheric conditions and state-of-the-art technology that allow it to observe radio waves at millimeter and submillimeter wavelengths better than any other telescope on Earth. 

"By setting this international standard, NRAO is taking a leadership role in ensuring that our global partners can efficiently utilize and share astronomical data," said Jan-Willem Steeb, the technical lead of the new data processing program at the NSF NRAO. "This foundational work is crucial as we prepare for the immense data volumes anticipated from projects like the Wideband Sensitivity Upgrade (WSU) to ALMA and the Square Kilometer Array Observatory in Australia and South Africa."  

To ensure compatibility and functionality for a wide range of instruments, the team used datasets from approximately 10 observatories worldwide, with ALMA having the most considerable impact on the repository.   

"This new scheme is critical for ALMA to process the WSU data, which increases the total number of channels ALMA can provide from 15,000 to 1.2 million. ALMA data were used extensively to develop and test the new scheme," said Amanda Kepley, project director of RADPS-ALMA.  

The new data model lays the groundwork for efficient data sharing and processing across various current and future radio telescope platforms by focusing on these key aspects.   

"The new model is designed to address the limitations of aging models, in use for more than 30 years, and created when computing capabilities were vastly different," adds Jeff Kern, who leads software development for NSF-NRAO. "The new model updates the data architecture to align with current and future computing needs and is built to handle the massive data volumes expected from next-generation instruments. It will be scalable, which ensures the model can cope with the exponential growth in data from future developments in radio telescopes". 

As part of this initiative, the NSF NRAO plans to release additional materials, including guides for various instruments and example datasets from multiple international partners. "The new data model is completely open-source and integrated into the Python ecosystem, making it easily accessible and usable by the broader scientific community," explains Steeb, "Our project promotes accessibility and ease of use, which we hope will encourage widespread adoption and ongoing development." 
 
 

Additional information 

The original press release was published by the National Radioastronomy Observatory (NRAO) of United
States, an ALMA partner on behalf of North America.

ALMA is a partnership of ESO (representing its member states), NSF (USA) and NINS (Japan), together with NRC (Canada), NSTC and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ.

Images