ALMA: a Data-Driven Organization

ALMA observatory is one of the first industrial astronomical facilities. Fifteen years ago, Earth-based observatories, even the most advanced ones, resembled more a laboratory than industries producing data for astronomical research. Since the beginning of the millennium, more advanced facilities have been built, with multimillion-dollar budgets and international collaborations involving hundreds of people distributed worldwide. Today's advanced telescopes are producing terabytes and even petabytes of data to power scientific research and enable unprecedented discoveries in astronomy.

To produce 4,300 hours of useful scientific observations from the Universe each year, ALMA requires hundreds of people working worldwide. About the same amount of time must be allocated to maintenance and updating activities. Among the maintained infrastructure, there are 66 high-precision antennas and a supercomputer that allows them to work as one by combining their signal through interferometry. Behind all this hardware, there is a complex software system to control the antennas and collect and process the obtained data. In addition, there are sensors monitoring power supplies, weather in different locations of the array at the Chajnantor Plateau, and operational conditions to ensure observations are performed at a high-quality level.

With all this complexity, ALMA had to explore and learn from other industries to implement what seemed unavoidable: becoming a data-driven organization. To state the intention to transform into such type of organization, this paragraph was included in ALMA's fundamental statements: "We aim for excellence in everything we do, and we expect our staff to be data-centric: As a scientific organization, we make our decisions, create solutions and organize our work based on data-driven analysis and the facts that support them."

ALMA is probably the first Earth-based astronomical facility to make advancements to incorporate data science, analytics, and automation to improve its operations.

The data produced by the observatory's operations is being used by its staff, who knows the tools and platforms specially implemented to fulfill data-analysis requirements. Its value has been broadly recognized among the organization, triggering fruitful cross-functional collaborations between astronomers, archive managers, software engineers, and system engineers, among others.

The short-term plan aims to mature and integrate even further data science and analytics functions into our operations, taking advantage of industrial frameworks and tools and enabling collaborations with key actors in this field to accelerate our data strategy implementation.

The long-term objective is, first, to make critical data widely available to all decision-makers involved in the observatory operations to remove any limitations that might hinder executing swift and appropriate actions. Second, to create and spread a data culture across the organization to improve data awareness. And finally, to make hardware and software systems compatible with data interoperability to avoid unnecessary human-machine interfaces constraining the observatory's ability to make efficient and assertive decisions.