Almost 15K Reviews for the Most Competitive Proposals Process to Observe with ALMA

20 August, 2021 / Read time: 7 minutes

The new ALMA cycle of observations, Cycle 8, was one of the most competitive to date. The requested hours for observation in the 12-m Array exceed by six times the available hours. For this tough selection, the proposal review process included an unprecedented participation by the community of more than one thousand researchers in the new distributed peer review system and panel reviews for larger programs.

On August 6, 2021, the Cycle 8 2021 Call for Proposals results were announced to the astronomical community, completing a process that started last December with its pre-announcement. The proposal review process selected 253 high-priority programs that ALMA will observe beginning October 2021.

"Any proposal review process relies on the support and commitment of the community, especially in Cycle 8 with distributed peer review and the need for virtual panel meetings," explains ALMA Director Sean Dougherty. "ALMA is indebted to the community for their enthusiasm and support for this new process."

The original Cycle 8 proposal process was delayed due to the COVID-19 pandemic and restarted in 2021. It was extraordinary in many aspects:

• Despite the challenging work conditions imposed by the pandemic, the community submitted 1,735 proposals that requested over 26,000 hours on the 12-m Array, far exceeding the time requested in previous cycles (19,148 hours requested in cycle 7). With 4,300 hours available on the 12-m Array, Cycle 8 was one of the most competitive ALMA cycles to date. The total requested time for observations using the ACA 7-m array and Total Power array is about five times the available time, also significantly more than the requested time in previous cycles.
• The proposal review process had unprecedented participation by the community. With the introduction of distributed peer review and the continuation of panel reviews for proposals over 25 hours in the 12-m Array, or in excess of 150 hours in the Morita Array (aka. Atacama Compact Array, or ACA) in standalone mode, more than one thousand astronomers participated in the review process to help determine ALMA's observing program for the new cycle.
• For the first time, ALMA required proposals written in a dual-anonymous fashion, where the identity of the proposal team remains hidden to the reviewers.
• The ongoing travel restrictions imposed by the pandemic forced the panel reviews to happen virtually.

"Given the significant innovations in the proposal review process, we are conducting a detailed analysis of the results. Our objective is to examine the potential biases in the review results and carefully consider the feedback from the reviewers and the proposal teams," commented John Carpenter, ALMA Observatory Scientist and leader of the revised proposal review process. "In the coming months, a detailed analysis of the distributed peer review process will be made available."

Dual-anonymous Proposal Review

In previous cycles, ALMA has carefully monitored the results of the proposal process to identify systematics in the proposal rankings. The Observatory instituted a dual-anonymous proposal evaluation process to reduce biases and make the review process as fair as possible for all users.

"The vast majority of the community adapted to the guidelines and anonymized their proposals successfully," explains Andrea Corvillón, the Proposal Handling Team Lead. "The feedback from the reviewers was overwhelmingly positive as they thought dual-anonymous made the process fairer and allowed them to focus on the science and not the proposal team."

Distributed Peer Review

After the successful introduction of distributed peer review in the Cycle 7 ACA stand-alone Supplemental Call, ALMA applied this method for the first time in the main call for Cycle 8, for proposals requesting less than 25 hours on the 12-m Array or less than 150 hours on the Morita Array. For this cycle, 1,016 astronomers reviewed 1,497 proposals requesting less than 25 hours of observing time.

"Through the distributed peer review system, the community completed the impressive number of 14,960 reviews in a couple of months," signaled Carpenter. "Distributed peer review allows much broader participation by the community in the review process compared to the traditional method."

Most reviewers read either ten proposals (67%), or twenty (23%). The process ran smoothly, and there was widespread praise for the ease of use of the software tool. Valuable feedback from the reviewers will support improvements to the process in future cycles.

Panel Reviews

The ALMA Review Panels (ARPs) met from June 21 to 25, and the ALMA Proposal Review Committee (APRC) met from June 29 to July 1. The primary outcomes of the ARP discussions are a ranked list of the proposals requesting between 25 and 50 hours on the 12-m Array and a down-selection of the Large Programs. These Large Programs are then advanced to the APRC, which consists of the panel chairs and at-large members.

Statistics and Results

Users submitted 1,735 proposals for 26,325 hours of 12-m Array time, 14,846 hours of the ACA 7-m Array, and 13,802 hours of the ACA Total Power array in Cycle 8. While the number of Cycle 8 proposals is comparable to that in Cycle 7 (1,773), the requested time is 37% higher for the 12-m Array. The observation time request using ACA has also increased significantly; 67% and 99% higher for the ACA 7-m Array and the Total Power array, respectively. As a result, the oversubscription rate in Cycle 8 is the highest for ALMA since Cycle 0. Therefore, given the incredibly competitive cycle, many excellent proposals could not be scheduled. Overall, 15% of the proposals were accepted with priority Grade A or B.

Following advice from several committees, ALMA continued to encourage larger, more ambitious programs in Cycle 8. The cap in the amount of time allocated to Large Programs (15%) was replaced with a floor (10%) that represents the minimum amount of time prioritized in the queue-building process for proposals requesting more than 25 hours on the 12-m Array. The community responded enthusiastically to these changes. The number of Large Program proposals submitted increased to 40 in Cycle 8 from 14 in Cycle 7, of which six will be scheduled. Further, the number of submitted proposals between 25 and 50 hours increased to 198 from 97.

"We will begin observing the new projects on October 1. We are very excited to see the amazing discoveries that the community will find in Cycle 8." Says Head of ALMA Department of Science Operations Elizabeth Humphreys after checking the scientific program lined up for the new period.

The Joint ALMA Observatory is optimistic that this new process will reduce potential biases and maintain a high review standard while considerably reducing the carbon footprint of the process.

A more complete version of this announcement can be found on the ALMA Science Portal.

ALMA is a partnership of ESO (representing its member states), NSF (USA) and NINS (Japan), together with NRC (Canada), MOST and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ.

Tables & Figures