by Gillian Beltz-Mohrmann (Argonne National Laboratory)
19 March 2025
The first known blind experiment was conducted by the French Royal Commission on Animal Magnetism in 1784. In the experiment, the researchers literally blindfolded participants and asked them to identify objects that the experimenters had previously filled with “vital fluid,” which none of the participants were able to do. Today, many experiments rely on “blinding” techniques to prevent sources of bias from influencing the results. Many people have probably heard of blind analyses in the context of medical studies, in which the participants in some clinical trial do not know whether they are receiving a drug or a placebo. Withholding this information reduces the risk of confirmation bias (i.e., participants reporting better outcomes because they know they are receiving medication).
The purpose of blinding in other areas of research is much the same: to eliminate the risk of confirmation bias influencing the results of the analysis. The past several decades of cosmology have led us to a model known as LCDM, which describes a flat Universe made up of cold dark matter (CDM) undergoing accelerated expansion described by a cosmological constant (L). LCDM has been favored by numerous observational studies and has thus become the accepted “standard cosmological model.” However, the fact that the model is so widely accepted comes with risks: when we analyze a new data set, it might be tempting to assume that LCDM is going to be the preferred model. While this temptation might be subconscious, it can still lead us to unknowingly bias our results. To avoid this, we need to do a blind analysis!
DESI measures the positions of millions of galaxies in the Universe. Our understanding of the early Universe predicts that these galaxies will preferentially be separated by a particular distance, known as the baryon acoustic oscillation (BAO) scale. The size of this distance is determined by the fundamental ingredients of the Universe (e.g., the amount of dark matter, the amount of dark energy, and the nature of dark energy, just to name a few). By choosing different combinations of these ingredients (also known as cosmological parameters), we can make various predictions for the BAO scale, and determine which prediction agrees the best with our observations.
To perform this analysis blind, the measurements are first altered and the analysis methodology is developed using the altered data. As a result, the researchers don’t know what answer is preferred by their real data until the very end, when the alterations are undone (“unblinded”), the analysis is carried out on the real data, and the true results revealed. In other words, no one involved in the analysis of the data should be able to tell what amounts of dark matter and dark energy are preferred by the real data until the final step of the analysis. As the analysis pipeline is being developed, various checks are performed to ensure that there are no errors or bugs. Only once all of these checks are passed is the final analysis performed on the unblinded data and the cosmological parameters revealed.
In April 2024 DESI released the BAO results from its first year of data, reporting a surprising result: a slight preference for a model with a time-varying dark energy equation of state. This is in stark contrast to the standard LCDM cosmological model, which includes a constant dark energy density (i.e., not varying over time). The new preference for a time-evolving dark energy, a model known as w0waCDM, was strengthened further when the DESI data were combined with other datasets (Cosmic Microwave Background measurements and Type Ia Supernovae distances). Because the preference for w0waCDM based on the Year 1 data was moderate, DESI would have to collect more data to get a clearer picture of whether this was an anomalous result or whether we had discovered something new about the cosmos.
The December 2024 meeting of the DESI Collaboration came with an exciting prospect: the unblinding of the BAO results from the first 3 years of DESI data! This historic event would give us the first indication of whether the w0waCDM cosmology would still be preferred with the inclusion of more data.
The “live unblinding,” at which the collaboration would witness the results for the first time, was scheduled for Thursday evening. But before this could occur, a final consensus had to be reached about whether to indeed proceed with the unblinding. A Tuesday evening session was scheduled to discuss last minute analysis choices and decide whether all of the necessary checks had been passed in order to proceed with the unblinding. The stakes of the session were high: once the data was unblinded, it could not be blinded again. After a long discussion, which lasted about an hour longer than it was scheduled to, the decision was made: the unblinded analysis would proceed as scheduled!
Tuesday’s session was led by Sesh Nadathur, an STFC Ernest Rutherford Fellow at the Institute of Cosmology and Gravitation at the University of Portsmouth. Recalling the experience, Sesh said, “Tuesday was a long and exhausting day – the whole collaboration felt they had a stake in the results, and we must have had 100 people in the room or joining the call. That was a great feeling, but I also felt a lot of responsibility to ensure we took everyone with us and everyone bought into our decision to move ahead with the unblinding. At the same time, we knew the decision had to be agreed upon in that session otherwise the whole timeline would have been thrown off! By the end, my overwhelming emotion was just relief that we’d gotten there.”
The BAO analysis team spent the next two nights getting very little sleep in order to ensure the timely delivery of the results. Cristhian Garcia-Quintero, a NASA Einstein Fellow at Harvard University and a member of the analysis team, recalled, “Once we got the green light on Tuesday, I was very nervous because we needed to have the unblinded results ready to present by Thursday. There was a lot to do in a short amount of time, and everyone was waiting on the results.” Sesh added, “It was only possible to complete the analysis because of the incredible work that Cristhian and Uendert Andrade (a postdoctoral fellow at the University of Michigan) did on those two days, with massive help from many others in the months leading up to the meeting to set everything up to process all the results so quickly.”
By Thursday morning, Sesh and Cristhian were able to examine the final results for the first time. Over the next few hours, Sesh worked hard to put his presentation together in time for the evening unblinding session, a process he describes as “Actually the best experience of my career so far! It was that extremely rare combination of learning the answer to a very important scientific question – and recognizing immediately that it was very important, rather than slowly realizing this over time, which is the more usual case – and for a few hours being one of only two people in the whole world who knows it.”
Sesh and Cristhian spent all of Thursday deftly deflecting questions about the status of the unblinded results. Cristhian found that “Lots of people were joking with me throughout the day, asking for a sneak preview of the results.” Sesh had a similar experience: “All day people asked me subtle questions about the results and tried to read my expression to guess which way things had gone. I had to try to keep a poker face!”
By the end of the day, the two had artfully managed to keep the results concealed from everyone. Sesh even went so far as to refuse to upload his slides to the meeting website until after his presentation. He explained, “I really wanted to make the result reveal in the plenary talk as theatrical and memorable an experience for everyone as possible. This was not just about keeping the results secret until then, but also about trying to present as comprehensive an overview of the cosmology results as possible, and being prepared to answer all questions on the spot. I didn’t want people to come away with questions, and only slowly discover things in the weeks afterwards, because this could dilute the experience.”
As Sesh prepared to deliver his talk, the energy in the room was palpable. Cristhian recalled, “Once we were in the unblinding session, I could see the anticipation in everyone’s faces and feel it in the room. The biggest difference compared to the Year 1 unblinding was that in Year 1 the dark energy results came as a surprise, but in Year 3 there was a lot of expectation from the collaboration.” As an attendee at the meeting, I can honestly say that I have never been so excited about a conference presentation. It was a privilege to bear witness to the unveiling of results that had the potential to reveal something new and strange about the Universe.
Building the drama of the moment, Sesh made us wait for the results while he explained the science behind the BAO measurement (and then added a couple of extra slides just for fun). Finally, we arrived at the moment we had all been waiting for: the big reveal of the constraints on the dark energy equation of state. Was the preference for evolving dark energy from the Year 1 analysis a fluke? Or is DESI was onto something big?
At last, the results were revealed: the preference for a time-evolving dark energy was confirmed! With the increase in data from Year 1 to Year 3, the constraints on the cosmological parameters governing the dark energy equation of state were tightened, strengthening the results and making history in the process. The findings provide further indication that a cosmological constant is not the origin of cosmic acceleration, but rather that dark energy is a kind of dynamically evolving fluid that pervades all of space. With this result, a whole new era of cosmology begins.
Cristhian Garcia-Quintero said of the results, “It seems like in the end, the data is hinting more strongly that there is something wrong with the standard model of cosmology, and dark energy is potentially that ‘something,’ but we need to be patient, keep paying attention to these tensions in the data, and keep performing careful analyses.”
Enrique Paillas, a postdoctoral fellow at the University of Arizona and a member of the analysis team, said “The Year 3 unblinding was at the same time, one of the most difficult and exciting periods of the last year. We needed to be very diligent with all the tests required to ensure our cosmological constraints were robust against potential systematic errors. Delivering the results in our allocated timeframe was only possible thanks to the contribution of many talented scientists across all career stages. After all the hard work, we couldn’t really contain the excitement once we finally unveiled the results. It was a very rewarding experience on many levels.”
Reflecting on the time since the unblinding, Sesh said, “The past three months getting the results ready for publication have been the hardest I’ve worked — a stretch of about 120% effort! I was anticipating this of course. With a subject is as sensitive and important as this, we’ve really had to make sure every argument is absolutely watertight and backed up by multiple calculations and tests. It’s a very important and impactful result, but one that is naturally going to face a lot of skepticism. Working to present the arguments in the most logical and convincing form, and crafting a paper that people will hopefully also enjoy reading — that’s a challenge I have actually enjoyed thinking about. I hope we have achieved it!”
— Edited by Joan Najita
To learn more:
Read about how blinding works and what it’s like to work this way in an interview with Sam Brieden, Uendert Andrade, and Juan Mena-Fernández.
Read about the Year 3 BAO results in the LBNL press release.
