Last time, there was a bit of excitement about the possibility of phosphine in the Venusian atmosphere. This time, I’d like to update the current status of this result and explain how science progresses in practise.
Reproducibility
For any measurement or experimental finding to be considered true and reliable, the results must be reproducible. Independent teams must be able to reach the same conclusion with independent methods or an independent analysis. So upon the news of this exciting finding about phosphine in the Venusian atmosphere and its potential implications for life in our solar system, this is exactly what the scientific community attempted to do – to check and verify the results.
No phosphine? Mis-interpretation?
- 15/10/2020 Encrenaz et al. “A stringent upper limit of the PH3 abundance at the cloud top of Venus”
This team attempted to search for the same spectral feature using data from a different instrument – and did not detect phosphine, instead only placing a limit on the abundance. - 19/10/2020 Snellen et al. “Re-analysis of the 267-GHz ALMA observations of Venus: No statistically significant detection of phosphine”
This team reanalysed the data used in the original study (from ALMA), and criticised the method used, suggesting that it could create spurious signals that seem to be significant, but are false. They conclude the original results are unreliable. - 28/10/2020 Thompson “The statistical reliability of 267 GHz JCMT observations of Venus: No significant evidence for phosphine absorption“
This person reanalysed the other dataset used in the original study (from ALMA) and also has the same criticism of the original method used – finding that it can create falsely significant features and that in their reanalysis no indication for phosphine is found. - 27/10/2020 Villanueva et al. “No phosphine in the atmosphere of Venus”
Possibly the most robust refutal of phosphine detection came from the 26 member strong team claiming no detection of phosphine. The original dataset was, apparently, subject to severe calibration issues. Following their independent calibration and analysis (with different methods), they also did not find any evidence for phosphine. This team also considers that the most significant spectral feature indicating phosphine (PH3) could actually be explained by sulphur dioxide (SO2) — a far less controversial molecule to find in that environment.
Response of the original team
With criticism like that, the original team had to respond and defend their findings. On 16/11/2020, Greaves et al. did just that: “Re-analysis of Phosphine in Venus’ Clouds” in which they re-calibrate and re-analyse the data (removing the aforementioned issues). The detection of phosphine is “tentatively” recovered – at a level 7 times lower than that of their first paper. They also dispute the SO2 interpretation.
(At this point, it is worth noting that at least one independent group supported the detection of phosphine.)
Conclusions:
reliable, reproducible, repeatable.
Scientists don’t always agree, especially when results are new. Findings must be considered reliable enough for others to trust; reproducible by independent groups and methods; and repeatable with different experiments and new data.
So far, it seems the detection of phosphine is neither reproducible, nor reliable – failing two of these tests. Repeatable with new data? Let’s wait and see.
Progress in science is made by trial and error; consensus reached by scientific debate; it’s not always black and white.