Introduction
The search for life on other worlds, in the absence of ‘a herd of elephants stampeding across the field of view of the lander camera’ (Oró, Reference Oró and Schopf2002), relies on the detection and interpretation of biosignatures. A biosignature is a measurement that indicates the presence of life (Seager and Bains, Reference Seager and Bains2015; Catling et al., Reference Catling, Krissansen-Totton, Kiang, Crisp, Robinson, DasSarma, Rushby, Del Genio, Bains and Domagal-Goldman2018; Schwieterman et al., Reference Schwieterman, Kiang, Parenteau, Harman, DasSarma, Fisher, Arney, Hartnett, Reinhard and Olson2018; Walker et al., Reference Walker, Bains, Cronin, DasSarma, Danielache, Domagal-Goldman, Kacar, Kiang, Lenardic and Reinhard2018). In principle, a wide range of measurements could be used, but atmospheric gases that may be made by life are most widely discussed (Seager et al., Reference Seager, Schrenk and Bains2012; Seager and Bains Reference Seager and Bains2015), in part because those are the only exoplanet biosignatures we are likely to be able to detect in the foreseeable future.
However, an individual measurement of a biosignature gas on its own is unlikely to be irrefutable proof of the presence of life on another world, and the interpretation of any candidate biosignature is likely to be controversial (NAS, 2019). The degree of certainty that a biosignature provides about the presence of life depends on what else is known about a planet, about the biosignature, and about life. Even oxygen, the paradigmatic atmospheric marker for life for nearly 100 years (Jeans, Reference Jeans1930) can be generated abiologically, and is only a strong biosignature in the context of other atmospheric and planetary properties (reviewed in Meadows et al., Reference Meadows, Reinhard, Arney, Parenteau, Schwieterman, Domagal-Goldman, Lincowski, Stapelfeldt, Rauer and DasSarma2018).
The detection of a biosignature has to be placed into context, both of the reliability of its detection (based on instrumental as well as signal characteristics) and our certainty that is can be associated with life (Neveu et al., Reference Neveu, Hays, Voytek, New and Schulte2018). There are several approaches that can be used to evaluate how useful a biosignature will be (Pohorille and Sokolowska Reference Pohorille and Sokolowska2020), but the framework most often cited as being applicable to evaluating what a biosignature detection means once it has been detected is that of Bayesian statistics (reviewed in Catling et al., Reference Catling, Krissansen-Totton, Kiang, Crisp, Robinson, DasSarma, Rushby, Del Genio, Bains and Domagal-Goldman2018; NAS, 2019). Bayesian statistics are statistics where probability reflects a degree of belief in an event. In frequentist statistics (the type of statistical analysis more familiar to most people), a probability of an event of 0.5 means that in a large number of trials half the time that event would happen. In Bayesian statistics, a probability of 0.5 means that we think there is a 50 : 50 chance that the event would happen. Bayesian statistics is particularly useful when we want to see how a new piece of evidence changes our beliefs that a hypothesis is likely to be true. Bayesian statistics can tell us how much our confidence that life is present on a planet or moon changes as we add new data. This is particularly useful in a field such as astrobiology, where there is little actual data to support a wide range of hypotheses.
The recent report of a preliminary detection of phosphine in the atmosphere of Venus (Greaves et al., Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020; Bains et al., Reference Bains, Petkowski, Seager, Ranjan, Sousa-Silva, Rimmer, Zhan, Greaves and Richards2021) provides a test case both to apply Bayesian statistics in a well characterized example and to see if this is actually how astrobiologists reason about biosignatures. Phosphine has been postulated to be a strong biosignature, in that on rocky planets it is only expected to be made by life (Bains et al., Reference Bains, Petkowski, Sousa-Silva and Seager2019; Sousa-Silva et al., Reference Sousa-Silva, Seager, Ranjan, Petkowski, Zhan, Hu and Bains2020). However, the Venusian environment is both the antithesis of terrestrial ecosystems that make phosphine (Bains et al., Reference Bains, Petkowski, Sousa-Silva and Seager2019) and highly inimical to terrestrial life (Seager et al., Reference Seager, Petkowski, Gao, Bains, Bryan, Ranjan and Greaves2020). How can these conflicting observations be reconciled?
This paper reports on using the tentative discovery of phosphine on Venus as an observational experiment to test whether astrobiologists actually use Bayesian reasoning in evaluating a new piece of astrobiological knowledge about a planet. The case study is particularly useful as all the other facts known about Venus were the same before and after the discovery. There is only one new piece of data to consider. This contrasts to the situation were a biosignature is discovered on an exoplanet, where the discovery of the biosignature would be made alongside the discovery of other data about the planet's temperature, atmosphere and other properties (Catling et al., Reference Catling, Krissansen-Totton, Kiang, Crisp, Robinson, DasSarma, Rushby, Del Genio, Bains and Domagal-Goldman2018; Schwieterman et al., Reference Schwieterman, Kiang, Parenteau, Harman, DasSarma, Fisher, Arney, Hartnett, Reinhard and Olson2018). Thus, the case of phosphine on Venus is a ‘pure’ case where any change in our estimate of the chance of life is due solely to the candidate biosignature discovery.
We approach the Bayesian analysis of responses to the Greaves et al. paper in two ways.
Firstly, we carried out two surveys of opinions on the possibility of indigenous life on Venus. To give some context we also asked about four other Solar System bodies that are frequently discussed as targets for astrobiological missions: Mars, Europa, Titan and Enceladus. We asked about life in two ways: firstly, how likely respondents thought it was that life was present, and secondly whether they thought it worthwhile investing $1 billion in a life-finding mission to that body. The first survey was done before the Greaves et al. result was published, the second survey was done after Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) was published (and very extensively publicized) to test how that result changed the responses.
Secondly, we derived a Bayesian formulation for how our expectation of the presence of life on another world should be influenced by a new piece of necessarily noisy and uncertain data. We then compared this change in expectation with the change actually seen in the survey data.
We conclude that astrobiologists as a community made a balanced and well-considered conclusions from Greaves et al.'s paper, but that their reasoning probably was not purely Bayesian.
Method
Two surveys were created on Google Forms (full survey text is shown in Appendix 1 in the Supplementary material). Participants were invited to participate by an e-mail. E-mails were kindly sent out by the Astrobiological Society of Great Britain, the European Astrobiology Network and The International Journal of Astrobiology (Cambridge University Press). We are very grateful to these organizations for sending out e-mails on our behalf. The first survey e-mail was complemented by a personal e-mail from WB to some North American researchers, as no equivalent source of North American e-mails could be identified for large-scale e-mailing. The second survey e-mail was complemented by e-mails to the same personal e-mail list, and a list compiled from all those who gave e-mails in the first survey.
The first survey was e-mailed to potential respondents in the middle of June (e-mails being sent out 11 and 12 June 2020). The majority of responses came in the first 3 days, with a smaller number over the next 4 days. The survey was closed to respondents on 26 June 2020. Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) was published, and Bains et al. (Reference Bains, Petkowski, Seager, Ranjan, Sousa-Silva, Rimmer, Zhan, Greaves and Richards2021) put online in ArXiv, on 14 September 2020, and significant publicity followed over the following week. The second survey was then e-mailed to potential respondents on 2 October (for ASB, ENEA and personal lists) and 19 October (Int. J. Astrobio. list). The survey was closed to respondents on 25 October 2020.
The surveys asked respondents to estimate the chance that life was present on five Solar System bodies: Enceladus, Europa, Mars, Titan and Venus, which were listed in this alphabetical order in the first survey and reverse distance from the Sun in the second (Venus last on the list in both cases). The respondents were asked to select one of nine bins, arranged in a log scale. Ideally for Bayesian statistics the respondents should have been asked to provide a number for the probability, but we did not do this for three reasons. Firstly, we wanted the survey to be simple to fill in, and to require as little conscious calculation as possible. Secondly, a very large number (1000s) of respondents would be needed to populate a continuous scale from 0 to 100, especially if results clustered at one end. Finally, and probably most importantly, even if we had asked for a continuous number, we expected that respondents would chose a simple fraction as a probability, such as 10, 25, 33%, and the likelihood that any respondent would chose (e.g.) 27 or 19% was very small. (The same bias is seen in how people select ‘random’ numbers (Bains, Reference Bains2008).) By pre-defining the bins we constrained that tendency to a set of bins that were consistent.
For ranking and averaging of data, the ranges were converted to single values as listed in Table 1.
Table 1. Conversion values for ranges
Values used to convert ranges cited in the survey to discrete values for averaging and ranking. Note that a probability of zero would mean that the chance of life on a planet is absolutely ruled out on first principles, which seems an implausibly extreme position, so the ‘0%’ band was assigned a discrete probability of 0.5%.
Survey results
General enthusiasm for Solar System life
The first survey collected 121 valid responses, and the second survey collected 85. We found a strong correlation between respondents' enthusiasm for life on all the Solar System bodies (SSBs). The results of Pearson rank correlation coefficients between scores are shown in Table 2. There is a strong correlation between the scores for all the different SSBs. Correlation is extreme between scores for Europa and Enceladus (suggesting a group of researchers with an enthusiasm for life on icy moons), but strongly and significantly positive for all scores (the chance of a correlation across 120 values of >0.3 is ≲ ˆ0.001). This suggests that respondents had an overall view of how likely life was on other solar system bodies, separate from their belief about the relative likelihood of life on a specific body. This was confirmed by the observation that, with the exception of the Europa : Enceladus correlation, all SSBs' scores were correlated more strongly with the average of a respondent's score of all bodies than with any individual score. Despite this correlation, the tentative detection of phosphine on Venus did not significantly affect the perception of the chances for life on any other world.
Table 2. Correlations between scores for different SSBs
How individual respondents in the first survey scored the likelihood of life on other worlds is highly correlated. Values are rank correlation coefficients for the numerical score (from Table 1) for the five Solar System bodies, and the per-respondent average score across all bodies.
There was a hint that this was related to the research interests of the respondents (Fig. 1), the physical sciences being less enthusiastic about solar system life than the biological sciences; however this is a weak effect, and cannot explain the strong correlations seen in Table 2.
Fig. 1. Average results by discipline. Average score for all five Solar System bodies, by discipline of respondent. Y axis – number of respondents (note that many listed more than one discipline). Error bars – standard error the mean. (Only two respondents ticked ‘mathematics’ in both surveys, which is too few to analyse.)
The survey was not confined to full-time, professional astrobiologists. Respondent were not required to give an e-mail address, but 77 did (and another three filled in their names, not their e-mail address, illustrating that even professional scientists can fail to follow simple instructions on occasion). Of the respondents to the first survey, 44 provided institutional e-mails from research institutions or universities, or were known to us as professional scientists. Chi-squared test showed that the responses of this known academic subset did not differ significantly from the whole survey response. We conclude that there is no evidence that the respondents who did not give an institutional e-mail address were different from those who did.
Greaves et al. slightly boosted expectations of life on Venus
The proportional responses in each survey to the first question are shown in Fig. 2. As expected, favoured astrobiology targets Mars, Enceladus and Europa have a substantial fraction of the respondents saying that they think there is a substantial chance of life there. Titan and Venus have generally less positive responses, with nearly 50% of respondents in the first survey saying that they believed there was 0% chance of life on Venus. The second survey showed a very similar pattern, but with fewer respondents saying the chance of life on Venus was 0%, and more saying it was 1%. The difference between surveys is significant only for Venus (Table 3), which is rational – there is no reason why the tentative detection of phosphine on Venus should alter our expectation of life on Mars.
Fig. 2. Survey responses to first question. Responses to the first question. Y axis – fraction of respondents giving each answer. Each respondent had to choose one probability for each Solar System body, so the columns for each Solar System body sum to 1. A: first survey, B: second survey.
Table 3. Chi-squared test of differences between the two surveys
Only Venus shows significant difference between first and second surveys. Chi-squared test for the null hypothesis that the second survey would have the same distribution of expectations as the first survey, calculated separately for each of the five Solar System bodies. For 8 degrees of freedom the critical values for chi-squared p < 0.05 = 15.51, p < 0.01 20.09.
Despite this slight increase in confidence that there was life on Venus, there was no increased enthusiasm for a mission to Venus (Fig. 3). Rather, enthusiasm for a mission to Mars seemed to increase. This may be because respondents felt that there was enough ‘chatter’ about Venus missions already in the weeks following Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020), or because results coming from Mars supported a new mission there (such as the report of subglacial ‘lakes’ (Lauro et al., Reference Lauro, Pettinelli, Caprarelli, Guallini, Rossi, Mattei, Cosciotti, Cicchetti, Soldovieri, Cartacci, Di Paolo, Noschese and Orosei2020), which appeared after Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) but before the second survey). So, this question was less informative than had been hoped.
Fig. 3. Survey responses to second question. There has been no change in enthusiasm for an astrobiological mission to Venus since the Greaves et al.'s paper. Respondents had to select one Solar System body to send a $1 billion mission to in addition to other planned missions. Blue columns therefore sum to 1, as do red columns. Y axis – fraction of respondents selecting that solar system body. X axis – Solar System body.
Icy versus non-icy worlds
Table 2 suggests a group of respondents who shared a high expectation of life on the icy moons Europa and Enceladus. We therefore separated the responses of potential ice moon enthusiasts from others to see if their response to the Greaves et al. discovery was different. Figure 4 shows that those with enthusiasm for life on ice moons Europa and Enceladus systematically have a lower belief in life on Venus than those with low belief in life on (or in) ice moons, but that both groups increased their belief in life on Venus after the Greaves et al. result.
Fig. 4. p(L) for Venus from respondents with enthusiasm for icy moons. The relative enthusiasm for life on icy moons was calculated as I = average probability recorded for Europa and Enceladus divided by the average probability for all five Solar System bodies. The median value was ~1.3. Respondents to first and second survey were categorized into ‘Ice Fans’ (I > 1.3) or non-ice-fans (I ≤ 1.3), and the average belief of the two classes for life on Venus calculated for both surveys. Error bars = standard error of the Mean. (The irony of using frequentist statistics to analyse Bayesian results is not lost on the authors.)
Bayesian treatment
The results seen above suggest that the community has taken a measured, thoughtful approach to the Greaves et al.'s result, and to the subsequent intensive, informal online discussion about whether it really is phosphine and whether there is an abiotic explanation for the presence of phosphine. As a whole, the community has increased their estimate of the chances of life on Venus slightly, but still considers it the least likely place of the five bodies surveyed to harbour indigenous life. But, on what basis is this judgement made?
There is a growing line of thought that Bayesian statistics provide a basis for judgements such as the one analysed here – deciding how a new piece of evidence changes our expectation of life on another world. We explored whether the astrobiological community could actually be shown to be using Bayesian reasoning.
An excellent introduction to the use of Bayesian statistics in biosignature analysis is provided by Catling et al. (Reference Catling, Krissansen-Totton, Kiang, Crisp, Robinson, DasSarma, Rushby, Del Genio, Bains and Domagal-Goldman2018), and we will not repeat that here. In this paper, we follow and extend Catling et al.'s analysis, deviating from that paper only in formulation by using a more conventional notation, that probability of the presence of life is p(L), not p(Life), and the probability of the absence of life is $p( {\bar{L}} )$
, rather than p(NoLife). In summary, take the basic Bayesian equation (Catling et al., Reference Catling, Krissansen-Totton, Kiang, Crisp, Robinson, DasSarma, Rushby, Del Genio, Bains and Domagal-Goldman2018):
And expand it to derive an equation for our expectation of the probability of life on a planet given the planetary context and our prior expectation of life on that planet:
where p(A|B) is the conditional probability of A given B, L is the presence of life on the planet, $\overline {L\;}$
is the absence of life on the planet, $L + \bar{L} = 1$
(life is either present or it is not present), C is the context (i.e. all the other things we know about the planet before gathering a new piece of data about the planet) and D is the new piece of data that we now have about the planet.
For our purposes, we simplify this as
With the implicit understanding that all probability terms apply to Venus only, i.e. C is a given.
Catling et al.'s model implicitly assumes that D is the detection of a biosignature on a planet, in our case the detection of an atmospheric gas. We then chose p(D|L) and $p( {D{\rm \vert }\bar{L}} ) \;$
to reflect how likely it is that life and abiological processes respectively would make a specific gas on a specific planet. The astrobiological literature abounds with such estimates.
However, this neglects the link between datum and the gas. For a real observation, we need to modify the method to reflect that what we measure is not the presence of a gas, it is a signal in an instrument from which we infer the presence of a gas. Usually that signal is faint and noisy (inevitably, as we are always pushing the limits of detectability to make new, publishable observations). Estimates of the instrumental requirements for gas detection (e.g. as described in Benneke and Seager, Reference Benneke and Seager2012; Seager et al., Reference Seager, Bains and Hu2013) quantify this uncertainty but do not eliminate it. So, we have two uncertainties: whether a gas on a planet is made by life, and whether our signal actually shows that gas is present on the planet. This is not a trivial point: several ‘detections’ of gases in exoplanets have subsequently been suggested to be over-interpretation of marginal, noisy data (e.g. Gibson et al., Reference Gibson, Pont and Aigrain2011; Deming and Seager Reference Deming and Seager2017). So, equation (2) is a combination of an expectation of the properties and presence of life and of our interpretation of the actual signal we have obtained.
We therefore need to modify equation (2) to take into account the potential uncertainty in interpretation of the data as well as interpretation of the inferred observation. We do this below for the first time, so that
where D c means the data and our interpretation of that data are both correct, and D i means that either the data or its interpretation is incorrect, and D c + D i = 1. (In verbal terms, the probability that there is life given the observed data is the sum of the probability of life being present and our interpretation that the data show the biosignature gas is present is correct, and the probability that there is life and our interpretation of the data is not correct.) In the specific example of Greaves et al., their spectral signal might not be real, or it might not be due to phosphine, but there could still be life on Venus, so p(L, D i|D) is not necessarily = 0. Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) are at pains to stress that theirs is a preliminary detection, and while they went to considerable lengths to confirm that the signal they observed was a detection of phosphine, the signal is only evidence of phosphine on Venus, not proof of it.
Equation (3) can be expanded to
The formulation in equation (4) separates out issues of detection and interpretation of the gas from issues of interpretations of the gas's source.
We now expand equation (2) according to equation (4), assuming we have data (i.e. p(D) = 1) to give
where p(D c) is the probability that we have interpreted the data correctly. If an observation is in error, or our interpretation of it is wrong, then we do not know what the observation means in terms of a gas on planet, and so $p( {D_{\rm i}{\rm \vert }L} ) = p( D_{\rm i}\vert \bar{L})$
, as we have no evidence as to whether life is or is not involved in the signalFootnote 1. Then the second term in equation (3) becomes
And so equation (5) simplifies to
If we are completely confident in Greaves et al.'s detection and analysis (p(D c) = 1), equation (6) simplifies to equation (2). If we discount Greaves et al. completely (p(D c) = 0), then equation (5) simplifies to
i.e. the data are worthless and adds nothing to our knowledge of whether there is life on Venus.
We do not wish to suggest that Greaves et al.'s detection is dubious. Equation (6) just quantifies the effect of any doubt that readers of Greaves et al. have on their estimate of the chances of life in Venus' atmosphere. Some reviewers of Greaves et al. were quite forceful in expressing their doubt, as were some subsequent online commentators and at least two un-refereed arXiv posts, so it is appropriate to acknowledge that doubt.
Are astrobiologists actually Bayesian?
We can now ask whether astrobiologists actually use this type of analysis – consciously or otherwise – in their interpretation of the new data presented in September 2020.
p(L) is our prior expectation of life on Venus. The first part of the survey quantified the community's prior belief in the presence of life on Venus. (In Catling et al.'s treatment, this is p(Life|C), the probability that there is life given the context of Venus.) Our survey showed that in June 2020 it was widely considered that Venus had a low probability of hosting indigenous life.
The survey asked respondents to select a probability range, for reasons discussed above. Equations (1) through (6) relate to continuous probability functions, not binned values. We therefore generated a function that provided a continuous p(L) value that fitted the observed binned data. (See Appendix 2 in the Supplementary Materials for details.) (We note that using a 12-parameter equation to match a 9 point data set is over-fitting; however this is a pure curve-fitting exercise to provide a smooth, continuous function for Bayesian statistical methods, the function is not meant to have theoretical significance.) This probability density function was used as the distribution of p(L) in subsequent calculations.
p(D c|L) and $p( D_{\rm c}\vert \bar{L})$
are the probabilities that phosphine is present on a rocky planet in the presence and absence of life respectively (more formally, they are the probabilities that the signal is seen and its assignment to phosphine is correct, and life is or is not present). Excluding Venus, only the sub-Saturn mass body in the Solar System is known to have phosphine on it or in it is Earth, where it is the exclusive product of life (reviewed in Bains et al., Reference Bains, Petkowski, Sousa-Silva and Seager2019; Sousa-Silva et al., Reference Sousa-Silva, Seager, Ranjan, Petkowski, Zhan, Hu and Bains2020). However, the at times heated online discussion following Bains et al. (Reference Bains, Petkowski, Seager, Ranjan, Sousa-Silva, Rimmer, Zhan, Greaves and Richards2021) and Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) suggests that there is no consensus on p(D c|L) or $p( D_{\rm c}\vert \bar{L})$
despite the arguments for a high p(D c|L) and a low $p( D_{\rm c}\vert \bar{L})$
presented by Bains et al. (Reference Bains, Petkowski, Sousa-Silva and Seager2019, Reference Bains, Petkowski, Seager, Ranjan, Sousa-Silva, Rimmer, Zhan, Greaves and Richards2021); Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) and Sousa-Silva et al. (Reference Sousa-Silva, Seager, Ranjan, Petkowski, Zhan, Hu and Bains2020). $p( D_{\rm c}\vert \bar{L})$
cannot be larger than p(D c|L) for a ‘positive’ biosignature (one that is made by life; Seager et al., Reference Seager, Schrenk and Bains2012), because if there is a probability that a gas is made by life or by non-biological processes, then there cannot be a greater chance that the gas is present if it is made solely by geological processes than the chance that it made by geological processes and by life.
We modelled the expected distribution of answers to the second survey assuming (i) the p(L) distribution from the first survey, (ii) all possible combinations of p(D c|L) and $p( D_{\rm c}\vert \bar{L})$
such that 0 < p(D c|L) < 1 and $0 < p( D_{\rm c}\vert \bar{L})$
≤ p(D c|L), and (iii) all p(D c) such that 0 < p(D c) ≤ 1, a total of 3971 combinations. The result is summarized in Fig. 5. The observed result from the survey lies outside the area of possible results predicted from the Bayesian analysis. This suggests that a Bayesian approach does not explain the second survey results. We explored what might be causing the deviation by doubling each of the survey results in turn (i.e. adding new ‘votes’ to each bin, without reducing the others). Only doubling the ‘2%’ bin (purple dot in Fig. 5) moved the result into the space of expected Bayesian results.
Fig. 5. Observed second survey result does not match any Bayesian outcome. Principal components analysis of the predicted second survey results based on the first survey p(L). The number of respondents in each of the nine probability bands was calculated for combinations of p(D c|L), $p( D_{\rm c}\vert \bar{L})$
and p(D c) as described in the text. The nine-dimensional output was then converted to a two-dimensional plot using principal components analysis using RealStatistics (https://www.real-statistics.com) in Excel. X and Y axes – first and second principal components. Circle area is proportional to $p( D_{\rm c}\vert \bar{L})$
(i.e. the probability that, if there is phosphine, there, it is not made by life). Blue dots – modelled values. Orange dot – actual result from second survey. Red ellipse – 95% confidence on principal components analysis coordinates of survey results, based on Poisson counting error estimates. Purple dot – principle components plot of second survey result in which the ‘2%’ bin had 16 instead of 8 counts, all other counts being equal.
Discussion
If phosphine is a biosignature gas as suggested by Bains et al. (Reference Bains, Petkowski, Sousa-Silva and Seager2019) and Sousa-Silva et al. (Reference Sousa-Silva, Seager, Ranjan, Petkowski, Zhan, Hu and Bains2020), then its detection in the atmosphere of Venus should have had a substantial impact on the estimate of the probability of the existence of life on Venus. The observed impact was quite modest, although significant. This suggests that the community is sceptical about the detection, the possibility of life on Venus, the strength of the association of phosphine with life, or some combination of these. Although we stand fully behind the results reported in Bains et al. (Reference Bains, Petkowski, Seager, Ranjan, Sousa-Silva, Rimmer, Zhan, Greaves and Richards2021) and Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020), we understand the community scepticism at this early stage.
The lack of match between the second survey result and any Bayesian model based on the first survey suggests that the community does not weigh the probabilities of a biosignature gas in a purely Bayesian fashion. This appears to be a significant difference. Our tentative exploration suggests that the response of individuals was polarized to ‘very low probability’ or ‘moderate probability’, such that artificially increasing the ‘2%’ score moved the results into the space of results predicted by the Bayesian analysis above. This is consistent with the observation of strong correlations between scores for different worlds – some respondents had a low expectation of life on other solar system bodies, and any tentative result would only shift them from ‘ruled out’ (0%) to ‘highly unlikely’ (1%). It is possible this is related to the apparent existence of a sub-community with more enthusiasm for life on ice moons and less enthusiasm for life on Mars or Venus. We emphasize this is only a suggestion, and that a much larger and more detailed survey would be needed to confirm any such speculation. However, this does point to differences in how the community evaluates biosignatures, which might be worth exploring when developing biosignature evaluation criteria in the future
Notes for future surveys
Responses to the survey, including several follow-up e-mail discussions, suggested that future surveys could improve on the surveys reported here. A major issue is the ‘bins’ of probability used, which were intuitively pleasing but proved hard to convert to a continuous scale for Bayesian analysis. Any future survey should consider bins based on the statistical model to be tested. One respondent commented that there should be some sort of ‘don't know’ option for Question 1; more generally, an approach such as mathematical Expert Elicitation techniques (O'Hagan et al., Reference O'Hagan, Buck, Daneshkhah, Eiser, Garthwaite, Jenkinson, Oakley and Rakow2006) could be useful, although more complex than the simply tick-box used here. Asking whether there is life there and whether there is not life there will give complementary answers which might be revealing, and is related to the concept of ‘indigenous’ life, which several respondents questioned as ambiguous. A future survey might ask about life that arose on a body versus life that was transferred there.
All of these improvements would help dissect the reasons for respondents' answers, but would also require a more complex survey. Even with this survey there was a decline in interest in filling in a second form. A future survey might therefore best be conducted as an in person interview rather than as a rather basic Google form.
Conclusion
We conducted two surveys of opinion among astrobiologists as to the likelihood of there being life on five Solar System bodies. The result was consistently that Mars, Europa and Enceladus were considered to have a roughly equal chance of hosting life, Titan was less probable and Venus was judged to have the lowest chance. The tentative detection of the biosignature gas phosphine on Venus by Greaves et al. (Reference Greaves, Richards, Bains, Rimmer, Sagawa, Clements, Seager, Petkowski, Sousa-Silva, Ranjan, Drabek-Maunder, Fraser, Cartwright, Mueller-Wodarg, Zhan, Friberg, Coulson, Lee and Hoge2020) improved the perception of Venus' status as a potential candidate for inhabitance, but still did not raise it above Titan in the ranking. Preliminary analysis suggested that although respondents showed reasonable caution, even scepticism, about the phosphine result, how they applied this to the case of Venus was not Bayesian. Better understanding of how scientists involved in astrobiology use data to determine inhabitance would improve how such results are presented, and could lead to biosignature choice and evaluation approaches that better reflect how we think of life on other worlds.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1473550421000185
Acknowledgements
We are very grateful to Cambridge University Press and to Enrico Mancinelli, editor of the International Journal of Astrobiology, to Manish Patel and Jennifer Wadsworth and the Astrobiology Society of Great Britain, and to Lena Noack, Frances Westall, Barbara Cavalazzi, Kai Finster and Tetyana Milojevic and the European Association of Astrobiology Networks for their help in mailing out requests for this survey to their mailing lists. Without their help this survey would not have been possible. We are also grateful to the community members who filled in the survey. We emphasize that the survey and the interpretation of its results are ours, and are not necessarily endorsed by these organizations or the survey respondents. We are also very grateful to Prof. David Kipping (Columbia, NY, USA) for help with Bayesian math, to Anna Ohanjanyan for introducing WB to Expert Elicitation and to several survey respondents for their constructive comments. This work was not funded.
Conflict of interest
The authors have no conflicts of interest with respect to this work.
