On how science happens - Case Study: NASA, Arsenic, and Controversy

If you read Alex Bradley's guest post calling into question the claims of the recent Science paper stating the existence of microbes that can substitute arsenic for phosphorus in their DNA you might be wondering what to take away from it all, if the scientists can't even agree on whether the study is valid or not. I would like to take this opportunity to reflect on the scientific process and hopefully explain why this type of intellectual discourse demonstrates the strength of the scientific process rather than being an example of scientists not knowing what they are talking about. Here's the deal. In terms of the scientific process, this is how major discoveries happen, or dont.

  1. Scientist thinks they have made major new discovery
  2. Scientists writes up said discovery and submits paper to peer-reviewed journal (the bigger the claim, the fancier the journal)
  3. If editors of journal like the paper, they send it off to 3 or so experts in the field who review it, and provide the editor with feedback as to whether they feel the work is valid or not.
  4. Eventually, lets say the paper makes its way through the reviewers and gets published.
  5. Hold a massive press conference hyped up with hints of extraterrestrial life and blow things totally out of proportion. Hey, how'd that get in there?
  6. Now the fun begins.
  7. The vultures start to circle, and by that I mean other scientists start to pick at the paper (again, the bigger the claim, the more criticism typically surfaces).
  8. Original author might perform additional experiments that address these claims.
  9. Other scientists might carry out experiments that disprove the claims.
  10. Original claim either does, or does not, stand the test of time.

This paper makes a very bold claim. It claims that a bacterium grown in the lab under very particular chemical conditions eventually started building its DNA in a way previously assumed to be impossible. Please note that the claim of the paper is NOT that this organisms does this in nature, (although the authors do imply this, there is no evidence for it), it is merely claiming that the switch itself was shown to happen, and therefore is possible in nature.

Understandably there are many scientists out there (chemists in particular) who don't like it for many of the reasons that Alex pointed out (see link above) in the previous post on this blog. I am not a chemist, and don't feel qualified to determine the validity of the paper's claims. I do feel (perhaps naively) that errors as fundamental as the ones pointed out previously should have been (and may have been) addressed during peer review, and it is possible that Wolfe-Simon and her team have done more work than the space of the short Science communication allowed. I imagine we will be seeing more thorough publications from them soon which may strengthen their case. I am certainly looking forward to these publications.

I hope it turns out that they are correct. It would certainly be very cool, and would likely reinvigorate astrobiology research and funding for astrobiology research. Which is something I would very much like to see. Also, if it ends up that this claim is erroneous, I fear serious negative impacts on the astrobiology field because of all the fanfare that NASA gave the publication of this paper. I would blame NASA for this though, not Wolfe-Simon and her team. Scientific findings that have not had the chance to withstand the test of at least a bit of time do not generally get their own press conferences. Basically, I worry that NASA shot themselves in the foot by touting the enormity of this discovery too early.

In summary this study is, at this point, neither bad science nor good science. It is unproved science. The jury is still out. Publishing potentially controversial findings is part of the scientific process. Big name journals like Science tend to publish the higher profile claims, and inherent in that is that these claims are sometimes disproved. That does not mean the the journal is publishing bad science or is in someway flawed. It does not mean that the science itself is bogus. Even when findings are disproved, they often inspire other studies that make groundbreaking discoveries. That is, they have an inherent value to science even if the claims do not stand the test of time. Sometimes the initial way an experiment was run actually does support one claim. This is why we repeat experiments and do follow up experiments to confirm. There are always more tests you could do, but scientists walk a fine line between definitively proving their claims, and publishing exciting results so that others can hear about them. Obviously you need to be confident in your data, but you also need to consider the benefits of publishing quickly.

The point is that we will have an answer, and we will be able to confidently say that the answer is correct. Either this microbe can do what has been claimed, or it can't, and there are some simple analyses that can be done to determine which is the case. We just need to be patient and see what happens... and realize that this IS how science happens.

[NOTE: Because this post is now a magnet for spam, commenting has been closed. If you want to leave a comment, please send it to me via e-mail: webeastiesblog (at) gmail.com]