What is Scientific Consensus?

When a theory is put forward, it takes time for the scientific community to evaluate its merits. Ultimately, one hopes that the theory is able to not only explain past data, but to be able to predict the outcome of future experiments as well. When the dust settles, we hope that we reach “scientific consensus” regarding a theory. But what does this mean?

Since this is a condensed matter blog, let us take BCS theory as an example. When BCS was formulated, it was able to explain numerous experimental observations, such as the evolution of the electronic gap as a function of temperature as well as the specific heat anomaly among several other observations. However, there were also apparent problems with BCS theory. Many physicists were concerned with the non-conservation of particle number and with some aspects of broken gauge symmetry (pdf!) in the theory. Notably also, there were materials that did not conform exactly to the BCS formulas, such as Pb (lead), where the predicted 2\Delta/k_BT_c=3.5 relation and was found instead to be around 4.38.

So the question is, how were these issues resolved and how did the community reach the general consensus that BCS theory was applicable for the existing superconductors at that point in history?

This question actually leads to a more general scientific question: how do we reach a consensus concerning a theory? The answer to this question involves a Bayesian approach. We start with a prior probability based on our biases and update this prior probability as we begin to examine more and more data, making predictions as we go along. If physicist A had spent the past 10 years working actively on a theory of superconductivity and may secretly hope that BCS theory is wrong, s/he may start out with only 3% confidence that BCS theory is correct. On the other hand, physicist B may be completely neutral and would have a prior probability of 50%. Another physicist C would perhaps be swayed by the fact that Bardeen had just won a Nobel prize in physics for the invention of the transistor and therefore has a initial confidence level of 85% that BCS is correct. These constitute these physicists’ prior probabilities or “biases”.

What happens with time? Well, BCS predicted the existence of the Hebel-Slichter peak in the NMR spectrum, which was then observed shortly thereafter. Furthermore, Anderson showed that one could project out a particle-conserving part of the ground-state, which resolved some theoretical issues pertaining to particle-number conservation. Gorkov was also able to show that the phenomenological equations of Ginzburg and Landau were derivable from BCS theory (pdf!). McMillan and Rowell then conducted their famous experiments where they analyzed the second derivative of tunneling spectra, which exhibited phonon anomalies, to explain why lead did not obey the simple BCS formalism, but required a small extension.

As these data points accumulated, confidence in BCS theory grew for physicists A, B and C. In a Bayesian picture, we update our beliefs as we get more and more data points that are consistent with (or resolve questions pertaining to) a particular theory. Ultimately, the members of the scientific community would asymptotically approach a place where they understand the domain of validity of BCS theory and understand what it can predict. The picture I have in mind to represent this process is plotted below:


This plot is of a Bayesian updating scheme based on prior beliefs. The convergence of the viewpoints of physicists A, B and C is what is crudely meant by scientific consensus. Note that a person that starts out with a dogmatic 0% belief in the correctness of BCS theory will not change his/her mind with time.

It is important to emphasize that what I have called the 100% confidence level in my plot is meant to indicate a place where we understand the limitations and validity of a theory and how/when to apply this theory. For example, we can have 99.9% confidence that Newton’s theory of gravity will enable us to solve simple kinematics problems on the surface of the earth. While we know that Newton’s theory of gravity requires corrections from Einstein’s theory of general relativity, our confidence in Newton’s theory is not diminished when used in the correct limits. Therefore, in this Bayesian scheme, we get closer and closer to being 100% confident in a theory, but never quite reach it.

This is a rather Popperian view of scientific consensus and we know the limits of such a view in light of Kuhn’s work, but I think it nonetheless serves as a valuable guide as to how to think about the concept which is so often corrupted, especially in regard to the climate change discussion. Therefore, in the future, when people talk about scientific consensus, think convergence and think Bayes.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s