When I first learned quantum mechanics, I was told that we knew that the photon existed because of Einstein’s explanation of the photoelectric effect in 1905. As the frequency of light impinging on the cathode material was increased, electrons came out with higher kinetic energies. This led to Einstein’s famous formula:
where is the kinetic energy of the outgoing electron, is the photon energy and is the material-dependent work function.
Since the mid-1960s, however, we have known that the photoelectric effect does not definitively imply the existence of photons. From the photoelectric effect alone, it is actually ambiguous whether it is the electronic levels or the impinging radiation that should be quantized!
So, why do we still give the photon explanation to undergraduates? To be perfectly honest, I’m not sure whether we do this because of some sort of intellectual inertia or because many physicists don’t actually know that the photoelectric effect can be explained without invoking photons. It is worth noting that Willis E. Lamb, who played a large role in the development of quantum electrodynamics, implored other physicists to be more cautious when using the word photon (see for instance his 1995 article entitled Anti-Photon, which gives an interesting history of the photon nomenclature and his thoughts as to why we should be wary of its usage).
Let’s return to 1905, when Einstein came up with his explanation of the photoelectric effect. Just five years prior, Planck had heuristically explained the blackbody radiation spectrum and, in the process, evaded the ultraviolet catastrophe that plagued explanations based on the classical equipartition theorem. Planck’s distribution consequently provided the first evidence of “packets of light”, quantized in units of . At the time, Bohr had yet to come up with his atomic model that suggested that electron levels were quantized, which had to wait until 1913. Thus, from Einstein’s vantage point in 1905, he made the most reasonable assumption at the time — that it was the radiation that was quantized and not the electronic levels.
Today, however, we have the benefit of hindsight.
According to Lamb’s aforementioned Anti-Photon article, in 1926, G. Wentzel and G. Beck showed that one could use a semi-classical theory (i.e. electronic energy levels are quantized, but light is treated classically) to reproduce Einstein’s result. In the mid- to late 1960’s, Lamb and Scully extended the original treatment and made a point of emphasizing that one could get out the photoelectric effect without invoking photons. The main idea can be sketched if you’re familiar with the Fermi golden rule treatment to a harmonic electric field perturbation of the form , where is the frequency of the incoming photon. In the dipole approximation, we can write the potential as and we get that the transition rate is:
Here, is the difference in energies between the initial and final states. Now, there are a couple things to note about the above expression. Firstly, the term in brackets (containing the sinusoidal function) peaks up when . This means that when the incoming light is resonant between the ground state and a higher energy level, the transition rate sharply increases.
Let us now interpret this expression with regard to the photoelectric effect. In this case, there exists a continuum of final states which are of the form , and as long as , where is the work function of the material, we recover , where represents the energy given to the electron in excess of the work function. Thus, we recover Einstein’s formula from above!
In addition to this, however, we also see from the above expression that the current on the photodetector is proportional to , i.e. the intensity of light impinging on the cathode. Therefore, this semi-classical treatment improves upon Einstein’s treatment in the sense that the relation between the intensity and current also naturally falls out.
From this reasoning, we see that the photoelectric effect does not logically imply the existence of photons.
We do have many examples that non-classical light does exist and that quantum fluctuations of light play a significant role in experimental observations. Some examples are photon anti-bunching, spontaneous emission, the Lamb shift, etc. However, I do agree with Lamb and Scully that the notion of a photon is indeed a challenging one and that caution is needed!
A couple further interesting reads on this subject at a non-technical level can be found here: The Concept of the Photon in Physics Today by Scully and Sargent and The Concept of the Photon – Revisited in OPN Trends by Muthukrishnan, Scully and Zubairy (pdf!)