10.6 Dark energy by Benjamin Crowell, Light and Matter licensed under the Creative Commons Attribution-ShareAlike license.
Until recently, physicists thought they understood gravity fairly well. Einstein had modified Newton's theory, but certain characteristics of gravitational forces were firmly established. For one thing, they were always attractive. If gravity always attracts, then it is logical to ask why the universe doesn't collapse. Newton had answered this question by saying that if the universe was infinite in all directions, then it would have no geometric center toward which it would collapse; the forces on any particular star or planet exerted by distant parts of the universe would tend to cancel out by symmetry. More careful calculations, however, show that Newton's universe would have a tendency to collapse on smaller scales: any part of the universe that happened to be slightly more dense than average would contract further, and this contraction would result in stronger gravitational forces, which would cause even more rapid contraction, and so on.
When Einstein overhauled gravity, the same problem reared its ugly head. Like Newton, Einstein was predisposed to believe in a universe that was static, so he added a special repulsive term to his equations, intended to prevent a collapse. This term was not associated with any interaction of mass with mass, but represented merely an overall tendency for space itself to expand unless restrained by the matter that inhabited it. It turns out that Einstein's solution, like Newton's, is unstable. Furthermore, it was soon discovered observationally that the universe was expanding, and this was interpreted by creating the Big Bang model, in which the universe's current expansion is the aftermath of a fantastically hot explosion. An expanding universe, unlike a static one, was capable of being explained with Einstein's equations, without any repulsion term. The universe's expansion would simply slow down over time due to the attractive gravitational forces. After these developments, Einstein said woefully that adding the repulsive term, known as the cosmological constant, had been the greatest blunder of his life.
This was the state of things until 1999, when evidence began to turn up that the universe's expansion has been speeding up rather than slowing down! The first evidence came from using a telescope as a sort of time machine: light from a distant galaxy may have taken billions of years to reach us, so we are seeing it as it was far in the past. Looking back in time, astronomers saw the universe expanding at speeds that were lower, rather than higher. At first they were mortified, since this was exactly the opposite of what had been expected. The statistical quality of the data was also not good enough to constitute ironclad proof, and there were worries about systematic errors. The case for an accelerating expansion has however been supported by high-precision mapping of the dim, sky-wide afterglow of the Big Bang, known as the cosmic microwave background. This is discussed in more detail in section 27.4.
So now Einstein's “greatest blunder” has been resurrected. Since we don't actually know whether or not this self-repulsion of space has a constant strength, the term “cosmological constant” has lost currency. Nowadays physicists usually refer to the phenomenon as “dark energy.” Picking an impressive-sounding name for it should not obscure the fact that we know absolutely nothing about the nature of the effect or why it exists.
Dark energy is discussed in more detail on p. 808.
10.6 Dark energy by Benjamin Crowell, Light and Matter licensed under the Creative Commons Attribution-ShareAlike license.