The normal distribution and the Galton board

What is the normal distribution?

Picture a board studded with pegs in a triangle, and a ball dropped in at the top. Every time the ball hits a peg it bounces left or right, more or less at random, like flipping a coin. After a whole run of these little bounces it drops into one of the bins along the bottom.

One ball on its own is anyone's guess. But drop thousands, and something tidy happens. They pile up tall in the middle and thin out towards the edges. The heap always comes out the same shape: a smooth hump that people call the bell curve.

Why the middle? To land far out on the left, a ball has to bounce left nearly every single time, which almost never happens. Most balls get a rough mix of lefts and rights, and that mix lands them near the centre. The lopsided runs are rare, so the ends stay short.

This is not just a toy. Lots of things in the real world are built from many small, independent nudges added together, and they end up bell-shaped too. People's heights, scores on a big test, and the little errors you get when you measure the same thing over and over all tend to look like this same friendly hump.

Why is the bell curve everywhere?

The board has a name, the Galton board, after Francis Galton, and it is really a probability machine you can watch. Say there are \(n\) rows of pegs. At each row the ball makes a fair left-or-right choice, a fifty-fifty bounce, and those choices are independent of each other. Count how many times it went right and you know which bin it lands in. So the bin a ball ends up in follows the binomial distribution: the chance of exactly \(k\) rights out of \(n\) rows is

\[ P(k) = \binom{n}{k}\left(\tfrac{1}{2}\right)^{n}. \]

Those binomial heights are the bars in the histogram. Here is the striking part. As \(n\) grows, the jagged binomial staircase smooths into a continuous curve, the normal, or Gaussian, distribution. Two numbers describe it. The mean \(\mu\) is where the peak sits, the middle of the pile. The standard deviation \(\sigma\) measures how wide it spreads. Its formula is

\[ f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-(x-\mu)^2/2\sigma^2}. \]

That exponential of a negative square is what gives the curve its shape: high in the middle where \(x\) is near \(\mu\), and falling away fast as you move out, because the exponent grows with the square of the distance.

The spread comes with a handy rule of thumb, the 68-95-99.7 rule. About 68 percent of the values land within one \(\sigma\) of the mean, about 95 percent within two, and about 99.7 percent within three. So a value more than three standard deviations from the middle is genuinely unusual, roughly three in a thousand.

This is why so many measured quantities come out bell-shaped. A person's height, for instance, is the combined result of many genes and many environmental factors, each adding a small push one way or the other. Whenever a quantity is a sum of lots of small independent contributions, the Galton board is a fair picture of what is going on, and the normal curve is what you should expect to see.

The central limit theorem

The theorem behind the curve

The reason the normal turns up again and again is the Central Limit Theorem. Take independent random variables \(X_1, X_2, \dots, X_n\) drawn from the same distribution, with finite mean \(\mu\) and finite variance \(\sigma^2\). Form their average \(\bar{X}_n = \tfrac{1}{n}\sum_i X_i\). The theorem says that as \(n \to \infty\), the standardised average \(\sqrt{n}\,(\bar{X}_n - \mu)/\sigma\) converges in distribution to the standard normal \(\mathcal{N}(0,1)\), whatever the shape of the original \(X_i\). The individual pieces can be skewed, lumpy, or discrete; the sum forgets all of that and tends to the same bell curve.

How fast the average tightens

The average has standard deviation \(\sigma/\sqrt{n}\), the standard error. That \(\sqrt{n}\) in the denominator is worth dwelling on. To halve the spread of your estimate you need four times as much data, not twice. It is why large samples are steadier than small ones, and why polling a few thousand people can pin a proportion down surprisingly well. The Galton board is the theorem made physical: each ball is a sum of \(n\) independent left-or-right steps, and the pile is the distribution of that sum, already close to normal by a dozen rows.

Where it breaks

The theorem leans on that finite-variance assumption, and it is not a formality. If the underlying variables have heavy tails, so heavy that the variance is infinite, the classical Central Limit Theorem does not apply and the averages do not settle to a normal. The Cauchy distribution is the standard cautionary case: the average of \(n\) Cauchy variables has the same Cauchy distribution as a single one, no matter how large \(n\) gets, so it never concentrates. Real data with rare but enormous outliers, some financial returns for example, can behave this way, which is exactly when assuming a bell curve leads you badly astray. The normal is the default for sums of well-behaved, finite-variance pieces, and it is worth checking that your pieces really are that well-behaved.