Law of Large Numbers and Central Limit Theorem

The Law of Large Numbers (LLN) says that sample averages converge to population means. The Central Limit Theorem (CLT) says that the fluctuations around convergence are approximately normal. Together, they explain why Monte Carlo simulation works, why the normal distribution appears everywhere in finance, and --- critically --- when these comforting results break down.

This article completes the probability theory module by bringing together the tools from expectation-variance-and-mgfs (moments, MGFs) and conditional-expectation (the measure-theoretic machinery). The LLN justifies Monte Carlo as a computational method; the CLT tells you how many samples you need for a given precision; and the failure modes of both theorems explain why quantitative models blew up in 2008.

Key Topics

• Weak Law of Large Numbers: ${\bar{X}}_n\stackrel{P}{\to }\mu$ --- convergence in probability. Proof via Chebyshev’s inequality
• Strong Law of Large Numbers: ${\bar{X}}_n\stackrel{\text{a.s.}}{\to }\mu$ --- almost sure convergence. The distinction matters: “eventually always close” vs “usually close”
• Central Limit Theorem: $\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}\stackrel{d}{\to }N(0,1)$. Proof sketch via moment generating functions (or characteristic functions)
• Berry-Esseen theorem: quantifies the rate of convergence in the CLT. The error is $O(1/\sqrt{n})$, with the constant depending on the third moment
• When the CLT fails:
  • Fat tails: if $E[X^2]=\infty$ (infinite variance), the CLT does not apply. Cauchy-distributed returns have no mean at all
  • Stable distributions: the generalization of the normal that serves as the limit when variance is infinite (Mandelbrot’s program)
  • Dependence: the CLT as stated requires independence. Correlated samples (e.g., autocorrelated returns) converge more slowly or to different limits

Finance Connections

• Why Monte Carlo works: to price $C_0=e^{-rT}{E}^Q[\text{payoff}]$, simulate $N$ paths, average the discounted payoffs. The LLN guarantees convergence; the CLT gives the error: $\text{SE}=\sigma /\sqrt{N}$. Halving the error requires quadrupling the samples
• Why normal appears everywhere: the CLT explains why aggregate returns (sums of many small shocks) look roughly normal, and why the normal distribution dominates financial modeling
• Why it is dangerous: daily equity returns have kurtosis $\approx 5\text{–}10$, not 3 (the normal value). The 2008 crisis exposed models --- particularly Gaussian copula CDO pricing --- that assumed normality in the tails. A “25-sigma event” under the normal is a “4-sigma event” under a fat-tailed distribution. The CLT tells you the center is normal; it says nothing about the tails, which is exactly where risk lives
• Variance reduction: since Monte Carlo error is $O(1/\sqrt{N})$, brute-force sampling is expensive. Techniques like antithetic variates, control variates, and importance sampling reduce variance without more samples --- all motivated by understanding the CLT’s error structure

Prerequisites

• conditional-expectation

---

This article is a roadmap --- content to be developed in future sessions.