Law of Large Numbers

Let \(\{X_i\}_{i=1}^{N}\) be a sequence of identically and independently distributed random variables with a finite expected value \(\mu\). For each \(n\le N\), define the random variable \(\overline{X}_n\) as:

\[ \overline{X}_n=\frac{X_1+X_2+\cdots+X_n}{n} \]

Then,

\[ \forall\epsilon>0,\ \lim_{n\to\infty}P\left(\left|\overline{X}_n-\mu\right|\le\epsilon\right)=1 \]

For sufficiently large \(n\), the mean of independently and identically distributed (i.i.d.) \(n\) random variables will almost surely be arbitrarily close to the expected value of the individual random variables.