Chapter 4
Sampling distributions

This chapter bridges our knowledge of the probability theory to statistical inference. Sampling distributions is the key to our understanding of how a small portion of a whole can represent the whole.

4.1 Chebyshev’s theorem

For any random variable X with a finite expected value μ and finite variance σ²,

or, alternatively, by setting k = ,

When σ² is known, one of the k and 𝜖 can be arbitrarily picked.

⁰ Checkpoint
No: 58

4.2 Law of large numbers theorem

Let _i=1^N be a sequence of identically and independently distributed random variables with a finite expected value μ. For each n ≤ N, define the random variable X_n as:

Then,

For sufficiently large n, the mean of independently and identically distrbuted (iid) n random variables will almost surely be arbitrarily close to the expected value of the individual random variables.

Noting that, E = μ and Var = Then, ∀n ≤ N, ∀𝜖 > 0,

as implied by the Chebyshev’s theorem. As n approaches infinity, this expression reduces to:

which is known as the Law of large numbers and is true even when the variance of X_i is not finite.

⁰Checkpoint
No: 59

4.3 Central limit theorem

Let _i=1^N be a sequence of identically and independently distributed random variables with a finite expected value μ, and a finite and positive variance σ². For each n, define the random variable X_n as:

Let Z be a standard normal random variable. For any z ∈R, we have

Informally, CLT states that for sufficiently large n, the random variable

is approximately standard normal distributed regardless of the distribution of X_i and exactly standard normal distributed if X_i are normally distributed.

⁰Checkpoint
No: 60

4.4 Distribution of sample means

Consider _i=1ⁿ ⊂_i=1^N which is a random sample of n observations coming from a population with mean μ and variance σ². X_n being X_n =

If the population is distributed normally, then the distribution of the sample means is also normal. So,

has a standard normal distribution.

4.4.1 Essence of sampling distributions

⁰Go to Teaching page & experiment to reveal the relationship between the population and sampling distributions using the file named ‘Confidence intervals.xlsx’.

⁰ Checkpoint
No: 61

4.4.2 Distribution of sample proportions

Consider _iⁿ ⊂_i^N which is a random sample of n observations coming from a Bernoulli population. X_n being X_n = =

If n is large,

is approximately distributed as a standard normal.

⁰Checkpoint
No: 62

4.4.3 Distribution of sample variances

Let s² denote the sample variance for a random sample of n observations from a population with a variance of σ².

So,

Given a random sample of n observations from a normally distributed population whose variance is σ², the sample variance s² has a χ² distribution with (n− 1) degrees of freedom?

is distributed as the Chi-squared distribution with degrees of freedom.

⁰Checkpoint
No: 63

4.1 EXERCISES ___________________________________________________________     

1. 

We have two data sets consisting of identical values: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. We will use x_i denote the i^th value in one data set and y_j to denote the j_th value in the other data set. We construct a new data set by taking a number from each data set and finding their average, i.e.. the new data set consists of values of the form:

The constructed data set will consist of one 0, two 0.5’s, three 1’s, etc.

i. Find all values in the new data set and their corresponding frequencies

ii. Construct a graph that summarize your finding

iii. Find the mean and variance of the initial data set and the new data set

Solution: Solve yourself to explore the Central Limit Theorem.

2. 

We make 100 independent observations from a population with mean 40 and standard deviation 20. Approximately, what is the probability that the mean of these observations will be greater than 37?

Solution: Population X ∼⋅(40, 20²). We take n = 100 observations and calculate X₁₀₀.

3. 

The following table gives the relative frequency distribution of a population:

Value	Rel. Freq.
2	0.1
4	0.3
6	0.2
8	0.3
10	0.1

i. A number is selected from this population at random, what is the probability that the number selected is greater than or equal to 8?

ii. If we select two numbers at random (with replacement), what is the probability that the mean of these two numbers is less than or equal to 5?

iii. If 25 numbers are selected from the population at random (with replacement), what is the probability (approximately) that the mean these 25 numbers is less than 6.5?

Solution:

1.

The only trick in this exercise is to begin with calculating E(X) and Var(X). Calculate and see E(X) = 6 and Var(X) = 5.6. There is no sampling as n = 1. Simply calculate P(X ≥ 8).

2.

Calculate P.

3.

Calculate P. Remember that E = μ and Var = σ²/n.

4. 

We choose 36 numbers, with replacement, at random (i.e.., we take a random sample of size 36) from the interval . Let X be the random variable that assigns to each sample (outcome) the mean of the sample.

i. Find the expected value and variance of X

ii. Find (an approximate value for) the probability that the sample mean, i.e.. X, will be less than or equal to 2.3

Solution:

1.

Since nothing further is instructed, assume that the population is Uniform(0, 4). Then,

X, here, is the mean of our 36 observations, i.e., X₃₆ in our usual notation.

2.

Using the parameters in part (i), calculate P(X ≤ 2.3), i.e., P(z ≤ 1.56). The answer is 0.94062.

5. 

We choose 9 numbers from a normally distributed population of numbers. The mean of the population is unknown but the variance is know to be equal to 16. If μ denotes the mean of the population, then what is the probability that the mean of the 9 numbers that we choose will be in the interval ?

Solution: You do not need the value of μ in this exercise. The key to solution is that Var = 16/9. So, performing the intermediate steps, the problem reduces to finding P(−1.5 ≤ z ≤ 1.5) and the answer is 0.86638.

6. 

In a certain university the CGPA’s of students only takes the values 0, 1, 2, 3, 4. The distribution of CGPA’s of students of this university is given below:

CGPA	Freq
0	5, 000
1	10, 000
2	20, 000
3	5, 000
4	10, 000
Total	50, 000

i. Let X₁ denote the CGPA of a student who was chosen at random from the population of all students of this university. Tabulate the PDF of this random variable.

ii. Let X₂ denote the average (mean) of the CGPA’s of two randomly selected students from the population of all students of this university. What is the probability that the average CGPA of the two students is less than or equal to 1?

iii. Now we choose 36 students at random. What is the probability, approximately, that the average CGPA of these students (X₃₆) is less than or equal to 2.3?

Solution: This exercise is left as self-study.

7. 

Consider a large population of which only 20% know basic concepts of statistics. We take a random sample of size 81 from this population and count the number of individuals, in the sample, who knows basic concepts of statistics. What is the probability that the sample will have between 15 and 18 (inclusive) individuals who know basic concepts of statistics?

Solution: This exercise is left as self-study.

8. 

A four sided fair die is rolled several times and we calculate the average (mean) of the values observed.

i. Let X₁ denote the random variable that gives the value observed when the die is rolled once. Find the expected value and variance of X₁.

ii. Let X_n denote the mean of the values observed when the die is rolled n times. What is the minimum number of times that the die should be rolled so that the mean (the value of X_n) takes a value in the interval with at least a probability of 0.9? Use Chebyshev′sTheorem to answer this problem.

iii. Using the CentralLimitTheorem and the value for n you found above, find an approximate value for the probability of X_n taking a value in the interval .

iv. Do the answers you found in item and item contradict each other? If there is a difference in what the two answers suggest, explain the reason for this.

Solution: i. When the die is rolled once, by definition we produce an outcome directly of the population (that is we don’t do any sampling at all). So,

So, X₁ ∼⋅(2.5, 1.25)

ii. X_n is the RV denoting the sample mean, when We roll the die n times.

Midpoint of the interval [2.1, 2.9] is 2.5

Notice that in this part, we use the Chebyshev’s theorem only. We get:

and

iii.

Notice that in this part, we use the CLT only.

iv. No, they don’t contradict each other: While Chebyshev’s theorem sets a lowerbound of 0.90 here (in ii), (iii) gives the actual probability as 0.99852 which is larger than 0.90 (as it should be).

Though I am not a fan of such hints, here a useful hint may be underlined: When an interval is given in a question like this, always begin with observing (calculating) its midpoint. In this question, the midpoint was (2.1 + 2.9)/2 = 2.5, which is nothing but E and E. Once you have noticed this, it will be trivial to find 𝜖 to figure out the rest of the steps.

One additional point is:

So, sampling with n = 1 simply refers to population’s distribution.

⁰Checkpoint
No: 64