PRACTICAL MATHEMATICS

Statistics

Population Statistics
Histograms
Analysis of Variance
Standardized Variables
Two Dimensional Populations
Analysis of Variance in Two Dimensions

Population Statistics

A complete set of observations is called a population. A statistic is a number that summarizes some property of these observations.

EXAMPLE

Twenty people are male or female as follows:

F F F F M M F F M F F M F M F M M F F F

This population is divided into two categories M and F with frequencies:

7 males, 13 females.

The frequency distribution may be expressed as percentages (35% males, 65% females) or as relative frequencies:

f₁ = 0.35, f₂ = 0.65.

In a relative frequency distribution with n categories we have:

f₁ + ... + f_n = 1.

EXAMPLE

The following ten numbers make a population:

1.7, 4.5, 3.8, 2.7, 0.6, 4.8, 1.1, 5.7, 3.4, 2.2.

Statistics commonly used with populations of numbers x₁, ... , x_n are the mean

mu = (x₁ + ... + x_n)/n,

and the variance

sigma² = [(x₁ - mu)² + ... + (x_n - mu)²] / n.

The statistic sigma is called the standard deviation. The mean is a measure of the center of the population, and the standard deviation is a measure of the scattering of the numbers about the mean. The most convenient formula for calculating the variance is:

sigma² = (x₁² + ... + x_n²)/n - mu².

EXERCISE

Show that for the ten numbers in the above example: mu = 3.05, and sigma = 1.59.

These results can be shown in a diagram as in Fig. 1.

Fig. 1.

Fig. 1. A population of ten numbers with their mean and standard deviation.

EXERCISE

Show that the mean square deviation s² of the numbers x₁, ..., x_n from the value x, defined by

s² = [(x₁ - x)² + ... + (x_n - x)²]/n,

is a minimum when x = mu. The minimum value of s² is the variance sigma².

Histograms

Observed numerical data may be grouped into classes with a given class interval. The number of observations in each class is a frequency distribution of the data.

A histogram is a diagram which consists of rectangles with areas proportional to the relative frequencies of the classes.

EXAMPLE

Table 1. One Hundred Numbers Grouped into Classes
Class No. i	Class Interval	Central Value x_i	Frequency n_i	Relative Frequency f_i
1	40 - 50	45	5	0.05
2	50 - 60	55	18	0.18
3	60 - 70	65	42	0.42
4	70 - 80	75	27	0.27
5	80 - 90	85	8	0.08

Fig. 2.

Fig. 2. A histogram of the data in Table 1. The symbol above the histogram shows the mean and standard deviation of the data.

The mean and standard deviation of the data are calculated by summations over the classes as follows:

n = n₁ + ... + n₅, f_i = n_i/n
mu = f₁x₁ + ... + f₅x₅ = (n₁x₁ + ... + n₅x₅)/n,
sigma² = f₁(x₁ - mu)² + ... + f₅(x₅ - mu)² = (n₁x₁² + ... + n₅x₅²)/n - mu².

EXERCISE

Show that, for the data in Table 1, mu = 66.5, sigma = 9.73.

Analysis of Variance

Suppose a population of n measurements x_i,j is divided into m categories of sizes n_i having means mu_i and variances sigma_i², where i = 1, ..., m, as follows:

mu_i = (x_i,1 + ... + x_i,ni)/n_i,
sigma_i² = [(x_i,1 - mu_i)² + ... + (x_i,ni - mu_i)²]/n_i = (x_i,1² + ... + x_i,ni²)/n_i - mu_i².

For the whole population we have:

n = n₁ + ... + n_m,
mu = [(x_1,1 + ... + x_1,n1) + ... + (x_m,1 + ... + x_m,nm)]/n = [n₁mu₁ + ... + n_mmu_m]/n,
sigma² = [((x_1,1 - mu)² + ... + (x_1,n1 - mu)²) + ... + ((x_m,1 - mu)² + ... + (x_m,nm - mu)²)]/n = [n₁sigma₁² + ... + n_msigma_m²]/n + [n₁(mu₁ - mu)² + ... + n_m(mu_m - mu)²]/n.

The first term in the last expression for sigma² is the mean of the category variances, and the second term is the variance of the category means.

EXERCISE

Prove the formula:

sigma² = [n₁sigma₁² + ... + n_msigma_m²]/n + [n₁(mu₁ - mu)² + ... + n_m(mu_m - mu)²]/n.

EXAMPLE

Table 2. Twenty Numbers Divided into Three Categories
Number and Category	4.0	4.5	0.5	3.5	4.9	5.6	9.4	1.6	1.0	3.4
Number and Category	A	A	A	A	B	A	C	A	C	A
Number and Category	0.5	7.1	7.9	5.1	4.3	5.5	8.2	9.8	2.2	3.3
Number and Category	B	C	A	C	B	B	C	A	B	B

Using the above formulas we get:

Table 3. Statistics of the Twenty Numbers Divided into Three Categories
Category	n_i	mu_i	sigma_i²	sigma_i
A	9	4.533	7.524	2.743
B	6	3.433	2.892	1.701
C	5	6.160	8.658	2.943
Whole population	20	4.610	7.437	2.727
Mean of category variances = 6.418
Variance of category means = 1.019

Figure 3 shows the mean and standard deviation of each category and the whole population. The variance of the category means is much smaller than the mean of the category variances, so the division of this population into three categories may not be significant.

Fig. 3.

Fig. 3. Statistics of the categories and the whole population for the data in Table 2. The symbols above the diagrams show the means and standard deviations of the data.

EXERCISE

Analyse the data below in this way. Is the division of the population into two categories likely to be significant?

x:	3.5	6.1	13.4	6.9	16.8	2.9	10.1	8.7	16.6	15.2	4.0	6.7
Category	G	G	H	G	H	G	G	H	H	H	G	G

Standardized Variables

We may represent each measurement x by its position relative to the mean mu as a multiple of the standard deviation sigma. This gives us the measurement in the standardized form

z = (x - mu)/sigma.

A distribution of standardized observations has mean mu_z = 0, and standard deviation sigma_z = 1.

EXERCISE

Calculate the standardized values

z_i = (x_i - mu)/sigma

of the numbers

1.7, 4.5, 3.8, 2.7, 0.6, 4.8, 1.1, 5.7, 3.4, 2.2.

and check that mu_z = 0 and sigma_z = 1 by direct calculation.

Two Dimensional Populations

Suppose a population consists of a number of points (x₁, y₁), ... , (x_n, y_n) in two dimensions.

EXAMPLE

Table 4. A Two-Dimensional Population
x:	6.24	6.42	6.56	6.41	6.57	6.64	6.62	6.76	6.72	6.76
y:	3.21	3.26	3.39	3.31	3.41	3.21	3.29	3.31	3.39	3.39

x:	6.62	6.72	6.92	6.94	6.84	6.88	6.82	6.98	7.16	7.08
y:	3.43	3.51	3.35	3.31	3.44	3.49	3.51	3.51	3.52	3.54

These points may be plotted on a scatter diagram as in Fig. 4.

Fig. 4.
Fig. 4. Scatter diagram of data points in two dimensions. The centroid and a line summarizing the data points are also shown.

We may also compute for the variables x and y the means mu_x, mu_y and the variances sigma_x², sigma_y² as before thus:

mu_x = (x₁ + ... + x_n)/n
sigma_x² = (x₁² + ... + x_n²)/n - mu_x²
mu_y = (y₁ + ... + y_n)/n
sigma_y² = (y₁² + ... + y_n²)/n - mu_y²

EXAMPLE

For the data in Table 4 we have:

mu_x = 6.73
sigma_x² = 0.0510
sigma_x = 0.2258
mu_y = 3.39
sigma_y² = 0.0105
sigma_y = 0.1024

The data in this example can be summarized by the centroid (mu_x, mu_y), and the line (see Fig. 4) given by the equation

(x - mu_x)/sigma_x = (y - mu_y)/sigma_y.

Another statistic is the covariance sigma_xy defined by:

sigma_xy = [(x₁ - mu_x)(y₁ - mu_y) + ... + (x_n - mu_x)(y_n - mu_y)]/n = (x₁y₁ + ... + x_ny_n)/n - mu_xmu_y.

This is a measure of how much changes in x are associated with changes in y. If sigma_xy > 0 then x and y tend to increase or decrease together. If sigma_xy < 0, then y tends to decrease as x increases, and vice versa. If sigma_xy = 0, or is very small, then x and y tend to be independent of each other.

Because the magnitude of the covariance depends on the dispersion of x and y, a better measure of these associations is given by the correlation coefficient rho_xy. The correlation coefficient is the covariance of the data in standardized form:

rho_xy = sigma_xy/(sigma_x.sigma_y).

The correlation coefficient may also be calculated from the formula:

rho_xy = [n.Sum(xy) - Sum(x).Sum(y)] / sqrt[(n.Sum(x²) - (Sum x)²).(n.Sum(y²) - (Sum y)²)],

where Sum(xy) = x₁y₁ + ... + x_ny_n, Sum(x) = x₁ + ... + x_n, and so on.

The correlation coefficient can have values in the range -1 to +1.

EXAMPLE

For the data in Table 4 we obtain:

sigma_xy = 0.0158 and rho_xy = 0.682.

EXERCISE

Draw a scatter diagram and calculate the means, variances, covariance, and correlation coefficient for the following data.

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

Analysis of Variance in Two Dimensions

Standardized Scatter Diagram

In order to analyse the relation between x and y in two-dimensional populations we plot the scatter diagram in standardized form. The standardized variables for x and y are

z_x = (x - mu_x)/sigma_x, z_y = (y - mu_y)/sigma_y.

The left diagram in Fig. 5 shows the standardized scatter diagram for the points in Table 4. The centroid of the points in Fig. 5(left) is the origin z_x = 0, z_y = 0, and the standard deviations are sigma_zx = 1, sigma_zy = 1. The correlation coefficient is rho_zx,zy = 0.682, which is the same as rho_xy.

Fig. 5.

Fig. 5. Left: Standardized plot of the data in Table 4. Right: The same data plotted relative to the principle axes.

Principle Axes

The uv-axes at angles 45° to the z_xz_y-axes in Fig. 5(left) are the principle axes for the population. Relative to these axes the data in the population are represented by points (u,v) as in the right diagram in Fig. 5. The points (u,v) have the following statistics:

mu_u = 0
mu_v = 0
sigma_u² = 1 + rho_xy
sigma_v² = 1 - rho_xy
rho_uv = 0

In this example the variances of u and v are therefore:

sigma_u² = 1.682, sigma_v² = 0.318.

These statistics give an alternative description of the standardized data in terms of the uncorrelated variables u and v relative to the principle axes.

Analysis of Variance

The total variance of the standardized two-dimensional population may be defined as:

sigma_total² = sigma_zx² + sigma_zy² = 2.

The total variance is unchanged by a rotation of the axes. Therefore, we also have

sigma_total² = sigma_u² + sigma_v² = 2.

When rho_xy > 0, we have sigma_u² > sigma_v². We then think of sigma_u² as the variance in the direct relation between x and y, and sigma_v² as the variance of the scatter due to other causes outside the relation between x and y.

If rho_xy < 0, we have sigma_u² < sigma_v². In this case we think of sigma_v² as the variance in the inverse relation between x and y, and sigma_u² as the variance of the scatter due to other causes.

For the data in Table 4, rho_xy > 0, and we have

sigma_u²/sigma_total² = 1.682/2 = 0.841,
sigma_v²/sigma_total² = 0.318/2 = 0.159.

Therefore 84.1% of the total variance is in the relation between x and y, and 15.9% of the total variance has causes outside this relation.

EXERCISE

Draw a standardized scatter diagram for the following data. Analyze the total variance of the data.

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

Home Page

By R. H. B. Exell, 2001, modified 2003. King Mongkut's University of Technology Thonburi.

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

Statistics

Contents

Population Statistics

EXAMPLE

EXAMPLE

EXERCISE

EXERCISE

Histograms

EXAMPLE

EXERCISE

Analysis of Variance

EXERCISE

EXAMPLE

EXERCISE

Standardized Variables

EXERCISE

Two Dimensional Populations

EXAMPLE

EXAMPLE

EXAMPLE

EXERCISE

Analysis of Variance in Two Dimensions

Standardized Scatter Diagram

Principle Axes

Analysis of Variance

EXERCISE

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152

x:	10	17	8	20	14	10	6	15	18	13	15	12
y:	155	150	180	135	156	168	178	160	132	145	139	152