Variance and Covariance

• A mean does not give adequate description of the shape of a random variable (probability distribution).
• We need to characterize the variability (or dispersion) of the random variable $X$ in the distribution.

  Figure 1: Distributions with equal means and unequal dispersions.

  
  
• Definition 4.3:
  

• Example 4.8:Let the random variable $X$ represent the number of automobiles that are used for official business purposes on any given workday.
• The probability distribution for company A and B is as follows.

  $x$	1	2	3
  $f(x)$	0.3	0.4	0.3

  $x$	0	1	2	3	4
  $f(x)$	0.2	0.1	0.3	0.3	0.1

• Show that the variance of the probability distribution for company B is greater than that of company A.
• Solution:
  $$\mu_A=E(X)=1*0.3+2*0.4+3*0.3=2.0$$
  $$\sigma_A^2=\sum_{x=1}^3 (x-2.0)^2f(x)=(1-2)^2*0.3+(2-2)^2*0.4+(3-2)^2*0.3=0.6$$
  $$\mu_B=2.0~ \&~ \sigma_B^2=1.6$$

• Theorem 4.2:
  
• Example 4.9: Let the random variable $X$ represent the number of defective parts for a machine when 3 parts are sampled from a production line and tested.
• Calculate $\sigma^2$ using the following probability distribution.

  $x$	0	1	2	3
  $f(x)$	0.51	0.38	0.10	0.01

• Solution:
  $$\mu=E(X)=0*0.51+\ldots=0.61$$
  $$E(X^2)=\sum_{x=0}^3 x^2f(x)=0^2*0.51+\ldots=0.87$$
  $$\sigma^2=E(X^2)-\mu^2=0.87-0.61^2=0.4979$$

• Theorem 4.3:
  
• Example 4.11: Calculate the variance of $g(X)=2X+3$, where $X$ is a random variable with probability distribution.

  $x$	0	1	2	3
  $f(x)$	1/4	1/8	1/2	1/8

• Solution:
  $$\mu_{2X+3}=E(2X+3)=\sum_{x=0}^3 (2X+3)f(x)= 6$$
  $$\sigma^2_{2X+3}=E\left\lbrace [2X+3-\mu_{2X+3}]^2 \right\rbrace =E\left\lbrace [2X+3-6]^2 \right\rbrace$$
  $$=E(4X^2-12X+9)=\sum_{x=0}^3 (4X^2-12X+9)f(x)=4$$

• Definition 4.4:
  
• The covariance between two random variables is a measurement of the nature of the association between the two.
• The sign of the covariance indicates whether the relationship between two dependent random variables is positive or negative.
• When $X$ and $Y$ are statistically independent, it can be shown that the covariance is zero.
• The converse, however, is not generally true. Two variables may have zero covariance and still not be statistically independent.

• The covariance only describe the linear relationship between two random variables.
• If a covariance between $X$ and $Y$ is zero, $X$ and $Y$ may have a nonlinear relationship, which means that they are not necessarily independent.
• Theorem 4.4:
  

• Definition 4.5:
  
• Exact linear dependency: $Y = a + bX$
  $$\rho_{XY} =1, ~ if~ b> 0 ~~;~~ \rho_{XY} =-1, ~ if~ b< 0$$