Mathematical Statistics

Exercises

1 Review Problems in Probability Theory

Elementary Probability Calculations

1. A fair coin is tossed ten times. Find the probability of each of the following events: (a) the number of heads equals the number of tails.

(b) the number of heads is either 3 or 4.

(c) any two consecutive results are distinct.

(d) the first two tosses give the same result, the second two tosses give the same

result, etc.

2. Three dice are tossed. Find the probability that the sum of the points will be 3. Same for 4; for 5.

3. Six cards are drawn randomly from a full deck of 52 cards. Find the probability of: (a) no aces.

(b) exactly two aces and three kings.

(c) two aces, two kings and two queens.

4. Fifty people are chosen randomly. What is the probability that no two of them have the same birthday?

5. An urn U_{1} contains six white and three black balls and an urn U_{2} contains five white and two black balls.

(a) One ball is drawn from each urn. What is the probability that one of the balls is white and the other black?

(b) An urn is chosen randomly and two balls are drawn from it without replacement. What is the probability that one of them is white and the other black?

(c) Suppose in the experiment described in (b) both balls turned out to be black. What is the probability that U_{1} was the selected urn?

6. Reuven is tossing n + 1 coins and Shimon n coins. What is the probability for Reuven to have (strictly) more heads than Shimon?

7. Let Z_{p} be the field {0, 1, …, p−1} with addition and multiplication modulo p, where p is a prime. We select randomly n elements a_{1}, a_{2}, …, a_{n} from Z_{p}.

(a) Find the probability that ∑_{i=1}^{n} a_{i} = 0.

(b) Find the probability that ∏_{i=1}^{n} a_{i} = 0.

(c) Find the probability that both ∑_{i=1}^{n} a_{i} = 0 and ∏_{i=1}^{n} a_{i} = 0.

8. Let F be a finite field with s elements, and n a fixed positive integer. An n × n matrix A over F is randomly selected.

(a) Find the probability for A to be invertible.

(b) For any f ∈ F find the probability P (det(A) = f).

9. A family has two children. Find the probability for both to be boys if it is known that at least one is a boy.

10. Two dice are tossed. What is the probability of the sum of the points to be 12 if it is known to be at least 10?

11. The numbers 1,2,3,4 are marked on the faces of a regular tetrahedron. In addition to the tetrahedron we have two dice. One of the three bodies is selected randomly and tossed twice.

(a) Find the probability for at least one 3.

(b) If it is known that at least one of the tosses resulted in a 3, what is the probability that the tetrahedron was selected?

Distribution and Density Functions

12. The following two functions are probability functions of certain random variables. In each case find the parameter c and calculate the distribution function F:

(a)

p(x) = \{ \begin{matrix}c(10 − x), & x = 0, 1, …, 9, \\ 0, & otherwise. \end{matrix}

(b)

p(x) = \{ \begin{matrix}\frac{c}{x(x+2)}, & x = 1, 2, …, \\ 0, & otherwise. \end{matrix}

(c)

p(x) = \{ \begin{matrix}\frac{c}{x(x+1)(x+2)}, & x = 1, 2, …, \\ 0, & otherwise. \end{matrix}

p(x) = \{ \begin{matrix}\frac{c}{x}, & x = 2^{k}3^{l}, k, l = 0, 1, …, \\ 0, & otherwise. \end{matrix}

13. A mixture of two radioactive elements contains 10^{20} atoms of element A and 10^{21} atoms of element B. The probability of disintegration of a given atom of element A within one second is 2 · 10^{−21}, and that of a given atom of element B is 8 · 10^{−22}. Distinct atoms are independent.

(a) Find an exact expression for the probability that within one second exactly three atoms in the mixture will disintegrate.

(b) Find a convenient good estimate for this probability.

14. The following functions are density functions of certain random variables. In each case find the parameter c and calculate the distribution function F:

(a)

f(x) = \{ \begin{matrix}cx^{3}, & 3 ≤ x ≤ 5, \\ 0, & otherwise. \end{matrix}

(b)

f(x) = \{ \begin{matrix}c(e^{−2x} + e^{−3x}), & x ≥ 0, \\ 0, & x < 0. \end{matrix}

(c)

f(x) = c(e^{−2|x|} + e^{−3|x|}), −∞ < x < ∞ .

(d)

f(x) = \{ \begin{matrix}0, & x < −1, \\ −2cx, & −1 ≤ x ≤ 0, \\ 3ce^{−x}, & 0 < x. \end{matrix}

(e)

f(x) = c(2e^{−x^{2}} + 3e^{−2x^{2}}), −∞ < x < ∞ .

(Here find c explicitly, but express F in terms of Φ – the distribution function of the standard normal distribution.)

Independence

15. Show that for any k there exist k dependent random variables, any k − 1 of which are independent.

16. Explain without performing any calculations why, if X ∼ B(n_{1}, p) and Y ∼ B(n_{2}, p) are independent, then X + Y ∼ B(n_{1} + n_{2}, p).

17. Show that, if X ∼ P (λ) and Y ∼ P (μ) are independent, then X + Y ∼ P (λ + μ).

18. Let X, Y be as in the preceding question. Show that the conditional probability function of X given that X + Y = n is B (n, \frac{λ}{λ + μ}).

19. Let X ∼ G(p_{1}) and Y ∼ G(p_{2}) be independent. Find the probability function of X + Y.

20. Do the same if X ∼ U(2, 3) and Y ∼ U(3, 5).

21. Find the density function of X/Y if X, Y ∼ U(0, 1) are independent.

22. Let X_{i} ∼ Exp(θ), i = 1, 2, …, n, be independent. Find the density function of ∑_{i=1}^{n} X_{i}.

Expectation, Variance, Covariance

23. Find E(X) and V (X) if:

(a) X ∼ B(300, \frac{1}{6}).

(b) X ∼ P (5).

(c) X ∼ G(0.3).

(d) X is distributed uniformly on the integers between 5 and 14.

(e) X ∼ U(5, 14).

(f) X ∼ Exp(4).

(g) X ∼ N(−5, 13).

24. Find E(X) and V (X) if X is distributed according to the distributions defined in questions 12 and 14.

25. Find E(X^{3}) if:

(a) X ∼ B(n, p).

(b) X ∼ P (λ).

(c) X ∼ G(p).

(d) X ∼ U(a, b).

(e) X ∼ Exp(θ).

(f) X ∼ N(μ, σ^{2}).

26. Find:

(a) ρ(X^{2}, X^{3}) if X ∼ U(0, 1).

(b) ρ (X, e^{X}) if X ∼ U(0, 1).

(c) ρ(X, X^{2}) if X ∼ P (λ).

27. An urn contains a white balls and b black balls. Suppose k random balls are taken out of the urn without replacement, and then l more are taken out, where k + l ≤ min(a, b). Let X and Y denote the number of white balls in the first and the second batch, respectively.

(a) Calculate Cov(X, Y).

(b) Calculate ρ(X, Y).

(c) Is ρ(X, Y) positive or negative? Why should you have expected this without performing any calculations?

28. Let X, Y_{1}, …, Y_{k}, Z_{1}, …, Z_{k} be independent random variables, each with mean 0 and finite variance σ^{2}. The random variable N is independent of all these variables, and its probability function is given by:

p_{N}(j) = \{ \begin{matrix}p_{j}, & j = 1, 2, …, k \\ 0, & otherwise, \end{matrix}

(where p_{j} ≥ 0, ∑_{j=1}^{k} p_{j} = 1). Define random variables U, V by:

U = X + ∑_{j=1}^{N} Y_{j}, V = X + ∑_{j=1}^{N} Z_{j} .

(a) Show that N, UV are not independent, yet uncorrelated.

(b) Find ρ(U, V).

Convergence in Probability and in Distribution

29. Show that if a sequence of random variables converges in probability then the limit is unique.

30. Show that if a sequence of random variables converges in distribution to a constant then it converges to this constant in probability; however, the implication is false in general (i.e., for convergence to a non-constant random variable).

31. Let (X_{n}) be a sequence of random variables with E(X_{n}) = μ_{n} and V (X_{n}) = σ_{n}^{2}. Suppose that Cov(X_{m}, X_{n}) ≤ 0 and that ∑_{i=1}^{n} σ_{i}^{2}/n^{2} ⟶_{n→∞} 0. Then:

\frac{X_{1} + X_{2} + … + X_{n}}{n} − \frac{μ_{1} + μ_{2} + … + μ_{n}}{n} ⟶_{n→∞}^{P} 0 .

32. Let (X_{n}) and (Y_{n}) be two sequences of random variables. Suppose that X_{n} ⟶_{n→∞}^{D} X

and Y_{n} ⟶_{n→∞}^{D} Y. Do we necessarily have X_{n} + Y_{n} ⟶_{n→∞}^{D} X + Y ?

33. A sequence (F_{n}) of distribution functions is defined by:

F_{n}(x) = \{ \begin{matrix}0, & x ≤ −n, \\ \frac{x+n}{2n}, & −n < x < n, \\ 1, & x ≥ n. \end{matrix}

Show that the sequence converges pointwise to a function F. Is F a distribution function?

Central Limit Theorem

34. Let (X_{n})_{n=1}^{∞} be a sequence of i.i.d. random variables with mean μ and variance σ^{2}. Let (Y_{n})_{n=1}^{∞} be a sequence of random variables, each taking the values ±1 with probabilities \frac{1}{2}, independent of the X_{n}’s. Form the sequence:

S_{n} = Y_{1}X_{1} + Y_{2}X_{2} + … + Y_{n}X_{n}, n = 1, 2, ….

Show that S_{n}/\sqrt{n} is asymptotically normal, and find its asymptotic mean and variance.

35. A random sample of size 54 is drawn from a population distributed according to the probability function

P (x) = \{ \begin{matrix}\frac{1}{3}, & x = 3, 5, 8, \\ 0, & otherwise. \end{matrix}

Estimate the probability for the sample mean to be between 5 and 5.2.

36. A random sample of size 200 is drawn from an Exp(2) population. Find a d such that the probability for the sample mean to deviate from the population mean by more than d is approximately 0.05.

2 Estimation

Method of Moments

37. Use the method of moments to determine the estimator of the parameter v in the Rayleigh distribution, with density:

ρ(x; v) = \{ \begin{matrix}\frac{x}{v}e^{− \frac{x^{2}}{2v}} , & x > 0, \\ 0, & otherwise. \end{matrix}

38. Let (X_{k})_{k=1}^{n} be a sequence of i.i.d. random variables from the Pareto density

ρ(x; θ) = \{ \begin{matrix}θ(1 + x)^{−θ−1}, & x > 0, \\ 0, & otherwise, \end{matrix}

where θ > 0. For what values of θ can the method of moments be used for its estimation? Find the estimator of θ by this method.

39. A coin, having probability p of landing heads, is flipped until a head appears for the first time. Let X denote the number of required flips. Let X_{1}, X_{2}, …, X_{n} be an independent random sample distributed with the same distribution as X. Estimate p by the method of moments.

40. Consider a system of m components, each of which either functions or fails. The series system works only if all its components function and the parallel system works if at least one of its components functions. Suppose now that the states of the components (X_{i})_{i=1}^{m} are i.i.d. random variables such that each component functions with P (X_{i} = 1) = p and fails with P (X_{i} = 0) = 1 − p. Suppose that the value of the parameter p is unknown. For the series system and parallel system, denote the random variables which determine the state of the system by

Y = min_{1≤i≤m}X_{i}

and

Z = max_{1≤i≤m}X_{i},

correspondingly.

(a) Let Y_{1}, Y_{2}, …, Y_{n} be a random independent sample of the system states of the series system. Use the method of moments to estimate the parameter p.

(b) Let Z_{1}, Z_{2}, …, Z_{n} be a random independent sample of the system states of the parallel system. Use the method of moments to estimate the parameter p.

41. Use the method of moments to determine the estimator of the parameter θ = (λ, m) in the Erlang distribution, with density

ρ(x; θ) = \{ \begin{matrix}\frac{λ^{m}x^{m−1}e^{−λx}}{m!} , & x > 0, \\ 0, & otherwise, \end{matrix}

where λ > 0 and m ∈ N.

42. For a normal population with unknown values of mean and variance, use the method of moments to obtain an estimator for θ = θ_{3} − θ_{1}, where θ_{1} and θ_{3} are the first and third quartiles, respectively.

43. Let (X_{k})_{k=1}^{n} be a sequence of i.i.d. random variables from the lognormal density

ρ(x; μ, σ^{2}) = \{ \begin{matrix}\frac{1}{σ\sqrt{2π}x}e^{− \frac{(ln x−μ)^{2}}{2σ^{2}}} , & x > 0, \\ 0, & otherwise, \end{matrix}

where σ > 0. Find an estimator for the parameter (μ, σ^{2}) by the method of moments.

44. Use the method of moments to find estimators for the unknown parameters in case of the following distributions:

(a)

f(x; a) = \{ \begin{matrix}\frac{2(a − x)}{a^{2}}, & 0 ≤ x ≤ a, \\ 0, & otherwise, \end{matrix}

where a > 0.

(b)

f(x; a) = \{ \begin{matrix}\frac{2(a + x)}{3a^{2}}, & −a ≤ x ≤ 0, \\ \frac{2a − x}{3a^{2}}, & 0 < x ≤ 2a, \\ 0, & otherwise, \end{matrix}

where a > 0.

45. For a uniform population U(−a, a) with unknown parameter a > 0, use the method of moments to obtain an estimator for the parameter a. (Hint: What order of moment do you need?)

46. Same as the preceding question for the following populations:

(a) U(a, 2a) with unknown parameter a.

(b) U(a, a^{2}) with unknown parameter a. (Explain!)

(c) N(0, σ^{2}) with unknown σ.

(d) with density function

f(x; a) = \{ \begin{matrix}\frac{a − |x|}{2a^{2}}, & −a ≤ x ≤ a, \\ 0, & otherwise, \end{matrix}

where a > 0.

47. An estimator Θ̂_{1} is better than Θ̂_{2} if |Θ̂_{1} − θ| ≤ |Θ̂_{2} − θ|, for all values of the parameter θ and for all x_{1}, x_{2}, …, x_{n}, with strict inequality in at least one point. For the uniform population U(0, a) find an estimator for the parameter a better than the estimator obtained by the method of moments.

Maximum Likelihood Estimator

48. Consider a system that will either operate or fail in a certain mission, and let p denote the probability of successful operation. Eight independent trials are conducted, with the results: S,F,S,S,S,F,S,S. What is the maximum likelihood estimate of the parameter p in this situation?

49. Consider a Pareto population (see question 38). Find the maximum likelihood estimator for θ and \frac{1}{θ}.

50. Find the maximum likelihood estimator for the parameter p if the population comes from:

(a) a B(1, p) distribution.

(b) a G(p) distribution. How does this agree with (a)?

51. Sometimes, there may be relevant information regarding some parameter from several samples, each from a different population, where the distribution functions of all populations depend on the same unknown parameter. One can define in this case a maximum likelihood estimator for the parameter exactly as in the case of a single population. Find this estimator in the following cases:

(a) X_{1}, X_{2}, …, X_{n_{1}} ∼ B(1, p), Y_{1}, Y_{2}, …, Y_{n_{2}} ∼ G(p).

(b) X_{1}, X_{2}, …, X_{n_{1}} ∼ N(μ, σ^{2}), Y_{1}, Y_{2}, …, Y_{n_{2}} ∼ N(μ + 1, σ^{2}).

(c) X_{1}, X_{2}, …, X_{n_{1}} ∼ B(1, p), Y_{1}, Y_{2}, …, Y_{n_{2}} ∼ B(1, 1 − p).

(d) X_{1}, X_{2}, …, X_{n_{1}} ∼ P (λ), Y_{1}, Y_{2}, …, Y_{n_{2}} ∼ P (2λ).

(e) X_{1}, X_{2}, …, X_{n_{1}} ∼ N(μ, σ^{2}), Y_{1}, Y_{2}, …, Y_{n_{2}} ∼ N(μ, 4σ^{2}).

52. For a uniform population U(a − \frac{1}{2}, a + \frac{1}{2}), find a maximum likelihood estimator for a, based on a random sample of size n. Suggest at least two (distinct) such estimators.

53. For a uniform population U((a, b) ⋃(c, d)), a ≤ b ≤ c ≤ d, find a maximum likelihood estimator for (a, b, c, d), based on a random sample of size n.

54. For a uniform population U((0, a) ⋃(2a, 3a)), a > 0, find a maximum likelihood estimator for a, based on a random sample of size n.

55. (a) Suppose that Y_{1}, Y_{2}, …, Y_{m}, the counts in cells 1, …, m, follow a multinomial distribution with a total count of n and cell probabilities p_{1}, …, p_{m}. The joint probability function of Y_{1}, Y_{2}, …, Y_{m} is:

P (y_{1}, y_{2}, …, y_{m}; p_{1}, p_{2}, …, p_{m}) = \frac{n!}{∏_{i=1}^{m} y_{i}!} ∏_{i=1}^{m} p_{i}^{y_{i}},

where ∑_{i=1}^{m} y_{i} = n and ∑_{i=1}^{m} p_{i} = 1. Note that the marginal distribution of each Y_{i} is B(n, p_{i}), and therefore the Y′_{i}s are not independent (they are constrained to sum to n). For given n, find the maximum likelihood estimator for the parameter (p_{1}, p_{2}, ..., p_{m}).

(b) If gene frequencies are in equilibrium, the genotypes AA, Aa and aa occur in a population with frequencies (1 − θ)^{2}, 2θ(1 − θ) and θ^{2}, according to the Hardy-Weinberg Law. In a sample from the Chinese population of Hong Kong in 1937, blood types occurred with the following frequencies, where M and N are erythrocyte antigens: 342 for M blood type, 500 for MN blood type and 187 for N blood type. Find the maximum likelihood estimator for the parameter θ. (Hint: Use part (a) with p_{i} = p_{i}(θ).)

Properties of Point Estimators

56. The radius of a circle is measured with an error of measurement which is distributed N(0, σ^{2}), where σ^{2} is unknown. Given n independent measurements of the radius, find an unbiased estimator of the area of the circle.

57. A random sample of size n is drawn from a uniform population U(0, a).

(a) Construct an unbiased estimator for a depending on the maximum likelihood estimator Θ̂ = max_{1≤i≤n}X_{i}.

(b) Construct an unbiased estimator based on Θ̂′ = min_{1≤i≤n}X_{i}. Compare it to the estimator found in (a).

(c) Do the same based on any order statistic of the sample.

58. For a uniform population U(−a, a), suggest an unbiased estimator for a, based on a random sample of size n.

59. As the preceding question for a population distributed as in Exercise 44.a.

60. For a normal population N(0, σ^{2}), suggest an unbiased estimator for σ^{4}, based on a random sample of size n.

61. A random sample of size n is drawn from an Exp(θ) population.

(a) Show that the statistic Θ̂ = n · min_{1≤i≤n}X_{i} is an unbiased estimator for \frac{1}{θ}.

(b) Construct an unbiased estimator based on X̄_{n}. Compare it to the estimator found in (a).

(c) Find an unbiased estimator for \frac{1}{θ^{2}} .

(d) Find an unbiased estimator for e^{−θ}.

(e) Find an unbiased estimator for e^{−2θ}.

62. A random sample of size 1 is drawn from a P (λ) population. Can you find an unbiased estimator for the parameter \frac{1}{λ} ? (Hint: Assume there exists such an estimator and use the Taylor series for e^{λ}.)

63. A random sample of size 1 is drawn from a G(p) population, where 0 < p < 1. Find an unbiased estimator for p. Does it take possible values (i.e., values in the interval (0, 1)) ?

64. Let X be a single observation from a Bernoulli population. Let Θ̂_{1} = X and Θ̂_{2} = \frac{1}{2}. Are both estimators unbiased? Is either? Compare the mean-squared error of Θ̂_{1} with that of Θ̂_{2}.

65. Let X ∼ ρ(x; θ) = θx^{θ−1}, 0 ≤ x ≤ 1, θ > 0. Find, based on X_{1}, X_{2}, …, X_{n}, an unbiased estimator for \frac{1}{θ}.

66. Θ̂_{1} and Θ̂_{2} are two unbiased estimators, based on a random sample of size n, for an unknown parameter θ of some distribution. Suppose they are known to have the same variance for each θ in the parameter space.

(a) What can you say about the estimator Θ̂_{3} = \frac{Θ̂_{1}+Θ̂_{2}}{2} ?

(b) The same if Θ̂_{1} and Θ̂_{2} are independent.

Sufficient Statistics

67. Find a parametric family of distribution functions, such that T = X_{1}^{2} + X_{2}^{2} + ... + X_{n}^{2} forms a sufficient statistic for the parameter.

68. Find sufficient statistics, based on a random sample of size n, for the following populations:

(a) G(p).

(b) Pareto distribution with parameter θ (question 38).

(c) Erlang distribution with known value of m (question 41).

(d) possible values 0,1,2 with probabilities P (X = 0) = p, P (X = 1) = p and P (X = 2) = 1 − 2p, where 0 < p < \frac{1}{2}.

(e) U((−b, −a) ⋃(a, b)), 0 < a < b.

69. Let X_{1}, X_{2}, X_{3} be independent identically distributed Bernoulli random variables with parameter p. Show that X_{1} + 2X_{2} + 3X_{3} is not sufficient for p.

70. Suggest at least 5 sufficient statistics, based on a random sample of size n, for the normal populations N(0, σ^{2}). (At least one of them should be 1-dimensional.)

71. Let X_{1}, X_{2}, …, X_{n} be a random sample from a Beta population with the density function:

ρ(x; θ_{1}, θ_{2}) = \{ \begin{matrix}\frac{1}{β(θ_{1},θ_{2})}x^{θ_{1}−1}(1 − x)^{θ_{2}−1}, & 0 < x < 1, \\ 0, & otherwise, \end{matrix}

where β(θ_{1}, θ_{2}) = ∫_{0}^{1} t^{θ_{1}−1}(1 − t)^{θ_{2}−1}dt, (θ_{1}, θ_{2} > 0), is the beta function.

(a) Find the mean and variance of the population.

Hint: The beta function can be shown to satisfy

β(θ_{1}, θ_{2}) = \frac{Γ(θ_{1}) · Γ(θ_{2})}{Γ(θ_{1} + θ_{2})},

where the gamma function Γ is given by

Γ(θ) = ∫_{0}^{∞} e^{−t}t^{θ−1}dt, θ > 0.

The gamma function has the following property:

Γ(θ + 1) = θ · Γ(θ).

(b) Which distribution is obtained in the special case θ_{1} = θ_{2} = 1? Verify that your results in the preceding part agree with the formulae in this case.

(c) Find a 2-dimensional sufficient statistic for the parameter (θ_{1}, θ_{2}).

72. Let X_{1}, X_{2}, …, X_{n} be a random sample from a population with density function:

ρ(x; θ) = \{ \begin{matrix}θx^{−2}, & x > θ, \\ 0, & otherwise, \end{matrix}

where θ > 0. Is Y_{1} = min(X_{1}, X_{2}, …, X_{n}) a sufficient statistic?

73. Is the family of multinomial distributions (question 55(a)), with arbitrary fixed m and n, an exponential family?

74. Which of the families of distributions considered in question 68 are exponential families?

Unbiased Estimators with Minimal Variance

75. Suppose T is a biased estimator of h(θ), and its bias b(θ) = E_{θ}(T) − h(θ) is a differentiable function. Prove a generalized form of the Cramér-Rao inequality:

V (T) ≥ \frac{(h′(θ) + b′(θ))^{2}}{nE ((\frac{∂}{∂θ} log f(X; θ))^{2})},

where f(X; θ) is a density function, satisfying the regularity conditions.

76. Let X_{1}, X_{2}, …, X_{n} be a random sample from a B(1, p) population.

(a) Find the Cramér-Rao lower bound for the variance of any unbiased estimator of p(1 − p).

(b) Suggest an unbiased estimator and compare its variance with the Cramér-Rao lower bound.

77. Let X_{1}, X_{2}, …, X_{n} be a random sample from an Erlang population with a known value of m (question 41). Find the UMVUE of λ if such exists.

78. Let X_{1}, X_{2}, …, X_{n} be a random sample from a population with density function

ρ(x; θ) = \{ \begin{matrix}θx^{θ−1}, & 0 < x < 1, \\ 0, & otherwise, \end{matrix}

where θ > 0. Find the UMVUE of θ.

79. Let X_{1}, X_{2}, … X_{n} be a random sample from a U(0, a) population.

(a) Find the Cramér-Rao lower bound for the variance of an unbiased estimator of a.

(b) Find the variance of the estimator found in question 57 and compare it with the corresponding Cramér-Rao bound. Explain!

80. Let X_{1}, X_{2}, …, X_{n} be a random sample from an N(μ, 1) population. Find the UMVUE of μ.

81. Let X_{1}, X_{2}, …, X_{n} be a random sample from an N(0, σ^{2}) population. Find the UMVUE of σ^{2}.

82. Suppose that θ is a 1-dimensional parameter in a regular model (i.e., the regularity conditions are valid). Prove:

(a) If T is an unbiased estimator of some h(θ), whose variance coincides with the Cramér-Rao lower bound, then the family of densities f(x; θ) is an exponential family (i.e., f(x; θ) = a(θ)b(x)e^{c(θ)d(x)}).

(b) If the family of densities f(x; θ), as θ varies over the parameter space, is an expo-

nential family and T = \frac{1}{n} ∑_{i=1}^{n} d(X_{i}) is an unbiased estimator of h(θ) = −\frac{a′(θ)}{c′(θ)a(θ)},

then its variance coincides with the Cramér-Rao lower bound.

Sufficiency and Completeness

83. For discrete random variables X and Y prove:

(a)

E(E(X|Y)) = E(X)

(b)

V (X) = E(V (X|Y)) + V (E(X|Y)).

(c)

V (X) ≥ V (E(X|Y)).

84. For a B(r, p) population with known r, find the Cramér-Rao lower bound for an unbiased estimator of p^{s}, s ≤ n, and find an unbiased estimator having this variance.

85. Find the Cramér-Rao lower bound for an unbiased estimator of λ^{s} based on a random sample of size n from a P (λ) population. Find an unbiased estimator with this variance.

86. Let X_{1}, X_{2}, … X_{n} be a random sample from a Pareto distribution. Suggest an estimator for \frac{1}{θ} based on a sufficient statistic. Find its variance and compare it with the corresponding Cramér-Rao lower bound.

87. Let X_{1}, X_{2}, … X_{n} be a random sample from a Exp(θ) distribution. Find the UMVUE estimator for θ^{2}and \frac{1}{θ^{2}} . Find its variance and compare it with the corresponding Rao-Cramér lower bound.

88. Find an unbiased estimator of e^{2λ} and (1 + λ)e^{−λ} based on the first observation in a random sample of size n from a P (λ) population. Improve it by Rao-Blackwell theorem. Is it a UMVUE estimator?

89. Let X be a single observation from a Beta population (see question 71). Is X a sufficient statistic? Is it complete?

90. Let X be a discrete random variable with P (X = −1) = p and P (X = k) = p^{k}(1−p)^{2}, (where 0 < p < 1) and k = 0, 1, 2, ... . For a random sample of size 1, is X a complete statistic?

91. Let X_{1}, X_{2}, …, X_{n} be a random sample from the density

ρ(x; θ) = \frac{2x}{θ^{2}}1_{(0,θ)}(x), θ > 0.

(a) Is Y_{n} = max(X_{1}, X_{2}, ..., X_{n}) a sufficient statistic? Is it complete?

(b) Is there a UMVUE estimator of θ? If so, find it.

92. Let X_{1}, X_{2}, ..., X_{n} be a random sample from a B(1, p) population.

(a) Prove directly from the definition that T = ∑_{i=1}^{n} X_{i} is a complete statistic.

(b) Find a UMVUE estimator for the parameter h(p) = ∑_{i=1}^{k} a_{j}p^{j}, (k ≤ n).

93. Let X_{1}, X_{2}, ..., X_{n} be a random sample from the density

f(x; θ) = \frac{log θ}{θ − 1} · θ^{x}1_{(0,1)}(x),

where θ > 1.

(a) Find a complete sufficient statistic.

(b) Find a function of θ for which there exists an unbiased estimator whose variance coincides with the Cramér-Rao lower bound.

94. Let X_{1}, X_{2}, ..., X_{n} be a random sample from the discrete uniform distribution

f(x; θ) = \frac{1}{θ} · 1_{\{1,2,...,θ\}}(x),

where θ = 1, 2, ... .

(a) Find a complete sufficient statistic.

(b) Let T = max_{1≤i≤n}X_{i}. Show that the UMVUE of θ is \frac{T^{n+1} − (T − 1)^{n+1}}{T^{n} − (T − 1)^{n}} .

3 Parametric Interval Estimation

Confidence Intervals

95. Let X_{1}, X_{2}, ..., X_{n} be a random sample from a U(a − 1, a + 1) population.

(a) Find P (Y_{1} < a < Y_{n}), where Y_{1} and Y_{n} are the smallest and the largest order statistics.

(b) For what sample size n is the interval (Y_{1}, Y_{n}) a 95 percent (or more) confidence interval for a?

96. Let X_{1}, X_{2}, ..., X_{n} be a large random sample from a P (λ) population. Find a 95 percent confidence interval for λ based on the sample mean.

97. Find a 90 percent confidence interval for the mean of a normal distribution with σ = 3 given the sample (3.3, −0.3, −0.6, −0.9). What would be the confidence interval if σ were unknown?

98. Consider random sampling from an N(μ_{0}, σ^{2}) population. Derive a confidence interval estimator of σ^{2} when μ_{0} is known.

99. Let X_{1}, X_{2}, ..., X_{n} be a random sample from an N(θ, θ^{2}) population, θ > 0. Give an example of a pivotal quantity, and use it to obtain a confidence interval for θ.

100. Let X_{i} ∼ χ^{2}(n_{i}), i = 1, 2, be independent random variables. Prove that X_{1} + X_{2} ∼ χ^{2}(n_{1} + n_{2}).

101. A sample was drawn from each of five independent populations assumed to be normal with the same variance. The values of Y_{j} = ∑_{i=1}^{n_{j}} (X_{i} − X̄_{n_{j}} )^{2}, j = 1, 2, ..., 5, and n_{j} are:

\begin{matrix}Y_{j} & 40 & 30 & 20 & 42 & 50 \\ n_{j} & 6 & 4 & 3 & 7 & 8 \end{matrix}

Find a 99 percent confidence interval for the common variance.

102. Let X be a single observation from the density

ρ(x; θ) = \{ \begin{matrix}θx^{θ−1}, & 0 < x < 1 \\ 0, & otherwise, \end{matrix}

where θ > 0.

(a) Find a pivotal quantity, and use it to find a confidence interval for θ.

(b) Define Y = −\frac{1}{log X}. Find a confidence level of a following confidence interval

(\frac{Y}{2} , Y ) for θ. Also find a shorter confidence interval for θ, having the same confidence level.

103. Let X_{1}, X_{2}, ..., X_{n} be a random sample from an Exp(θ) population.

(a) Find a confidence interval for:

(i) the mean of the population. (ii) the variance of the population. (iii) P (X > 1).

(b) Find a pivotal quantity based on min_{1≤i≤n}X_{i} and use it to find a confidence

interval for θ.

104. Let X be a single observation from an Exp(θ) population.

(a) Consider the confidence interval (X, 2X) for \frac{1}{θ}. What is its confidence level?

(b) Find another confidence interval with the same confidence level but smaller expected length.

Some Special Distriution Families

105. Show that

∫0^{\frac{π}{2}} sin^{α−1} x cos^{β−1} xdx = \frac{1}{2}B (\frac{α}{2}, \frac{β}{2} ) ,

where α, β > 0 and B(., .) is the beta function.

106. Prove that the beta function is related to the gamma function according to the following formula:

B(θ_{1}, θ_{2}) = \frac{Γ(θ_{1}) · Γ(θ_{2})}{Γ(θ_{1} + θ_{2})}.

(Hint: Consider the integral ∫_{0}^{∞} ∫_{0}^{∞} e^{−x−y}x^{θ_{1}−1}y^{θ_{2}−1}dxdy and use the substitution x = u(1 − v), y = uv. The Jacobian is equal to u.)

107. Prove that

Γ(θ) · Γ(1 − θ) = \frac{π}{sin πθ}, 0 < θ < 1.

108. Show that the even-order moments of the standard normal distribution can be expressed in terms of gamma functions, as follows:

E (Z^{2k}) = \frac{2^{k}}{\sqrt{π}} · Γ(k + \frac{1}{2}).

109. Show that if U ∼ U(0, 1), then −2 log U ∼ χ^{2}(2).

110. Find all moments of the χ^{2} distribution.

111. Show that the family of chi-square distributions is an the exponential family.

112. Find the mean and variance of the t distribution. (For what values of ν do they exist?)

113. Obtain the following approximations for χ_{α}^{2}(ν) for large values of ν:

(a) χ_{α}^{2}(ν) ≈ \sqrt{2ν} · z_{α} + ν.

(b) χ_{α}^{2}(ν) ≈ \frac{(z_{α} + \sqrt{2ν − 1})^{2}}{2}

114. Using a normal approximation, estimate:

(a) χ_{α}^{2}(50).

(b) P (X > 60), where X ∼ χ^{2}(50).

115. (a) Let X and Y be independent random variables, and E(Y) ≥ 1. Prove that

V (XY) ≥ V (X).

(b) Let Z ∼ N(0, 1) and T ∼ t(ν). Use part (a) to prove that:

V (T) ≥ V (Z)

Confidence Intervals, II

116. Assume that the nicotine contents of a certain brand of cigarettes is distributed normally. In a random sample of size 8 the results where 2.0, 1.7, 2.1, 1.9, 2.2, 2.1, 2.0 and 1.6 milligram. Find a 95 percent confidence interval for σ^{2}.

117. Let \frac{X}{σ} ∼ χ^{2}(10). Determine a 90 percent confidence interval for σ^{2} based on a single observation of X.

118. What is the probability that the length of a 95 percent confidence interval for μ, when sampling from a normal distribution, will be less than σ for samples of size 20?

119. Let X_{1}, X_{2}, ..., X_{m} be a random sample of size m from an N(μ_{1}, σ^{2}) population, and let Y_{1}, Y_{2}, ..., Y_{n} be a random sample of size n from an N(μ_{2}, kσ^{2}) population, where the value of k > 0 is known and μ_{1}, μ_{2} and σ^{2} are unknown. Assume that the two samples are independent. Find a confidence interval for μ_{2} − μ_{1}.

120. Let X_{1}, X_{2}, ..., X_{m} be a random sample of size m from an N(μ_{1}, σ_{0}^{2}) population, and let Y_{1}, Y_{2}, ..., Y_{n} be a random sample of size n from an N(μ_{2}, σ_{0}^{2}) population, where σ_{0}^{2} is known. Assume that the two samples are independent. What happens if we use the technique, used in the case of unknown variance, to find a confidence interval for μ_{2} − μ_{1}, in this case?

121. Show that the length and the variance of the length of the confidence interval for μ, when sampling from an N(μ, σ^{2}) population, (both values (μ, σ^{2}) are unknown), approach 0 as the sample size increases.

122. Let X_{1}, X_{2}, ..., X_{n} be a large random sample from a B(1, p) population. Suggest a more accurate confidence interval of confidence level 1 − α for p, than the one presented in class.

123. Let X_{1}, X_{2}, ..., X_{n} and Y_{1}, Y_{2}, ..., Y_{n} be large paired random samples from B(1, p_{1}) and B(1, p_{2}) populations.

(a) Suggest a 1 − α confidence interval for p_{2} − p_{1}.

(b) Compare the length of the interval in part (a) with that of the interval resulting if the samples are independent.

124. Let X_{1}, X_{2}, ..., X_{n} be a random sample from an Exp(θ) population.

(a) Suggest a 1 − α confidence interval for \frac{1}{θ} based on the sample mean.

(b) Suggest a 1 − α confidence interval for θ based on the sample mean.

(c) Let N_{≤1} be the number of observations which are equal to or less than 1. Find a 1 − α confidence interval for θ based on N_{≤1}.

125. Let X_{1}, X_{2}, ..., X_{n} be an independent random sample from the population

\begin{matrix}x & 2 & 3 & 5 \\ P (x) & p & 1-2p & p \end{matrix}

where 0 < p < \frac{1}{2} is unknown parameter.

(a) Suggest an unbiased estimator for p based on X̄_{n}.

(b) Suggest an unbiased estimator for p based on N_{=2} (the number of observations equal to 2).

(c) Suggest an unbiased estimator for p based on N_{≠3}(the number of observations not equal to 3).

(d) Which of the estimators from parts (a)-(c) is the best? Explain!

(e) Suggest three distinct 1 − α confidence intervals for p, based on the estimators from the parts (a)-(c). Compare them.

(f) For the last confidence interval I, find the required number of observations, such that its length |I| will not exceed 2e, assuming that

(i) we have an initial estimate for p. (ii) we do not have such an estimate.

126. Let X_{1}, X_{2}, ..., X_{n} be an independent random sample from the population

\begin{matrix}x & 0 & 1 & 2 & 3 \\ P (x) & 3p & p & 2p & 1-6p \end{matrix}

where 0 < p < \frac{1}{6} is unknown parameter. Denote by N_{i}, i = 0, 1, 2, 3, the number of observations equal to i.

(a) Suggest four distinct 1 − α confidence intervals for p, based on the N_{i}’s. Compare them.

(b) Suppose that the only information a statistisian has is the value of N_{2} − N_{1}. He wants to find a 1 − α confidence interval for p. He claims that he has N_{1} “successes” in a sample of size n from a B(1, p) population and N_{2} “successes” in a sample of size n from a B(1, 2p) population. Since the difference between the two proportions is 2p − p = p, he can construct a 1 − α confidence interval for this difference. Obtain this confidence interval for p. What is the problem with this method?

(c) Find a 1 − α confidence interval for p, based only on N_{2} − N_{1}.

(d) Compare the length of the interval based on N_{1} from part (a) with that of the interval obtained in (c).

Tests of Hypotheses

127. The proportion of college graduates among adults in some small town is estimated as p = 0.3. To test this hypothesis, a random sample of size 15 is selected. If the number of college graduates in the sample is anywhere from 2 to 7, we accept the null hypothesis that p = 0.3.

(a) Find α.

(b) Find β for the alternatives p = 0.2 and p = 0.4

128. Suppose X_{1}, X_{2}, ..., X_{n} is a random sample from a P (λ) population with unknown parameter λ > 0. We want to test the hypothesis:

\begin{matrix}H_{0} : & λ = λ_{0}, \\ H_{1} : & λ > λ_{0}. \end{matrix}

Denote X = X_{1} + X_{2} + ... + X_{n}.

(a) Consider the tests R′ = {X ≥ x_{0}} and R′′ = {X < x_{0}}, where x_{0} > 0 is a fixed value. Which is better? Explain!

(b) Suppose λ_{0} = 1, n = 1, R_{1} = {X ≥ 1}. Find α.

(c) Sketch the power function of R_{1}.

(d) Suppose R_{2} = {X = 0}. Find α and sketch the power function.

(d) Same for R_{3} = {X ≥ 2}.

(e) Suppose λ_{0} = \frac{1}{4}, n = 4, R_{4} = \{X ≥ 1\}. Find α and sketch the power function.

129. Suppose X_{1}, X_{2}, ..., X_{n} is a random sample from a G(p) population with unknown parameter 0 < p < 1. We want to test the hypothesis:

\begin{matrix}H_{0} : & p = p_{0}, \\ H_{1} : & p ≠ p_{0}. \end{matrix}

Denote X = X_{1} + X_{2} + ... + X_{n}.

(a) Suppose p_{0} = \frac{1}{2}, n = 1, R_{1} = \{X ≥ 3\}. Find α and sketch the power function.

(b) Same for R_{2} = {X = 1}.

(c) Same for R_{3} = {X ≠ 2}.

(d) Suppose n = 2 and R_{4} = {X ≠ 3, 4, 5}. Find α and sketch the power function.

(e) Suppose R_{5} = {X > n}. Find α and the algebraic expression for the power function.

(f) Same for R_{6} = {X > n + 1}.

130. Suppose X is distributed with the density function:

ρ(x; θ) = \{ \begin{matrix}θx^{θ−1}, & 0 < x < 1 \\ 0, & otherwise, \end{matrix}

where θ > 0. To test

\begin{matrix}H_{0} : & θ = 1, \\ H_{1} : & θ > 1, \end{matrix}

a sample size of 2 is selected, and the critical region

R = \{(X_{1}, X_{2}) : \frac{3}{4X_{1}} < X_{2}\}

is used . Find α and the power function.

131. Suppose X_{1}, X_{2}, ..., X_{n} is a random sample from a U(0, a) population with unknown parameter a > 0. We want to test the hypothesis:

\begin{matrix}H_{0} : & a = a_{0}, \\ H_{1} : & a ≠ a_{0}. \end{matrix}

Find such b that the test {max_{1≤i≤n}X_{i} ≤ b} has significance level α. Calculate and sketch the power function.

132. Suppose X_{1}, X_{2}, ..., X_{n} is a random sample from an N(μ, 3^{2}) population with unknown parameter μ. We want to test the hypothesis:

\begin{matrix}H_{0} : & μ = 1, \\ H_{1} : & μ > 1. \end{matrix}

Find the critical region and sample size n required in order to provide the significance level α = 0.05 and power 1 − β(2) = 0.9.

133. Suppose X_{1}, X_{2}, ..., X_{m} and Y_{1}, Y_{2}, ..., Y_{n} are large random samples of sizes m and n from Exp(θ) and P (λ) populations, respectively. We want to test:

\begin{matrix}H_{0} : & θ = λ, \\ H_{1} : & θ > λ. \end{matrix}

(a) Test using a single mean test.

(b) Test using a difference of means test.

(c) Find the power function of the test in (b) and of the test suggested in class.

134. Let X_{1}, X_{2}, ..., X_{12} be a random sample of size 12 from a B(1, p) population. We want to test the hypothesis:

\begin{matrix}H_{0} : & p = 0.5, \\ H_{1} : & p > 0.5. \end{matrix}

(a) Suggest a test with significance level α ≈ 0.06.

(b) Find the values of the power function for p = 0.6, 0.8, 0.9.

135. Suppose:

\begin{matrix}H_{0} : & μ = μ_{0}, \\ H_{1} : & μ > μ_{0}. \end{matrix}

(a) Suppose that σ = cμ is an unknown parameter, where c is a known constant. For a large sample size, find a test of significance level α based only on X̄_{n}.

(b) Suppose σ = f(μ), where the function f is known. For a large sample size, find a test of significance level α based only on X̄_{n}. Apply the general test to

(i) an Exp(θ) population, (ii) a P (λ) population, (iii)a G(p) population, (iv) a U(0, a) population.

(c) Under the conditions of part (b), obtain an (approximate) expression for the power

function of the test.

136. Suppose X_{1}, X_{2}, ..., X_{n} is a random sample from a U(0, a) population with unknown parameter a > 0. We want to test the hypothesis:

\begin{matrix}H_{0} : & a = a_{0}, \\ H_{1} : & a > a_{0}. \end{matrix}

Denote Y_{n} = max_{1≤i≤n}X_{i} and Y_{1} = min_{1≤i≤n}X_{i}.

(a) Suggest an α test based on Y_{n}. Find its power function.

(b) Same for Y_{1}.

(c) For n = 1, 2 compare the power functions from part (a) and (b). Intuitively, what should happen for any n?

137. Consider question 126. We want to test the hypothesis:

\begin{matrix}H_{0} : & p = p_{0}, \\ H_{1} : & p > p_{0}. \end{matrix}

(a) Suggest four distinct α tests, each one based only on one of the N′_{i}s, i = 0, 1, 2, 3.

(b) Find the power functions of the tests in part (a). Compare the tests.

(c) Suggest an α test based on all the N′_{i}s, i = 0, 1, 2, 3. Find its power function.

138. Let X_{1}, X_{2}, ..., X_{m} and Y_{1}, Y_{2}, ..., Y_{n} be random samples from B(1, p_{1}) and B(1, p_{2}) populations. Suggest tests for checking

H_{0} : p_{2} − p_{1} = d_{0},

against various alternatives.

139. Let X_{1}, X_{2}, ..., X_{m} and Y_{1}, Y_{2}, ..., Y_{n} be random samples from G(p) and Exp(θ) populations. Suggest tests for checking the null hypothesis

\begin{matrix}H_{0} : & p = e^{−θ}, \\ H_{0} : & (1 − p)^{2} = 1 − e^{−θ}, \\ H_{0} : & p = θ, \\ H_{0} : & p = \frac{1}{θ}, \end{matrix}

against various alternatives.

Goodness-of-Fit Test

140. Mendelian theory indicates that the shape and color of a certain variety of pea ought to be grouped into four groups, “round and yellow”, “round and green”, “angular and yellow”, and “angular and green”, according to the rations 9:3:3:1. For n = 556 peas, the following were observed:

round and yellow 315
round and green  108
angular and yellow 101
angular and green 32

Test the null hypothesis

H_{0} : p_{1} = \frac{9}{16}, p_{2} = \frac{3}{16}, p_{3} = \frac{3}{16}, p_{4} = \frac{1}{16},

with significance level α = 0.05.

141. A genetic law states that children having one parent of blood group M and the other parent of blood group N will always have one of the three blood groups M, MN, N, and the expected number of children in these three groups will be in the ratio 1:2:1. Of 300 children tested, each having one M parent and one N parent, 30 percent were found to be of type M, 45 percent of type MN, and 25 percent of type N. Do the data conform to the genetic law?

142. In 50 flips of a coin a head appears 20 times. Can we believe that the coin is fair? Assume α = 0.1.

143. Consider the random variable

X^{2} = ∑_{i=1}^{k} \frac{(O_{i} − e_{i})^{2}}{e_{i}} ,

where e_{i} and O_{i} are as defined in class. Show that

E(X^{2}) = k − 1.

144. In the case k = 2 show that

X^{2} = ∑_{i=1}^{k} \frac{(O_{i} − e_{i})^{2}}{e_{i}} ,

is distributed χ^{2}(1)

145. In the case k = 2 the chi-square goodness-of-fit test may be substituted by the test for proportions. Show that both of the tests coincide.

146. Formulate the goodness-of-fit test in terms of the statistic ∑_{i=1}^{k} \frac{O_{i}^{2}}{p_{i}}. In particular, for

H_{0} : p_{1} = p_{2} = ... = p_{k},

base the test on ∑_{i=1}^{k} O_{i}^{2}.