Final #2 - Questions and Solutions

1. A population is distributed according to the density function

f(x; a) = \{ \begin{matrix}\frac{2}{3a}, & 0 ≤ x ≤ a, \\ \frac{1}{3a}, & −a ≤ x < 0, \\ 0, & otherwise, \end{matrix}

where a > 0 is an unknown parameter. A random sample X_{1}, X_{2}, …, X_{n} is drawn from the population.

(a) Find the method of moments estimator of a.

The expectation of the population is

E(X) = ∫_{−∞}^{∞} xf(x; a) dx = ∫_{−a}^{0} \frac{1}{3a}x dx + ∫_{0}^{a} \frac{2}{3a}x dx = \frac{a}{6} .

The method of moments estimator is found by solving

\frac{a}{6} = X̄_{n} ,

which yields Â = 6X̄_{n} .

(b) Find a sufficient statistic of dimension as small as possible.

We claim that S = max_{1≤i≤n} |X_{i}| is a sufficient statistic. In fact, write

the density function of the population in the form

f(x; a) = \frac{1}{3a} · 1_{[0,a]}(|x|) · 2^{1_{[0,∞)}(x)} .

Now write the joint density function in the form:

fX⃗(x⃗; a) = \frac{1}{(3a)^{n}} · 1_{[0,a]}( max_{1≤i≤n} |x_{i}|) · 2^{#\{1≤i≤n: x_{i}>0\}} .

The representation shows that S is indeed a sufficient statistic.

(c) Find the maximum likelihood estimator of a.

Consider the likelihood function, namely the joint density function viewed as a function of the parameter a. It vanishes for a < max_{1≤i≤n} |x_{i}|, attains a positive value for a = max_{1≤i≤n} |x_{i}|, and decreases

as a increases from this point on. Hence the maximum likelihood estimator of a is:

Â = max_{1≤i≤n} |X_{i}| .

2. A population is distributed B(r, 1/3), where r is an unknown positive integer parameter. A random sample X_{1}, X_{2}, …, X_{n} is drawn from the population.

(a) Find an unbiased estimator for r based on the statistic ∑_{i=1}^{n} X_{i}.

Denote the statistic in question by T_{1}. Clearly E(T_{1}) = n · r/3, and therefore 3T_{1}/n is an unbiased estimator for r.

(b) Find another unbiased estimator for r, based on the statistic ∑_{i=1}^{n} (X_{i} − X̄_{n})^{2}.

Denoting this statistic by T_{2}, we find that

E(T_{2}) = (n − 1) · r · \frac{1}{3} · \frac{2}{3} ,

and therefore 9T_{2}/(2(n − 1)) forms an unbiased estimator as required.

(c) Is the estimator found in (a) of uniformly minimal variance among all unbiased estimators? If yes – prove it, if not – find a better (although not necessarily unbiased) estimator.

The estimator in (a) is probably not of uniformly minimal variance. To present a better estimator, we first note that r ≥ 1, so that if 3T_{1}/n happens to assume a value below 1, we will be better off by selecting the value 1 instead. Secondly, we must have r ≥ max_{1≤i≤n} X_{i}, whence if max_{1≤i≤n} X_{i} assumes a value smaller than that

of 3T_{1}/n then it would be better to take the first of the two values. Altogether, the estimator

T = max \{\frac{3T_{1}}{n} , 1, max_{1≤i≤n} X_{i}\}

is a better estimator than 3T_{1}/n.

3. A population has a density function f(x; θ), where the parameter θ is unknown. A random sample X_{1}, X_{2}, …, X_{n} is drawn from the population. Let h_{1}(θ) and h_{2}(θ) be two functions of θ. Prove or disprove each of the following claims:

(a) If the statistics T_{1} and T_{2} are unbiased estimators for h_{1}(θ) and h_{2}(θ), respectively, of uniformly minimal variance among all unbiased estimators, and S is a complete sufficient statistic, then T_{1} + T_{2} is an unbiased estimator of uniformly minimal variance among all unbiased estimators for h_{1}(θ) + h_{2}(θ).

Clearly, T_{1}+T_{2} is an unbiased estimator for h_{1}(θ)+h_{2}(θ). Since T_{1} and T_{2} are of uniformly minimal variance, they must be functions of the complete sufficient statistic S. Therefore T_{1} + T_{2} is also a function of S. Hence T_{1} + T_{2} is of uniformly minimal variance among all unbiased estimators.

(b) If T is an unbiased estimator of uniformly minimal variance among all unbiased estimators for h_{1}(θ)+h_{2}(θ), then there exist statistics T_{1} and T_{2} such that T = T_{1} + T_{2}, where T_{1} and T_{2} are unbiased estimators for h_{1}(θ) and h_{2}(θ), respectively, of uniformly minimal variance among all unbiased estimators.

Take a population where some functions of the unknown parameter admit no unbiased estimator. For example, take a population distributed P (λ) with unknown λ, and h_{1}(λ) = 1/λ, h_{2}(λ) = λ − 1/λ. We have seen that h_{1}(λ) + h_{2}(λ) = λ admits an unbiased estimator of uniformly minimal variance T (namely, T = X̄_{n}). However, T cannot be decomposed as required since h_{1}(λ) (same as h_{2}(λ)) does not admit any unbiased estimator. (An even simpler counter-example would be obtained by taking h_{2}(λ) = −1/λ.)

4. A population has the probability function

\begin{matrix}x & −1 & 0 & 1 \\ P (x) & 1/3 − b & 1/3 & 1/3 + b \end{matrix}

where b > 0 is an unknown parameter. A random sample X_{1}, X_{2}, …, X_{n} is drawn from the population, where n is large. Consider the statistics:

T_{1} = ∑_{i=1}^{n} X_{i},

T_{2} = ∑_{i=1}^{n} 1_{{1}}(X_{i}), T_{3} = ∑_{i=1}^{n} 1_{{−1,1}}(X_{i}) .

(a) Find a confidence interval of confidence level 1 − α for b which is based on one of the T_{i}’s.

It is possible to find a confidence interval based on either one of the first two T_{i}’s. We shall do it based on T_{2}. Obviously, 1_{{1}}(X_{i}) is distributed B(1, 1/3+b), and consequently a confidence interval of confidence level 1 − α for 1/3 + b is given by

I = [\frac{T_{2}}{n} − z\frac{α}{2} \sqrt{\frac{T_{2}}{n} (1 − \frac{T_{2}}{n} )/ n}, \frac{T_{2}}{n} + z\frac{α}{2} \sqrt{\frac{T_{2}}{n} (1 − \frac{T_{2}}{n} )/ n} ] .

To obtain a confidence interval for b, we have to subtract 1/3 from both endpoints of the interval. Denote

T = \frac{T_{2}}{n} − \frac{1}{3}

and

E = z_{α/2}\sqrt{\frac{T_{2}}{n} (1 − \frac{T_{2}}{n} )/ n} .

The required confidence interval is:

I′ = [T − E , T + E] .

(b) Suggest a test, based on one of the other two T_{i}’s, for testing

\begin{matrix}H_{0} : & b = 1/4, \\ H_{1} : & b > 1/4, \end{matrix}

with significance level α.

A routine calculation gives the mean and variance of the population,

E(X) = (\frac{1}{3} − b) · (−1) + \frac{1}{3} · 0 + (\frac{1}{3} + b) · 1 = 2b ,

and:

V (X) = E (X^{2}) − E^{2}(X) = \frac{2}{3} − 4b^{2} .

Consequently, we accept the null hypothesis if T_{1} assumes a sufficiently small value and reject otherwise. Specifically, since for b = 1/4 the mean and variance of the population are 1/2 and 5/12, respectively, the following test is as required:

R = \{X̄_{n} > \frac{1}{2} + z_{α}\sqrt{\frac{5}{12n}}\} .

Or:

R = \{T_{1} > \frac{n}{2} + z_{α}\sqrt{\frac{5n}{12}}\} .

(c) Is the remaining statistic sufficient? If yes – find if it is complete, if not – what information (if any) can you derive from it regarding b?

The statistic T_{3} is not sufficient. In fact, we have for example:

P (X⃗ = (1, 0, …, 0) | T_{3} = 1) = \frac{P (X⃗ = (1, 0, …, 0))}{P (T_{3} = 1)} .

Obviously, T_{3} ∼ B(n, 2/3), and therefore

P (X⃗ = (1, 0, …, 0) | T_{3} = 1) = \frac{(1/3 + b)(1/3)^{n−1}}{n · 2/3 · (1/3)^{n−1}} = \frac{1 + 3b}{2n} .

Since the right hand side depends on b, the statistic T_{3} is not sufficient. As the distribution of T_{3} does not depend on b, we cannot derive any information regarding b from T_{3}.