Description
1. (7.44). Let X1, . . . , Xn be iid N(θ, 1). Show that the best unbiased estimator of θ
2
is X¯ 2
n − (1/n).
Calculate its variance, and show that it is greater than the Cramer-Rao Lower Bound.
Hints: When compute the variance V ar(δ) = E(δ
2
) − [E(δ)]2
, you can write X¯
n = a + bZ with
Z ∼ N(0, 1) and suitably constants a, b and then use the fact that for Z ∼ N(0, 1), we have E(Z) =
0, E(Z
2
) = 1, E(Z
3
) = 0 and E(Z
4
) = 3.
2. (7.38). For each of the following distributions, let X1, . . . , Xn be a random sample. Is there a function
of θ, say g(θ), for which there exists an unbiased estimator whose variance attains the Cramer-Rao
Lower Bound? If so, find it. If not, show why not.
(a) fθ(x) = θxθ−1
, 0 < x < 1, θ > 0;
(b) fθ(x) = log θ
θ−1
θ
x
, 0 < x < 1, θ > 1.
3. (Modified by Problem 7.10). The random variables X1, · · · , Xn are iid with probability density
function [motivated from a “practical point of view” at the end of this problem]
fθ1,θ2
(x) =
θ
−θ1
2
θ1x
θ1−1
, if 0 < x ≤ θ2;
0, otherwise.
where θ1 > 0, θ2 > 0, and Ω will be completed specified later.
(a) Assume θ1 is known (positive) and Ω = {θ2 : θ2 > 0}. Find the MLE of θ2.
(b) Assume θ2 is known (positive) and Ω = {θ1 : 0 < θ1 < ∞}. Find the MLE of θ1.
(c) Show that the estimator in (a) is biased, but in case (b) the MLE of 1/θ1 is unbiased.
[Hints: −
R 1
0
x
α−1
(log x)dx = α
−2
; incidentally, the MLE of θ1 itself is biased.]
(d) Assume both θ1 and θ2 are unknown, and Ω = {(θ1, θ2) : 0 < θ1 < ∞, 0 < θ2 < ∞}.
i. Find a two-dimensional sufficient statistic for (θ1, θ2).
ii. Find the MLEs of θ1 and θ2.
iii. Find the MLE estimator of ϕ(θ1, θ2) = Pθ1,θ2
(X1 > 1).
iv. The length (in millimeters) of cuckoos’ eggs found in hedge sparrow nests can be modelled
with this distribution. For the data
22.0, 23.9, 20.9, 23.8, 25.0, 24.0, 21.7, 23.8, 22.8, 23.1, 23.1, 23.5, 23.0, 23.0,
Compute the value of MLE in parts (ii) and (iii).
[Model that could yield such a problem: There are iid random variables Yj , uniformly distributed from
0 to θ2. You send an observer out on each of n successive days to observe some Yj ’s. He does not record
the Yj ’s. Instead, knowing that “the maximum of the Yj ’s is sufficient and an MLE,” he decides to
observe a certain number, θ1, of the Yj ’s each day and computes the maximum of these θ1 observations.
He reports you the value of Xi
, the maximum he computes on the i-th day. Unfortunately, he forgets
to tell you the θ1 he used. Then the Xi has the density function stated for this problem, where we
have simplified matters by allowing θ1 to be any positive value instead of restricting it to integers.]
4. Recall that in problem 6.3 of our text (i.e., problem #6 of HW #7 with special cases in problem #5
of HW #7 and problem #2 of HW #8), X1, . . . , Xn are assumed to be a random sample from the pdf
f(x|µ, σ) = 1
σ
e
−(x−µ)/σ, µ ≤ x < ∞, 0 < σ < ∞.
In each of the following three scenarios, estimate the parameter(s) using both the maximum likelihood
estimator (MLE) and the best unbiased estimator:
(a) Assume that σ is known. Find both MLE and the best unbiased estimator of µ.
(b) Assume that µ is known. Find both MLE and the best unbiased estimator of σ.
(c) Assume that both µ and σ are unknown. Find both MLE and the best unbiased estimator of µ
and σ.
5. (Modified from Problem 7.9). Let X1, . . . , Xn be iid with pdf
fθ(x) = 1
θ
, 0 ≤ x ≤ θ, θ > 0.
(a) Estimate θ using both the method of moments and maximum likelihood.
(b) Calculate the means and variances of the two estimators in part (a). Which one should be preferred
and why?
(c) One can improve the MLE θbMLE to an unbiased estimator of the form δc = cθbMLE. Find a constant
c such that Eθ(δc) = θ, i.e., δc = cθbMLE is an unbiased estimator of θ. Is it the best unbiased
estimator of θ?
(d) The best estimator of the form of δc = cθbMLE is the one that uniformly minimizes the risk function
Rδc
(θ) = Eθ(δc − θ)
2
. Find such constant c.
6. (This is to show that sometimes MLE has poor performance). Suppose that X1, . . . , Xn are
iid with density
fθ(x) =
2θ
2
(x+θ)
3 , if x > 0;
0, if x ≤ 0.
where Ω = {θ : θ > 0}.
(a) If n = 1, show that an MLE estimator of θ is θba = 2X1.
(b) Show that θba in part (a) is not an unbiased estimator of θ.
[Verify or believe: R ∞
0
x
(x+1)3 dx =
R ∞
1
u−1
u3 du =
1
2 with u = x + 1.]
(c) Under the squared error loss function L(θ, d) = (θ − d)
2
, show that θba in part (a) is much worse
than the constant estimator θb∗ ≡ 17. [Hints:
R ∞
0
x
2
(1+x)
3 dx = +∞.]
(d) If n = 2, show that an MLE of θ is θbb =
1
4
[X1 + X2 +
p
X2
1 + 34X1X2 + X2
2
].
[If you want, you can consider the general n by yourself. For general n, describe the computation of
the MLE in terms of solving a polynomial equation of some degree, checking whether a local maximum
is a global maximum, etc.]