Description
1. (Modified from 10.1). This question illustrates that the Method of Moments (MOM) estimators are
typically consistent). A random sample X1, . . . , Xn is drawn from a population with pdf
fθ(x) = 1
2
(1 + θx), −1 < x < 1, −1 < θ < 1.
(a) Find the population mean, Eθ(X1), and population variance, V arθ(X1).
(b) Find the MOM estimator θbMOM of θ, by equating the sample mean to the population mean.
(c) Compute the bias and variance of θbMOM, and show that θbMOM is a consistent estimator of θ.
2. (Modified from 10.3). A random sample X1, . . . , Xn (n ≥ 2) is drawn from N(θ, θ), where θ > 0.
(a) Show that the MLE of θ, θ, b is a root of the quadratic equation θ
2 + θ − W = 0, where W =
(1/n)
Pn
i=1 X2
i
, and determine which root equals the MLE.
(b) Compute Fisher information numbers In(θ) and I1(θ), and find the asymptotic distribution of θ. b
(c) Find the approximate variance of θb (using the techniques of Section 10.1.3 or other methods).
(d) Suppose that we observe the following n = 10 observations:
2.84, 0.93, 4.73, 5.69, 2.11, 2.88, 2.01, 1.17, 2.82, 4.49
For your convenience, the sample mean ¯x = 2.9670, the sample variance s
2 = 2.4352, and the
sample second-moment Pn
i=1 x
2
i = 109.9475. Since ¯x and s
2 are close, it is reasonable to assume
that this data set is a random sample from N(θ, θ) distribution. For this dataset, calculate the
MLE θb and find its approximate variance.
Remark: If interested, you can further compute a so-called 95% confidence interval on the true θ, which
is given [θb− 1.96p
V , b θb+ 1.96p
Vb], where Vb is the approximate variance of MLE θ. b
3. (Modified from 10.9). Assume that X1, . . . , Xn are iid Poisson(λ). As in Midterm II, please feel
free to use the fact that T(X) = Pn
i=1 Xi
is a complete sufficient statistic for λ. Suppose now that we
are interested in estimating ϕ(λ) = λe−λ
, the probability that X = 1.
(a) Find the best unbiased estimator of ϕ(λ) = λe−λ
, the probability that X = 1.
(b) Calculate the Fisher Information numbers In(λ) and I1(λ).
(c) Derive the Cramer-Rao lower bound for any unbiased estimator of ϕ(λ) = λe−λ
.
(d) Find the MLE of ϕ(λ) = λe−λ
, and derive its asymptotic distribution.
(e) A preliminary test of a possible carcinogenic compound can be performed by measuring the mutation rate of microorganisms exposed to the compound. An experimenter places the compound
in 15 petri dishes and records the following number of mutant colonies:
10, 7, 8, 13, 8, 9, 5, 7, 6, 8, 3, 6, 6, 3, 5.
For this data set, calculate both the best unbiased estimator and the MLE of ϕ(λ) = λe−λ
, the
probability that one mutant colony will emerge. In addition, find the approximate variance of
the MLE of ϕ(λ) = λe−λ
(using the techniques of Section 10.1.3 or other methods).
Remark: all your answers in part (e) should be numerical values.
4. (MLE and other topics). Assume that X1, . . . , Xn are iid with density
fθ(x) = 1
θ
e
−x/θ
, if x > 0;
0, if x ≤ 0.
and we want to estimate ϕ(θ) = θ
2 under the squared error loss. Here Ω = {θ : θ > 0}, D = {: d > 0},
and L(θ, d) = (ϕ(θ) − d)
2
. Recall that T =
Pn
i=1 Xi
is a complete sufficient statistic for θ, and has a
Gamma(n, θ) distribution and Eθ(T
k
) = θ
k
(n+k −1)!/(n−1)! for k an integer (= ∞ if n+k −1 < 0.)
(a) Find that the MLE estimator of ϕ(θ) = θ
2
.
(b) Show that the estimator of θ
2 of the form δc = cT2
(with c constant) has risk function
Rδc
(θ) = Eθ(δc − θ
2
)
2 = θ
4
{c
2n(n + 1)(n + 2)(n + 3) − 2cn(n + 1) + 1}.
In particular, the MLE estimator of ϕ(θ) = θ
2
corresponds to δc = cT2 with c =
1
n2 and show
that the MLE in (a) yields risk function θ
4
n
4n
2+11n+6
n3
o
.
(c) One can improve the MLE to an unbiased estimator of the form δc = cT2
. Show that the best
unbiased estimator of θ
2
corresponds to c =
1
n(n+1) and yields risk function θ
4
n
4n+6
n(n+1)o
.
(d) The best estimator of the form of δc = cT2
is the one that uniformly minimizes the risk function.
For the estimator of the form δc = cT2
, show that the “best” choice of c is c =
1
(n+2)(n+3) and
that the resulting estimator has the smallest risk function θ
4
n
4n+6
(n+2)(n+3)o
.
(e) Compute the Fisher information In(θ) and I1(θ), and show that the Cramer-Rao lower bound for
any unbiased estimator of θ
2
is Hn(θ) = 4θ
4/n. Consequently, assuming that biased estimators
also cannot have risk much smaller than this bound when n is large, we define a sequence of
estimators δ(X1, . . . , Xn) = δn to be asymptotically efficient if Rδn
(θ)/Hn(θ) → 1 as n → ∞, for
each θ in Ω. Here Rδn
(θ) is the risk function of the estimator δn. Show that under this definition,
each of the sequences of estimators in (a), (c), (d) is asymptotically efficient.
(f) Another type of estimators can be obtained by a Bayes procedure relative to some prior density
π(θ). One possible choice of π(θ) is π(θ) =
θ
−2
e
−1/θ
, if θ > 0;
0, otherwise.
First show that this is indeed a probability density function by integrating it, using the transformation u = θ
−1
. Next, recall our lectures on Bayes procedure, and show that for n ≥ 2, the
Bayes procedure (under the squared error loss L(θ, d) = (θ
2 − d)
2
) is δ
∗ =
(T +1)2
n(n−1) .
Remark: Note that, for fixed “true” θ > 0, T /n is very likely to be close to θ when n is large, so that
the Bayes estimator δ
∗
is likely to be close in values to the estimators of (b), (c), (d).
5* (Optional, Extra Questions: NOT Required and No credits, as the TA will not be able
to grade these optional questions). To enhance your understanding, you may work many other exercises/questions from our text such as: 7.15 & 7.16 (MLE), 7.46, 7.47 & 7.48 (the best unbiased
estimator), 7.49 (estimation via sufficient statistic), 7.50 & 7.51 (how to improve an estimator), 7.58
(Unbiased), 7.62 (Bayes), 7.66 (Jackknife), 10.7 & 10.8 (proofs for MLE asymptotic), 10.10 & 10.11,
10.17 (a)(d)(e), 10.22, 10.23 (Asymptotic properties).
Below is a research type problem for those students who are interested in research.
(a) (Modified from 7.19). Suppose that we observe the real-valued random variables Y1, · · · , Yn
that satisfies the AR(1) model: Yi = βYi−1 + ϵi for i ≥ 1 and Y0 = 0, where the ϵi
’s are iid
N(0, σ2
), σ2 unknown. Find the MLE of β, and derive the asymptotic distribution of βbMLE.
Remark: The asymptotic distribution will depend on the true value of β : |β| < 1, = 1 or > 1.
(b) In practice, the initial value Y0 is generally unobservable, how would you modify your answers
in (a)? That is, when we are not sure whether Y0 = 0 or not, how to estimate β based on
Y1, Y2, · · · , Yn, and what is the asymptotic distribution of your estimator?
(c) Another extension is the AR(2) model: assume that the observed data Y1, . . . , Yn(n ≥ 4) are from
the AR(2) model: Yi = β1Yi−1 + β2Yi−2 + ϵi
, where the ϵi
’s are iid N(0, σ2
), σ2 unknown. Find
a good estimator of θ = (β1, β2), and derive its asymptotic distribution.