Description
1. (Finding minimax procedures, case I). Suppose that Y is Bernoulli random variable with
Pθ(Y = 1) = θ and Pθ(Y = 0) = 1 − θ, and it is desired to guess the value of θ on the basis
of X under the squared error loss function L(θ, d) = (θ − d)
2
.
Assume that the domain of θ is
Ω = {
4
9 ≤ θ ≤
5
9
} (since we still want to estimate θ, the decision space D can still be [0, 1]).
Find the minimax procedure under the squared error loss function when Ω = {
4
9 ≤ θ ≤
5
9
}.
2. (Finding minimax procedures, case II). When finding the minimax procedures, sometimes the desired prior distribution might not necessarily put the point mass at the boundary
of Ω. To illustrate this, let us consider the setting of the previous problem with a single
Bernoulli random variable Y , but now we assume that the domain of θ is Ω = [ 1
9
,
8
9
].
Find
the minimax estimator of θ under the squared error loss function L(θ, d) = (θ − d)
2 when
Ω = [ 1
9
,
8
9
]. For that purpose, let us consider two kinds of Bayesian procedures.
(a) Assume that θ has a prior distribution on two endpoints of Ω with probability mass
function πa(θ =
1
9
) = 1
2
and πa(θ =
8
9
) = 1
2
.
For the corresponding Bayes procedure,
denoted by δa, show that its Bayes risk rδa
(πa) < supθ∈Ω Rδa
(θ), and thus this direction
does not work.
(b) Consider another prior distribution πb(θ =
2−
√
2
4
) = 1
2
and πb(θ =
2+√
2
4
) = 1
2
, which
is a well-defined prior over Ω = [ 1
9
,
8
9
], since 1
9 <
2−
√
2
4 <
2+√
2
4 <
8
9
.
Show that the
corresponding Bayes procedure, denoted by δb, satisfies rδb
(πb) = supθ∈Ω Rδb
(θ), and
thus we can conclude that δb is minimax on Ω = [ 1
9
,
8
9
].
3. (Finding minimax procedures, case III). When finding the minimax procedures, sometimes the desired prior distribution might not necessarily be symmetric! To see this, under
the setting of Problem 1 with a single Bernoulli random variable Y , but we now assume that
the domain of θ is Ω = [0,
1
2
].
Find the minimax estimator of θ under the squared error loss
function L(θ, d) = (θ−d)
2 when Ω = [0,
1
2
]. To help you find the minimax estimator/procedure
when Ω = [0,
1
2
], we can split into the following steps.
(a) Assume that θ has a prior distribution on two endpoints of Ω = [0,
1
2
] with probability
mass function π(θ =
1
2
) = γ and π(θ = 0) = 1 − γ. Find the posterior distribution of θ
given Y = y for y = 0 or 1. [Note that your answer for π(θ|Y = 0) should be different
from π(θ|Y = 1), and θ only has two values: 0 and 1
2
].
(b) Show that the Bayes procedure under the assumption of part (a) is δB(0) = γ
4−2γ
and
δB(1) = 1
2
.
(c) Computing the risk function of δB and solving the equation RδB
(θ = 0) = RδB
(θ =
1
2
).
Show that γ = 2 −
√
2. Thus the Bayes procedure in part (b) becomes
δ
∗
(Y ) = ( √
2−1
2
, if Y = 0;
1
2
, if Y = 1.
(d) Show that the procedure δ
∗
in part (c) is minimax on Ω = [0,
1
2
].
4. (Finding minimax procedures, case IV). Sometimes the desired prior distribution for the
minimax procedure might not exist, but we can use a sequence of priors to find the minimax
procedures.
(a) Let πk, k = 1, 2, . . . , be a sequence of prior distributions on Ω. Let δk denote a Bayes
procedure with respect to πk, and define the Bayes risks of the Bayes procedures
rk =
Z
Ω
Rδk
(θ)πk(θ)dθ.
Show that if the sequence rk converges to a real-valued number r and if δ∗ is a statistical
procedure with its risk function Rδ∗
(θ) ≤ r for all θ ∈ Ω, then δ∗ is minimax.
(b) Let Y1, . . . , Yn be iid N(θ, σ2
) with σ known. Consider estimating θ using squared error
loss. Show that Y¯ = (Y1 + · · · + Yn)/n is a minimax procedure.
5. (Finding minimax procedures, case V). Sometimes the minimax properties can be extended from a smaller domain to a larger domain.
(a) Let Y1, . . . , Yn be iid with distribution F and finite unknown expectation θ, where F
belongs to a set F1 of distributions. Suppose we want to estimate θ under a given loss
function L(θ, d).
Show that if δ∗ is a minimax procedure when F is restricted to some
subset F0 of F1, and if supF ∈F0 Rδ∗
(F) = supF ∈F1 Rδ∗
(F) (i.e., sup risk of δ∗ over F1
is the same as sup risk over F0), then δ∗ is also minimax when F is permitted to vary
over F1.
(b) Suppose that Y1, . . . , Yn are iid with unknown mean θ. We further assume that the Yi
’s
can take any values in the interval [0, 1], i.e., 0 ≤ Yi ≤ 1, and that Ω = {θ : 0 ≤ θ ≤ 1}
and D = {d : 0 ≤ d ≤ 1}. Show that
δ∗ =
√
n
1 + √
n
Y¯ +
1
1 + √
n
1
2
is minimax for estimating θ under the squared error loss function.
6. (Impact of Loss function on minimax properties). In HW#1, we assume that Y1, . . . , Yn
are iid normal N(θ, 1) with θ ∈ Ω = (−∞, ∞) and one of the proposed procedures of estimating θ is
δ3,n(Y1, . . . , Yn) =
√
n Y¯
n
1 + √
n
.
(a) Under the loss function L(θ, d) = (θ − d)
2/(1 + θ
2
), show that k can be chosen in the
prior density πa(θ) = const.×(1+θ
2
)k
−1ϕ(θ/k), where ϕ is the standard normal density,
in such a way that δ3,n is Bayes relative to πa.
(b) Show that δ3,n is actually minimax under the loss function L(θ, d) = (θ − d)
2/(1 + θ).
(c) Let us still consider the problem of estimating the normal mean θ, but now under the
squared error loss function L(θ, d) = (θ−d)
2
. Show that δ3,n is still Bayes but no longer
minimax under the squared error loss function L(θ, d) = (θ − d)
2
.
2
Hints for problems 1 and 2: you can consider a general case of finding the Bayes procedure when
the prior distribution is given by π(θ = r) = 1
2
and π(θ = 1 − r) = 1
2
for some 0 ≤ r ≤
1
2
.
The
corresponding Bayes procedure is of the form δ(0) = u and δ(1) = 1 − u, where u depends on r.
Next, the risk function of this Bayes procedure is of the form
Aθ2 − Aθ + B,
where A = 4u − 1 and B = u
2
. When r =
4
9
, the coefficient A > 0, which allows us to prove the
minimax properties in Problem 2. However, when r =
1
9
, the coefficient A < 0, which is the case
in Problem 2(a). Meanwhile, problem 2(b) corresponds to the case of u = 1/4 and A = 0.
Hints for problem 4: In (a), it is possible that rk < r but limk→∞ rk = r. In (b) consider the
priors N(µ, τ 2
) for θ. We have shown in class that the Bayes risk of the corresponding Bayes
procedure is 1
n/σ2+1/τ 2 , denoted by rτ . What is r = limτ→∞ rτ ? What is the risk function of
δ∗ = Y¯ ? Does the condition in part (a) hold?
Hints for problem 5 (b): Let the subset F0 = the set of Bernoulli distributions, and let F1 =
the class of all distribution functions F with F(0) = 0 and F(1) = 1, i.e., 0 ≤ Yi ≤ 1. We have
shown in class that δ∗ is minimax as F varies over F0 (i.e., Bayes with constant risk). In order
to use part (a), we need to consider the risk function Rδ∗
(F) when F varies over F1. Note that if
0 ≤ Yi ≤ 1, then Y
2
i ≤ Yi and EF (Y
2
i
) ≤ EF (Yi) = θ, with equality if F ∈ F0. Use this to prove
that when F varies over F1, VarF (Yi) ≤ θ − θ
2
, and thus Rδ∗
(F) ≤
1
4(1+√
n)
2
, where the right-hand
side is the (constant) risk of δ∗ over F0.
Hints for problem 6(c): will it be Bayes relative to the prior density πb(θ) = C1πa(θ)/(1 +θ
2
) =
k
−1ϕ(θ/k) under the squared error loss? What are supθ∈Ω Rδ3,n
(θ) and supθ∈Ω RY¯ (θ) under the
squared error loss?]