## Description

1. (Finding minimax procedures, case I). Suppose that Y is Bernoulli random variable with

Pθ(Y = 1) = θ and Pθ(Y = 0) = 1 − θ, and it is desired to guess the value of θ on the basis

of X under the squared error loss function L(θ, d) = (θ − d)

2

.

Assume that the domain of θ is

Ω = {

4

9 ≤ θ ≤

5

9

} (since we still want to estimate θ, the decision space D can still be [0, 1]).

Find the minimax procedure under the squared error loss function when Ω = {

4

9 ≤ θ ≤

5

9

}.

2. (Finding minimax procedures, case II). When finding the minimax procedures, sometimes the desired prior distribution might not necessarily put the point mass at the boundary

of Ω. To illustrate this, let us consider the setting of the previous problem with a single

Bernoulli random variable Y , but now we assume that the domain of θ is Ω = [ 1

9

,

8

9

].

Find

the minimax estimator of θ under the squared error loss function L(θ, d) = (θ − d)

2 when

Ω = [ 1

9

,

8

9

]. For that purpose, let us consider two kinds of Bayesian procedures.

(a) Assume that θ has a prior distribution on two endpoints of Ω with probability mass

function πa(θ =

1

9

) = 1

2

and πa(θ =

8

9

) = 1

2

.

For the corresponding Bayes procedure,

denoted by δa, show that its Bayes risk rδa

(πa) < supθ∈Ω Rδa

(θ), and thus this direction

does not work.

(b) Consider another prior distribution πb(θ =

2−

√

2

4

) = 1

2

and πb(θ =

2+√

2

4

) = 1

2

, which

is a well-defined prior over Ω = [ 1

9

,

8

9

], since 1

9 <

2−

√

2

4 <

2+√

2

4 <

8

9

.

Show that the

corresponding Bayes procedure, denoted by δb, satisfies rδb

(πb) = supθ∈Ω Rδb

(θ), and

thus we can conclude that δb is minimax on Ω = [ 1

9

,

8

9

].

3. (Finding minimax procedures, case III). When finding the minimax procedures, sometimes the desired prior distribution might not necessarily be symmetric! To see this, under

the setting of Problem 1 with a single Bernoulli random variable Y , but we now assume that

the domain of θ is Ω = [0,

1

2

].

Find the minimax estimator of θ under the squared error loss

function L(θ, d) = (θ−d)

2 when Ω = [0,

1

2

]. To help you find the minimax estimator/procedure

when Ω = [0,

1

2

], we can split into the following steps.

(a) Assume that θ has a prior distribution on two endpoints of Ω = [0,

1

2

] with probability

mass function π(θ =

1

2

) = γ and π(θ = 0) = 1 − γ. Find the posterior distribution of θ

given Y = y for y = 0 or 1. [Note that your answer for π(θ|Y = 0) should be different

from π(θ|Y = 1), and θ only has two values: 0 and 1

2

].

(b) Show that the Bayes procedure under the assumption of part (a) is δB(0) = γ

4−2γ

and

δB(1) = 1

2

.

(c) Computing the risk function of δB and solving the equation RδB

(θ = 0) = RδB

(θ =

1

2

).

Show that γ = 2 −

√

2. Thus the Bayes procedure in part (b) becomes

δ

∗

(Y ) = ( √

2−1

2

, if Y = 0;

1

2

, if Y = 1.

(d) Show that the procedure δ

∗

in part (c) is minimax on Ω = [0,

1

2

].

4. (Finding minimax procedures, case IV). Sometimes the desired prior distribution for the

minimax procedure might not exist, but we can use a sequence of priors to find the minimax

procedures.

(a) Let πk, k = 1, 2, . . . , be a sequence of prior distributions on Ω. Let δk denote a Bayes

procedure with respect to πk, and define the Bayes risks of the Bayes procedures

rk =

Z

Ω

Rδk

(θ)πk(θ)dθ.

Show that if the sequence rk converges to a real-valued number r and if δ∗ is a statistical

procedure with its risk function Rδ∗

(θ) ≤ r for all θ ∈ Ω, then δ∗ is minimax.

(b) Let Y1, . . . , Yn be iid N(θ, σ2

) with σ known. Consider estimating θ using squared error

loss. Show that Y¯ = (Y1 + · · · + Yn)/n is a minimax procedure.

5. (Finding minimax procedures, case V). Sometimes the minimax properties can be extended from a smaller domain to a larger domain.

(a) Let Y1, . . . , Yn be iid with distribution F and finite unknown expectation θ, where F

belongs to a set F1 of distributions. Suppose we want to estimate θ under a given loss

function L(θ, d).

Show that if δ∗ is a minimax procedure when F is restricted to some

subset F0 of F1, and if supF ∈F0 Rδ∗

(F) = supF ∈F1 Rδ∗

(F) (i.e., sup risk of δ∗ over F1

is the same as sup risk over F0), then δ∗ is also minimax when F is permitted to vary

over F1.

(b) Suppose that Y1, . . . , Yn are iid with unknown mean θ. We further assume that the Yi

’s

can take any values in the interval [0, 1], i.e., 0 ≤ Yi ≤ 1, and that Ω = {θ : 0 ≤ θ ≤ 1}

and D = {d : 0 ≤ d ≤ 1}. Show that

δ∗ =

√

n

1 + √

n

Y¯ +

1

1 + √

n

1

2

is minimax for estimating θ under the squared error loss function.

6. (Impact of Loss function on minimax properties). In HW#1, we assume that Y1, . . . , Yn

are iid normal N(θ, 1) with θ ∈ Ω = (−∞, ∞) and one of the proposed procedures of estimating θ is

δ3,n(Y1, . . . , Yn) =

√

n Y¯

n

1 + √

n

.

(a) Under the loss function L(θ, d) = (θ − d)

2/(1 + θ

2

), show that k can be chosen in the

prior density πa(θ) = const.×(1+θ

2

)k

−1ϕ(θ/k), where ϕ is the standard normal density,

in such a way that δ3,n is Bayes relative to πa.

(b) Show that δ3,n is actually minimax under the loss function L(θ, d) = (θ − d)

2/(1 + θ).

(c) Let us still consider the problem of estimating the normal mean θ, but now under the

squared error loss function L(θ, d) = (θ−d)

2

. Show that δ3,n is still Bayes but no longer

minimax under the squared error loss function L(θ, d) = (θ − d)

2

.

2

Hints for problems 1 and 2: you can consider a general case of finding the Bayes procedure when

the prior distribution is given by π(θ = r) = 1

2

and π(θ = 1 − r) = 1

2

for some 0 ≤ r ≤

1

2

.

The

corresponding Bayes procedure is of the form δ(0) = u and δ(1) = 1 − u, where u depends on r.

Next, the risk function of this Bayes procedure is of the form

Aθ2 − Aθ + B,

where A = 4u − 1 and B = u

2

. When r =

4

9

, the coefficient A > 0, which allows us to prove the

minimax properties in Problem 2. However, when r =

1

9

, the coefficient A < 0, which is the case

in Problem 2(a). Meanwhile, problem 2(b) corresponds to the case of u = 1/4 and A = 0.

Hints for problem 4: In (a), it is possible that rk < r but limk→∞ rk = r. In (b) consider the

priors N(µ, τ 2

) for θ. We have shown in class that the Bayes risk of the corresponding Bayes

procedure is 1

n/σ2+1/τ 2 , denoted by rτ . What is r = limτ→∞ rτ ? What is the risk function of

δ∗ = Y¯ ? Does the condition in part (a) hold?

Hints for problem 5 (b): Let the subset F0 = the set of Bernoulli distributions, and let F1 =

the class of all distribution functions F with F(0) = 0 and F(1) = 1, i.e., 0 ≤ Yi ≤ 1. We have

shown in class that δ∗ is minimax as F varies over F0 (i.e., Bayes with constant risk). In order

to use part (a), we need to consider the risk function Rδ∗

(F) when F varies over F1. Note that if

0 ≤ Yi ≤ 1, then Y

2

i ≤ Yi and EF (Y

2

i

) ≤ EF (Yi) = θ, with equality if F ∈ F0. Use this to prove

that when F varies over F1, VarF (Yi) ≤ θ − θ

2

, and thus Rδ∗

(F) ≤

1

4(1+√

n)

2

, where the right-hand

side is the (constant) risk of δ∗ over F0.

Hints for problem 6(c): will it be Bayes relative to the prior density πb(θ) = C1πa(θ)/(1 +θ

2

) =

k

−1ϕ(θ/k) under the squared error loss? What are supθ∈Ω Rδ3,n

(θ) and supθ∈Ω RY¯ (θ) under the

squared error loss?]