Description
Question 1
Consider a data set in which each data point is associated with a weighting factor , so
that the sum-of-squares error function becomes
Find an expression for the solution that minimizes this error function.
Give two alternative interpretations of the weighted sum-of-squares error function in terms of (i)
data dependent noise variance and (ii) replicated data points.
Question 2
We saw in Section 2.3.6 that the conjugate prior for a Gaussian distribution with unknown mean
and unknown precision (inverse variance) is a normal-gamma distribution. This property also
holds for the case of the conditional Gaussian distribution of the linear regression
model. If we consider the likelihood function,
then the conjugate prior for and is given by
Show that the corresponding posterior distribution takes the same functional form, so that
and find expressions for the posterior parameters , , , and .
Question 3
Show that the integration over in the Bayesian linear regression model gives the result
Hence show that the log marginal likelihood is given by
Question 4
Consider real-valued variables and . The variable is generated, conditional on , from the
following process:
where every is an independent variable, called a noise term, which is drawn from a Gaussian
distribution with mean 0, and standard deviation . This is a one-feature linear regression model,
where is the only weight parameter. The conditional probability of has distribution
, so it can be written as
Assume we have a training dataset of pairs ( ) for , and is known.
Derive the maximum likelihood estimate of the parameter in terms of the training example ‘s
and ‘s. We recommend you start with the simplest form of the problem:
Question 5
If a data point follows the Poisson distribution with rate parameter , then the probability of a
single observation is
You are given data points independently drawn from a Poisson distribution with
parameter . Write down the log-likelihood of the data as a function of .
Question 6
Suppose you are given observations, , independent and identically distributed with
a ) distribution. The following information might be useful for the problem.
If , then and
The probability density function of is ,
where the function is only dependent on and not .
Suppose, we are given a known, fixed value for . Compute the maximum likelihood estimator for
.