Homeworks - THU



Homework

Homework 1

1. M. D. Computing describes the use of Bayes’ theorem and the use of conditional probability in medical diagnosis. Prior probabilities of diseases are based on the physician’s assessment of such things as geographical location, seasonal influence, occurrence of epidemics, and so forth. Assume that a patient is believed to have one of two diseases, denoted [pic] and [pic] with [pic] and [pic] and that medical research has determined the probability associated with each symptom that may accompany the diseases. Suppose that given diseases [pic] and [pic], the probabilities that the patient will have symptoms [pic], [pic] or [pic] are as follows.

| |[pic] |[pic] |[pic] |

|[pic] |0.15 |0.1 |0.15 |

|[pic] |0.80 |0.15 |0.03 |

After a certain symptom is found to be present, the medical diagnosis may be aided by finding the revised probabilities of each particular disease. Compute the posterior probabilities of each disease given the following medical findings.

a) The patient has symptom [pic].

b) The patient has symptoms [pic] or [pic].

c) For the patient with symptom [pic] in part (a), suppose we also find symptom [pic]. What are the revised probabilities of [pic] and [pic].

2. Let [pic], with [pic] known. For the prior distribution for [pic], we shall assume the normal distribution [pic].

a) Find the maximum likelihood estimate.

b) Find the posterior distribution of [pic] and posterior mode as estimate of . [pic].

c) Compare the estimates in (a) and (b).

3. Let [pic],

[pic]

For the prior distribution for [pic], we shall assume the gamma distribution

[pic].

a) Find the maximum likelihood estimate.

b) Find the posterior distribution of [pic] and posterior mean as estimate of . [pic].

c) Compare the estimates in (a) and (b).

Solution 1:

1.

(a) By Bayes’ theorem,

[pic]

and

[pic].

(b)Suppose [pic]. Then,

[pic],

[pic]

and

[pic]

Also,

[pic]

and

[pic]

Therefore,

[pic]

and

[pic].

(c)

We can use [pic] and [pic] as new revised probability for the diseases. That is,

[pic]

and

[pic].

Thus,

[pic]

and

[pic].

2.

[pic], [pic] known. [pic].

(a) The likelihood function is

[pic].

The log-likelihood function is

[pic]

(b)

[pic]

Therefore, the posterior distribution is

[pic].

Thus, the mode of the posterior distribution is

[pic].

(c)

[pic].

The larger [pic] is (i.e., the prior [pic] varies dramatically), the less contribution of the prior mean [pic] is for the Bayes estimate. On the other hand, the larger [pic] is (i.e., the m.l.e. [pic] varies dramatically), the less contribution of the m.l.e. [pic] based on the data is for the Bayes estimate.

3.

(a) The likelihood function is

[pic].

The log-likelihood function is

[pic]

(b)

[pic]

Therefore, the posterior distribution is

[pic].

Thus, the mean of the posterior distribution is

[pic].

(c)

[pic].

The larger [pic] is (i.e., more data), the more contribution of the m.l.e. [pic] based on the data is for the Bayes estimate. On the other hand, the larger [pic] is (i.e., the prior [pic] varies dramatically), the less contribution of the prior mean [pic] for the Bayes estimate.

Homework 2

4. Let X have a [pic] distribution, [pic]. The loss function is [pic]. Consider decision rules of the form [pic]. Assume the prior is [pic].

a) Calculate [pic], and find the Bayes action. (Note that this is the optimal Bayes action for the no-data problem in which x is not observed).

b) Find [pic].

c) Show that [pic] is inadmissible if [pic].

d) Find [pic].

e) Find the value of c which minimizes [pic]

f) Is there a best rule of the form [pic] in terms of the minimax principle?

5. A insurance company is faced with taking one of the following 3 actions: [pic]: increase sales force by 10%; [pic]: maintain present sales force; [pic]: decrease sales force by 10%.

Depending on whether or not the economy is good ([pic]), mediocre ([pic]), or bad ([pic]), the company would expect to lose the following amounts of money in each case:

| |[pic] |[pic] |[pic] |

|[pic] |-10 |-5 |-3 |

|[pic] |-5 |-5 |-2 |

|[pic] |1 |0 |-1 |

a) Determine if each action is admissible or inadmissible.

b) The company believes that [pic] has the probability distribution [pic]. Order the action according to their Bayesian expected loss (equivalent to Bayes risk, here), and state the Bayes action.

c) Order the actions according to the minimax principle and find the minimax action.

6. A professional baseball team is concerned about attendance for the upcoming year. They must decide whether or not to implement a 0.5 million dollar promotional campaign. If the team is a contender, they feel that 4 million in attendance revenues will be earned (regardless of whether or not the promotional campaign is implemented). Letting [pic] denote the team’s proportion of wins, they feel the team will be a contender if [pic]. If [pic], they feel their attendance revenues will be [pic] million dollars without the promotional campaign, and [pic] million dollars with the promotional campaign. It is felt that [pic] have a [pic] distribution.

a) Describe A, [pic], and [pic].

b) What is the Bayes action?

c) What is the minimax action?

7. The owner of a ski shop must order skis for the upcoming season. Orders must be placed in quantities of 25 pairs of skis. The cost per pair of skis is $50 if 25 are ordered, $45 if 50 are ordered, and $40 if $75 are ordered. The skis will be sold at $75 per pair. Any skis left over at the end of the year can be sold (for sure) at $25 per pair. If the owner runs out of skis during the season, he will suffer a loss of “goodwill” among unsatisfied customers. He rates this loss at $5 per unsatisfied customer. For simplicity, the owner feels that demand for the skis will be 30, 40, 50, or 60 pair of skis, with probabilities 0.2, 0.4, 0.2, and 0.2, respectively.

a) Describe A, [pic], the loss matrix, and the prior distribution.

b) Which actions are admissible?

c) What is the Bayes action?

d) What is the minimax action?

Solution 2:

1.

(a)

[pic]

since [pic].

As [pic], [pic] achieves its minimum. Therefore, 1 is the Bayes action.

(b)

[pic]

(c)

Since

[pic]

[pic] achieves its minimum at [pic]. As [pic],

[pic].

Therefore, [pic] is inadmissible if [pic].

(d)

[pic]

(e)

[pic]

(f)

Since

[pic],

there is no minimax estimate!!

2.

(a)

In no data problem, [pic]. Since

[pic],

[pic] are all admissible.

(b)

[pic].

Therefore, the Bayes action is [pic].

(c)

Since

[pic]

the minimax action is [pic].

3.(a)

[pic]

(b)

[pic]

Since [pic], [pic] is the Bayes action.

(c)

[pic].

Since

[pic],

[pic] is the minimax action.

4.

(a)

[pic]

and the loss matrix is

| |[pic] |[pic] |[pic] |

|[pic] |-600 |-500 |-375 |

|[pic] |-550 |-1000 |-875 |

|[pic] |-500 |-1500 |-1375 |

|[pic] |-450 |-1450 |-1875 |

where [pic] is to order 25 pairs of skis, [pic] is to order 50 pairs of skis, [pic] is to order 75 pairs of skis and [pic] represents the demand will be 30 pairs of skis, [pic] represents the demand will be 40 pairs, [pic] represents the demand will be 50 pairs, and [pic] represents the demand will be 60 pairs. The prior distribution is

[pic].

(b)

[pic] are all admissible.

(c)

[pic]The Bayes action is [pic].

(d)

[pic]

The minimax action is [pic].

Homework 3

1. A large shipment of parts is received, out of which 5 are tested for defects. The number of defective parts, X, is assumed to have a [pic]. From past shipments, it is known that [pic]. Find the Bayes rule (or Bayes estimate) under loss

(a) [pic] (b) [pic] (c) [pic]

(d) [pic].

2. In the IQ example, where [pic] and [pic]. Assume it is important to detect particularly high or low IQs. Indeed the weighted loss

[pic],

is deemed appropriate. Find the Bayes rule.

3. A device has been created which can supposedly classify blood as type A, B, AB, or O. The device measures a quantity X, which has density

[pic].

If [pic], the blood is of type AB; if [pic], the blood is of type A; if [pic], the blood is of type B; and if [pic], the blood is of type O. In the population as a whole, [pic]. The loss in misclassifying the blood is given in the following table.

| |AB |A |B |O |

|AB |0 |1 |1 |2 |

|A |1 |0 |2 |2 |

|B |1 |2 |0 |2 |

|O |3 |3 |3 |0 |

If [pic] is observed, what is the (posterior) Bayes action.

Solution 3:

1.

[pic]

Therefore, the posterior density is [pic].

(a)

[pic].

(b)

Given [pic], the posterior density is [pic]. Thus, the Bayes estimate is the posterior median of [pic]. Suppose the posterior median is denoted by m, then

[pic]

(c) [pic]. Thus, the Bayes estimate is

[pic]

Since

[pic]

and

[pic]

then

[pic].

(d) Given [pic], the Bayes estimate is the [pic] percentile of [pic]. Similar to (b), suppose [pic] percentile of [pic] is p. Then,

[pic]

2. [pic]

Since [pic], then

[pic]

and similarly

[pic]

then

[pic]

3.

[pic]

Then,

[pic]

So, the posterior Bayes action is AB.

Homework 4

1. If [pic] and

[pic].

Find the Bayes estimator of [pic] under loss

[pic].

( If a random variable [pic], then

[pic]

2. A business man must decide how to finance the investment of some business. It costs $100000 to invest. The man has available 3 options: [pic]--finance the investment himself; [pic]--accept $70000 from investors in return for paying them 50% of the business profits; [pic]--accept $120000 from investors in return for paying them 90% of the business profits. The business profits will be [pic], where [pic] is the number of goods sold in the business. From past data, it is believed that [pic] with probability 0.9, while the [pic] have density [pic].

A financial assessment is performed to determine the likelihood of goods sold . The assessment tells which status [pic] is present. It is known that the probabilities of the [pic] given [pic] are

[pic]

a) For mometary loss, what is the Bayes action if [pic] is observed?

For mometary loss, what is the Bayes action if [pic] is observed?

Solution 4:

1. Please replace the original loss function with

[pic].

Note that for a random variable [pic], then

[pic][pic],

[pic]

and

[pic].

The posterior is

[pic]

The Bayes estimate under the loss function is

[pic]

Note:

Homework 5, 1, (a) can be solved by finding the posterior, which is also distributed as inverse gamma!!

2.

(a)

The joint distribution function is

[pic]

Since

[pic]

the posterior is

[pic]

Since the loss function for [pic] is

[pic],

the Bayesian (posterior) expected loss for [pic] is

[pic].

Similarly, the loss function for [pic] is

[pic]

the Bayesian (posterior) expected loss for [pic] is

[pic]

Since the loss function for [pic] is

[pic]

the Bayesian (posterior) expected loss for [pic] is

[pic]

Therefore, the Bayes action is [pic].

(b) Similar to (a),

The joint distribution function is

[pic]

Since

[pic]

the posterior is

[pic]

Since the loss function for [pic] is

[pic],

the Bayesian (posterior) expected loss for [pic] is

[pic]

Similarly, the loss function for [pic] is

[pic]

the Bayesian (posterior) expected loss for [pic] is

[pic]

Since the loss function for [pic] is

[pic]

the Bayesian (posterior) expected loss for [pic] is

[pic] Therefore, the Bayes action is [pic].

Homework 5

3. A production lot of 5 electronic components is to be tested to determine [pic], the mean lifetime. A sample of 5 components is drawn, and the lifetimes [pic] are observed. It is known that [pic]. From past records it is known that, among production lots, [pic] is distributed according to an [pic] distribution. The 5 observations are 15, 12, 14, 10, 12.

a) Find the generalized maximum likelihood estimate of [pic], the posterior mean of [pic], and their respective posterior variance.

b) Find the approximate 90% HPD credible set, using the normal approximation to the posterior.

4. The weekly number of fires, X, in a town has a [pic] distribution. It is desired to find a 90% HPD credible set for [pic]. Nothing is known a priori about [pic], so the noninformative prior [pic] is deemed appropriate. The number of fires observed for 5 weekly periods was 0, 1, 1, 0, 0.

a) What is the desired credible set?

b) Find the approximate 90% HPD credible set, using the normal approximation to the posterior.

5. Suppose [pic], and that a 90% HPD credible set for [pic] is desired. The prior information is that [pic] has a symmetric unimodal density with median 0 and quartiles [pic]. The observation is [pic].

a) If the prior information is modeled as a [pic] prior, find the 90% HPD credible set.

b) If the prior information is modeled as a [pic] prior, find the 90% HPD credible set.

6. A large shipment of parts is received, out of which 5 are tested for defects. The number of defective parts, X, is assumed to have a [pic] distribution. From past shipments, it is known that [pic] has a [pic] prior distribution. Find the 95% HPD credible set for [pic], if [pic] is observed.

5. From path perturbations of a nearby sun, the mass [pic] of a neutron star is to be determined. 5 observations 1.2, 1.6, 1.3, 1.4 and 1.4 are obtained. Each observation is (independently) normally distributed with mean [pic] and unknown variance [pic]. A priori nothing is known about [pic] and [pic], so the noninformative prior density [pic] is used. Find a 90% HPD credible set for [pic].

Solution 5:

1. [pic] and [pic] Then,

[pic]

Since [pic], thus [pic].

(a) Since

[pic]

The posterior mean is

[pic],

and the posterior variance is

[pic].

[pic]

(b) The approximate 90% HPD credible set for [pic] is

[pic]

2. [pic] and [pic]. Then,

[pic]

(b) Since [pic], the posterior mean is

[pic],

and the posterior variance is

[pic].

Therefore, the approximate 90% HPD credible set for [pic] is

[pic]

3.

(a)

[pic] and [pic]. Then,

[pic]

Since [pic], the posterior mean is [pic]and the posterior variance is [pic]. Thus, 90% HPD credible set for [pic] is

[pic]

4. [pic] and [pic]. Then,

[pic]

Since [pic], the posterior mean is [pic] and

the posterior variance is

[pic].

Thus, the approximate 95% HPD credible set for [pic] is

[pic]

5. [pic] and [pic]. Then,

[pic]

Further,

[pic]

Note that

[pic],

where [pic] is the sample variance.

Thus, a [pic] HPD credible set is

[pic]

Since [pic] and [pic], a 90% HPD credible set for [pic] is

[pic]

Note:

For [pic] and [pic]. Then,

[pic].

Further,

[pic],

where [pic].

Homework 6

7. For 2. (a) and 3. (b) in homework 5, please write programs to calculate 90% HPD credible sets (not approximate).

8. A large shipment of parts is received, out of which 5 are tested for defects. The number of defective parts, X, is assumed to have a [pic] distribution. From past shipments, it is known that [pic] has a [pic] prior distribution. Suppose [pic] is observed. It is desired to test [pic] versus [pic]. Find the posterior probabilities of the two hypotheses, the posterior odds ratio, and the Bayes factor.

9. The waiting time for a bus at a given corner at a certain time of day is known to have a [pic] distribution. It is desired to test [pic] versus [pic]. From other similar routes, it is known that [pic] has a [pic] distribution. If waiting times of 10, 3, 2, 5, and 14 are observed at the given corner, calculate the posterior probability, the posterior odds ratio, and the Bayes factor.

(If [pic], then [pic].

[pic] )

Solution 6:

2. [pic]

Then,

[pic]

and

[pic].

The posterior odds ratio is

[pic].

Since

[pic]

and

[pic],

the Bayes factor is

[pic].

3.

[pic]

Then,

[pic]

and

[pic].

The posterior odds ratio is

[pic].

Since

[pic]

and

[pic],

the Bayes factor is

[pic].

Homework 7

10. Theory predicts that [pic], the melting point of a particular substance under a pressure of [pic] atmospheres, is 4.01. The procedure for measuring this melting point is fairly inaccurate, due to the high pressure. Indeed it is known that an observation X has a [pic] distribution. 5 independent experiments give observations of 4.9, 5.6, 5.1, 4.6, and 3.6. The prior probability that [pic] is 0.5. The remaining values of [pic] are given the density [pic].

a) Assume [pic]. Formulate and conduct a Bayesian test of the proposed theory.

b) Calculate the p-value against [pic].

c) Calculate the lower bound on the posterior probability of [pic] for any [pic], and find the corresponding lower bound on the Bayes factor.

d) Assume [pic], and graph the posterior probability of [pic] as a function of [pic].

11. Let [pic]. Suppose the prior for [pic] is [pic].

(a) Find the MLE for [pic].

(b)Find the posterior density [pic] for [pic]

c) Find the posterior density [pic] for [pic] and calculate the posterior mean.

Solution 7:

1. The hints are given in the following:

(a) Please calculate [pic] for the hypothesis

[pic].

(b) Please use classical t-test to obtain the p-value!!

(c) Please use the theorem (p.32, course notes) to obtain the lower bounds.

(d) Please use the formula on p. 28 (course notes) to solve this problem.

2.

(a) The MLE is the least square estimate

[pic].

(b)

[pic] and [pic] Then,

[pic]

where

[pic],

[pic]

[pic]

is a bivariate Student’s t-distribution.

(c)

[pic]

where

[pic]

Since

[pic]

the posterior mean is

[pic]

Homework 8

12. Determine the Jeffreys noninformative prior for the unknown parameter in each of the following distributions:

(a) [pic]; (b) [pic]; (c) [pic];

(d) [pic] ([pic] known).

13. Determine the Jeffreys noninformative prior for the unknown vector of parameters in each of the following distributions:

(a) [pic]; (b) [pic] ([pic] unknown).

(c) [pic];

Solution 8:

1..

(a) [pic]

(b) [pic]

(c) [pic]

(d) [pic],

where [pic] and [pic] are the first and the second derivatives of [pic], respectively.

2.

(a) [pic]

(b) [pic]

(c) [pic]

Homework 9

Observations [pic], are independent given two parameters [pic] and [pic], and normally distributed with respective means [pic] and variance [pic], where the [pic] are specified explanatory variables.

a) Suppose [pic] and [pic] are unknown and the prior is [pic].

• Find the marginal posterior density of [pic].

• Find the posterior mean based on the above marginal posterior.

• Find the generalized maximum likelihood estimate of [pic] based on the marginal posterior.

b) Suppose [pic] is known and the prior is [pic].

• Find the marginal posterior density of [pic].

• Find the generalized maximum likelihood estimate of [pic] based on the above estimate.

• For the following data,

|[pic] |1 |-1 |0 |-1 |1 |

|[pic] |3 |2 |0 |3 |2 |

and the hypothesis [pic], find the posterior probability for [pic] and the posterior odds ratio.

Solution 9:

1..

(a) [pic] and [pic] Then,

[pic]

where

[pic].

Thus,

[pic]

The posterior mean is

[pic].

The generalized M.L.E. can be obtained by solving

[pic]

(b)

[pic] and [pic] Then,

[pic]

Since [pic], the posterior distribution is

[pic].

The posterior probability for [pic] is

[pic],

where Z is the standard normal random variable. The posterior odds ratio then is [pic].

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download