AMS572 - Stony Brook University



AMS570.01 Final Exam Spring 2018Name: _______________________________________________ ID: __________________________________ Signature: ________________________Instruction: This is a close book exam. You are allowed two pages 8x11 formula sheet (2-sided). No cellphone or calculator or computer or smart watch is allowed. Cheating shall result in a course grade of F. Please provide complete solutions for full credit. The exam goes from 11:15am-1:45pm. Good luck! 1. Let X1,…,Xnbe iid Poisson(λ), let λ be a gammaα,β distribution, the conjugate family for the Poisson. (a) Find the posterior distribution of λ, and subsequently the Bayesian estimator of λ.(b) Derive the mode of the posterior distribution of λ, and then discuss when and how we can derive the 100(1-α)% Bayesian HPD credible set for λ. Solution:(a) The Poisson pdf is:fx|λ=e-λλxx!, x=0, 1, …Therefore the likelihood function is:fx| λ=fx1,…,xn|λ=e-nλλixiixi!=e-nλλixi?1ixi!=gixi,λ?h(x1,…,xn)By the factorization theorem, Y=iXi is a sufficient statistic for λ.The mgf of Y|λ is:MY|λt=i=1ne-λet-1=e-nλet-1Therefore we know Y|λ ~ Poisson(nλ).I. For the prior:πλ=1Γαβαλα-1e-λβThe marginal pdf of Y ismy=-∞∞fy|λπλdλ=0∞nλye-nλy!1Γαβαλα-1e-λβdλ=nyy!Γαβα 0∞λy+α-1 e-λβ/(nβ+1)dλ=nyy!ΓαβαΓy+αβnβ+1y+aThus the posterior distribution is,πλy=f(y|λ)π(λ)m(y)=λy+α-1e-λβ/(nβ+1)Γy+αβnβ+1y+a~Gammay+α,βnβ+1.The Bayesian estimator for λ is:Eλy=y+αβnβ+1=βnβ+1y+1nβ+1αβ=y+αβnβ+1(b) lnπλy=c+y+α-1lnλ-λβ/(nβ+1)?lnπλy?λ=y+α-1λ-1βnβ+1=0?λ=y+α-1βnβ+1, for y+α-1>0Therefore when y+α-1>0, the mode exists, we can obtain the Bayesian HPD [a, b] as illustrated below by the Theorem of Optimal Interval. II. For the prior:πλ=βαΓαλα-1e-λβThe posterior distribution is,πλy=f(y|λ)π(λ)m(y)~Gammay+α,n+β.The Bayesian estimator for λ is:Eλy=y+αn+βThe mode for the posterior of λ is:λ=y+α-1n+β, for y+α-1>02. Let X1,…,Xn be a random sample from an exponential distribution exp θ with pdf:f x;θ=1θexp-xθ, where θ>0 and x>0,Please deriveThe maximum likelihood estimator for θ . Is the above MLE unbiased for θ ?Please derive the distribution of the first order statistic X1 and further show whether Y=nX1 is an unbiased estimator of θ or not.Please calculate the MSE (mean squared error) of Y, and the MSE of the MLE for θ , which one is smaller?Is there a best estimator (UMVUE) for θ? Please show the entire derivation.Is there an efficient estimator for θ? Please show the entire derivation.Hint: Cramér-Rao Inequality: Let be unbiased for, where is a random sample from a population with pdf satisfying all regularity conditions. ThenSolution:The likelihood function is L=i=1nf xi;θ=i=1n1θexp-xiθ=1θnexp-i=1nxiθThe log likelihood function is l=lnL=ln1θnexp-i=1nxiθ=-nlnθ-i=1nxiθSolving ?l?θ=-nθ+i=1nxiθ2=0We obtain the MLE for θ:θ=XSinceEX=EX=0∞x1θexp-xθdx=θWe know the MLE θ=X is an unbiased estimator for θ.Now we derive the general formula for the pdf of the first order statistic as follows:PX1>x=PX1>x,…,Xn>x=i=1nPXi>xTherefore we have 1-FX1x=i=1n1-Fx=1-FxnDifferentiating with respect to x, and then multiplying by (-1) at both sides leads to:fX1x=nfx1-Fxn-1Now we can derive the pdf of the first order statistic for a random sample from the given exponential family. First, the population cdf is:Fx=0x1θexp-uθdu=-exp-uθ0x=1-exp-xθNow plugging in the population pdf and cdf we have: fX1x=n1θexp-xθexp-xθn-1=nθexp-nxθ,when θ>0 and x>0. Thus we know that X1~exp θn, and its mean should beEX1=θnTherefore we have: EY=EnX1=nθn=θThat is, Y=nX1 is an unbiased estimator of θ.Since both estimators are unbiased, MSE = Variance. We can easily derive the population variance for X~exp θ to be: VarX=θ2Thus we have: VarX=VarXn=θ2nSimilar we know VarX1=θn2=θ2n2Thus we have: VarY=VarnX1=n2θ2n2=θ2It is obvious that the variance, and thus the MSE, of the MLE X is smaller.The exponential family exp θ with pdf:f x;θ=1θI0,∞exp-xθ== c(θ)h(x) expwθtxis a regular exponential family with and therefore TX=i=1nXiis a complete sufficient statistic for θ. Thus the MLE of θ:θ=TXn=Xis a function of the complete sufficient statistic, and is unbiased for θ. By the Lehmann Scheffe Theorem, the MLE is a best estimator (UMVUE) for θ.Now we calculate the Cramer-Lower bound for the variance of an unbiased estimator for θ:lnf x;θ=ln1θexp-xθ=-lnθ-xθ?lnf x;θ?θ=-1θ+xθ2?2lnf x;θ?θ2=1θ2-2xθ3-nE?2lnf x;θ?θ2=-nE1θ2-2Xθ3=-n1θ2-21θ3θ=nθ2Thus the Cramer-Rao lower bound is: θ2nTherefore we claim that the MLE is an efficient estimator for θ.***Note: You can also first prove (f) the MLE is an efficient estimator for θ, and subsequently immediately claim it is the (e) best estimator (UMVUE) for θ.3. To test whether the birth rates of boys and girls are equal, a random sample of 300 families with exactly three children within the age of 18 was taken and the distribution of the children’s gender is provided below. Please test whether the birth rates are equal or not at the significance level of α = 0.05. Please first derive your test using the pivotal quantity approach, including all steps for full points. 3 girls2 girls, 1 boy1 girl, 2 boys3 boysNo. of families 3611411733Solution:This problem can be done in several ways including, a large sample Z-test for one population proportion, or a Chi-square goodness of fit test. Here we simply use the large sample Z-test.Part 1. Derivation of the general large sample Z-test for one population proportion.Sampling from the Bernoulli DistributionToss a coin, and get the result as following: Head(H), H, Tail(T), T, T, H, …Let A proportion of p is head, in the population. , =0, 1; i = 1, 2, …, n.(*Binomial distribution with n = 1)Inference on p – the population proportion:Point estimator: , is the sample proportion and also, the sample meanLarge sample inference on p:Large sample (the original) pivotal quantity for the inference on p.Alternative P.Q. Z*=p-pp1-pn~N0,1*** You can use either of the pivotal quantity. In the following, we will use the first pivotal quantity, and illustrate the procedure for a two-sided test.Large Sample Test: Test statistic (large sample – using the original P.Q.): At the significance level , we reject in favor of Ha:p≠p0 if:Reject if Part 2. Now we apply the above general test to the given problem.Here we have n=900. Let Y be the total number of girls among the 900 children, we have y=453. Let p be the proportion of entrepreneurs with domestic cars, we have p=453900, and we are testing: versus The test statistics is:Z0=p-0.50.51-0.5/900=0.2Since Z0=0.2<1.96, we can not reject the null hypothesis of equal birth rates at the significance level of 0.05. 4. Suppose we have a simple random sample from a normal population as follows: X1, X2,?, Xn~Nμ, σ2, where μ and σ2 are both unknown.At the significance level α, please construct a test using the pivotal quantity approach to test H0:σ2=σ02 versus Ha:σ2>σ02 where σ02 is a positive constant (*Please include the derivation of the pivotal quantity, the proof of its distribution, and the derivation of the rejection region for full credit.) At the significance level α, please derive the likelihood ratio test for testing H0:σ2=σ02 versus Ha:σ2>σ02. Subsequently, please show whether this test is equivalent to the one derived in part (a). Is the test in part (b) a uniformly most powerful test? Please provide detailed justifications for your answer. Solution:ω=μ,σ2: -∞<μ<∞, σ2=σ02Ω=μ,σ2: -∞<μ<∞, σ2≥σ02Note: Now that we are not just dealing with the two-sided hypothesis, it is important to know that the more general definition of ω is the set of all unknown parameter values under H0, while Ω is the set of all unknown parameter values under the union of H0 and Ha.Therefore we have:Lω=fx1, x2, ? , xnμ, σ2=σ02=i=1nfxi|μ, σ2=σ02=i=1n12πσ02e-xi-μ22σ02=2πσ02-n2e-12σ02i=1nxi-μ2LΩ=fx1, x2, ? , xnμ, σ2≥σ02=i=1nfxi|μ, σ2≥σ02=i=1n12πσ2e-xi-μ22σ2=2πσ2-n2e-12σ2i=1nxi-μ2∵?lnLω?μ=nσ02x-μ=0?μω=xSo,supLω=2πσ02-n2e-12σ02i=1nxi-x2∵?logLΩ?μ=nσ2x-μ=0?μΩ=x?logLΩ?σ2=n2σ2+12σ4i=1nxi-μ2=0?σ2=1ni=1nxi-x2?2logLΩ?σ22=-n2σ4-1σ6i=1nxi-μ2<0∴supLΩ=2πσ2-n2e-n2, if σ2>σ022πσ02-n2e-12σ02i=1nxi-x2, if σ2≤σ02∴LR=supLωsupLΩ=i=1nxi-x2nσ02-n2en2-12σ02i=1nxi-x2, if σ2>σ021, if σ2≤σ02.∴PLR≤c*H0=α?PW0=i=1nxi-x2σ02≥c**H0=α.∴ Reject H0 in favor of Ha if W0=i=1nxi-x2σ02≥χn-1,α,upper2 Furthermore, it is easy to show that for each σ12>σ02, the likelihood ratio test of H0:σ2=σ02 versus Ha:σ2=σ12 rejects the null hypothesis at the significance level α for W0=i=1nxi-x2σ02≥χn-1,α,upper2By the Neyman-Pearson Lemma, the likelihood ratio test is also the most powerful test.Now since for each σ12>σ02, the most powerful size α test of H0:σ2=σ02 versus Ha:σ2=σ12 rejects the null hypothesis for W0=i=1nxi-x2σ02≥χn-1,α,upper2. This rejection region does not depend on σ12 -- it is the same rejection region, hence test, for each σ12. Since this same test is most powerful for each σ12>σ02, this test is UMP for H0:σ2=σ02 versus Ha:σ2>σ02, by the definition of the UMP test. 5. Suppose that the random variables Y1, Y2,?, Yn satisfy Yi=βxi+εi, i=1,?,n , where x1, x2,?, xn are fixed constants, and ε1, ε2,?, εn are independent Gaussian random variables with mean 0 and variance c1σ2, c2σ2, ?,cnσ2, respectively (ci>0, i= 1,?,n, are known constants, while σ2is also a known constant). Please derive:The (ordinary) least squares estimator (LSE) for β.The maximum likelihood estimator (MLE) for β. Is the MLE the same as the LSE?At the significance level α, please derive a test for H0:β=0 versus Ha:β≠0. Solution: To minimize SS=i=1n(yi-βxi)2:?SS?β=0?i=1nxiyi-βxi=0?β=i=1nxiYii=1nxi2y=βx is the least squares regression line through the origin.Let wi=yici vi=xici ei=εiciwe havewi=βvi+ei, i=1, 2, … ,n, vi is given, and eiiid~N0, σ2,Wi~Nβvi,σ2, i=1,?,nSince W1, W2,?, Wn are independent to each other, the likelihood is:L=i=1n12πσ2e-(wi-βvi)22σ2=2πσ2-n/2exp-i=1n(wi-βvi)22σ2The log likelihood is: lnL=-n2ln2πσ2-i=1n(wi-βvi)22σ2Take derivative with respect to β, and set it to zero, we obtain the MLE:β=i=1nviWii=1nvi2=i=1nxiYi/cii=1nxi2/ciTherefore we can easily see that the MLE and the LSE are no longer identical when the ci's are not all equal to each other.Let θi=xi/cii=1nxi2/ci, i=1,?,nThen θiYi~Nθiβxi,θi2ciσ2, i=1,?,nFurthermore, they are independent to each other. We have the moment generating function for β:Mβt=Eexptβ=Eexpti=1nθiYi=i=1nEexptθiYi=i=1nexptθiβxi+12t2θi2ciσ2=exptiθiβxi+12t2iθi2ciσ2=exptβ+12t2σ2i=1nxi2/ciTherefore we found:β~Nβ,σ2i=1nxi2/ciThe pivotal quantity for β is:Z=β-βσ2i=1nxi2/ci~N0,1For the test of: H0:β=0 versus Ha:β≠0,The test statistic is: Z0=β-0σ2i=1nxi2/ci H0~ N0,1At the significance level , we reject H0 in favor of Ha if:Reject if *** That’s all, folks! *** ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download