Applied Stats and Data Analysis EN.553.413-613, Spring 2024 Feb 21, 2024 Exam 1

Question 1 (18 pts). The following TRUE/FALSE questions concern the Simple Linear Regression model Yi = β0 + β1Xi + εi , E(εi) = 0, V ar(εi) = σ 2 , cov(εi , εj ) = 0, for i = j. (a) TRUE or FALSE. For the least squares estimates b0, b1 we require the errors to be normally distributed. (b) TRUE or FALSE. The estimated mean of the response variable at Xi is defined as b0 + b1Xi . (c) TRUE or FALSE. One of the Gauss Markov conditions is P n i=1 ei = 0. (d) TRUE or FALSE. Plotting e 2 i vs Yˆ i is one of the diagnostic plots. (e) TRUE or FALSE. QQ plot of the Yi ’s is one of the diagnostic plots. (f) TRUE or FALSE. Low R2 means that X and Y are not related. (g) TRUE or FALSE. The s 2 is an estimate of the variance of Yi . (h) TRUE or FALSE. Coefficient of simple determination R2 measures the proportion of the explained variation in Y over the unexplained variation in Y . (i) TRUE or FALSE. In the Correlation model of the regression Xi ’s are random variables.

Question 2 (18 pts). Let X, Y, Z ∼ iid N(0, 1), i.e. they are independent, identically distributed standard normal random variables. For the following random variables state whether they follow a normal distribution, a t- distribution, a χ 2 distribution, an F distribution, or none of the above. State relevant parameters (e.g. degrees of freedom, and means and variances for normal RVs)

(a) 3Y − Z (b) X + Y + Z. (c) X2 + Y 2 + Z 2 . X2 + Y 2 (d) 2Z2 X2 (e) √ Y 2 + Z2 (X + Y ) 2 (f) 2

Question 3 (20 pts). Suppose a data set {(Xi , Yi) : 1 ≤ i ≤ n} is fit to a linear model of the form. Yi = β0 + β1xi + εi where εi are independent, mean zero, and normal with common variance σ 2 . Here we treat Y as the response variable and X as the predictor variable. The output of the lm function is given. Some values are hidden by ‘XXXXX’. We provide you with additional value: X¯ = 1.11. Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 1.9412 0.4593 4.226 0.000508 *** x 0.7042 0.3697 1.905 0.072911 . --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.9221 on 18 degrees of freedom Multiple R-squared: 0.1678, Adjusted R-squared: ----- F-statistic: XXXXX on XX and XX DF, p-value: XXXXXX (a) (2 points) How many data points are there (what is n, the sample size)? What is the estimated mean of the response variable Y at Xh = 2 for this dataset? (b) (3 points) Based on all of this output, do you reject H0 : β1 = 0 in favour of Ha : β1 = 0 at level α = 0.05 significance? What does the test tell us about the relationship between X and Y ? (c) (3 points) Based on all of this output, do you reject the H0 : β1 = 0 vs Ha : β1 > 0 at level α = 0.05 significance? Briefly explain why, or why not. (d) (4 points) The degrees of freedom, the p-value and the value of the F statistic are hidden. Is it possible to reconstruct all of them based on the data shown? Recover as many values as you can. (e) (4 points) Based on the data above find SSTo, SSR and SSE. Hint: Residual standard error may be useful here. (f) (4 points) Find the 95% confidence interval for the mean of the response function at Xh = 2. Write your answer in the form. A ± B · t(C, D), specify values A, B, C, D as precise as you can (i.e. find values of as many terms as you can).

Question 4 (14 pts). Consider the following diagnostic plots for two models (Model 1 and Model 2). Two simple linear regression models Y = β0 + β1X + ε are fitted to the two different datasets (X, Y ) observations of each Model. For each model 3 diagnostic plots are shown: plot of Yi vs Xi , plot of semi-studentized residuals e ∗ i versus fitted values Yˆ i , QQ-plot of the semi-studentized residuals e ∗ i .

(a) (5 points) What is the main issue do you diagnose with the Model 1, if any? Why? Which plot was the most useful in diagnosing this problem? Be as specific in describing the issue as you can. (b) (5 points) What is the main issue do you diagnose with the Model 2, if any? Why? Which plot was the most useful in diagnosing this problem? Be as specific in describing the issue as you can. (c) (4 points) This question is unrelated to the above plots. Explain in what cases the transformation of the predictor variable X is more appropriate than the transformation of the response variable Y .

Question 5 (20 points). For the dataset of n = 200 observations a simple linear regression model Yi = β0 + β1Xi + εi is fit. The following estimates are obtained. b0 = 2, b1 = 1 We have listed additional information here

(a) (2 points) What is the estimated variance s 2 of the error term based on the data above? (b) (3 points) Find a 90% confidence interval for β1. Write it in the form. A ± B · t(C, D), compute values of A, B, C, D if possible. (c) (4 points) Find the joint confidence intervals with confidence at least 90% for β0, β1 in the form. Ai ± Bi · t(Ci , Di). Compute values of Ai , Bi , Ci , Di if possible. Without any computation how does the interval for β1 for this part compare to the one in part (a)? (d) (4 points) Find the joint confidence intervals using Bonferroni procedure with confidence at least 90% for the mean of the response variable at Xh = 2 and Xh′ = 0. Find it in the form. Ai ± Bi · t(Ci , Di). (e) (4 points) Set up a General Linear Test for the data provided: specify the reduced and full model, compute the value of the F-statistic, specify its distribution under the null hypothesis. (f) (3 points) An Aspiring Data Scientist (ADS) noticed that one of the observed data points (Xi , Yi) = (2, 15) lies outside of the 99% Working-Hotelling band (we assume everything was computed correctly). They claim it is an issue. Briefly justify if their concern is correct or not.

Question 6 (20 points). Suppose Yi follows the model Yi = βXi + εi where εi is independent, identically distributed N(0, σ2 ). Note, there is no intercept term. You observe a collection {(Xi , Yi)} of data from this model, i = 1, . . . , n. (a) (5 points) Write the objective function to be minimized and the equations that need to be solved to get the least squares estimate of β. (b) (5 points) Solve the equation in (a) and express the answer as a linear combination of Yi ’s. (c) (5 points) What is the distribution of b? Find the mean, variance. Justify your steps (d) (5 points) Write the log-likelihood that needs to be maximized to obtain the estimate of β. DO NOT MAXIMIZE IT. (a) Function to be minimized

Equations to be solved:

(b) Solving the equation:

(c) Since b = P i ciYi , a linear combination of normal RVs, it will be a normal RV itself. The mean is

The variance is

We have shown that

(d) The log-likelihood is