Intro to Statistical Learning (2nd Ed), Solution to Problem 6.7C | Bayes connection w lasso/ridge

6.7C
We will now derive the Bayesian connection to the lasso and ridge regression discussed in Section 6.2.2.

(a) Suppose that yi=β0+∑pj=1xijβj+ϵj where ϵ1,...,ϵn are independent and identically distributed from a N(0,σ2) distribution. Write out the likelihood for the data.

(b) Assume the following prior for β:β1,...,βp are independent and identically distributed according to a double-exponential distribution with mean 0 and common scale parameter b : i.e. p(β)=12bexp(−|β|/b) . Write out the posterior for β in this setting.

(c) Argue that the lasso estimate is the mode for β under this posterior distribution.

(d) Now assume the following prior for β:β1,...,βp are independent and identically distributed according to a normal distribution with mean zero and variance c . Write out the posterior for β in this setting.

(e) Argue that the ridge regression estimate is both the mode and the mean for β under this posterior distribution.


Colab (R): https://colab.research.google.com/dri...
My GitHub (R): https://github.com/nicklausmillican/I...
My GitHub (Python): https://github.com/nicklausmillican/I...
Download Book: https://www.statlearning.com/
Authors' Lectures (R):    • Statistical Learning with R  
Authors' Lectures (Python):    • Statistical Learning with Python