The Frisch–Waugh–Lovell theorem for the lasso and the ridge regression

The Frisch–Waugh–Lovell (FWL) (partitioned regression) theorem is essential in regression analysis. This is partly because it is quite useful to derive theoretical results. The lasso regression and the ridge regression, both of which are penalized least-squares regressions, have become popular statistical techniques. This article describes that the FWL theorem remains valid for these penalized least-squares regressions. More precisely, we demonstrate that the covariates corresponding to unpenalized regression parameters in these penalized least-squares regression can be projected out. Some other results related to the FWL theorem in such penalized least-squares regressions are also presented.