fsml_ridge Interface

public interface fsml_ridge

The multiple linear Ridge regression models the relationship or linear dependence between a dependent (predictand) variable and one or more independent (predictor) variables, incorporating a penalty term on the size of the regression coefficients to reduce multicollinearity and overfitting.

The procedure estimates the linear regression coefficients by minimising the sum of squared residuals plus a penalty proportional to the square of the magnitude of coefficients:

$$\hat{\beta} = (X^\top X + \lambda I)^{-1} X^\top y$$

where $\lambda$ (lambda) is the ridge penalty parameter, and $I$ is the identity matrix with the first diagonal element corresponding to the intercept set to zero (no penalty on intercept).

The estimated regression model is of the form:

$$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_m x_m + \dots + \beta_M x_M$$

where $y$ is the predictand variable, $x_m \ (m = 1 \dots M)$ are the predictor variables (x) with nd observations, $\beta_0$ is the y-intercept (b0), $\beta_m \ (m = 1 \dots M)$ (b) are the ridge regression coefficients, and $M$ (nv) is the number of predictors (excluding the intercept).

The subroutine constructs a full matrix internally by prepending a column of ones to account for the intercept. The coefficient of determination $R^2$ (r2), predicted values (y_hat), ridge-adjusted standard errors (se) of the coefficients, and the ridge-adjusted covariance matrix of the predictors (cov_b) can optionally be returned. The covariance matrix and standard errors are adjusted for the ridge penalty as:

$$\mathrm{cov}(\hat{\beta}) = \sigma^2 (X^\top X + \lambda I)^{-1} X^\top X (X^\top X + \lambda I)^{-1}$$

where $\sigma^2$ is the residual variance estimate.

Note: This subroutine uses eigh from the stdlib_linalg module.

Module Procedures