Online Coordinate Coordinate Descent

Overview

This submodule holds the functions related to the online coordinate descent (OCO).

API Reference

rolch.online_coordinate_descent

online_coordinate_descent(x_gram: np.ndarray, y_gram: np.ndarray, beta: np.ndarray, regularization: float, is_regularized: bool, beta_lower_bound: np.ndarray, beta_upper_bound: np.ndarray, selection: Literal['cyclic', 'random'] = 'cyclic', tolerance: float = 0.0001, max_iterations: int = 1000) -> Tuple[np.ndarray, int]

The parameter update cycle of the online coordinate descent.

Parameters:

Name	Type	Description	Default
x_gram	ndarray	X-Gramian \(\(X^TX\)\)	required
y_gram	ndarray	Y-Gramian \(\(X^TY\)\)	required
beta	ndarray	Current beta vector	required
regularization	float	Regularization parameter lambda	required
is_regularized	bool	Vector of bools indicating whether the coefficient is regularized	required
beta_lower_bound	ndarray	Lower bounds for beta	required
beta_upper_bound	ndarray	Upper bounds for beta	required
selection	Literal['cyclic', 'random']	Apply cyclic or random coordinate descent. Defaults to "cyclic".	'cyclic'
tolerance	float	Tolerance for the beta update. Defaults to 1e-4.	0.0001
max_iterations	int	Maximum iterations. Defaults to 1000.	1000

Returns:

Type	Description
ndarray	Tuple[np.ndarray, int]: Converged $$ \beta $$

rolch.online_coordinate_descent_path

online_coordinate_descent_path(x_gram: np.ndarray, y_gram: np.ndarray, beta_path: np.ndarray, lambda_path: np.ndarray, is_regularized: np.ndarray, beta_lower_bound: np.ndarray, beta_upper_bound: np.ndarray, which_start_value: Literal['previous_lambda', 'previous_fit', 'average'] = 'previous_lambda', selection: Literal['cyclic', 'random'] = 'cyclic', tolerance: float = 0.0001, max_iterations: int = 1000) -> Tuple[np.ndarray, np.ndarray]

Run coordinate descent on a grid of regularization values.

Parameters:

Name	Type	Description	Default
x_gram	ndarray	X-Gramian \(\(X^TX\)\)	required
y_gram	ndarray	Y-Gramian \(\(X^TY\)\)	required
beta_path	ndarray	The current coefficent path	required
lambda_path	ndarray	The lambda grid	required
is_regularized	bool	Vector of bools indicating whether the coefficient is regularized	required
beta_lower_bound	ndarray	Lower bounds for beta	required
beta_upper_bound	ndarray	Upper bounds for beta	required
which_start_value	Literal['previous_lambda', 'previous_fit', 'average']	Values to warm-start the coordinate descent. Defaults to "previous_lambda".	'previous_lambda'
selection	Literal['cyclic', 'random']	Apply cyclic or random coordinate descent. Defaults to "cyclic".	'cyclic'
tolerance	float	Tolerance for the beta update. Will be passed through to the parameter update. Defaults to 1e-4.	0.0001
max_iterations	int	Maximum iterations. Will be passed through to the parameter update. Defaults to 1000.	1000

Returns:

Type	Description
ndarray	Tuple[np.ndarray, np.ndarray]: Tuple with the updated coefficient path and the iteration count.

rolch.soft_threshold

soft_threshold(value: float, threshold: float)

The soft thresholding function.

For value \(x\) and threshold \(\lambda\), the soft thresholding function \(S(x, \lambda)\) is defined as:

\[S(x, \lambda) = sign(x)(|x| - \lambda)\]

Parameters:

Name	Type	Description	Default
value	float	The value	required
threshold	float	The threshold	required

Returns:

Name	Type	Description
out	float	The thresholded value