Online Coordinate Coordinate Descent

Overview

This submodule holds the functions related to the online coordinate descent (OCO).

API Reference

ondil.coordinate_descent.online_coordinate_descent

online_coordinate_descent(
    x_gram: ndarray,
    y_gram: ndarray,
    beta: ndarray,
    regularization: float,
    is_regularized: bool,
    alpha: float,
    beta_lower_bound: ndarray,
    beta_upper_bound: ndarray,
    selection: Literal["cyclic", "random"] = "cyclic",
    tolerance: float = 0.0001,
    max_iterations: int = 1000,
) -> Tuple[np.ndarray, int]

The parameter update cycle of the online coordinate descent.

Parameters:

• x_gram (ndarray) –

  X-Gramian \(\(X^TX\)\)

• y_gram (ndarray) –

  Y-Gramian \(\(X^TY\)\)

• beta (ndarray) –

  Current beta vector

• regularization (float) –

  Regularization parameter lambda

• is_regularized (bool) –

  Vector of bools indicating whether the coefficient is regularized

• beta_lower_bound (ndarray) –

  Lower bounds for beta

• beta_upper_bound (ndarray) –

  Upper bounds for beta

• selection (Literal['cyclic', 'random'], default: 'cyclic' ) –

  Apply cyclic or random coordinate descent. Defaults to "cyclic".

• tolerance (float, default: 0.0001 ) –

  Tolerance for the beta update. Defaults to 1e-4.

• max_iterations (int, default: 1000 ) –

  Maximum iterations. Defaults to 1000.

Returns:

• Tuple[ndarray, int] –

  Tuple[np.ndarray, int]: Converged $$ \beta $$

ondil.coordinate_descent.online_coordinate_descent_path

online_coordinate_descent_path(
    x_gram: ndarray,
    y_gram: ndarray,
    beta_path: ndarray,
    lambda_path: ndarray,
    is_regularized: ndarray,
    alpha: float,
    early_stop: int,
    beta_lower_bound: ndarray,
    beta_upper_bound: ndarray,
    which_start_value: Literal[
        "previous_lambda", "previous_fit", "average"
    ] = "previous_lambda",
    selection: Literal["cyclic", "random"] = "cyclic",
    tolerance: float = 0.0001,
    max_iterations: int = 1000,
) -> Tuple[np.ndarray, np.ndarray]

Run coordinate descent on a grid of regularization values.

Parameters:

• x_gram (ndarray) –

  X-Gramian \(\(X^TX\)\)

• y_gram (ndarray) –

  Y-Gramian \(\(X^TY\)\)

• beta_path (ndarray) –

  The current coefficent path

• lambda_path (ndarray) –

  The lambda grid

• is_regularized (bool) –

  Vector of bools indicating whether the coefficient is regularized

• early_stop (int) –

  Early stopping criterion. 0 implies no early stopping. Defaults to 0.

• beta_lower_bound (ndarray) –

  Lower bounds for beta

• beta_upper_bound (ndarray) –

  Upper bounds for beta

• which_start_value (Literal['previous_lambda', 'previous_fit', 'average'], default: 'previous_lambda' ) –

  Values to warm-start the coordinate descent. Defaults to "previous_lambda".

• selection (Literal['cyclic', 'random'], default: 'cyclic' ) –

  Apply cyclic or random coordinate descent. Defaults to "cyclic".

• tolerance (float, default: 0.0001 ) –

  Tolerance for the beta update. Will be passed through to the parameter update. Defaults to 1e-4.

• max_iterations (int, default: 1000 ) –

  Maximum iterations. Will be passed through to the parameter update. Defaults to 1000.

Returns:

• Tuple[ndarray, ndarray] –

  Tuple[np.ndarray, np.ndarray]: Tuple with the updated coefficient path and the iteration count.

ondil.coordinate_descent.soft_threshold

soft_threshold(value: float, threshold: float)

The soft thresholding function.

For value \(x\) and threshold \(\lambda\), the soft thresholding function \(S(x, \lambda)\) is defined as:

\[S(x, \lambda) = sign(x)(|x| - \lambda)\]

Parameters:

• value (float) –

  The value

• threshold (float) –

  The threshold

Returns:

• out ( float ) –

  The thresholded value