Linear Model APIs#

class aoclda.linear_model.linmod(mod, intercept='False', solver='auto', scaling='auto', max_iter=None, constraint='ssc', precision='double')#

Linear models.

Parameters
  • mod (str) –

    Which linear model to compute.

    • If linmod_model='mse' then \(\ell_2\) norm linear regression is calculated.

    • If linmod_model='logistic' then logistic regression is calculated.

  • intercept (bool, optional) – Controls whether to add an intercept variable to the model. Default=False.

  • solver (str, optional) –

    Which solver to use to compute the coefficients. It can take values ‘auto’, ‘svd’, ‘cholesky’, ‘sparse_cg’, ‘qr’, ‘coord’, ‘lbfgs’. Some solvers are not suitable for some regularization types.

    • 'auto' chooses the best solver based on regression type and data.

    • 'svd' works with normal and ridge regression. The most robust solver but the least efficient.

    • 'cholesky' works with normal and ridge regression. Will return an error when a singular matrix is encountered.

    • 'sparse_cg' works with normal and ridge regression. You may need to set a smaller tol if a badly conditioned matrix is encountered.

    • 'qr' works with normal linear regression only. Will return an error when an underdetermined system is encountered.

    • 'coord' works with all regression types. Requires data to have variance of 1 column-wise (this can be achieved with the scaling option set to scale only). In the case of normal linear regression and an underdetermined system it will converge to a solution that is not necessarily a minimum norm solution.

    • 'lbfgs' works with normal and ridge regression. In the case of normal linear regression and an underdetermined system it will converge to a solution that is not necessarily a minimum norm solution.

  • scaling (str, optional) – What type of preprocessing you want to apply on the dataset. Available options are: ‘none’, ‘centering’, ‘scale_only’, ‘standardize’.

  • max_iter (int, optional) – Maximum number of iterations. Applies only to iterative solvers: ‘sparse_cg’, ‘coord’, ‘lbfgs’. The default value depends on the solver. For ‘sparse_cg’ it is 500, for ‘lbfgs’ and ‘coord’ it is 10000.

  • constraint (str, optional) –

    Affects only multinomial logistic regression. The type of constraint put on coefficients. This will affect the number of coefficients returned.

    • 'rsc' means we choose a reference category whose coefficients will be set to all 0. This results in K-1 class coefficients for K class problems.

    • 'ssc' means the sum of coefficients class-wise for each feature is 0. It will result in K class coefficients for K class problems.

  • precision (str, optional) – Whether to compute the linear model in double or single precision. It can take the values ‘single’ or ‘double’. Default = ‘double’.

property coef#

contains the output coefficients of the model. If an intercept variable was required, it corresponds to the last element.

Type

numpy.ndarray

fit(X, y, reg_lambda=0.0, reg_alpha=0.0, x0=None, tol=0.0001, progress_factor=None)#

Computes the chosen linear model on the feature matrix X and response vector y

Parameters
  • X (numpy.ndarray) – The feature matrix on which to compute the model. Its shape is (n_samples, n_features).

  • y (numpy.ndarray) – The response vector. Its shape is (n_samples).

  • reg_lambda (float, optional) – \(\lambda\), the magnitude of the regularization term. Default=0.0.

  • reg_alpha (float, optional) – \(\alpha\), the share of the \(\ell_1\) term in the regularization.

  • x0 (numpy.ndarray, optional) – Initial guess for solution. Applies only to iterative solvers

  • tol (float, optional) – Convergence tolerance for iterative solvers. Applies only to iterative solvers: ‘sparse_cg’, ‘coord’, ‘lbfgs’.

  • progress_factor (float, optional) – Applies only to ‘lbfgs’ and ‘coord’ solver. Factor used to detect convergence of the iterative optimization step.

Returns

Returns the instance itself.

Return type

self (object)

property loss#

The value of loss function \(L(\beta_0, \beta)\).

Type

numpy.ndarray of shape (1, )

property n_iter#

The number iterations performed to find the solution. Only valid for iterative solvers.

Type

int

property nrm_gradient_loss#

The norm of the gradient of the loss function. Only valid for iterative solvers.

Type

numpy.ndarray of shape (1, )

predict(X)#

Evaluate the model on a data set X.

Parameters

X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

Returns

The prediction vector, where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

property time#

Compute time (wall clock time in seconds).

Type

numpy.ndarray of shape (1, )

typedef enum linmod_model_ linmod_model#

Alias for the linmod_model_ enum.

enum linmod_model_#

Defines which linear model is computed.

Values:

enumerator linmod_model_undefined#

No linear model set.

enumerator linmod_model_mse#

\(L_2\) norm linear regression.

enumerator linmod_model_logistic#

Logistic regression.

da_status da_linmod_select_model_s(da_handle handle, linmod_model mod)#
da_status da_linmod_select_model_d(da_handle handle, linmod_model mod)#

Select which linear model to compute.

The last suffix of the function name marks the floating point precision on which the handle operates (see precision section).

The model definition can be further enhanced with elements such as a regularization term by setting up optional parameters. See the linear model options section for more information.

Parameters
Returns

da_status. The function returns:

da_status da_linmod_define_features_s(da_handle handle, da_int n_samples, da_int n_features, float *X, float *y)#
da_status da_linmod_define_features_d(da_handle handle, da_int n_samples, da_int n_features, double *X, double *y)#

Define the data to train a linear model.

The last suffix of the function name marks the floating point precision on which the handle operates (see precision section).

Pass pointers to a data matrix X containing n_samples observations (rows) over n_features features (columns) and a response vector y of size n_samples.

Only the pointers to X and y are stored; no internal copy is made.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_linmod.

  • n_samples – [in] the number of observations (rows) of the data matrix X. Constraint: n_samples \(\ge\) 1.

  • n_features – [in] the number of features (columns) of the data matrix, X. Constraint: n_features \(\ge\) 1.

  • X – [in] the n_samples \(\times\) n_feat data matrix, in column major format.

  • y – [in] the response vector, of size n_samples.

Returns

da_status. The function returns:

da_status da_linmod_fit_s(da_handle handle)#
da_status da_linmod_fit_d(da_handle handle)#

Fit the linear model defined in the handle.

Compute the linear model defined by da_linmod_select_model_? on the data passed by the last call to the function da_linmod_define_features_?.

Note that you can customize the model before using the fit function through the use of optional parameters, see this section for a list of available options (e.g., the regularization terms).

Parameters

handle – [inout] a da_handle object, initialized with type da_handle_linmod.

Returns

da_status. The function returns:

da_status da_linmod_fit_start_s(da_handle handle, da_int n_coefs, float *coefs)#
da_status da_linmod_fit_start_d(da_handle handle, da_int n_coefs, double *coefs)#

Fit the linear model defined in the handle using a custom starting estimate for the model coefficients.

Compute the same model as da_linmod_fit_?, starting the fitting process with the custom values defined in coefs.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_linmod.

  • n_coefs – [in] the number of coefficients provided in coefs. It must match the number of expected coefficients for the model defined in handle to be taken into account.

  • coefs – [in] the initial coefficients.

Returns

da_status. The function returns:

da_status da_linmod_evaluate_model_s(da_handle handle, da_int nsamples, da_int nfeat, float *X, float *predictions, float *observations, float *loss)#
da_status da_linmod_evaluate_model_d(da_handle handle, da_int nsamples, da_int nfeat, double *X, double *predictions, double *observations, double *loss)#

Evaluate the model previously computed on a new set of data X and observations y.

After a model has been fitted using da_linmod_fit_?, it can be evaluated on a new set of data and observations. This function returns the model evaluation (loss) in the array loss and the predictions in predictions.

In the case where the model chosen solves a classification problem (e.g., logistic regression), the predictions computed will be categorical. For each data point i, prediction[i] will contain the index of the most likely class according to the model.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_linmod.

  • nfeat – number of columns of X or equivalently the number of features of the test data. It must match the number features of the data defined in the handle.

  • nsamples – number of rows of X or equivalently the number of samples to estimate the model on.

  • X – the nsamples \(\times\) nfeat data matrix to evaluate the model on, in column major format.

  • predictions – vector of size nsamples containing the model’s prediction.

  • observations – vector of size nsamples containing new observations; may be NULL if none are provided.

  • loss – scalar containing the model’s loss given the new data X and the new observations y; may be NULL if no observations are provided. Note that either both observations and loss parameters are NULL or both must contain a valid address.

Returns

da_status