Decision tree and decision forest APIs#

This chapter contains two sets of APIs, one for classification using a single decision tree and another one for the ensemble method decision forests (also known as random forests).

Decision trees#

class aoclda.decision_tree.decision_tree(criterion='gini', seed=-1, max_depth=10, max_features=0, min_samples_split=2, build_order='breadth first', precision='double')#

A decision tree classifier.

Parameters
  • seed (int, optional) – Set the random seed for the random number generator. If the value is -1, a random seed is automatically generated. In this case the resulting classification will create non-reproducible results. Default = -1.

  • max_depth (int, optional) – Set the maximum depth of the tree. Default = 29.

  • max_features (int, optional) – Set the number of features to consider when splitting a node. 0 means take all the features. Default 0.

  • criterion (str, optional) – Select scoring function to use. It can take the values ‘cross-entropy’, ‘gini’, or ‘misclassification’

  • min_samples_split (int, optional) – The minimum number of samples required to split an internal node. Default 2.

  • build_order (str, optional) – Select in which order to explore the nodes. It can take the values ‘breadth first’ or ‘depth first’. Default ‘breadth first’.

  • precision (str, optional) – Whether to initialize the PCA object in double or single precision. It can take the values ‘single’ or ‘double’. Default = ‘double’.

fit(X, y, min_impurity_decrease=0.0, min_split_score=0.0, feat_thresh=1e-06)#

Computes the decision tree on the feature matrix X and response vector y

Parameters
  • X (numpy.ndarray) – The feature matrix on which to compute the model. Its shape is (n_samples, n_features).

  • y (numpy.ndarray) – The response vector. Its shape is (n_samples).

  • min_impurity_decrease (float, optional) – Minimum score improvement needed to consider a split from the parent node. Default 0.0

  • min_split_score (float, optional) – Minimum score needed for a node to be considered for splitting. Default 0.0.

  • feat_thresh (float, optional) – Minimum difference in feature value required for splitting. Default 1.0e-06

Returns

Returns the instance itself.

Return type

self (object)

property n_leaves#

The number of nodes in the trained tree

Type

int

property n_nodes#

The number of nodes in the trained tree

Type

int

predict(X)#

Generate labels using fitted decision forest on a new set of data X.

Parameters
  • X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have

  • columns. (n_features) –

Returns

The prediction vector, where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

predict_log_proba(X)#

Generate class log probabilities using fitted decision forest on a new set of data X.

Parameters

X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

Returns

The prediction vector,

where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

predict_proba(X)#

Generate class probabilities using fitted decision forest on a new set of data X.

Parameters
  • X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have

  • columns. (n_features) –

Returns

The prediction vector, where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

score(X, y)#

Calculates score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data.

Parameters
  • X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

  • y (numpy.ndarray) – The response vector. It must have shape (n_samples).

Returns

The mean accuracy of the model on the test data.

Return type

float

da_status da_tree_set_training_data_s(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, float *X, da_int ldx, da_int *y)#
da_status da_tree_set_training_data_d(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, double *X, da_int ldx, da_int *y)#

Pass a data matrix and a label array to the da_handle object in preparation for fitting a decision tree.

Parameters
  • handle – a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – number of observations in X.

  • n_features – number of features in X.

  • n_class – number of distinct classes in y. Will be computed automatically if n_class is set to 0.

  • X – array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx – leading dimension of X. Constraint: ldx \(\ge\) n_samples.

  • y – array containing the n_samples labels. The label values are expected to range from 0 to n_class - 1.

Returns

da_status. The function returns:

da_status da_tree_fit_s(da_handle handle)#
da_status da_tree_fit_d(da_handle handle)#

Fit the decision tree defined in the handle.

Compute the decision tree parameters given the data passed by da_tree_set_training_data_?. Note that you can customize the model before using the fit function through the use of optional parameters, see this section for a list of available options.

Parameters

handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

Returns

da_status. The function returns:

Post

After successful execution, da_handle_get_result_? can be queried with the following enum:

  • da_rinfo - return an array of size 5 containing n_features, n_samples, the number of samples the tree was trained on, the value of the random seed used to fit the tree and the depth of the tree.

da_status da_tree_predict_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_pred)#
da_status da_tree_predict_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_pred)#

Generate labels using fitted decision tree on a new set of data X_test.

After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree predictions in the array y_pred.

For each data point i, y_pred[i] will contain the label of the most likely class according to the decision tree, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_pred – [out] - array of size at least n_samples. On output, will contain the predicted class labels.

Returns

da_status

da_status da_tree_predict_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_proba, da_int n_class, da_int ldy)#
da_status da_tree_predict_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_proba, da_int n_class, da_int ldy)#

Generate class probabilities using fitted decision tree on a new set of data X_test.

After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree class probabilities in the array y_pred.

For each data point i, and class j, y_proba[i + j*ldy] will contain the class probability according to the decision tree, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_proba – [out] - array of size at least n_samples \(\times\) n_class . On output, will contain the predicted class probabilities.

  • n_class – [in] - number of classes in y_proba.

  • ldy – [in] leading dimension of y_proba. Constraint: ldy \(\ge\) n_samples.

Returns

da_status

da_status da_tree_predict_log_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_log_proba, da_int n_class, da_int ldy)#
da_status da_tree_predict_log_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_log_proba, da_int n_class, da_int ldy)#

Generate class log probabilities using fitted decision tree on a new set of data X_test.

After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree class log probabilities in the array y_pred.

For each data point i, and class j, y_proba[i + j*ldy] will contain the class log probability according to the decision tree, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_log_proba – [out] - array of size at least n_samples \(\times\) n_class . On output, will contain the predicted class log probabilities.

  • n_class – [in] - number of classes in y_log_proba.

  • ldy – [in] leading dimension of y_log_proba. Constraint: ldy \(\ge\) n_samples.

Returns

da_status

da_status da_tree_score_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_test, float *mean_accuracy)#
da_status da_tree_score_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_test, double *mean_accuracy)#

Calculate score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data X_test.

To be used after a model has been fitted using da_tree_fit_?.

For each data point i, y_test[i] will contain the label of the test data, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test. It must match the number of features from the training data set.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_test – [in] - actual class labels.

  • mean_accuracy – [out] - proportion of observations where predicted label matches actual label.

Returns

da_status

Decision forests#

class aoclda.decision_forest.decision_forest(criterion='gini', bootstrap=True, n_trees=100, features_selection='sqrt', max_features=0, seed=-1, max_depth=10, min_samples_split=2, build_order='breadth first', precision='double')#

Decision forest classifier.

An ensemble classifier based on decision trees.

Parameters
  • seed (int, optional) – Set the random seed for the random number generator. If the value is -1, a random seed is automatically generated. In this case the resulting classification will create non-reproducible results. Default = -1.

  • n_trees (int, optional) – Set the number of trees to compute. Default = 100.

  • criterion (str, optional) – Select scoring function to use. It can take the values ‘cross-entropy’, ‘gini’, or ‘misclassification’

  • max_depth (int, optional) – Set the maximum depth of the trees. Default = 29.

  • features_selection (str, optional) – Select how many features to use for each split. It can take the values ‘all’, ‘sqrt’, ‘log2’, or ‘custom’. If set to ‘custom’, the number of features to consider is set by the optional argument max_features. Default ‘sqrt’.

  • max_features (int, optional) – Set the number of features to consider when splitting a node. Only taken into account when features_selection is set to ‘custom’. 0 means take all the features. Default 0.

  • min_samples_split (int, optional) – The minimum number of samples required to split an internal node. Default 2.

  • build_order (str, optional) – Select in which order to explore the nodes. It can take the values ‘breadth first’ or ‘depth first’. Default ‘breadth first’.

  • bootstrap (bool, optional) – Select whether to bootstrap the samples in the trees. Default True.

  • precision (str, optional) – Whether to initialize the PCA object in double or single precision. It can take the values ‘single’ or ‘double’. Default = ‘double’.

fit(X, y, samples_factor=0.8, min_impurity_decrease=0.0, min_split_score=0.0, feat_thresh=1e-06)#

Computes the decision forest on the feature matrix X and response vector y

Parameters
  • X (numpy.ndarray) – The feature matrix on which to compute the model. Its shape is (n_samples, n_features).

  • y (numpy.ndarray) – The response vector. Its shape is (n_samples).

samples_factor (float, optional): Proportion of samples to draw from

the data set to build each tree if ‘bootstrap’ was set to True. Default 0.8.

min_impurity_decrease (float, optional): Minimum score improvement

needed to consider a split from the parent node. Default 0.0

min_split_score (float, optional): Minimum score needed for a node

to be considered for splitting. Default 0.0.

feat_thresh (float, optional): Minimum difference in feature value

required for splitting. Default 1.0e-06

Returns

Returns the instance itself.

Return type

self (object)

predict(X)#

Generate labels using fitted decision forest on a new set of data X.

Parameters

X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

Returns

The prediction vector,

where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

predict_log_proba(X)#

Generate class log probabilities using fitted decision forest on a new set of data X.

Parameters

X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

Returns

The prediction vector,

where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

predict_proba(X)#

Generate class probabilities using fitted decision forest on a new set of data X.

Parameters

X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

Returns

The prediction vector,

where n_samples is the number of rows of X.

Return type

numpy.ndarray of length n_samples

score(X, y)#

Calculates score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data.

Parameters
  • X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.

  • y (numpy.ndarray) – The response vector. It must have shape (n_samples).

Returns

The mean accuracy of the model on the test data.

Return type

float

da_status da_forest_set_training_data_s(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, float *X, da_int ldx, da_int *y)#
da_status da_forest_set_training_data_d(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, double *X, da_int ldx, da_int *y)#

Pass a data matrix and a label array to the da_handle object in preparation for fitting a decision forest.

Parameters
  • handle – a da_handle object, initialized with type da_handle_decision_forest.

  • n_samples – number of observations in X.

  • n_features – number of features in X.

  • n_class – number of distinct classes in y. Will be computed automatically if n_class is set to 0.

  • X – array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx – leading dimension of X. Constraint: ldx \(\ge\) n_samples.

  • y – array containing the n_samples labels. The label values are expected to range from 0 to n_class - 1.

Returns

da_status. The function returns:

da_status da_forest_fit_s(da_handle handle)#
da_status da_forest_fit_d(da_handle handle)#

Fit the decision forest defined in the handle.

Compute the decision forest parameters given the data passed by da_forest_set_training_data_?. Note that you can customize the model before using the fit function through the use of optional parameters, see this section for a list of available options.

Parameters

handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.

Returns

da_status. The function returns:

Post

After successful execution, da_handle_get_result_? can be queried with the following enum:

  • da_rinfo - return an array of size 5 containing n_features, n_samples, the number of samples the tree was trained on, the value of the random seed used by the RNG and n_tree, the total number of trees in the forest.

da_status da_forest_predict_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_pred)#
da_status da_forest_predict_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_pred)#

Generate labels using fitted decision forest on a new set of data X_test.

After a model has been fitted using forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest predictions in the array y_pred.

For each data point i, y_pred[i] will contain the label of the most likely class according to the decision forest; x[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_pred – [out] - array of size at least n_samples. On output, will contain the predicted class labels.

Returns

da_status

da_status da_forest_predict_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_proba, da_int n_class, da_int ldy)#
da_status da_forest_predict_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_proba, da_int n_class, da_int ldy)#

Generate class probabilities using fitted decision forest on a new set of data X_test.

After a model has been fitted using da_forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest class probabilities in the array y_pred.

For each data point i, and class j, y_proba[i*n_class + j] will contain the class probability according to the decision forest, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_proba – [out] - array of size at least n_samples \(\times\) n_class . On output, will contain the predicted class probabilities.

  • n_class – [in] - number of classes in y_proba.

  • ldy – [in] leading dimension of y_proba. Constraint: ldy \(\ge\) n_samples.

Returns

da_status

da_status da_forest_predict_log_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_log_proba, da_int n_class, da_int ldy)#
da_status da_forest_predict_log_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_log_proba, da_int n_class, da_int ldy)#

Generate class log probabilities using fitted decision forest on a new set of data X_test.

After a model has been fitted using da_forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest class log probabilities in the array y_pred.

For each data point i, and class j, y_proba[i*n_class + j] will contain the class log probability according to the decision forest, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_log_proba – [out] - array of size at least n_samples \(\times\) n_class . On output, will contain the predicted class log probabilities.

  • n_class – [in] - number of classes in y_log_proba.

  • ldy – [in] leading dimension of y_log_proba. Constraint: ldy \(\ge\) n_samples.

Returns

da_status

da_status da_forest_score_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_test, float *mean_accuracy)#
da_status da_forest_score_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_test, double *mean_accuracy)#

Calculate score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data X_test.

To be used after a model has been fitted using da_forest_fit_?.

For each data point i, y_test[i] will contain the label of the test data, X_test[i + j*ldx_test] should contain the feature j for observation i.

Parameters
  • handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.

  • n_samples – [in] - number of observations in X_test.

  • n_features – [in] - number of features in X_test. It must match the number of features from the training data set.

  • X_test – [in] array containing n_samples \(\times\) n_features data matrix, in column-major format.

  • ldx_test – [in] leading dimension of X_test. Constraint: ldx_test \(\ge\) n_samples.

  • y_test – [in] - actual class labels.

  • mean_accuracy – [out] - proportion of observations where predicted label matches actual label.

Returns

da_status