Decision tree and decision forest APIs#
This chapter contains two sets of APIs, one for classification using a single decision tree and another one for the ensemble method decision forests (also known as random forests).
Decision trees#
- class aoclda.decision_tree.decision_tree(criterion='gini', seed=-1, max_depth=10, max_features=0, min_samples_split=2, build_order='breadth first', precision='double')#
A decision tree classifier.
- Parameters
seed (int, optional) – Set the random seed for the random number generator. If the value is -1, a random seed is automatically generated. In this case the resulting classification will create non-reproducible results. Default = -1.
max_depth (int, optional) – Set the maximum depth of the tree. Default = 29.
max_features (int, optional) – Set the number of features to consider when splitting a node. 0 means take all the features. Default 0.
criterion (str, optional) – Select scoring function to use. It can take the values ‘cross-entropy’, ‘gini’, or ‘misclassification’
min_samples_split (int, optional) – The minimum number of samples required to split an internal node. Default 2.
build_order (str, optional) – Select in which order to explore the nodes. It can take the values ‘breadth first’ or ‘depth first’. Default ‘breadth first’.
precision (str, optional) – Whether to initialize the PCA object in double or single precision. It can take the values ‘single’ or ‘double’. Default = ‘double’.
- fit(X, y, min_impurity_decrease=0.0, min_split_score=0.0, feat_thresh=1e-06)#
Computes the decision tree on the feature matrix
Xand response vectory- Parameters
X (numpy.ndarray) – The feature matrix on which to compute the model. Its shape is (n_samples, n_features).
y (numpy.ndarray) – The response vector. Its shape is (n_samples).
min_impurity_decrease (float, optional) – Minimum score improvement needed to consider a split from the parent node. Default 0.0
min_split_score (float, optional) – Minimum score needed for a node to be considered for splitting. Default 0.0.
feat_thresh (float, optional) – Minimum difference in feature value required for splitting. Default 1.0e-06
- Returns
Returns the instance itself.
- Return type
self (object)
- property n_leaves#
The number of nodes in the trained tree
- Type
int
- property n_nodes#
The number of nodes in the trained tree
- Type
int
- predict(X)#
Generate labels using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have
columns. (n_features) –
- Returns
The prediction vector, where n_samples is the number of rows of
X.- Return type
numpy.ndarray of length n_samples
- predict_log_proba(X)#
Generate class log probabilities using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
- Returns
- The prediction vector,
where n_samples is the number of rows of X.
- Return type
numpy.ndarray of length n_samples
- predict_proba(X)#
Generate class probabilities using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have
columns. (n_features) –
- Returns
The prediction vector, where n_samples is the number of rows of
X.- Return type
numpy.ndarray of length n_samples
- score(X, y)#
Calculates score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data.
- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
y (numpy.ndarray) – The response vector. It must have shape (n_samples).
- Returns
The mean accuracy of the model on the test data.
- Return type
float
-
da_status da_tree_set_training_data_s(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, float *X, da_int ldx, da_int *y)#
-
da_status da_tree_set_training_data_d(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, double *X, da_int ldx, da_int *y)#
Pass a data matrix and a label array to the da_handle object in preparation for fitting a decision tree.
- Parameters
handle – a da_handle object, initialized with type da_handle_decision_tree.
n_samples – number of observations in
X.n_features – number of features in
X.n_class – number of distinct classes in
y. Will be computed automatically ifn_classis set to 0.X – array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx – leading dimension of
X. Constraint:ldx\(\ge\)n_samples.y – array containing the
n_sampleslabels. The label values are expected to range from 0 ton_class- 1.
- Returns
da_status. The function returns:
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_memory_error - internal memory allocation encountered a problem.
-
da_status da_tree_fit_d(da_handle handle)#
Fit the decision tree defined in the
handle.Compute the decision tree parameters given the data passed by da_tree_set_training_data_?. Note that you can customize the model before using the fit function through the use of optional parameters, see this section for a list of available options.
- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
- Returns
da_status. The function returns:
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_incompatible_options - some of the options set are incompatible with the model defined in
handle. You can obtain further information using da_handle_print_error_message.da_status_memory_error - internal memory allocation encountered a problem.
da_status_internal_error - an unexpected error occurred.
- Post
After successful execution, da_handle_get_result_? can be queried with the following enum:
da_rinfo- return an array of size 5 containingn_features,n_samples, the number of samples the tree was trained on, the value of the random seed used to fit the tree and the depth of the tree.
-
da_status da_tree_predict_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_pred)#
-
da_status da_tree_predict_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_pred)#
Generate labels using fitted decision tree on a new set of data
X_test.After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree predictions in the array
y_pred.For each data point
i,y_pred[i]will contain the label of the most likely class according to the decision tree,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_pred – [out] - array of size at least
n_samples. On output, will contain the predicted class labels.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_tree_predict_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_proba, da_int n_class, da_int ldy)#
-
da_status da_tree_predict_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_proba, da_int n_class, da_int ldy)#
Generate class probabilities using fitted decision tree on a new set of data
X_test.After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree class probabilities in the array
y_pred.For each data point
i, and classj,y_proba[i + j*ldy]will contain the class probability according to the decision tree,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_proba – [out] - array of size at least
n_samples\(\times\)n_class. On output, will contain the predicted class probabilities.n_class – [in] - number of classes in
y_proba.ldy – [in] leading dimension of
y_proba. Constraint:ldy\(\ge\)n_samples.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_tree_predict_log_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_log_proba, da_int n_class, da_int ldy)#
-
da_status da_tree_predict_log_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_log_proba, da_int n_class, da_int ldy)#
Generate class log probabilities using fitted decision tree on a new set of data
X_test.After a model has been fitted using da_tree_fit_?, it can be used to generate predicted labels on new data. This function returns the decision tree class log probabilities in the array
y_pred.For each data point
i, and classj,y_proba[i + j*ldy]will contain the class log probability according to the decision tree,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_log_proba – [out] - array of size at least
n_samples\(\times\)n_class. On output, will contain the predicted class log probabilities.n_class – [in] - number of classes in
y_log_proba.ldy – [in] leading dimension of
y_log_proba. Constraint:ldy\(\ge\)n_samples.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_tree_score_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_test, float *mean_accuracy)#
-
da_status da_tree_score_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_test, double *mean_accuracy)#
Calculate score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data
X_test.To be used after a model has been fitted using da_tree_fit_?.
For each data point
i,y_test[i]will contain the label of the test data,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test. It must match the number of features from the training data set.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_test – [in] - actual class labels.
mean_accuracy – [out] - proportion of observations where predicted label matches actual label.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
Decision forests#
- class aoclda.decision_forest.decision_forest(criterion='gini', bootstrap=True, n_trees=100, features_selection='sqrt', max_features=0, seed=-1, max_depth=10, min_samples_split=2, build_order='breadth first', precision='double')#
Decision forest classifier.
An ensemble classifier based on decision trees.
- Parameters
seed (int, optional) – Set the random seed for the random number generator. If the value is -1, a random seed is automatically generated. In this case the resulting classification will create non-reproducible results. Default = -1.
n_trees (int, optional) – Set the number of trees to compute. Default = 100.
criterion (str, optional) – Select scoring function to use. It can take the values ‘cross-entropy’, ‘gini’, or ‘misclassification’
max_depth (int, optional) – Set the maximum depth of the trees. Default = 29.
features_selection (str, optional) – Select how many features to use for each split. It can take the values ‘all’, ‘sqrt’, ‘log2’, or ‘custom’. If set to ‘custom’, the number of features to consider is set by the optional argument max_features. Default ‘sqrt’.
max_features (int, optional) – Set the number of features to consider when splitting a node. Only taken into account when features_selection is set to ‘custom’. 0 means take all the features. Default 0.
min_samples_split (int, optional) – The minimum number of samples required to split an internal node. Default 2.
build_order (str, optional) – Select in which order to explore the nodes. It can take the values ‘breadth first’ or ‘depth first’. Default ‘breadth first’.
bootstrap (bool, optional) – Select whether to bootstrap the samples in the trees. Default True.
precision (str, optional) – Whether to initialize the PCA object in double or single precision. It can take the values ‘single’ or ‘double’. Default = ‘double’.
- fit(X, y, samples_factor=0.8, min_impurity_decrease=0.0, min_split_score=0.0, feat_thresh=1e-06)#
Computes the decision forest on the feature matrix
Xand response vectory- Parameters
X (numpy.ndarray) – The feature matrix on which to compute the model. Its shape is (n_samples, n_features).
y (numpy.ndarray) – The response vector. Its shape is (n_samples).
- samples_factor (float, optional): Proportion of samples to draw from
the data set to build each tree if ‘bootstrap’ was set to True. Default 0.8.
- min_impurity_decrease (float, optional): Minimum score improvement
needed to consider a split from the parent node. Default 0.0
- min_split_score (float, optional): Minimum score needed for a node
to be considered for splitting. Default 0.0.
- feat_thresh (float, optional): Minimum difference in feature value
required for splitting. Default 1.0e-06
- Returns
Returns the instance itself.
- Return type
self (object)
- predict(X)#
Generate labels using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
- Returns
- The prediction vector,
where n_samples is the number of rows of X.
- Return type
numpy.ndarray of length n_samples
- predict_log_proba(X)#
Generate class log probabilities using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
- Returns
- The prediction vector,
where n_samples is the number of rows of X.
- Return type
numpy.ndarray of length n_samples
- predict_proba(X)#
Generate class probabilities using fitted decision forest on a new set of data
X.- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
- Returns
- The prediction vector,
where n_samples is the number of rows of X.
- Return type
numpy.ndarray of length n_samples
- score(X, y)#
Calculates score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data.
- Parameters
X (numpy.ndarray) – The feature matrix to evaluate the model on. It must have n_features columns.
y (numpy.ndarray) – The response vector. It must have shape (n_samples).
- Returns
The mean accuracy of the model on the test data.
- Return type
float
-
da_status da_forest_set_training_data_s(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, float *X, da_int ldx, da_int *y)#
-
da_status da_forest_set_training_data_d(da_handle handle, da_int n_samples, da_int n_features, da_int n_class, double *X, da_int ldx, da_int *y)#
Pass a data matrix and a label array to the da_handle object in preparation for fitting a decision forest.
- Parameters
handle – a da_handle object, initialized with type da_handle_decision_forest.
n_samples – number of observations in
X.n_features – number of features in
X.n_class – number of distinct classes in
y. Will be computed automatically ifn_classis set to 0.X – array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx – leading dimension of
X. Constraint:ldx\(\ge\)n_samples.y – array containing the
n_sampleslabels. The label values are expected to range from 0 ton_class- 1.
- Returns
da_status. The function returns:
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
-
da_status da_forest_fit_d(da_handle handle)#
Fit the decision forest defined in the
handle.Compute the decision forest parameters given the data passed by da_forest_set_training_data_?. Note that you can customize the model before using the fit function through the use of optional parameters, see this section for a list of available options.
- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.
- Returns
da_status. The function returns:
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_incompatible_options - some of the options set are incompatible with the model defined in
handle. You can obtain further information using da_handle_print_error_message.da_status_memory_error - internal memory allocation encountered a problem.
da_status_internal_error - an unexpected error occurred.
- Post
After successful execution, da_handle_get_result_? can be queried with the following enum:
da_rinfo- return an array of size 5 containingn_features,n_samples, the number of samples the tree was trained on, the value of the random seed used by the RNG andn_tree, the total number of trees in the forest.
-
da_status da_forest_predict_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_pred)#
-
da_status da_forest_predict_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_pred)#
Generate labels using fitted decision forest on a new set of data
X_test.After a model has been fitted using forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest predictions in the array
y_pred.For each data point
i,y_pred[i]will contain the label of the most likely class according to the decision forest;x[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_tree.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_pred – [out] - array of size at least
n_samples. On output, will contain the predicted class labels.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_forest_predict_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_proba, da_int n_class, da_int ldy)#
-
da_status da_forest_predict_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_proba, da_int n_class, da_int ldy)#
Generate class probabilities using fitted decision forest on a new set of data
X_test.After a model has been fitted using da_forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest class probabilities in the array
y_pred.For each data point
i, and classj,y_proba[i*n_class + j]will contain the class probability according to the decision forest,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_proba – [out] - array of size at least
n_samples\(\times\)n_class. On output, will contain the predicted class probabilities.n_class – [in] - number of classes in
y_proba.ldy – [in] leading dimension of
y_proba. Constraint:ldy\(\ge\)n_samples.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_forest_predict_log_proba_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, float *y_log_proba, da_int n_class, da_int ldy)#
-
da_status da_forest_predict_log_proba_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, double *y_log_proba, da_int n_class, da_int ldy)#
Generate class log probabilities using fitted decision forest on a new set of data
X_test.After a model has been fitted using da_forest_fit_?, it can be used to generate predicted labels on new data. This function returns the decision forest class log probabilities in the array
y_pred.For each data point
i, and classj,y_proba[i*n_class + j]will contain the class log probability according to the decision forest,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_log_proba – [out] - array of size at least
n_samples\(\times\)n_class. On output, will contain the predicted class log probabilities.n_class – [in] - number of classes in
y_log_proba.ldy – [in] leading dimension of
y_log_proba. Constraint:ldy\(\ge\)n_samples.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.
-
da_status da_forest_score_s(da_handle handle, da_int n_samples, da_int n_features, float *X_test, da_int ldx_test, da_int *y_test, float *mean_accuracy)#
-
da_status da_forest_score_d(da_handle handle, da_int n_samples, da_int n_features, double *X_test, da_int ldx_test, da_int *y_test, double *mean_accuracy)#
Calculate score (prediction accuracy) by comparing predicted labels and actual labels on a new set of data
X_test.To be used after a model has been fitted using da_forest_fit_?.
For each data point
i,y_test[i]will contain the label of the test data,X_test[i + j*ldx_test]should contain the featurejfor observationi.- Parameters
handle – [inout] a da_handle object, initialized with type da_handle_decision_forest.
n_samples – [in] - number of observations in
X_test.n_features – [in] - number of features in
X_test. It must match the number of features from the training data set.X_test – [in] array containing
n_samples\(\times\)n_featuresdata matrix, in column-major format.ldx_test – [in] leading dimension of
X_test. Constraint:ldx_test\(\ge\)n_samples.y_test – [in] - actual class labels.
mean_accuracy – [out] - proportion of observations where predicted label matches actual label.
- Returns
da_status
da_status_success - the operation was successfully completed.
da_status_wrong_type - the floating point precision of the arguments is incompatible with the
handleinitialization.da_status_invalid_pointer - the
handlehas not been correctly initialized.da_status_invalid_input - one of the arguments had an invalid value. You can obtain further information using da_handle_print_error_message.
da_status_out_of_date - the model has not been trained yet.