Datastore APIs#
Datastores#
-
typedef struct _da_datastore *da_datastore#
The main structure used by AOCL-DA to store data. All functions on this page operate on this data structure.
Initialize and destroy datastores#
-
da_status da_datastore_init(da_datastore *store)#
Initialize an empty da_datastore.
- Parameters
store – the da_datastore to initialize.
-
void da_datastore_destroy(da_datastore *store)#
Free all memory linked to a da_datastore.
Note
Memory leaks can occur if da_datastore structures are not destroyed after use.
- Parameters
store – the da_datastore to destroy.
Load data into a datastore#
-
da_status da_data_load_from_csv(da_datastore store, const char *filename)#
Read data from a CSV file into a da_datastore object. The data type of each column will be automatically detected.
Prior to calling this function, the following options can be set using calls to da_options_set_int or da_options_set_string.
Option Name
Type
Default
Description
Constraints
csv integers as floats
integer
\(i=0\)
Whether or not to interpret integers as floating point numbers when using autodetection.
\(0 \le i \le 1\)
csv datastore precision
string
\(s=\) double
The precision used when reading floating point numbers using autodetection.
\(s=\) double, or single.
csv use header row
integer
\(i=0\)
Whether or not to interpret the first row as a header.
\(0 \le i \le 1\)
csv warn for missing data
integer
\(i=0\)
If set to 0, return error if missing data is encountered; if set to 1, issue a warning and store missing data as either a NaN (for floating point data) or the maximum value of the integer type being used.
\(0 \le i \le 1\)
csv skip footer
integer
\(i=0\)
Whether or not to ignore the last line when reading a CSV file.
\(0 \le i \le 1\)
csv delimiter
string
\(s=\) ,
The delimiter used when reading CSV files.
csv whitespace delimiter
integer
\(i=0\)
Whether or not to use whitespace as the delimiter when reading CSV files.
\(0 \le i \le 1\)
csv decimal
string
\(s=\) .
The character used to denote a decimal point in CSV files.
csv skip initial space
integer
\(i=0\)
Whether or not to ignore initial spaces in CSV file lines.
\(0 \le i \le 1\)
csv line terminator
string
empty
The character used to denote line termination in CSV files (leave this empty to use the default).
csv row start
integer
\(i=0\)
Ignore the specified number of lines from the top of the file (note that line numbers in CSV files start at 1).
\(0 \le i\)
csv comment
string
\(s=\) #
The character used to denote comments in CSV files (note, if a line in a CSV file is to be interpreted as only containing a comment, the comment character should be the first character on the line).
csv quote character
string
\(s=''\)
The character used to denote quotations in CSV files.
csv scientific notation character
string
\(s=\) e
The character used to denote powers of 10 in floating point values in CSV files.
csv escape character
string
\(s=\) \
The escape character in CSV files.
csv thousands
string
empty
The character used to separate thousands when reading numeric values in CSV files.
csv skip rows
string
empty
A comma- or space-separated list of rows to ignore in CSV files.
csv datatype
string
\(s=\) auto
If a CSV file is known to be of a single datatype, set this option to disable autodetection and make reading the file quicker.
\(s=\) auto, boolean, double, float, integer, or string.
csv data storage
string
\(s=\) column major
Whether to store data from CSV files in row or column major format.
\(s=\) column major, or row major.
csv skip empty lines
integer
\(i=0\)
Whether or not to ignore empty lines in CSV files (note that caution should be used when using this in conjunction with options such as CSV skip rows since line numbers may no longer correspond to the original line numbers in the CSV file).
\(0 \le i \le 1\)
csv double quote
integer
\(i=0\)
Whether or not to interpret two consecutive quotechar characters within a field as a single quotechar character.
\(0 \le i \le 1\)
- Parameters
store – [inout] a _da_datastore object, initialized using da_datastore_init.
filename – [in] the relative or absolute path to a file or stream that can be opened for reading.
- Returns
da_status. The function returns:
-
da_status da_data_hconcat(da_datastore *store1, da_datastore *store2)#
Concatenate horizontally two da_datastore objects.
The two data stores must have a matching number of rows to successfully perform this operation.
If successful, on output,
store1will contain the concatenation of the two stores andstore2will be empty. No copy of the data is performed when this function is called.- Parameters
store1 – the da_datastore that will contain the concatenation on output.
store2 – the da_datastore to concatenate with
store1. Will be empty on output.
- Returns
da_status. The function returns:
da_status_success The operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_load_row_int(da_datastore store, da_int n_rows, da_int n_cols, da_int *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_row_str(da_datastore store, da_int n_rows, da_int n_cols, const char **block, da_ordering order)#
-
da_status da_data_load_row_real_s(da_datastore store, da_int n_rows, da_int n_cols, float *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_row_real_d(da_datastore store, da_int n_rows, da_int n_cols, double *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_row_uint8(da_datastore store, da_int n_rows, da_int n_cols, uint8_t *block, da_ordering order, da_int copy_data)#
Load new rows into a da_datastore from a dense matrix. The last suffix of the function name marks the type of the data to add.
The
da_data_load_row_?routines will try to add rows at the bottom of the data store passed in the argument. If data was already loaded in the store, the routines must be called repeatedly until the columns of the new blocks match the structure of the existing store (see the introduction section for more details on the store’s structure).The new data is expected to be provided as an
n_rows\(\times \)n_colsdense block and can be passed in row major or column major ordering.The data provided can be optionally copied into the store (for non C-string data blocks) if
copy_datawas set to true. Warning: ifcopy_datais set to false, the pointer will be copied as it is provided. Modifying or deallocating the memory before calling da_datastore_destroy can create unintended behaviour.- Parameters
store – [inout] the main structure.
n_rows – [in] number of rows of the new block.
n_cols – [in] number of columns of the new block.
block – [in] pointer to the raw data to add to the store.
order – [in] a da_ordering enumerated type. Specifies if the data block was stored in a column or row major ordering.
copy_data – [in] specifies if the data needs to be copied in the store.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_memory_error - internal memory allocation encountered a problem.
-
da_status da_data_load_col_int(da_datastore store, da_int n_rows, da_int n_cols, da_int *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_col_str(da_datastore store, da_int n_rows, da_int n_cols, const char **block, da_ordering order)#
-
da_status da_data_load_col_real_d(da_datastore store, da_int n_rows, da_int n_cols, double *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_col_real_s(da_datastore store, da_int n_rows, da_int n_cols, float *block, da_ordering order, da_int copy_data)#
-
da_status da_data_load_col_uint8(da_datastore store, da_int n_rows, da_int n_cols, uint8_t *block, da_ordering order, da_int copy_data)#
Load new columns into a da_datastore from a dense matrix. The last suffix of the function name marks the type of the data to add.
The
da_data_load_col_?routines will try to add columns to the right of the data store passed in the argument. If data was already loaded in the store, the number of rows of the new block must match with the number of rows already present.The new data is expected to be provided as an
n_rows\(\times \)n_colsdense block and can be passed in row major or column major ordering.The data provided can be optionally copied into the store (for non C-string data blocks) if
copy_datawas set to true. Warning: ifcopy_datais set to false, the pointer will be copied as it is provided. Modifying or deallocating the memory before calling da_datastore_destroy can create unintended behaviour.- Parameters
store – [inout] the main structure.
n_rows – [in] number of rows of the new block.
n_cols – [in] number of columns of the new block.
block – [in] pointer to the raw data to add to the store.
order – [in] a da_ordering enumerated type. Specifies if the data block was stored in a column or row major ordering.
copy_data – [in] specifies if the data needs to be copied in the store.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_memory_error - internal memory allocation encountered a problem.
Data selection#
-
da_status da_data_select_columns(da_datastore store, const char *key, da_int lbound, da_int ubound)#
Select all columns indexed between the values
lboundanduboundin the selection labeled bykey. Column indices are zero-based, meaning the index of the first column is 0 and the index of the last one isn_cols-1.Overlapping columns cannot be selected at this time.
- Parameters
store – [inout] the main data structure.
key – [in] label of the selection.
lbound – [in] lower bound of the column indices to select.
ubound – [in] upper bound of the column indices to select.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_select_rows(da_datastore store, const char *key, da_int lbound, da_int ubound)#
Select all rows indexed between the values
lboundanduboundin the selection labeled bykey. Row indices are zero-based, meaning the index of the first row is 0 and the index of the last one isn_rows-1.- Parameters
store – [inout] the main data structure.
key – [in] label of the selection.
lbound – [in] lower bound of the row indices to select.
ubound – [in] upper bound of the row indices to select.
- Returns
da_status. The function returns:
da_status_success The operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_select_slice(da_datastore store, const char *key, da_int row_lbound, da_int row_ubound, da_int col_lbound, da_int col_ubound)#
Select all rows indexed between the values
row_lboundandrow_uboundand all columns indexed between the valuescol_lboundandcol_uboundin the selection labeled bykey. Row and Column indices are zero-based.- Parameters
store – [inout] the main data structure.
key – [in] label of the selection.
row_lbound – [in] lower bound of the row indices to select.
row_ubound – [in] upper bound of the row indices to select.
col_lbound – [in] lower bound of the column indices to select.
col_ubound – [in] upper bound of the column indices to select.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_select_non_missing(da_datastore store, const char *key, uint8_t full_rows)#
Remove all the rows containing missing data from the selection labeled by
key. See da_datastore for more details on selections.If
keyhad a previously empty row selection, all rows are checked for missing elements.- Parameters
store – [inout] the main data structure.
key – [in] the label of the selection.
full_rows – [in] serves as a boolean variable indicating if only the columns already in the selection key are to be checked for missing data.
- Returns
da_status. The function returns:
da_status_success The operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_select_remove_columns(da_datastore store, const char *key, da_int lbound, da_int ubound)#
Remove the columns indexed by the values between
lboundanduboundfrom an existing selection. Column indices are zero-based, meaning the index of the first column is 0 and the index of the last one isn_rows-1.A warning da_status_invalid_input is returned if
keydoes not designate a valid selection.- Parameters
store – the main data structure.
key – the label of the selection.
lbound – lower bound of the column indices to select.
ubound – upper bound of the column indices to select.
- Returns
da_status. The function returns:
da_status_success The operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_select_remove_rows(da_datastore store, const char *key, da_int lbound, da_int ubound)#
Remove the rows indexed by the values between
lboundanduboundfrom an existing selection. Row indices are zero-based, meaning the index of the first row is 0 and the index of the last one isn_rows-1.A warning da_status_invalid_input is returned if
keydoes not designate a valid selection.- Parameters
store – the main data structure.
key – the label of the selection.
lbound – lower bound of the row indices to select.
ubound – upper bound of the row indices to select.
- Returns
da_status. The function returns:
da_status_success The operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
Data extraction#
-
da_status da_data_extract_selection_int(da_datastore store, const char *key, da_int *data, da_int lddata)#
-
da_status da_data_extract_selection_real_s(da_datastore store, const char *key, float *data, da_int lddata)#
-
da_status da_data_extract_selection_real_d(da_datastore store, const char *key, double *data, da_int lddata)#
-
da_status da_data_extract_selection_uint8(da_datastore store, const char *key, uint8_t *data, da_int lddata)#
Extract a selection labeled by
key. The last suffix of the function name marks the type of the data to be extracted.The data marked by the set of columns and rows in the selection
keyis extracted into a dense matrix of the corresponding type. The matrix is returned in column major ordering.- Parameters
store – [in] main data structure.
key – [in] label of the selection.
data – [out] output matrix.
lddata – [in] leading dimension of the output data.
- Returns
da_status. The function returns:
da_status_success the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_missing_block - the store contains incomplete row blocks.
da_status_internal_error - an unexpected error occurred.
-
da_status da_data_extract_column_int(da_datastore store, da_int idx, da_int dim, da_int *col)#
-
da_status da_data_extract_column_real_s(da_datastore store, da_int idx, da_int dim, float *col)#
-
da_status da_data_extract_column_real_d(da_datastore store, da_int idx, da_int dim, double *col)#
-
da_status da_data_extract_column_uint8(da_datastore store, da_int idx, da_int dim, uint8_t *col)#
-
da_status da_data_extract_column_str(da_datastore store, da_int idx, da_int dim, char **col)#
Extract a column from a store into a pre-allocated array. The last suffix of the function name marks the type of the data to add.
dimis the size of the output array provided to the function and must be at least the number or rows in the store.- Parameters
store – [in] main data structure.
idx – [in] index of the column to extract.
dim – [in] size of the vector provided.
col – [out] array the column will be exported to.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_missing_block - the store contains incomplete row blocks.
da_status_internal_error - an unexpected error occurred.
Column headers#
-
da_status da_data_label_column(da_datastore store, const char *label, da_int col_idx)#
Label a column.
- Parameters
store – [inout] main data structure.
label – [in] new label for the column
idx.col_idx – [in] index of the column to label.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_internal_error - an unexpected error occurred.
-
da_status da_data_get_col_idx(da_datastore store, const char *label, da_int *col_idx)#
Get the index of the column labeled by the input
label.- Parameters
store – [in] main data store.
label – [in] name of the column to look for.
col_idx – [out] on output contains the index of the column
label.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
da_status_internal_error - an unexpected error occurred.
-
da_status da_data_get_col_label(da_datastore store, da_int col_idx, da_int *label_sz, char *label)#
Get the label of a column from its index.
On output the C string
labelwill contain the label of the columnidx.label_szindicates the size of the C stringlabel. If it is smaller than the size of the column label, the function will return da_status_invalid_input andlabel_szwill be set to the minimum size acceptable forcol_idx.- Parameters
store – [in] main data structure.
col_idx – [in] index of the column to search for.
label_sz – [in] the size of the C string being provided to the function.
label – [out] if successful, contains the label of the column
idxon output.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
Data editing#
-
da_status da_data_get_n_rows(da_datastore store, da_int *n_rows)#
Get the number of rows in the store.
- Parameters
store – [in] main data structure.
n_rows – [out] contains the number of rows in
storeon output.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_get_n_cols(da_datastore store, da_int *n_cols)#
Get the number of rows in the store.
- Parameters
store – [in] main data structure.
n_cols – [out] contains the number of columns in
storeon output.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_get_element_int(da_datastore store, da_int i, da_int j, da_int *elem)#
-
da_status da_data_get_element_real_d(da_datastore store, da_int i, da_int j, double *elem)#
-
da_status da_data_get_element_real_s(da_datastore store, da_int i, da_int j, float *elem)#
-
da_status da_data_get_element_uint8(da_datastore store, da_int i, da_int j, uint8_t *elem)#
Get an individual element of a data store. The last suffix of the function name marks the type of the data to be extracted.
- Parameters
store – [in] main data structure.
i – [in] index of the row of the element to look for.
j – [in] index of the column of the element to look for.
elem – [out] contains the value of the element at indices
i,jon output.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
-
da_status da_data_set_element_int(da_datastore store, da_int i, da_int j, da_int elem)#
-
da_status da_data_set_element_real_d(da_datastore store, da_int i, da_int j, double elem)#
-
da_status da_data_set_element_real_s(da_datastore store, da_int i, da_int j, float elem)#
-
da_status da_data_set_element_uint8(da_datastore store, da_int i, da_int j, uint8_t elem)#
Set an individual element of a data store to a new value. The last suffix of the function name marks the type of the data to be set.
- Parameters
store – [inout] main data structure.
i – [in] index of the row of the element to modify.
j – [in] index of the column of the element to modify.
elem – [in] the new value for the element at index
i,j.
- Returns
da_status. The function returns:
da_status_success - the operation was successful.
da_status_invalid_input - some of the input data was not correct. Use da_handle_print_error_message to get more details.
da_status_invalid_pointer - the store was not correctly initialized.
Miscellaneous#
The da_datastore functionality also includes
option setting and
error handling capabilities.
-
typedef enum da_ordering_ da_ordering#
Alias for the da_ordering_ enum.