Datastore APIs#

Datastores#

typedef struct _da_datastore *da_datastore#

The main structure used by AOCL-DA to store data. All functions on this page operate on this data structure.

Initialize and destroy datastores#

da_status da_datastore_init(da_datastore *store)#

Initialize an empty da_datastore.

Parameters

store – the da_datastore to initialize.

void da_datastore_destroy(da_datastore *store)#

Free all memory linked to a da_datastore.

Note

Memory leaks can occur if da_datastore structures are not destroyed after use.

Parameters

store – the da_datastore to destroy.

Load data into a datastore#

da_status da_data_load_from_csv(da_datastore store, const char *filename)#

Read data from a CSV file into a da_datastore object. The data type of each column will be automatically detected.

Prior to calling this function, the following options can be set using calls to da_options_set_int or da_options_set_string.

Option Name

Type

Default

Description

Constraints

csv integers as floats

integer

\(i=0\)

Whether or not to interpret integers as floating point numbers when using autodetection.

\(0 \le i \le 1\)

csv datastore precision

string

\(s=\) double

The precision used when reading floating point numbers using autodetection.

\(s=\) double, or single.

csv use header row

integer

\(i=0\)

Whether or not to interpret the first row as a header.

\(0 \le i \le 1\)

csv warn for missing data

integer

\(i=0\)

If set to 0, return error if missing data is encountered; if set to 1, issue a warning and store missing data as either a NaN (for floating point data) or the maximum value of the integer type being used.

\(0 \le i \le 1\)

csv skip footer

integer

\(i=0\)

Whether or not to ignore the last line when reading a CSV file.

\(0 \le i \le 1\)

csv delimiter

string

\(s=\) ,

The delimiter used when reading CSV files.

csv whitespace delimiter

integer

\(i=0\)

Whether or not to use whitespace as the delimiter when reading CSV files.

\(0 \le i \le 1\)

csv decimal

string

\(s=\) .

The character used to denote a decimal point in CSV files.

csv skip initial space

integer

\(i=0\)

Whether or not to ignore initial spaces in CSV file lines.

\(0 \le i \le 1\)

csv line terminator

string

empty

The character used to denote line termination in CSV files (leave this empty to use the default).

csv row start

integer

\(i=0\)

Ignore the specified number of lines from the top of the file (note that line numbers in CSV files start at 1).

\(0 \le i\)

csv comment

string

\(s=\) #

The character used to denote comments in CSV files (note, if a line in a CSV file is to be interpreted as only containing a comment, the comment character should be the first character on the line).

csv quote character

string

\(s=''\)

The character used to denote quotations in CSV files.

csv scientific notation character

string

\(s=\) e

The character used to denote powers of 10 in floating point values in CSV files.

csv escape character

string

\(s=\) \

The escape character in CSV files.

csv thousands

string

empty

The character used to separate thousands when reading numeric values in CSV files.

csv skip rows

string

empty

A comma- or space-separated list of rows to ignore in CSV files.

csv datatype

string

\(s=\) auto

If a CSV file is known to be of a single datatype, set this option to disable autodetection and make reading the file quicker.

\(s=\) auto, boolean, double, float, integer, or string.

csv data storage

string

\(s=\) column major

Whether to store data from CSV files in row or column major format.

\(s=\) column major, or row major.

csv skip empty lines

integer

\(i=0\)

Whether or not to ignore empty lines in CSV files (note that caution should be used when using this in conjunction with options such as CSV skip rows since line numbers may no longer correspond to the original line numbers in the CSV file).

\(0 \le i \le 1\)

csv double quote

integer

\(i=0\)

Whether or not to interpret two consecutive quotechar characters within a field as a single quotechar character.

\(0 \le i \le 1\)

Parameters
  • store – [inout] a _da_datastore object, initialized using da_datastore_init.

  • filename – [in] the relative or absolute path to a file or stream that can be opened for reading.

Returns

da_status. The function returns:

da_status da_data_hconcat(da_datastore *store1, da_datastore *store2)#

Concatenate horizontally two da_datastore objects.

The two data stores must have a matching number of rows to successfully perform this operation.

If successful, on output, store1 will contain the concatenation of the two stores and store2 will be empty. No copy of the data is performed when this function is called.

Parameters
  • store1 – the da_datastore that will contain the concatenation on output.

  • store2 – the da_datastore to concatenate with store1. Will be empty on output.

Returns

da_status. The function returns:

da_status da_data_load_row_int(da_datastore store, da_int n_rows, da_int n_cols, da_int *block, da_ordering order, da_int copy_data)#
da_status da_data_load_row_str(da_datastore store, da_int n_rows, da_int n_cols, const char **block, da_ordering order)#
da_status da_data_load_row_real_s(da_datastore store, da_int n_rows, da_int n_cols, float *block, da_ordering order, da_int copy_data)#
da_status da_data_load_row_real_d(da_datastore store, da_int n_rows, da_int n_cols, double *block, da_ordering order, da_int copy_data)#
da_status da_data_load_row_uint8(da_datastore store, da_int n_rows, da_int n_cols, uint8_t *block, da_ordering order, da_int copy_data)#

Load new rows into a da_datastore from a dense matrix. The last suffix of the function name marks the type of the data to add.

The da_data_load_row_? routines will try to add rows at the bottom of the data store passed in the argument. If data was already loaded in the store, the routines must be called repeatedly until the columns of the new blocks match the structure of the existing store (see the introduction section for more details on the store’s structure).

The new data is expected to be provided as an n_rows \(\times \) n_cols dense block and can be passed in row major or column major ordering.

The data provided can be optionally copied into the store (for non C-string data blocks) if copy_data was set to true. Warning: if copy_data is set to false, the pointer will be copied as it is provided. Modifying or deallocating the memory before calling da_datastore_destroy can create unintended behaviour.

Parameters
  • store – [inout] the main structure.

  • n_rows – [in] number of rows of the new block.

  • n_cols – [in] number of columns of the new block.

  • block – [in] pointer to the raw data to add to the store.

  • order – [in] a da_ordering enumerated type. Specifies if the data block was stored in a column or row major ordering.

  • copy_data – [in] specifies if the data needs to be copied in the store.

Returns

da_status. The function returns:

da_status da_data_load_col_int(da_datastore store, da_int n_rows, da_int n_cols, da_int *block, da_ordering order, da_int copy_data)#
da_status da_data_load_col_str(da_datastore store, da_int n_rows, da_int n_cols, const char **block, da_ordering order)#
da_status da_data_load_col_real_d(da_datastore store, da_int n_rows, da_int n_cols, double *block, da_ordering order, da_int copy_data)#
da_status da_data_load_col_real_s(da_datastore store, da_int n_rows, da_int n_cols, float *block, da_ordering order, da_int copy_data)#
da_status da_data_load_col_uint8(da_datastore store, da_int n_rows, da_int n_cols, uint8_t *block, da_ordering order, da_int copy_data)#

Load new columns into a da_datastore from a dense matrix. The last suffix of the function name marks the type of the data to add.

The da_data_load_col_? routines will try to add columns to the right of the data store passed in the argument. If data was already loaded in the store, the number of rows of the new block must match with the number of rows already present.

The new data is expected to be provided as an n_rows \(\times \) n_cols dense block and can be passed in row major or column major ordering.

The data provided can be optionally copied into the store (for non C-string data blocks) if copy_data was set to true. Warning: if copy_data is set to false, the pointer will be copied as it is provided. Modifying or deallocating the memory before calling da_datastore_destroy can create unintended behaviour.

Parameters
  • store – [inout] the main structure.

  • n_rows – [in] number of rows of the new block.

  • n_cols – [in] number of columns of the new block.

  • block – [in] pointer to the raw data to add to the store.

  • order – [in] a da_ordering enumerated type. Specifies if the data block was stored in a column or row major ordering.

  • copy_data – [in] specifies if the data needs to be copied in the store.

Returns

da_status. The function returns:

Data selection#

da_status da_data_select_columns(da_datastore store, const char *key, da_int lbound, da_int ubound)#

Select all columns indexed between the values lbound and ubound in the selection labeled by key. Column indices are zero-based, meaning the index of the first column is 0 and the index of the last one is n_cols-1.

Overlapping columns cannot be selected at this time.

Parameters
  • store – [inout] the main data structure.

  • key – [in] label of the selection.

  • lbound – [in] lower bound of the column indices to select.

  • ubound – [in] upper bound of the column indices to select.

Returns

da_status. The function returns:

da_status da_data_select_rows(da_datastore store, const char *key, da_int lbound, da_int ubound)#

Select all rows indexed between the values lbound and ubound in the selection labeled by key. Row indices are zero-based, meaning the index of the first row is 0 and the index of the last one is n_rows-1.

Parameters
  • store – [inout] the main data structure.

  • key – [in] label of the selection.

  • lbound – [in] lower bound of the row indices to select.

  • ubound – [in] upper bound of the row indices to select.

Returns

da_status. The function returns:

da_status da_data_select_slice(da_datastore store, const char *key, da_int row_lbound, da_int row_ubound, da_int col_lbound, da_int col_ubound)#

Select all rows indexed between the values row_lbound and row_ubound and all columns indexed between the values col_lbound and col_ubound in the selection labeled by key. Row and Column indices are zero-based.

Parameters
  • store – [inout] the main data structure.

  • key – [in] label of the selection.

  • row_lbound – [in] lower bound of the row indices to select.

  • row_ubound – [in] upper bound of the row indices to select.

  • col_lbound – [in] lower bound of the column indices to select.

  • col_ubound – [in] upper bound of the column indices to select.

Returns

da_status. The function returns:

da_status da_data_select_non_missing(da_datastore store, const char *key, uint8_t full_rows)#

Remove all the rows containing missing data from the selection labeled by key. See da_datastore for more details on selections.

If key had a previously empty row selection, all rows are checked for missing elements.

Parameters
  • store – [inout] the main data structure.

  • key – [in] the label of the selection.

  • full_rows – [in] serves as a boolean variable indicating if only the columns already in the selection key are to be checked for missing data.

Returns

da_status. The function returns:

da_status da_data_select_remove_columns(da_datastore store, const char *key, da_int lbound, da_int ubound)#

Remove the columns indexed by the values between lbound and ubound from an existing selection. Column indices are zero-based, meaning the index of the first column is 0 and the index of the last one is n_rows-1.

A warning da_status_invalid_input is returned if key does not designate a valid selection.

Parameters
  • store – the main data structure.

  • key – the label of the selection.

  • lbound – lower bound of the column indices to select.

  • ubound – upper bound of the column indices to select.

Returns

da_status. The function returns:

da_status da_data_select_remove_rows(da_datastore store, const char *key, da_int lbound, da_int ubound)#

Remove the rows indexed by the values between lbound and ubound from an existing selection. Row indices are zero-based, meaning the index of the first row is 0 and the index of the last one is n_rows-1.

A warning da_status_invalid_input is returned if key does not designate a valid selection.

Parameters
  • store – the main data structure.

  • key – the label of the selection.

  • lbound – lower bound of the row indices to select.

  • ubound – upper bound of the row indices to select.

Returns

da_status. The function returns:

Data extraction#

da_status da_data_extract_selection_int(da_datastore store, const char *key, da_int *data, da_int lddata)#
da_status da_data_extract_selection_real_s(da_datastore store, const char *key, float *data, da_int lddata)#
da_status da_data_extract_selection_real_d(da_datastore store, const char *key, double *data, da_int lddata)#
da_status da_data_extract_selection_uint8(da_datastore store, const char *key, uint8_t *data, da_int lddata)#

Extract a selection labeled by key. The last suffix of the function name marks the type of the data to be extracted.

The data marked by the set of columns and rows in the selection key is extracted into a dense matrix of the corresponding type. The matrix is returned in column major ordering.

Parameters
  • store – [in] main data structure.

  • key – [in] label of the selection.

  • data – [out] output matrix.

  • lddata – [in] leading dimension of the output data.

Returns

da_status. The function returns:

da_status da_data_extract_column_int(da_datastore store, da_int idx, da_int dim, da_int *col)#
da_status da_data_extract_column_real_s(da_datastore store, da_int idx, da_int dim, float *col)#
da_status da_data_extract_column_real_d(da_datastore store, da_int idx, da_int dim, double *col)#
da_status da_data_extract_column_uint8(da_datastore store, da_int idx, da_int dim, uint8_t *col)#
da_status da_data_extract_column_str(da_datastore store, da_int idx, da_int dim, char **col)#

Extract a column from a store into a pre-allocated array. The last suffix of the function name marks the type of the data to add.

dim is the size of the output array provided to the function and must be at least the number or rows in the store.

Parameters
  • store – [in] main data structure.

  • idx – [in] index of the column to extract.

  • dim – [in] size of the vector provided.

  • col – [out] array the column will be exported to.

Returns

da_status. The function returns:

Column headers#

da_status da_data_label_column(da_datastore store, const char *label, da_int col_idx)#

Label a column.

Parameters
  • store – [inout] main data structure.

  • label – [in] new label for the column idx.

  • col_idx – [in] index of the column to label.

Returns

da_status. The function returns:

da_status da_data_get_col_idx(da_datastore store, const char *label, da_int *col_idx)#

Get the index of the column labeled by the input label.

Parameters
  • store – [in] main data store.

  • label – [in] name of the column to look for.

  • col_idx – [out] on output contains the index of the column label.

Returns

da_status. The function returns:

da_status da_data_get_col_label(da_datastore store, da_int col_idx, da_int *label_sz, char *label)#

Get the label of a column from its index.

On output the C string label will contain the label of the column idx. label_sz indicates the size of the C string label. If it is smaller than the size of the column label, the function will return da_status_invalid_input and label_sz will be set to the minimum size acceptable for col_idx.

Parameters
  • store – [in] main data structure.

  • col_idx – [in] index of the column to search for.

  • label_sz – [in] the size of the C string being provided to the function.

  • label – [out] if successful, contains the label of the column idx on output.

Returns

da_status. The function returns:

Data editing#

da_status da_data_get_n_rows(da_datastore store, da_int *n_rows)#

Get the number of rows in the store.

Parameters
  • store – [in] main data structure.

  • n_rows – [out] contains the number of rows in store on output.

Returns

da_status. The function returns:

da_status da_data_get_n_cols(da_datastore store, da_int *n_cols)#

Get the number of rows in the store.

Parameters
  • store – [in] main data structure.

  • n_cols – [out] contains the number of columns in store on output.

Returns

da_status. The function returns:

da_status da_data_get_element_int(da_datastore store, da_int i, da_int j, da_int *elem)#
da_status da_data_get_element_real_d(da_datastore store, da_int i, da_int j, double *elem)#
da_status da_data_get_element_real_s(da_datastore store, da_int i, da_int j, float *elem)#
da_status da_data_get_element_uint8(da_datastore store, da_int i, da_int j, uint8_t *elem)#

Get an individual element of a data store. The last suffix of the function name marks the type of the data to be extracted.

Parameters
  • store – [in] main data structure.

  • i – [in] index of the row of the element to look for.

  • j – [in] index of the column of the element to look for.

  • elem – [out] contains the value of the element at indices i, j on output.

Returns

da_status. The function returns:

da_status da_data_set_element_int(da_datastore store, da_int i, da_int j, da_int elem)#
da_status da_data_set_element_real_d(da_datastore store, da_int i, da_int j, double elem)#
da_status da_data_set_element_real_s(da_datastore store, da_int i, da_int j, float elem)#
da_status da_data_set_element_uint8(da_datastore store, da_int i, da_int j, uint8_t elem)#

Set an individual element of a data store to a new value. The last suffix of the function name marks the type of the data to be set.

Parameters
  • store – [inout] main data structure.

  • i – [in] index of the row of the element to modify.

  • j – [in] index of the column of the element to modify.

  • elem – [in] the new value for the element at index i, j.

Returns

da_status. The function returns:

Miscellaneous#

The da_datastore functionality also includes option setting and error handling capabilities.

typedef enum da_ordering_ da_ordering#

Alias for the da_ordering_ enum.

enum da_ordering_#

Enumeration used to set column or row major ordering.

Values:

enumerator row_major#

Use row major ordering.

enumerator col_major#

Use column major ordering.