Basic Statistics#
This chapter contains functions to compute basic statistical quantities such as the mean,
variance or quantiles of a data matrix. Utility routines for standardizing the data are
also available, and correlation and covariance matrices can also be computed. For more information on basic statistical quantities, see Rice [1995], Kokoska and Zwillinger [2000].
These functions operate on a n_rows \(\times\) n_cols matrix stored in column major format.
Choosing an axis#
Most statistical quantities can be computed by column, by row or for the data matrix overall.
This is specified using the axis function parameter.
The axis parameter can take the following values:
'col'- statistical quantities will be computed for each column of the data matrix'row'- statistical quantities will be computed for each row of the data matrix'all'- statistical quantities will be computed for the whole data matrix
For example, if the function mean() is called with the axis argument set
to col, then n_cols means will be computed, one for each column. If the
axis argument is set to all, then a single mean will be computed.
This is specified using the da_axis enum.
The da_axis enum can take the following values:
da_axis_col- statistical quantities will be computed for each column of the data matrixda_axis_row- statistical quantities will be computed for each row of the data matrixda_axis_all- statistical quantities will be computed for the whole data matrix
For example, if the routine da_mean_s() is called with the da_axis argument set
to da_axis_col, then n_cols means will be computed, one for each column. If the
da_axis argument is set to da_axis_all, then a single mean will be computed.
Examples#
The code below is supplied with your installation (see Python examples).
1# Copyright (C) 2024 Advanced Micro Devices, Inc. All rights reserved.
2#
3# Redistribution and use in source and binary forms, with or without modification,
4# are permitted provided that the following conditions are met:
5# 1. Redistributions of source code must retain the above copyright notice,
6# this list of conditions and the following disclaimer.
7# 2. Redistributions in binary form must reproduce the above copyright notice,
8# this list of conditions and the following disclaimer in the documentation
9# and/or other materials provided with the distribution.
10# 3. Neither the name of the copyright holder nor the names of its contributors
11# may be used to endorse or promote products derived from this software without
12# specific prior written permission.
13#
14# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
15# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
16# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
17# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
18# INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
19# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
20# OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
21# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
22# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
23# POSSIBILITY OF SUCH DAMAGE.
24#
25"""
26Basic statistics example python script
27"""
28
29from aoclda.basic_stats import harmonic_mean, mean, variance, quantile, covariance_matrix, standardize
30import numpy as np
31import sys
32
33
34def basic_stats_example():
35 """
36 Basic statistics examples
37 """
38 a = np.array([[1.1, 2.213, 3.3], [4, 5.013, 6]],
39 dtype=np.float32,
40 order='F')
41 print("Dataset:\n")
42 print(a)
43
44 means = mean(a, axis="row")
45 print(f"\nRow means: {means}")
46
47 harmonic_means = harmonic_mean(a, axis="col")
48 print(f"\nColumn harmonic means: {harmonic_means}")
49
50 var = variance(a, axis="all")
51 print(f"\nOverall data variance: {var}")
52
53 medians = quantile(a, 0.5, axis="row")
54 print(f"\nRow medians: {medians}")
55
56 covar = covariance_matrix(a)
57 print(f"\nCovariance matrix:\n{covar}")
58
59 standardized = standardize(a)
60 print(f"\nStandardized matrix:\n{standardized}")
61
62 print("\nBasic statistics calculations successful")
63 print("---------------------------")
64
65
66if __name__ == "__main__":
67 try:
68 basic_stats_example()
69 except RuntimeError:
70 sys.exit(1)
The code below can be found in basic_statistics.cpp in the examples folder of your installation.
1/*
2 * Copyright (C) 2023-2024 Advanced Micro Devices, Inc. All rights reserved.
3 *
4 * Redistribution and use in source and binary forms, with or without modification,
5 * are permitted provided that the following conditions are met:
6 * 1. Redistributions of source code must retain the above copyright notice,
7 * this list of conditions and the following disclaimer.
8 * 2. Redistributions in binary form must reproduce the above copyright notice,
9 * this list of conditions and the following disclaimer in the documentation
10 * and/or other materials provided with the distribution.
11 * 3. Neither the name of the copyright holder nor the names of its contributors
12 * may be used to endorse or promote products derived from this software without
13 * specific prior written permission.
14 *
15 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
16 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
17 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
18 * IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
19 * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
20 * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
21 * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
22 * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
24 * POSSIBILITY OF SUCH DAMAGE.
25 *
26 */
27
28#include "aoclda.h"
29#include <cmath>
30#include <iostream>
31
32/* Basic statistics example
33 *
34 * This example computes descriptive statistics
35 * for a small dataset, together with the
36 * correlation matrix and the standardized data
37 */
38
39int main() {
40
41 std::cout << "-----------------------------------------------------------------------"
42 << std::endl;
43 std::cout << "Basic statistics" << std::endl;
44 std::cout << "Descriptive statistics for a 4x5 data matrix" << std::endl << std::endl;
45 std::cout << std::fixed;
46 std::cout.precision(5);
47
48 // Problem data
49 double X[20]{1.0, 2.0, 3.0, 4.0, 4.0, 3.0, 2.0, 1.0, 2.0, 8.0,
50 4.0, 6.0, 9.0, 5.0, 4.0, 3.0, 1.0, 1.0, 2.0, 2.0};
51 da_int n_rows = 4, n_cols = 5, ldx = 4, ldcov = 5, dof = 0, mode = 0;
52
53 // Arrays for output data
54 double harmonic_mean[5], mean[4], variance[4], kurtosis[4];
55 double minimum[1], lower_hinge[1], median[1], upper_hinge[1], maximum[1];
56 double cov[25];
57 double *dummy = nullptr;
58
59 // Arrays for expected outputs
60 double harmonic_mean_exp[5]{1.92, 1.92, 3.84, 4.472049689440994, 1.3333333333333333};
61 double mean_exp[4]{3.4, 3.8, 3., 3.2};
62 double variance_exp[4]{9.04, 6.16, 0.8, 2.96};
63 double kurtosis_exp[4]{-0.4210588143159213, -0.9675324675324677, -1.7500000000000002,
64 -1.005478451424398};
65 double minimum_exp[1]{1}, lower_hinge_exp[1]{2}, median_exp[1]{3},
66 upper_hinge_exp[1]{4}, maximum_exp[1]{9};
67 double cov_exp[25]{// clang-format off
68 1.6666666666666665, -1.6666666666666665, 1.3333333333333333,
69 -3.1666666666666665, 0.6666666666666666, -1.6666666666666665,
70 1.6666666666666665, -1.3333333333333333, 3.1666666666666665,
71 -0.6666666666666666, 1.3333333333333333, -1.3333333333333333,
72 6.666666666666666, -4.333333333333333, 0.0,
73 -3.1666666666666665, 3.1666666666666665 ,-4.333333333333333,
74 6.916666666666666, -1.1666666666666665, 0.6666666666666666,
75 -0.6666666666666666, 0.0, -1.1666666666666665,
76 0.3333333333333333 };
77 // clang-format on
78 double X_exp[20]{-1.1618950038622251, -0.3872983346207417, 0.3872983346207417,
79 1.1618950038622251, 1.1618950038622251, 0.3872983346207417,
80 -0.3872983346207417, -1.1618950038622251, -1.1618950038622251,
81 1.1618950038622251, -0.3872983346207417, 0.3872983346207417,
82 1.4258795636800752, -0.0950586375786717, -0.4752931878933584,
83 -0.8555277382080452, -0.8660254037844387, -0.8660254037844387,
84 0.8660254037844387, 0.8660254037844387};
85
86 int exit_code = 0;
87 bool pass = true;
88
89 // Compute column-wise harmonic means
90 pass = pass && (da_harmonic_mean_d(da_axis_col, n_rows, n_cols, X, ldx,
91 harmonic_mean) == da_status_success);
92
93 // Compute row-wise mean, variance and kurtosis
94 pass = pass && (da_kurtosis_d(da_axis_row, n_rows, n_cols, X, ldx, mean, variance,
95 kurtosis) == da_status_success);
96
97 // Compute overall max/min, median and hinges
98 pass = pass && (da_five_point_summary_d(da_axis_all, n_rows, n_cols, X, ldx, minimum,
99 lower_hinge, median, upper_hinge,
100 maximum) == da_status_success);
101
102 // Compute covariance matrix
103 pass = pass && (da_covariance_matrix_d(n_rows, n_cols, X, ldx, dof, cov, ldcov) ==
104 da_status_success);
105
106 // Standardize the original data matrix
107 pass = pass && (da_standardize_d(da_axis_col, n_rows, n_cols, X, ldx, dof, mode,
108 dummy, dummy) == da_status_success);
109
110 // Check status (we could do this after every function call)
111 if (pass) {
112 std::cout << "Statistics computed successfully" << std::endl << std::endl;
113
114 // Print computed statistics
115 std::cout << "Column-wise harmonic means: ";
116 for (da_int i = 0; i < n_cols; i++)
117 std::cout << " " << harmonic_mean[i];
118 std::cout << std::endl << std::endl;
119
120 std::cout << "Row-wise means: ";
121 for (da_int i = 0; i < n_rows; i++)
122 std::cout << " " << mean[i];
123 std::cout << std::endl << std::endl;
124
125 std::cout << "Row-wise variances: ";
126 for (da_int i = 0; i < n_rows; i++)
127 std::cout << " " << variance[i];
128 std::cout << std::endl << std::endl;
129
130 std::cout << "Row-wise kurtoses: ";
131 for (da_int i = 0; i < n_rows; i++)
132 std::cout << " " << kurtosis[i];
133 std::cout << std::endl << std::endl;
134
135 std::cout << "Overall five-point summary statistics: " << minimum[0] << " "
136 << lower_hinge[0] << " " << median[0] << " " << upper_hinge[0] << " "
137 << maximum[0] << std::endl
138 << std::endl;
139
140 std::cout << "Covariance matrix:" << std::endl;
141 for (da_int j = 0; j < n_cols; j++) {
142 for (da_int i = 0; i < n_cols; i++) {
143 std::cout << cov[ldcov * i + j] << " ";
144 }
145 std::cout << std::endl;
146 }
147 std::cout << std::endl;
148
149 std::cout << "Standardized data matrix:" << std::endl;
150 for (da_int j = 0; j < n_rows; j++) {
151 for (da_int i = 0; i < n_cols; i++) {
152 std::cout << X[ldx * i + j] << " ";
153 }
154 std::cout << std::endl;
155 }
156 std::cout << std::endl;
157
158 // Check the outputs match the expected results
159 double tol = 1.0e-14;
160 double err = 0.0;
161 for (da_int i = 0; i < n_cols; i++)
162 err = std::max(err, std::abs(harmonic_mean[i] - harmonic_mean_exp[i]));
163 for (da_int i = 0; i < n_rows; i++)
164 err = std::max(err, std::abs(mean[i] - mean_exp[i]));
165 for (da_int i = 0; i < n_rows; i++)
166 err = std::max(err, std::abs(variance[i] - variance_exp[i]));
167 for (da_int i = 0; i < n_rows; i++)
168 err = std::max(err, std::abs(kurtosis[i] - kurtosis_exp[i]));
169 err = std::max(err, std::abs(minimum[0] - minimum_exp[0]));
170 err = std::max(err, std::abs(lower_hinge[0] - lower_hinge_exp[0]));
171 err = std::max(err, std::abs(median[0] - median_exp[0]));
172 err = std::max(err, std::abs(upper_hinge[0] - upper_hinge_exp[0]));
173 err = std::max(err, std::abs(maximum[0] - maximum_exp[0]));
174 for (da_int j = 0; j < n_cols; j++) {
175 for (da_int i = 0; i < n_cols; i++) {
176 err =
177 std::max(err, std::abs(cov[ldcov * j + i] - cov_exp[ldcov * j + i]));
178 }
179 }
180 for (da_int j = 0; j < n_cols; j++) {
181 for (da_int i = 0; i < n_rows; i++) {
182 err = std::max(err, std::abs(X[ldx * j + i] - X_exp[ldx * j + i]));
183 }
184 }
185 if (err > tol) {
186 std::cout << "Solution is not within the expected tolerance: " << err
187 << std::endl;
188 exit_code = 1;
189 }
190
191 } else {
192 exit_code = 1;
193 }
194
195 std::cout << "-----------------------------------------------------------------------"
196 << std::endl;
197
198 return exit_code;
199}