| Title: | Tests for Determining if the Covariance Structure of 2-Dimensional Data is Separable |
|---|---|
| Description: | Functions for testing if the covariance structure of 2-dimensional data (e.g. samples of surfaces X_i = X_i(s,t)) is separable, i.e. if covariance(X) = C_1 x C_2. A complete descriptions of the implemented tests can be found in the paper Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431--1461. <doi:10.1214/16-AOS1495> <https://projecteuclid.org/euclid.aos/1498636862> <arXiv:1505.02023>. |
| Authors: | Shahin Tavakoli [aut, cre], Davide Pigoli [ctb], John Aston [ctb] |
| Maintainer: | Shahin Tavakoli <[email protected]> |
| License: | GPL-2 |
| Version: | 1.1.0 |
| Built: | 2025-01-21 02:53:24 UTC |
| Source: | https://github.com/stavakol/covsep |
Marginal covariance matrix C1 used for simulations in the paper http://arxiv.org/abs/1505.02023
C1C1
An object of class matrix with 32 rows and 32 columns.
This is a 32x32 real-valued covariance matrix.
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
data(C1) str(C1)data(C1) str(C1)
Marginal covariance matrix C2 used for simulations in paper http://arxiv.org/abs/1505.02023
C2C2
An object of class matrix with 7 rows and 7 columns.
This is a 7x7 real-valued covariance matrix.
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
data(C2) str(C2)data(C2) str(C2)
This function performs the asymptotic test for the separability of the covariance operator for a random surface generated from a Gaussian process (described in the paper http://arxiv.org/abs/1505.02023).
clt_test(Data, L1, L2)clt_test(Data, L1, L2)
Data |
a (non-empty) |
L1 |
an integer or vector of integers in |
L2 |
an integer or vector of integers in |
The p-value of the test for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
If L1 and L2 are vectors, they need to be of the same length.
The function tests for separability using the projection of the covariance
operator in the separable eigenfunctions u_i tensor v_j : i = 1, ..., l1;
j = 1, ..., l2, for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
The test works by using asymptotics, and is only valid if the data is assumed to be Gaussian.
The surface data needs to be measured or resampled on a common regular grid or on common basis functions.
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
empirical_bootstrap_test, gaussian_bootstrap_test
data(SurfacesData) clt_test(SurfacesData, L1=c(1,2), L2=c(1,4))data(SurfacesData) clt_test(SurfacesData, L1=c(1,2), L2=c(1,4))
Functions for testing if the covariance structure of 2-dimensional data (e.g. samples of surfaces X_i = X_i(s,t)) is separable, i.e. if cov(X) = C_1 x C_2. A complete descriptions of the implemented tests can be found in the paper Aston et al. (2017); see references below.
The main functions are
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. <doi:10.1214/16-AOS1495>. https://projecteuclid.org/euclid.aos/1498636862
compute the difference between the full sample covariance and its separable approximation
difference_fullcov(Data)difference_fullcov(Data)
Data |
a (non-empty) |
A d1 x d2 x d1 x d2 array, where d1 = nrow(Data) and
d2 = ncol(Data).
This is an internal function.
This function performs the test for the separability of covariance structure of a random surface based on the empirical bootstrap procedure described in the paper http://arxiv.org/abs/1505.02023.
empirical_bootstrap_test(Data, L1 = 1, L2 = 1, studentize = "full", B = 1000, verbose = TRUE)empirical_bootstrap_test(Data, L1 = 1, L2 = 1, studentize = "full", B = 1000, verbose = TRUE)
Data |
a (non-empty) |
L1 |
an integer or vector of integers in |
L2 |
an integer or vector of integers in |
studentize |
parameter to specify which type of studentization is performed. Possible options are 'no', 'diag' or 'full' (see details section). |
B |
number of bootstrap replicates to be used. |
verbose |
logical parameter for printing progress |
The p-value of the test for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
This function performs the test of separability of the covariance structure for a random surface (introduced in the paper http://arxiv.org/abs/1505.02023), when generated from a Gaussian process. The sample surfaces need to be measured on a common regular grid. The test consider a subspace formed by the tensor product of eigenfunctions of the separable covariances. It is possible to specify the number of eigenfunctions to be considered in each direction.
If L1 and L2 are vectors, they need to be of the same length.
The function tests for separability using the projection of the covariance
operator in the separable eigenfunctions u_i x v_j : i = 1, ..., l1;
j = 1, ..., l2, for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
studentize can take the values
default & recommended method. Yhe projection coordinates are renormalized by an estimate of their joint covariance
NOT RECOMMENDED. No studentization is performed
NOT RECOMMENDED. Each projection coordinate is renormalized by an estimate of its standard deviation
B the number of bootstrap replicates (1000 by default).
verbose to print the progress of the computations (TRUE by default)
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
gaussian_bootstrap_test, clt_test
data(SurfacesData) empirical_bootstrap_test(SurfacesData) empirical_bootstrap_test(SurfacesData, B=100) empirical_bootstrap_test(SurfacesData,L1=2,L2=2, B=1000, studentize='full')data(SurfacesData) empirical_bootstrap_test(SurfacesData) empirical_bootstrap_test(SurfacesData, B=100) empirical_bootstrap_test(SurfacesData,L1=2,L2=2, B=1000, studentize='full')
This function performs the test for the separability of covariance structure of a random surface generated from a Gaussian process, based on the parametric bootstrap procedure described in the paper http://arxiv.org/abs/1505.02023
gaussian_bootstrap_test(Data, L1 = 1, L2 = 2, studentize = "full", B = 1000, verbose = TRUE)gaussian_bootstrap_test(Data, L1 = 1, L2 = 2, studentize = "full", B = 1000, verbose = TRUE)
Data |
a (non-empty) |
L1 |
an integer or vector of integers in |
L2 |
an integer or vector of integers in |
studentize |
parameter to specify which type of studentization is performed. Possible options are 'no', 'diag' or 'full' (see details section). |
B |
number of bootstrap replicates to be used. |
verbose |
logical parameter for printing progress |
The p-value of the test for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
This function performs the test of separability of the covariance structure for a random surface (introduced in the paper http://arxiv.org/abs/1505.02023), when generated from a Gaussian process. The sample surfaces need to be measured on a common regular grid. The test consider a subspace formed by the tensor product of eigenfunctions of the separable covariances. It is possible to specify the number of eigenfunctions to be considered in each direction.
If L1 and L2 are vectors, they need to be of the same length.
The function tests for separability using the projection of the covariance
operator in the separable eigenfunctions u_i x v_j : i = 1, ..., l1;
j = 1, ..., l2, for each pair (l1,l2) = (L1[k], L2[k]), for k = 1:length(L1).
studentize can take the values
default & recommended method. Yhe projection coordinates are renormalized by an estimate of their joint covariance
NOT RECOMMENDED. No studentization is performed
NOT RECOMMENDED. Each projection coordinate is renormalized by an estimate of its standard deviation
B the number of bootstrap replicates (1000 by default).
verbose to print the progress of the computations (TRUE by default)
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
empirical_bootstrap_test, clt_test
data(SurfacesData) gaussian_bootstrap_test(SurfacesData) gaussian_bootstrap_test(SurfacesData, B=100) gaussian_bootstrap_test(SurfacesData, L1=2,L2=2,B=1000, studentize='full')data(SurfacesData) gaussian_bootstrap_test(SurfacesData) gaussian_bootstrap_test(SurfacesData, B=100) gaussian_bootstrap_test(SurfacesData, L1=2,L2=2,B=1000, studentize='full')
Generate samples of surface data
generate_surface_data(N, C1, C2, gamma, distribution = "gaussian")generate_surface_data(N, C1, C2, gamma, distribution = "gaussian")
N |
sample size |
C1 |
row covariance |
C2 |
column covariance |
gamma |
parameter to specify how much the covariance is separable. |
distribution |
distribution of the data |
A N x dim(C1)[1] x dim(C2)[1] array containing the
generated data
gamma can take values between 0 and 1; gamma=0 corresponds to
a separable covariance, gamma=1 corresponds to a non-separable
covariance (described in the paper http://arxiv.org/abs/1505.02023).
Values of gamma between 0 and 1 corresponds to an interpolation between
these two covariances
distribution can take the values 'gaussian' or 'student'
Aston, John A. D.; Pigoli, Davide; Tavakoli, Shahin. Tests for separability in nonparametric covariance operators of random surfaces. Ann. Statist. 45 (2017), no. 4, 1431–1461. doi:10.1214/16-AOS1495. https://projecteuclid.org/euclid.aos/1498636862
Data = generate_surface_data(30, C1, C2, gamma=0)Data = generate_surface_data(30, C1, C2, gamma=0)
Empirical bootstrap test for separability of covariance structure using Hilbert–Schmidt distance
HS_empirical_bootstrap_test(Data, B = 100, verbose = TRUE)HS_empirical_bootstrap_test(Data, B = 100, verbose = TRUE)
Data |
a (non-empty) |
B |
number of bootstrap replicates to be used. |
verbose |
logical parameter for printing progress |
The p-value of the test.
This function performs the test of separability of the covariance structure for a random surface (introduced in the paper http://arxiv.org/abs/1505.02023), when generated from a Gaussian process. The sample surfaces need to be measured on a common regular grid. The test considers the Hilbert–Schmidt distance between the sample covariance and its separable approximation. WE DO NOT RECOMMEND THIS TEST, as it is does not have the correct level, nor good power.
data(SurfacesData) HS_empirical_bootstrap_test(SurfacesData) HS_empirical_bootstrap_test(SurfacesData, B = 100)data(SurfacesData) HS_empirical_bootstrap_test(SurfacesData) HS_empirical_bootstrap_test(SurfacesData, B = 100)
Gaussian (parametric) bootstrap test for separability of covariance structure using Hilbert–Schmidt distance
HS_gaussian_bootstrap_test(Data, B = 1000, verbose = TRUE)HS_gaussian_bootstrap_test(Data, B = 1000, verbose = TRUE)
Data |
a (non-empty) |
B |
number of bootstrap replicates to be used. |
verbose |
logical parameter for printing progress |
The p-value of the test.
This function performs the test of separability of the covariance structure for a random surface (introduced in the paper http://arxiv.org/abs/1505.02023), when generated from a Gaussian process. The sample surfaces need to be measured on a common regular grid. The test considers the Hilbert–Schmidt distance between the sample covariance and its separable approximation. WE DO NOT RECOMMEND THIS TEST, as it is does not have the correct level, nor good power.
data(SurfacesData) HS_gaussian_bootstrap_test(SurfacesData) HS_gaussian_bootstrap_test(SurfacesData, B = 100)data(SurfacesData) HS_gaussian_bootstrap_test(SurfacesData) HS_gaussian_bootstrap_test(SurfacesData, B = 100)
estimates marginal covariances (e.g. row and column covariances) of bi-dimensional sample
marginal_covariances(Data)marginal_covariances(Data)
Data |
a (non-empty) |
A list containing the row covariance (C1) and column covariance (C2)
Data <- rmtnorm(30, C1, C2) marginal.cov <- marginal_covariances(Data)Data <- rmtnorm(30, C1, C2) marginal.cov <- marginal_covariances(Data)
Compute the projection of the rescaled difference between the sample covariance and its separable approximation onto the separable eigenfunctions
projected_differences(Data, l1 = 1, l2 = 1, with.asymptotic.variances = TRUE)projected_differences(Data, l1 = 1, l2 = 1, with.asymptotic.variances = TRUE)
Data |
a (non-empty) |
l1 |
number of eigenfunctions to be used in the first (row) dimension for the projection |
l2 |
number of eigenfunctions to be used in the second (column) dimension for the projection |
with.asymptotic.variances |
logical variable; if TRUE, the function outputs the estimate asymptotic variances of the projected differences |
A list with
The projected differences
The row covariances of T.N
The column covariances of T.N
The function computes the projection of the rescaled difference between the sample covariance
and its separable approximation onto the separable eigenfunctions u_i
x v_j : i = 1, ..., l1; j = 1, ..., l2.
Data <- rmtnorm(30, C1, C2) ans <- projected_differences(Data, l1=1, l2=2)Data <- rmtnorm(30, C1, C2) ans <- projected_differences(Data, l1=1, l2=2)
renormalize a matrix normal random matrix to have iid entries
renormalize_mtnorm(X, C1, C2, type = "full")renormalize_mtnorm(X, C1, C2, type = "full")
X |
a matrix normal random matrix with mean zero |
C1 |
row covariance |
C2 |
column covariance |
type |
the type of renormalization to do. Possible options are 'no', 'diag' or 'full' (see details section). |
A matrix with renormalized entries
type can take the values
each entry of X is renormalized by its marginal standard deviation
X is renormalized by its root inverse covariance
Data <- rmtnorm(30, C1, C2) ans <- renormalize_mtnorm(Data[1,,], C1, C2)Data <- rmtnorm(30, C1, C2) ans <- renormalize_mtnorm(Data[1,,], C1, C2)
Generate a sample from a Matrix Gaussian distribution
rmtnorm(N, C1, C2, M = matrix(0, nrow(C1), nrow(C2)))rmtnorm(N, C1, C2, M = matrix(0, nrow(C1), nrow(C2)))
N |
sample size |
C1 |
row covariance |
C2 |
column covariance |
M |
mean matrix |
A N x dim(C1)[1] x dim(C2)[1] array containing the
generated data
Data = rmtnorm(30, C1, C2)Data = rmtnorm(30, C1, C2)
Dataset of 50 surfaces simulated from a Gaussian process with a separable covariance structure. SurfacesData[i,,] corresponds to the i-th surface observed on a 32x7 uniform grid.
SurfacesDataSurfacesData
An object of class array of dimension 50 x 32 x 7.
This is a 50 x 32 x 7 array.
data(SurfacesData) image(SurfacesData[1,,]) # color image of the first surface in the datasetdata(SurfacesData) image(SurfacesData[1,,]) # color image of the first surface in the dataset