bdtCrossprod_hdf5: Transposed Crossprod with hdf5 matrix

In BigDataStatMeth: Statistical Methods and Algorithms for Big Data

Transposed Crossprod with hdf5 matrix

Description

This function performs the transposed crossprod from a matrix inside and hdf5 data file

Usage

bdtCrossprod_hdf5(
  filename,
  group,
  A,
  groupB = NULL,
  B = NULL,
  block_size = NULL,
  paral = NULL,
  threads = NULL,
  mixblock_size = NULL,
  outgroup = NULL
)

Arguments

filename	string file name where dataset to normalize is stored
group,	string, group name where dataset A is stored
A	string name inside HDF5 file
groupB,	string, group name where dataset b is stored
B	string, dataset name for matrix B inside HDF5 file
block_size	(optional, defalut = 128) block size to make matrix multiplication, if 'block_size = 1' no block size is applied (size 1 = 1 element per block)
paral,	(optional, default = TRUE) if paral = TRUE performs parallel computation else performs seria computation
threads	(optional) only if bparal = true, number of concurrent threads in parallelization if threads is null then threads = maximum number of threads available
mixblock_size	(optional) only for debug pourpose
outgroup	(optional) group name to store results from Crossprod inside hdf5 data file

Value

If all process is ok, returns a list with :

• "filename" File name where results are storesd

• "dataset" route to results inside hdf5 data file

Examples

library(BigDataStatMeth)
library(rhdf5)

matA <- matA <- matrix(c(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15), nrow = 3, byrow = TRUE)
matB <- matrix(c(15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,5,3,4,5,2,6,2,3,4,
                 42, 23, 23, 423,1,2), ncol = 5, byrow = TRUE)

bdCreate_hdf5_matrix_file("BasicMatVect.hdf5", matA, "INPUT", "matA")
bdAdd_hdf5_matrix( matB, "BasicMatVect.hdf5", "INPUT", "matB")

res <- bdtCrossprod_hdf5("BasicMatVect.hdf5", "INPUT",
                                        "matA", block_size = 2)
res2 <- bdtCrossprod_hdf5("BasicMatVect.hdf5", "INPUT", 
                                        "matA", "INPUT","matB", block_size = 2)

# Open file
h5fdelay = H5Fopen("BasicMatVect.hdf5")

res <- h5fdelay$OUTPUT$tCrossProd_matAxmatA
res2 <- h5fdelay$OUTPUT$tCrossProd_matAxmatB

all.equal(tcrossprod(matA), res)
all.equal(tcrossprod(matA, matB), res2)

# Close delayed.hdf5 file
H5Fclose(h5fdelay)

# Remove file (used as example)
if (file.exists("BasicMatVect.hdf5")) {
  # Delete file if it exist
  file.remove("BasicMatVect.hdf5")
}

BigDataStatMeth documentation built on March 30, 2022, 1:07 a.m.