bdCrossprod_hdf5: Crossprod with hdf5 matrix

In BigDataStatMeth: Statistical Methods and Algorithms for Big Data

Crossprod with hdf5 matrix

Description

This function performs the crossprod from a matrix inside and hdf5 data file

Usage

bdCrossprod_hdf5(
  filename,
  group,
  A,
  groupB = NULL,
  B = NULL,
  block_size = NULL,
  paral = NULL,
  threads = NULL,
  mixblock_size = NULL,
  outgroup = NULL
)

Arguments

filename	string file name where dataset to normalize is stored
group,	string, group name where dataset A is stored
A	string name inside HDF5 file
groupB,	string, group name where dataset b is stored
B	string, dataset name for matrix B inside HDF5 file
block_size	(optional, defalut = 128) block size to make matrix multiplication, if 'block_size = 1' no block size is applied (size 1 = 1 element per block)
paral,	(optional, default = TRUE) if paral = TRUE performs parallel computation else performs seria computation
threads	(optional) only if bparal = true, number of concurrent threads in parallelization if threads is null then threads = maximum number of threads available
mixblock_size	(optional) only for debug pourpose
outgroup	(optional) group name to store results from Crossprod inside hdf5 data file

Value

If all process is ok, returns a list with :

• "filename" File name where results are storesd

• "dataset" route to results inside hdf5 data file

Examples

  
  library(BigDataStatMeth)
  library(rhdf5)
     
  matA <- matrix(c(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15), nrow = 3, byrow = TRUE)
  matB <- matrix(c(15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,5,3,4,5,2,6,2,3,4,
                   42, 23, 23, 423,1,2), nrow = 3, byrow = TRUE)
  
  bdCreate_hdf5_matrix_file("BasicMatVect.hdf5", matA, "INPUT", "matA")
  bdAdd_hdf5_matrix(matB, "BasicMatVect.hdf5", "INPUT", "matB")
  
  res <- bdCrossprod_hdf5("BasicMatVect.hdf5", "INPUT","matA", block_size = 3)
  res2 <- bdCrossprod_hdf5("BasicMatVect.hdf5", "INPUT",
                           "matA", "INPUT","matB", block_size = 3)
  
  # Examine hierarchy before open file
  h5ls("BasicMatVect.hdf5")
  
  # Open file
  h5fdelay = H5Fopen("BasicMatVect.hdf5")
  
  # Show hdf5 hierarchy (groups)
  h5fdelay
  
  res <- h5fdelay$OUTPUT$CrossProd_matAxmatA
  res2 <- h5fdelay$OUTPUT$CrossProd_matAxmatB
  
  all.equal(crossprod(matA), res)
  all.equal(crossprod(matA,matB), res2)
  
  # Close delayed.hdf5 file
  H5Fclose(h5fdelay)
  
  # Remove file (used as example)
  if (file.exists("BasicMatVect.hdf5")) {
    # Delete file if it exist
    file.remove("BasicMatVect.hdf5")
  }
  

BigDataStatMeth documentation built on March 30, 2022, 1:07 a.m.