ffmatrixmult: Matrix multiplication with "ff_matrix" or "matrix" inputs

In bootSVD: Fast, Exact Bootstrap Principal Component Analysis for High Dimensional Data

Description

A function for crossprod(x,y), for tcrossprod(x,y), or for regular matrix multiplication, that is compatible with ff matrices. Multiplication is done without creating new matrices for the transposes of x or y. Note, the crossproduct function can't be applied directly to objects with class ff.

Usage

ffmatrixmult(x, y = NULL, xt = FALSE, yt = FALSE, ram.output = FALSE,
  override.big.error = FALSE, ...)

Arguments

x	a matrix or ff_matrix
y	a matrix or ff_matrix. If NULL, this is set equal to x, although a second copy of the matrix x is not actually stored.
xt	should the x matrix be transposed before multiplying
yt	should the y matrix be transposed before multiplying (e.g. xt=TRUE, yt=FALSE leads to crossprod(x,y)).
ram.output	force output to be a normal matrix, as opposed to an object with class ff.
override.big.error	If the dimension of the final output matrix is especially large, ffmatrixmult will abort, giving an error. This is meant to avoid the accidental creation of very large matrices. Set override.big.error=TRUE to bypass this error.
...	passed to ff.

Value

A standard matrix, or a matrix with class ff if one of the input matrices has class ff.

Examples

## Not run: 
 library(ff)
	#Tall data
	y_tall<-matrix(rnorm(5000),500,10) #y tall
	x_tall<-matrix(rnorm(5000),500,10)
	y_wide<-t(y_tall)
	x_wide<-t(x_tall)
	y_tall_ff<-as.ff(y_tall) #y tall and ff
	x_tall_ff<-as.ff(x_tall) 
	y_wide_ff<-as.ff(y_wide) #y tall and ff
	x_wide_ff<-as.ff(x_wide) 

 #Set options to ensure that block matrix algebra is actually done,
 #and the entire algebra isn't just one in one step.
 #Compare ffmatrixmult against output from standard methods
 options('ffbytesize'=100)

 #small final matrices
	#x'x
	range(  crossprod(x_tall) - ffmatrixmult(x_tall_ff, xt=TRUE)  )
	range(  tcrossprod(x_wide) - ffmatrixmult(x_wide_ff, yt=TRUE)  )
	range(  crossprod(x_tall,y_tall) - ffmatrixmult(x_tall_ff,y_tall_ff, xt=TRUE)  )
	range(  tcrossprod(x_wide,y_wide) - ffmatrixmult(x_wide_ff,y_wide_ff, yt=TRUE)  )
	range(  (x_wide%*%y_tall) - ffmatrixmult(x_wide_ff,y_tall_ff)  )

	#ff + small data
	s_tall <- matrix(rnorm(80),10,8) 
	s_wide <- matrix(rnorm(80),8,10) 

	#tall output
	range(  crossprod(x_wide, s_tall) - ffmatrixmult(x_wide_ff, s_tall,xt=TRUE)[]  )
	range(  tcrossprod(x_tall, s_wide) - ffmatrixmult(x_tall_ff, s_wide,yt=TRUE)[]  )
	range( x_tall%*%s_tall - ffmatrixmult(x_tall_ff, s_tall)[])

	#Wide output
	range(  crossprod(s_tall, y_wide) - ffmatrixmult( s_tall, y_wide_ff,xt=TRUE)[]  )
	range(  tcrossprod(s_wide, y_tall) - ffmatrixmult( s_wide,y_tall_ff,yt=TRUE)[]  )
	range( s_wide%*%y_wide - ffmatrixmult(s_wide,y_wide_ff)[])

 #Reset options for more practical use
 options('ffbytesize'=16777216)


## End(Not run)

Example output

Loading required package: bit

Attaching package: ‘bit’

The following object is masked from ‘package:base’:

    xor

Attaching package ff
- getOption("fftempdir")=="/work/tmp/tmp/RtmpBd00tx/ff"

- getOption("ffextension")=="ff"

- getOption("ffdrop")==TRUE

- getOption("fffinonexit")==TRUE

- getOption("ffpagesize")==65536

- getOption("ffcaching")=="mmnoflush"  -- consider "ffeachflush" if your system stalls on large writes

- getOption("ffbatchbytes")==16777216 -- consider a different value for tuning your system

- getOption("ffmaxbytes")==536870912 -- consider a different value for tuning your system


Attaching package: ‘ff’

The following objects are masked from ‘package:utils’:

    write.csv, write.csv2

The following objects are masked from ‘package:base’:

    is.factor, is.ordered

[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0
[1] 0 0

bootSVD documentation built on Feb. 2, 2021, 5:06 p.m.