Convert input matrix into a distributed array.
input matrix that will be converted to darray.
size of each partition as a vector specifying number of rows and columns.
If partition size (psize) is missing then the input matrix is row partitioned and striped across the cluster, i.e., the returned distributed array has approximately as many partitions as the number of R instances in the session.
The last set of partitions may have fewer rows or columns if input matrix size is not an integer multiple of partition size. If 'A' is a 5x5 matrix, then 'as.darray(A, psize=c(2,5))' is a distributed array with three partitions. The first two partitions have two rows each but the last partition has only one row. All three partitions have five columns.
To create a distributed darray with just one partition, pass the dimension of the input frame, i.e. 'as.darray(A, psize=dim(A))'
Returns a distributed array with dimensions equal to that of the input matrix and partitioned according to argument 'psize'. Data may reside as partitions on remote nodes.
Prasad, S., Fard, A., Gupta, V., Martinez, J., LeFevre, J., Xu, V., Hsu, M., Roy, I. Large scale predictive analytics in Vertica: Fast data transfer, distributed model creation and in-database prediction. _Sigmod 2015_, 1657-1668.
Venkataraman, S., Bodzsar, E., Roy, I., AuYoung, A., and Schreiber, R. (2013) Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices. _EuroSys 2013_, 197-210.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
## Not run: ##Create 4x4 matrix mtx<-matrix(sample(0:1, 16, replace=T), nrow=4) ##Create distributed array spread across the cluster da<-as.darray(mtx) psize(da) ##Create distributed array with single partition db<-as.darray(mtx, psize=dim(mtx)) psize(db) ##Create distributed array with two partitions dc<- as.darray(mtx, psize=c(2,4)) psize(dc) ##Fetch first partition collect(dc,1) ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.