data_frame: function to create a data_frame object

In bigReg: Generalized Linear Models (GLM) for Large Data Sets

Description

function to create a data_frame object. The data_frame object is an object that is held on disk. It is written to a folder path on disk where the data is written to in blocks or chunks. The data is written in binary format using a C++ function in purely numerical data and a mapping to the table is held in a ".meta_data" file in the folder. The table object accomodates numeric, factor, and character (converted to factor).

Usage

data_frame(data = stop("data must be supplied"),
  chunkSize = stop("chunkSize must be specified, a good number is 50000"),
  path = stop("path must be specified"), nCores = parallel::detectCores(),
  ...)

Arguments

data	data.frame object to be converted into a data_frame object
chunkSize	number of rows to be used in each chunk
path	character to folder where the object will be created
nCores	the number of cores to use defaults to parallel::detectCores()
...	not currently used.

Details

Creates a data_frame object

Examples

irisA <- data_frame(iris[1:75,], 10, "irisA", nCores = 1)
irisA$append(iris[76:150,])
irisA$head()
irisA$tail(10)
irisA$delete(); rm(irisA)

Example output

Loading required package: Rcpp
Loading required package: parallel
Loading required package: uuid
Loading required package: MASS
  Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1          5.1         3.5          1.4         0.2  setosa
2          4.9         3.0          1.4         0.2  setosa
3          4.7         3.2          1.3         0.2  setosa
4          4.6         3.1          1.5         0.2  setosa
   Sepal.Length Sepal.Width Petal.Length Petal.Width   Species
1           6.7         3.1          5.6         2.4 virginica
2           6.9         3.1          5.1         2.3 virginica
3           5.8         2.7          5.1         1.9 virginica
4           6.8         3.2          5.9         2.3 virginica
5           6.7         3.3          5.7         2.5 virginica
6           6.7         3.0          5.2         2.3 virginica
7           6.3         2.5          5.0         1.9 virginica
8           6.5         3.0          5.2         2.0 virginica
9           6.2         3.4          5.4         2.3 virginica
10          5.9         3.0          5.1         1.8 virginica

bigReg documentation built on May 2, 2019, 6:43 a.m.