In privefl/bigstatsr: Statistical Tools for Filebacked Big Matrices

knitr::opts_chunk$set(echo = TRUE)

In this vignette, you learn how to read a Filebacked Big Matrix from a text file. Package {bigreadr} is required.

Data

library(bigstatsr)
library(bigreadr)

## LONG CSV
df <- datasets::mtcars
csv <- fwrite2(df[rep(seq_len(nrow(df)), 500000), ], 
               tempfile(fileext = ".csv"), 
               row.names = TRUE)
format(file.size(csv), big.mark = ",")

Check file content

nlines(csv)
(first_rows <- fread2(csv, nrows = 5))
sapply(first_rows, typeof)
ncol(first_rows)

What you can see with these first lines:

• there are some column names (header),
• each line has 12 elements,
• the first element of each line stores some row names,
• and the other elements seems to be all numeric.

Read those data

• Read all numeric columns in an FBM r (test <- big_read(csv, select = 2:12)) rbind(csv, test$backingfile) attr(test, "fbm_names")

• Read other non-numeric data afterwards r meta <- fread2(csv, select = 1) head(meta)

Read filtered data

## Get the filter data
filter <- fread2(csv, select = "cyl")[[1]] == 4
## Read only rows corresponding to 'filter'
(test2 <- big_read(csv, select = 2:12, filter = filter,
                   backingfile = tempfile()))

test2$is_saved
(rds <- test2$rds)

You need to read from the text file only once. To get the FBM object in another R session, just use big_attach():

(test3 <- big_attach(rds))

privefl/bigstatsr documentation built on March 29, 2024, 3:31 a.m.