mmap.csv: Memory Map Text File

In mmap: Map Pages of Memory

Description

Reads a file column by column and creates a memory mapped object.

Usage

mmap.csv(file, 
         header = TRUE, 
         sep = ",", 
         quote = "\"", 
         dec = ".", 
         fill = TRUE, 
         comment.char = "", 
         row.names,
         ...)

Arguments

file	the name of the file containing the comma-separated values to be mapped.
header	does the file contain a header line?
sep	field separator character
quote	the set of quoting characters
dec	the character used for decimal points in the file
fill	unimplemented
comment.char	unimplemented
row.names	what it says
...	additional arguments

Details

mmap.csv is meant to be the analogue of read.csv in R, with the primary difference being that data is read, by column, into memory-mapped structs on disk. The intention is to allow for comma-separated files to be easily mapped into memory without having to load the entire object at once.

Value

An mmap object containing the data from the file. All types will be set to the equivelant type from mmap as would be in R from a call to read.csv.

Warning

At present the memory required to memory-map a csv file will be the memory required to load a single column from the file into R using the traditional read.table function. This may not be adequately efficient for extremely large data.

Note

This is currently a very simple implementation to facilitate exploration of the mmap package. While the interface will remain consistent with read.csv from utils, more additions to handle various out-of-core types available in mmap as well as performance optimization will be added.

Author(s)

Jeffrey A. Ryan

See Also

mmap, read.csv

Examples

data(cars)
tmp <- tempfile()
write.csv(cars, file=tmp, row.names=FALSE)

m <- mmap.csv(tmp)

colnames(m) <- colnames(cars)

m[]

extractFUN(m) <- as.data.frame  # coerce list to data frame upon subset

m[1:3,]

munmap(m)

Example output

   speed dist
1      4    2
2      4   10
3      7    4
4      7   22
5      8   16
6      9   10
7     10   18
8     10   26
9     10   34
10    11   17
11    11   28
12    12   14
13    12   20
14    12   24
15    12   28
16    13   26
17    13   34
18    13   34
19    13   46
20    14   26
21    14   36
22    14   60
23    14   80
24    15   20
25    15   26
26    15   54
27    16   32
28    16   40
29    17   32
30    17   40
31    17   50
32    18   42
33    18   56
34    18   76
35    18   84
36    19   36
37    19   46
38    19   68
39    20   32
40    20   48
41    20   52
42    20   56
43    20   64
44    22   66
45    23   54
46    24   70
47    24   92
48    24   93
49    24  120
50    25   85
  speed dist
1     4    2
2     4   10
3     7    4

mmap documentation built on May 2, 2019, 4:45 p.m.