datamodels: Read and write data models for LaF
In djvanderlaan/LaF: Fast Access to Large ASCII Files

Description Usage Arguments Details Value See Also Examples

Using these routines data models can be written and read. These data models can be used to create LaF object without the need to specify all arguments (column names, column types etc.). Opening of files using the data models can be done using laf_open.

1
2
3

read_dm(modelfile, ...)

write_dm(model, modelfile)

`modelfile`	character containing the filename of the file the model is to be written to/read from.
`...`	additional arguments are added to the data model or, when they are also present in the file are used to overwrite the values specified in the file.
`model`	a data model or an object of type `laf`. See details for more information.

A data model is a list containing information which open routine should be used (e.g. laf_open_csv or laf_open_fwf), and the arguments needed for these routines. Required elements are ‘type’, which can (currently) be ‘csv’, or ‘fwf’, and ‘columns’, which should be a data.frame containing at least the columns ‘name’ and ‘type’, and for fwf ‘width’. These columns correspond to the arguments column_names, column_types and column_widths respectively. Other arguments of the laf_open_* routines can be specified as additional elements of the list.

write_dm can also be used to write a data model that is created from an object of type laf. This is probably one of the easiest ways to create a data model.

The data model is stored in a text file in YAML format which is a format in which data structures can be stored in a readable and editable format.

read_dm returns a data model which can be used by laf_open.

See detect_dm_csv for a routine which can automatically create a data model from a CSV-file. The data models can be used to open a file using laf_open.

# Create some temporary files
tmpcsv  <- tempfile(fileext="csv")
tmp2csv <- tempfile(fileext="csv")
tmpyaml <- tempfile(fileext="yaml")

# Generate test data
ntest <- 10
column_types <- c("integer", "integer", "double", "string")
testdata <- data.frame(
    a = 1:ntest,
    b = sample(1:2, ntest, replace=TRUE),
    c = round(runif(ntest), 13),
    d = sample(c("jan", "pier", "tjores", "corneel"), ntest, replace=TRUE)
    )
# Write test data to csv file
write.table(testdata, file=tmpcsv, row.names=FALSE, col.names=FALSE, sep=',')

# Create LaF-object
laf <- laf_open_csv(tmpcsv, column_types=column_types)

# Write data model to stdout() (screen)
write_dm(laf, stdout())

# Write data model to file
write_dm(laf, tmpyaml)

# Read data model and open file
laf2 <- laf_open(read_dm(tmpyaml))

# Write test data to second csv file
write.table(testdata, file=tmp2csv, row.names=FALSE, col.names=FALSE, sep=',')

# Read data model and open seconde file, demonstrating the use of the optional
# arguments to read_dm
laf2 <- laf_open(read_dm(tmpyaml, filename=tmp2csv))

# Cleanup
file.remove(tmpcsv)
file.remove(tmp2csv)
file.remove(tmpyaml)