partition_map: Apply function along the last dimension of an array and...

In dipterix/farray: Fast File-Based Out-of-Memory Array

Apply function along the last dimension of an array and aggregate the results

Description

Apply function along the last dimension of an array and aggregate the results

Usage

partition_map(x, map_fun, reduce, partitions, ...)

Arguments

x	R array or farray
map_fun	function that takes in a slice of array and an optional argument indicating current partition number
reduce	function that accept a list of results returned by map_fun, can be missing
partitions	integers of partitions, i.e. the slices of array to be applied to, can be missing. If missing, then applies to all partitions
...	internally used

Value

If reduce is missing, returns a list of results. Each result is returned by map_fun, and the total length equals to number of partitions mapped. If reduce is a function, that list of results will be passed to reduce and partition_map returns the results generated from reduce.

Author(s)

Zhengjia Wang

Examples

# -------------------------- Ordinary R array ---------------------------

x <- array(1:24, c(2,3,4))
partition_map(x, function(slice, part){
  sum(slice)
})

# When reduce and partitions are missing, the following code is equivalent
as.list(apply(x, 3, sum))

# When reduce is present
partition_map(x, function(slice, part){
  sum(slice)
}, function(slice_sum){
  max(unlist(slice_sum))
})

# equivalently, we could call
slice_sum <- partition_map(x, function(slice, part){
  sum(slice)
})
max(unlist(slice_sum))

# When partition is specified
# Partition 1, 2, and 4 exist but 5 is missing
# when a partition is missing, the missing slice will be NA
partition_map(x, function(slice, part){
  sum(slice)
}, partitions = c(1,2,4,5))

# -------------------------- farray ---------------------------
x <- farray(tempfile(), storage_format = 'integer', dim = c(2,3,4))
x[] <- 1:24

partition_map(x, function(slice, part){
  slice[1, ,] * slice[2, ,]
}, reduce = function(mapped_prod){
  mean(unlist(mapped_prod))
})

dipterix/farray documentation built on Oct. 16, 2022, 6:13 p.m.