mapreduce.control: Set mapreduce parameter for drsstl fitting

In XiaosuTong/drsstl: Spatial Seasonal Trend Loess analysis routine using divide and recombined

Description

Set control parameters of mapreduce for drsstl fits

Usage

mapreduce.control(reduceTask = 0, libLoc = NULL, BLK = 128,
  map_jvm = "-Xmx200m", reduce_jvm = "-Xmx200m", map_memory = 1024,
  reduce_memory = 1024, slow_starts = 0.5, spill_percent = 0.8,
  io_sort = 128, task_io_sort_factor = 100, reduce_parallelcopies = 5,
  reduce_shuffle_input_buffer_percent = 0.7,
  reduce_shuffle_merge_percent = 0.66, reduce_merge_inmem = 0,
  reduce_input_buffer_percent = 0, reduce_buffer_read = 150,
  map_buffer_read = 150, reduce_buffer_size = 10000,
  map_buffer_size = 10000)

Arguments

reduceTask	The reduce task number, also the number of output files. If set to be 0, then there is no shuffle and sort stage after map.
libLoc	The library path where each worker should go to load R packages. If all R packages have been push to HDFS, leave this parameter as default NULL.
BLK	The block size of output file on HDFS.
map_jvm, reduce_jvm	For mapreduce.map.java.opts and mapreduce.reduce.java.opts
map_memory, reduce_memory	For mapreduce.map.memory.mb and mapreduce.reduce.memory.mb
slow_starts	For mapreduce.job.reduce.slowstart.completedmaps
spill_percent	For mapreduce.sort.spill.percent parameter
io_sort	For mapreduce.task.io.sort.mb, the size, in megabytes, of the memory buffer to use while sorting map output.
task_io_sort_factor	For mapreduce.task.io.sort.factor
reduce_parallelcopies	For mapreduce.reduce.shuffle.parallelcopies
reduce_shuffle_input_buffer_percent	For mapreduce.reduce.shuffle.input.buffer.percent
reduce_shuffle_merge_percent	For mapreduce.reduce.shuffle.merge.percent
reduce_merge_inmem	For mapreduce.reduce.merge.inmem.shreshold
reduce_input_buffer_percent	For mapreduce.reduce.input.buffer.percent
reduce_buffer_read, map_buffer_read, reduce_buffer_size, map_buffer_size	For all rhipe arguments.

Value

A list with mapreduce tuning parameters.

Author(s)

Xiaosu Tong

Examples

    mapreduce.control()

XiaosuTong/drsstl documentation built on May 9, 2019, 11:06 p.m.