Description Usage Arguments Value Author(s) Examples
Set control parameters of mapreduce for drsstl fits
1 2 3 4 5 6 7 8 9 | mapreduce.control(reduceTask = 0, libLoc = NULL, BLK = 128,
map_jvm = "-Xmx200m", reduce_jvm = "-Xmx200m", map_memory = 1024,
reduce_memory = 1024, slow_starts = 0.5, spill_percent = 0.8,
io_sort = 128, task_io_sort_factor = 100, reduce_parallelcopies = 5,
reduce_shuffle_input_buffer_percent = 0.7,
reduce_shuffle_merge_percent = 0.66, reduce_merge_inmem = 0,
reduce_input_buffer_percent = 0, reduce_buffer_read = 150,
map_buffer_read = 150, reduce_buffer_size = 10000,
map_buffer_size = 10000)
|
reduceTask |
The reduce task number, also the number of output files. If set to be 0, then there is no shuffle and sort stage after map. |
libLoc |
The library path where each worker should go to load R packages. If all R packages have been push to HDFS, leave this parameter as default NULL. |
BLK |
The block size of output file on HDFS. |
map_jvm, reduce_jvm |
For mapreduce.map.java.opts and mapreduce.reduce.java.opts |
map_memory, reduce_memory |
For mapreduce.map.memory.mb and mapreduce.reduce.memory.mb |
slow_starts |
For mapreduce.job.reduce.slowstart.completedmaps |
spill_percent |
For mapreduce.sort.spill.percent parameter |
io_sort |
For mapreduce.task.io.sort.mb, the size, in megabytes, of the memory buffer to use while sorting map output. |
task_io_sort_factor |
For mapreduce.task.io.sort.factor |
reduce_parallelcopies |
For mapreduce.reduce.shuffle.parallelcopies |
reduce_shuffle_input_buffer_percent |
For mapreduce.reduce.shuffle.input.buffer.percent |
reduce_shuffle_merge_percent |
For mapreduce.reduce.shuffle.merge.percent |
reduce_merge_inmem |
For mapreduce.reduce.merge.inmem.shreshold |
reduce_input_buffer_percent |
For mapreduce.reduce.input.buffer.percent |
reduce_buffer_read, map_buffer_read, reduce_buffer_size, map_buffer_size |
For all rhipe arguments. |
A list with mapreduce tuning parameters.
Xiaosu Tong
1 |
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.