Description Usage Arguments Details Value See Also Examples
View source: R/mantaJob.launch.R
Submits R format Manta Job specification, runs Job, sends
inputs
if specified, closes inputs
, polls Job
status, returns Job status JSON.
1 2 | mantaJob.launch(inputs, job, batchsize = 500, watch = TRUE, sleep = 15,
watchtimeout = 600, silent = FALSE, verbose = FALSE)
|
inputs |
vector of character optional. List of
|
job |
required. The R job structure as created with
|
batchsize |
numeric. Maximum number of input object
paths to upload in one batch to the running job. This
function sends |
watch |
logical. When |
sleep |
integer. Number of seconds to wait between
status requests in polling mode when |
watchtimeout |
integer. Number of seconds after
which polling ends. Passed to |
silent |
logical. Supress console messages, does not affect verbose setting. |
verbose |
logical optional. Passed to |
Job is created by mantaJob.setup
and tasks as
defined therein by mantaMap
, and/or
mantaReduce
functions. Note that Manta tasks
are UNIX shell commands, not native R commands.
Returns a Manta status
JSON structure. The Manta Job
identifier is the "id": field - like this
"70c30bab-873b-66da-ebc8-ced12bd35ac4"
. This value
is the jobid
parameter to be used used by other
mantaJob
functions for inputs, status, errors and
outputs as Job lookup key. This key can also be used by
Node.js Manta command-line mjob
commands.
Other mantaJobs: mantaJob.cancel
;
mantaJob.done
;
mantaJob.errors.stderr
;
mantaJob.errors
;
mantaJob.failures
;
mantaJob.inputs
;
mantaJob.outputs.cat
;
mantaJob.outputs
;
mantaJob.setup
;
mantaJob.status
;
mantaJobs.running
;
mantaJobs.tail
; mantaJobs
;
mantaMap
; mantaReduce
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ## Not run:
## Example - Map/Reduce Unix Word Count
## Part 1.
## Job to download all of Shakespeare's plays to your account
plays <-
c("1kinghenryiv.txt", "1kinghenryvi.txt", "2kinghenryiv.txt",
"2kinghenryvi.txt", "3kinghenryvi.txt", "allswellthatendswell.txt",
"antonyandcleopatra.txt", "asyoulikeit.txt", "comedyoferrors.txt",
"coriolanus.txt", "cymbeline.txt", "hamlet.txt", "juliuscaesar.txt",
"kinghenryv.txt", "kinghenryviii.txt", "kingjohn.txt", "kinglear.txt",
"kingrichardii.txt", "kingrichardiii.txt", "loverscomplaint.txt",
"loveslabourslost.txt", "macbeth.txt", "measureforemeasure.txt",
"merchantofvenice.txt", "merrywivesofwindsor.txt",
"midsummersnightsdream.txt",
"muchadoaboutnothing.txt", "othello.txt", "periclesprinceoftyre.txt",
"rapeoflucrece.txt", "romeoandjuliet.txt", "sonnets.txt",
"tamingoftheshrew.txt",
"tempest.txt", "timonofathens.txt", "titusandronicus.txt",
"troilusandcressida.txt",
"twelfthnight.txt", "twogentlemenofverona.txt", "various.txt",
"venusandadonis.txt", "winterstale.txt")
file <- file("plays_list.txt", "wb")
# Important: This forces Windows to use /n instead of /r/n on write()
write(plays, file)
close(file)
rm(file)
mantaSetwd.stor()
mantaPut("plays_list.txt")
inputlist <- mantaLs.paths(grepfor = "plays_list.txt")
mantaMkdir("shakespeare")
mantaSetwd("shakespeare")
fileslocation <-
"https://us-east.manta.joyent.com/cwvhogue/public/shakespeare/"
destination <- mantaGetwd()
mantaJob.setup("Get Plays",
mantaMap(paste("xargs -I {} sh -c 'curl -ksL ",
fileslocation,
"{} | mput ",
destination,
"/{}'",
sep=""))) -> moveplays
## Launch the first job to download the plays:
mantaJob.launch(inputlist, moveplays)
## See if they arrived.
mantaLs()
mantaLs.n()
mantaLs.du()
## Copy all the plays to your local drive?
# mantaGet(mantaFind())
## Speedread all of Shakespeare?
# mantaCat(mantaFind())
## Part 2.
## Map/Reduce Count all the words with wc and awk
inputs <- mantaFind()
job <- mantaJob.setup(
name = "Word Count",
mantaMap("wc"),
mantaReduce("awk '{ l += $1; w += $2; c += $3 } END { print l, w, c }'")
)
mantaJob.launch(inputs, job) -> status
## Getting Job Results:
## These functions find the last Job run if no jobid provided.
# mantaJob.status() ## check to see if job is complete, as JSON information
# mantaJob.done() ## returns logical job done (TRUE/FALSE)
# mantaJob.inputs() ## returns list of inputs
mantaJob.outputs() ## retrieve list of paths to Manta output objects
# mantaJob.errors() ## retrieve JSON formatted job error information
mantaJob.outputs.cat() ## Print job output (text files) to console
# mantaJob.errors.stderr() ## Print any stderr messages to console
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.