In beauchamplab/raveio: File-System Toolbox for RAVE Project

# This code block sets up the engine environment
# Please do not remove me
raveio::pipeline_setup_rmd("TEMPLATE")

Introduction

A RAVE pipeline markdown is an interactive notebook that can keep your notes, code blocks, and corresponding results together, generating reports in various formats such as PDF, html, Word, PowerPoint.

The note parts are simply markdowns - the same as jupyter notebook, or github documentations. The code blocks support R, python, c++. When you hit the Knit button in this code editor panel, the r-markdown file will be compiled, generating reproducible documentation.

With carefully designed structures, this r-markdown file will automatically generate RAVE pipeline scripts during the compilation. The pipeline script can be used by RAVE to convert your pipeline into interactive dashboard application. (This feature is currently under development)

"RAVE" Pipeline Code Block

A RAVE pipeline markdown code block starts with ```{rave .... The block label following rave informative description of the target. After the target, the following RAVE-specific parameters configures how the block should be treated:

language: specifies the programming language used; choices are: R, python
export: variable name to be exported that will be available to the rest chunks
depends: indicates current block depends on variables generated from other blocks; this helps RAVE to build non-interactive pipeline scripts internally. For blocks written in R, the dependence can be automatically determined.

Other parameters are available at this rmarkdown book

An Example

Variables in the settings.yaml can be used directly

input_data <- data.frame(
  x = runif(n),
  y = runif(n)
)

Using cue="always" to never skip the target

plot_data <- Sys.time()
plot(input_data, pch = pch, col = col)

Build, Visualize, & Run

Please make sure the following code block is at the end of your pipeline file. This block will build the pipeline and generate a make-TEMPLATE.R script with your pipeline markdown file. RAVE will use the generated pipeline script to execute the pipeline in the dashboard application, or in massive production mode.

build_pipeline(make_file = "make-TEMPLATE.R")

Once the pipeline script make-TEMPLATE.R is built, you can visualize and execute the pipeline without the need of re-knit this document. Notice we use r block instead of rave. (This is because the code blocks are not part of pipeline targets.)

# Fixed usage, show pipeline graph
try({
  asNamespace("raveio")$pipeline_dependency_graph(
    pipeline_path = ".", glimpse = TRUE)
}, silent = TRUE)