knitr::opts_chunk$set(echo = TRUE, cache=FALSE, collapse=TRUE)
The following bullets describe problems or frustrations that scipiper
users might encounter, along with some ideas for how to interpret and fix those issues. Each "pitfall" is presented in this format:
scipiper
and remake
don't update their databases when you run a function outside of scipiper, even if the function creates the desired outputs. This may be frustrating if you've just run a long-running function outside of scipiper
and are now attempting to use scipiper
to build targets that depend on the function's output, because scipiper
will insist on running that function again. scipiper
target rather than calling the function directly.
scipiper
target that runs function A()
to create target a
, setting RStudio checkpoints or debug(A)
won't let you debug function A
when you call scmake('a')
. browser()
within the function instead.
*/tmp/*
folders and .gitignoring those folders. You may also choose to make exceptions, committing some data or configuration files that are very small and/or text-based, especially when their git diffs would be informative (e.g., when a file describes user settings or summarizes the shared status of a project build).
.ind
file promising the existence of the data file in the cache, and (2) a build/status/*.yml
file telling remake
how to decide whether the .ind
file is up to date. git commit
the .ind
and build/status/*.yml
files corresponding to all shared data files.
.rds
, .feather
, or .tsv
) and use the standard shared cache syntax.
.ind
files (and objects always lack .ind
files), which are the key to avoiding rebuilds. With a shared cache approach, every data file or object not represented by an .ind
file must be built locally by everybody who needs to build any target downstream of that file or object. .ind
file and make downstream targets depend on the .ind
file rather than the data file, (3) push the file to the shared cache, and (4) commit the .ind
file to the repository.
.ind
files than I want to manage..ind
file. If (a) you're comfortable requiring everybody to build a file or object locally, or (b) there are no declared downstream dependencies of the file/object, then you can skip creating an .ind
file for it. For example, you might have a task remake file whose execution (e.g., with loop_tasks()
) depends on X
and Y
and produces a single output file Z
. As long as the main remake file says that loop_tasks()
creates Z.ind
(not Z
), you can have just one person run the entire task remake file, and your team can share the output, without using any .ind
files within those tasks. .ind
files for targets that don't need them. Just remember that anything without an .ind
file must be rebuilt by anyone else trying to access that target.
.ind
. or build/status/*.yml
files, other teammates who are concurrently building targets that depend on your target will be forced to rebuild the target because their .ind
and build/status/*.yml
files show that they are outdated.options(scipiper.dry_put = TRUE)
to prevent files built from being pushed to the shared cache. Remember to revert to options(scipiper.dry_put = FALSE)
and force a rebuild when you are ready to push your changes to the code, .ind
, and build/status/*.yml
files.scipiper
targets, because what appears to be up to date on the originating OS will appear to be out of date on another OS, triggering unnecessary rebuilds. base::writeLines
, utils::write.csv
). Instead use readr
for all file writing.
.rds
files containing sf
objects with shared cache.sf
objects are stored differently by each OS; again, because the hash in the build/status/*.yml
file will then differ from the hash on the teammate's computer, this could trigger unnecessary rebuilds. Note that in some cases this might be the result of using two slightly different versions of sf
rather than two different OSes. sf
objects for intermediate data storage anyway (it might work), or save these objects in another format.
*.Rproj
file to use Insert spaces for tab: Tab width = 2; Strip trailing horizontal whitespace when saving = TRUE; and Line ending conversion: Posix (LF). If a shared *.Rproj
file is not an option, create a .gitattributes file declaring * text eol=lf instead.
.ind
and build/status/*.yml
files) to the shared cache.
Note:
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.