knitr::opts_chunk$set(collapse = TRUE, comment = "#>", eval = FALSE)
This vignette is the reference for running Python scripts as targets steps with reticulate: every tar_target_py() argument, how to choose the Python environment, and the common use cases. For a gentler tour start with vignette("get_started"); for Julia see vignette("julia").
Code blocks are illustrative and not executed when the vignette builds.
tar_target_py(): bare name (and unquoted pattern), for direct use in _targets.R.tar_target_py_raw(): string name, for use inside targets factories.Both return a single targets target and forward every targets::tar_target_raw() argument, so anything an ordinary target supports (pattern, format, iteration, deployment, resources, cue, memory, error, priority, description, ...) works exactly as usual.
These constructors depend on the reticulate R package. As a result, any limitation of reticulate is inherited by the constructors. Reading reticulate's own documentation is recommended.
| Argument | Meaning |
|------------------------------------|------------------------------------|
| script | Path to the Python script to run (required). |
| pre_script | Optional R script run before the Python script. Assign a named list to_py to push objects into Python's __main__. |
| post_script | Optional R script run after the Python script. py and py_get() are available; its last expression is the value (object mode) or file paths (file mode). |
Each of the three accepts either a literal path string (untracked: editing the file does not invalidate the target) or a tar_target_path("name") reference to an upstream format = "file" target (tracked: the step re-runs when the file changes). See "Tracking scripts" below.
| Argument | Meaning |
|------------------------------------|------------------------------------|
| inputs | Named character vector mapping the name seen in the step to an upstream target, e.g. c(x = "prepared_x"). Each becomes a real dependency (and is sliced under dynamic branching). |
| output | "object" (default) returns a converted R object; "file" returns a character vector of paths and defaults format to "file". |
| retrieve | Character vector of Python variable names to return when there is no post-script (object mode). One name returns that object; several return a named list. |
| files | Character vector of paths to return when there is no post-script (file mode). |
| Argument | Meaning |
|------------------------------------|------------------------------------|
| python | Path to a specific interpreter executable. |
| env + env_manager | Point at an existing environment built by a known tool. |
| python_version | A version string (e.g. "3.12"); reticulate provisions it via an ephemeral uv environment. |
These are alternatives: normally set only one. If several are given, the first in this precedence wins: python > env/env_manager > python_version > nothing. They are covered in detail in the next section.
upstream targets ─► pre_script (R) ─► script (.py) ─► post_script (R) ─► target value
builds `to_py` computes reads `py$name` /
`result` `py_get("name")`
script: the Python file. Runs in the __main__ module.pre_script: the inputs are already bound by name. Assign a named list to_py; each element is pushed as a top-level variable into __main__ using reticulate's conversion rules.post_script: py (the reticulate __main__ proxy) and py_get(name) (an explicit reticulate::py_to_r() shortcut) are available. Read a variable with py$result or py_get("result").py/compute.py:
# `x` was pushed from R by the pre-script. seq = list(x) if hasattr(x, "__iter__") else [x] result = {"sum": float(sum(seq)), "n": len(seq), "mean": float(sum(seq)) / len(seq)}
R/pre_push.R:
to_py <- list(x = x) # `x` came from inputs = c(x = "prepared_x")
R/post_result.R (last expression becomes the value):
res <- py_get("result") data.frame(sum = res$sum, n = res$n, mean = res$mean)
_targets.R:
list( tar_target(prepared_x, c(1, 2, 3, 4)), # (a) return a Python variable directly with `retrieve` (no post-script) tar_target_py( name = py_direct, script = "py/compute.py", inputs = c(x = "prepared_x"), pre_script = "R/pre_push.R", retrieve = "result" ), # (b) reshape the result in a post-script tar_target_py( name = py_prepost, script = "py/compute.py", inputs = c(x = "prepared_x"), pre_script = "R/pre_push.R", post_script = "R/post_result.R" ) )
Set output = "file" when the script writes to disk. The target format becomes "file" automatically, and the step has to report the path(s) it wrote. There are two ways to do that, covered in turn below.
Use this when the script chooses the path, for example a temporary file whose name is not known in advance.
py/write_csv.py: writes x to a CSV, keeps the path in out_path for the post-script to read back:
import csv import tempfile out_path = tempfile.mktemp(suffix=".csv") with open(out_path, "w", newline="") as f: csv.writer(f).writerow(x)
R/post_files.R:
py_get("out_path")
tar_target_py( name = py_file, script = "py/write_csv.py", inputs = c(x = "prepared_x"), pre_script = "R/pre_push.R", post_script = "R/post_files.R", # returns py_get("out_path") output = "file" )
filesUse this when the path is already known as you write _targets.R. The script writes to that fixed location, files declares it, and no post-script is needed at all.
py/write_fixed.py:
import csv import os os.makedirs("out", exist_ok=True) with open("out/values.csv", "w", newline="") as f: csv.writer(f).writerow(x)
tar_target_py( name = py_fixed, script = "py/write_fixed.py", inputs = c(x = "prepared_x"), pre_script = "R/pre_push.R", output = "file", files = "out/values.csv" # no post-script needed )
files and post_script are alternatives, not complements. When both are given the post-script wins and files is silently ignored:
| Supplied | Result |
|----|----|
| files only | the paths in files are returned |
| post_script only | the value of the post-script's last expression is returned |
| both | the post-script wins, files is ignored |
| neither | error: output = "file" needs a post_script returning paths, or files |
inputs are real dependencies, so pattern branches them:
list( tar_target(chunks, list(10, 20, 30)), tar_target_py( name = py_branch, script = "py/compute.py", inputs = c(x = "chunks"), pre_script = "R/pre_push.R", retrieve = "result", pattern = map(chunks) # 3 branches: x = 10, 20, 30 ) )
Run the branches in parallel by setting a crew controller (see vignette("get_started"), the "Parallelism and isolation with crew" section, which also covers polyglot_controller(log = polyglot_log(...)) for per-step stdout/stderr log files, including a caveat specific to branching there, since every branch of a pattern shares one log file).
Python's own multiprocessing / concurrent.futures.ProcessPoolExecutor work correctly through run_py_step(), confirmed both in the main session and inside a real crew worker, with one hard constraint: the pool has to be created and used at the top level of the script itself, guarded by the usual if __name__ == "__main__":, not inside a function that the script only defines and that a later post_script calls. That second shape breaks on Windows: reticulate::py_run_file() runs your script directly into __main__, and when the pool is created later from a separate reticulate call, the worker processes multiprocessing spawns cannot find the function it needs to unpickle, and the pool fails with a pickling error. Doing the parallel work at the top level of script, with retrieve (or a top-level result variable a post_script reads with py_get()) to bring the value back, avoids this entirely, and is also the standard, portable way to write this in plain Python:
# py/parallel_sqrt.py from concurrent.futures import ProcessPoolExecutor def _sqrt_one(v): return v ** 0.5 if __name__ == "__main__": with ProcessPoolExecutor(max_workers=n_workers) as ex: result = list(ex.map(_sqrt_one, x))
tar_target_py( name = py_parallel, script = "py/parallel_sqrt.py", inputs = c(x = "big_x", n_workers = "n_workers"), pre_script = tar_code({ to_py <- list(x = x, n_workers = n_workers) }), retrieve = "result" )
The same oversubscription concern from vignette("get_started") applies here too: a crew worker is a full process with access to every core, not a slice of one, so several workers each spawning their own multiprocessing pool at once can oversubscribe the machine. Put the parallel-heavy step on deployment = "main" if you have more than one, or cap n_workers per step, for the same reason described there.
The four selection arguments and when to reach for each:
python: a path to a specific interpreter executable (e.g. ".venv/Scripts/python.exe"). Use when you already have the exact interpreter and want no ambiguity.env + env_manager: an existing environment built by a known tool (env_manager says how to read env; see the table). Recommended for reproducible pipelines: install the packages once, every run reuses them.python_version (e.g. "3.12"), not an environment, just an interpreter version: reticulate provisions it through an ephemeral, uv-managed environment (py_require()), resolving packages on the fly. Use when only the version matters.The key contrast is reproducible vs throwaway: env/python reuse an environment you built and pinned; python_version lets reticulate build a fresh one keyed only on a version. One more difference: python and env/env_manager override an ambient RETICULATE_PYTHON (e.g. an RStudio project Python inherited by crew workers), while python_version and the default respect it.
| env_manager | what env is | reticulate call |
|------------------------|------------------------|------------------------|
| "system" (default) | none (optionally set python) | use_python(python) or default |
| "venv" | a python -m venv directory path | use_virtualenv(env) |
| "virtualenv" | same as venv | use_virtualenv(env) |
| "conda" | conda environment name | use_condaenv(env) |
| "uv" | a uv-created venv directory path | use_virtualenv(env) |
| "poetry" | a poetry project directory path | poetry env info -p, then use_virtualenv() |
"virtualenv", "venv", and "uv" all point at a standard Python virtualenv and behave identically: this includes one created by renv::use_python() (use env_manager = "venv" with the path under renv/python/virtualenvs/). For these managers, a relative env that is an existing directory is expanded to an absolute path, so env = ".venv" works; a bare name that is not a directory is passed through unchanged, so a genuine named environment still resolves. Setting env while env_manager = "system" is an error.
The use cases below are about the environment-selection arguments, not what each script computes, so the script bodies are deliberately trivial, one line assigning the variable named in retrieve.
Set nothing; reticulate uses the computer's default Python (on recent versions, an ephemeral uv-managed one provisioned on first use). Convenient for exploration; pin a real environment for reproducibility. py/probe.py (reused in use case 5 below):
import sys result = sys.executable
tar_target_py( name = probe, script = "py/probe.py", retrieve = "result" # no env / env_manager / python -> computer default Python )
Create it once, then point to it. Reproducible: the exact package set is reused every run.
uv venv .venv uv pip install --python .venv/Scripts/python.exe numpy pandas scikit-learn
py/fit.py:
model = {"coef": 1.5, "intercept": 0.2}
tar_target_py( name = fit, script = "py/fit.py", env_manager = "uv", env = ".venv", retrieve = "model" )
Give the project directory (the folder with pyproject.toml); its virtualenv is resolved for you with poetry env info -p. py/embed.py:
vectors = [[0.1, 0.2], [0.3, 0.4]]
tar_target_py( name = embed, script = "py/embed.py", env_manager = "poetry", env = "path/to/poetry_project", retrieve = "vectors" )
If the poetry on PATH is not the one you want, select a specific executable with options(tarpolyglot.poetry = "/path/to/poetry").
py/score.py:
result = 0.87
py/classify.py:
labels = ["setosa", "versicolor", "virginica"]
# python -m venv .venv tar_target_py( name = score, script = "py/score.py", env_manager = "venv", env = ".venv", retrieve = "result" ) # a named conda environment tar_target_py( name = classify, script = "py/classify.py", env_manager = "conda", env = "my_conda_env", retrieve = "labels" )
To fix the interpreter version without naming an environment, use python_version (leaves the environment at the default, uv-managed one):
tar_target_py( name = probe_ver, script = "py/probe.py", python_version = "3.12", # reticulate provisions 3.12 via uv retrieve = "result" )
One interpreter per session. reticulate binds a single Python per R session, so all Python targets that run in the same session share one environment. To use different environments in one pipeline, run those targets on separate
crewworkers (seevignette("get_started")).
By default script / pre_script / post_script are untracked literal paths. To have the step re-run when a script changes, track it with a format = "file" target and reference it with tar_target_path(). py/fit.py is the same script shown in "Use case 2" above; R/pre_fit.R:
to_py <- list(x = x) # `x` came from inputs = c(x = "data")
list( tar_target(fit_py, "py/fit.py", format = "file"), tar_target_py( name = fit, script = tar_target_path("fit_py"), # re-runs when py/fit.py changes pre_script = "R/pre_fit.R", # still an untracked literal path inputs = c(x = "data"), retrieve = "model" ) )
L (e.g. 42L) when Python needs an int.See ?tar_target_py and ?run_py_step for the full argument reference.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.