tar_resources_crew: Target resources: 'crew' high-performance computing

In targets: Dynamic Function-Oriented 'Make'-Like Declarative Pipelines

Target resources: crew high-performance computing

Description

Create the crew argument of tar_resources() to specify optional target settings.

Usage

tar_resources_crew(
  controller = targets::tar_option_get("resources")$crew$controller,
  scale = NULL,
  seconds_timeout = targets::tar_option_get("resources")$crew$seconds_timeout
)

Arguments

controller	Character of length 1. If tar_option_get("controller") is a crew controller group, the controller argument of tar_resources_crew() indicates which controller in the controller group to use. If you need heterogeneous workers, you can leverage this argument to send different targets to different worker groups.
scale	Deprecated in version 1.3.0.9002 (2023-10-02). No longer necessary.
seconds_timeout	Positive numeric of length 1, optional task timeout passed to the .timeout argument of mirai::mirai() (after converting to milliseconds).

Details

tar_resources_crew() accepts target-specific settings for integration with the crew R package. These settings are arguments to the push() method of the controller or controller group object which control things like auto-scaling behavior and the controller to use in the case of a controller group.

Value

Object of class "tar_resources_crew", to be supplied to the crew argument of tar_resources().

Resources

Functions tar_target() and tar_option_set() each takes an optional resources argument to supply non-default settings of various optional backends for data storage and high-performance computing. The tar_resources() function is a helper to supply those settings in the correct manner.

In targets version 0.12.2 and above, resources are inherited one-by-one in nested fashion from tar_option_get("resources"). For example, suppose you set tar_option_set(resources = tar_resources(aws = my_aws)), where my_aws equals tar_resources_aws(bucket = "x", prefix = "y"). Then, ⁠tar_target(data, get_data()⁠ will have bucket "x" and prefix "y". In addition, if new_resources equals ⁠tar_resources(aws = tar_resources_aws(bucket = "z")))⁠, then tar_target(data, get_data(), resources = new_resources) will use the new bucket "z", but it will still use the prefix "y" supplied through tar_option_set(). (In targets 0.12.1 and below, options like prefix do not carry over from tar_option_set() if you supply non-default resources to tar_target().)

See Also

Other resources: tar_resources(), tar_resources_aws(), tar_resources_clustermq(), tar_resources_custom_format(), tar_resources_feather(), tar_resources_fst(), tar_resources_future(), tar_resources_gcp(), tar_resources_network(), tar_resources_parquet(), tar_resources_qs(), tar_resources_repository_cas(), tar_resources_url()

Examples

# Somewhere in you target script file (usually _targets.R):
tar_target(
  name,
  command(),
  resources = tar_resources(
    crew = tar_resources_crew(seconds_timeout = 5)
  )
)

targets documentation built on Oct. 3, 2024, 1:11 a.m.