ds_write_tables: Write FHIR data to managed tables

In pathling: A Library for using 'Pathling'

Write FHIR data to managed tables

Description

Writes the data from a data source to a set of tables in the Spark catalog.

Usage

ds_write_tables(ds, schema = NULL, save_mode = SaveMode$OVERWRITE)

Arguments

ds	The DataSource object.
schema	The name of the schema to write the tables to.
save_mode	The save mode to use when writing the data - "overwrite" will overwrite any existing data, "merge" will merge the new data with the existing data based on resource ID.

Value

A list with element file_infos, containing a list of files created. Each file has fhir_resource_type and absolute_url.

See Also

Pathling documentation - Writing managed tables

SaveMode

Other data sink functions: ds_write_delta(), ds_write_ndjson(), ds_write_parquet()

Examples

# Create a temporary warehouse location, which will be used when we call ds_write_tables().
temp_dir_path <- tempfile()
dir.create(temp_dir_path)
sc <- sparklyr::spark_connect(master = "local[*]", config = list(
  "sparklyr.shell.conf" = c(
    paste0("spark.sql.warehouse.dir=", temp_dir_path),
    "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension",
    "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"
  )
), version = pathling_spark_info()$spark_version)

pc <- pathling_connect(sc)
data_source <- pc %>% pathling_read_ndjson(pathling_examples("ndjson"))

# Write the data to a set of Spark tables in the 'default' database.
data_source %>% ds_write_tables("default", save_mode = SaveMode$MERGE)

pathling_disconnect(pc)
unlink(temp_dir_path, recursive = TRUE)

pathling documentation built on June 3, 2026, 9:08 a.m.