add_filename: Add the data filename as a column

In arrow: Integration to 'Apache' 'Arrow'

Add the data filename as a column

Description

This function only exists inside arrow dplyr queries, and it only is valid when querying on a FileSystemDataset, such as one created by open_dataset(). Use it inside mutate() to add a column holding the path of the file each row was read from.

Usage

add_filename()

Details

The filename column can be used in later select(), arrange() and group_by() steps of the same query. However, it can't be used in filter(), and some functions (such as substr()) are not supported on it. In these cases, call compute() or collect() first. add_filename() must also be called before any aggregation or join. See Examples.

Value

A FieldRef Expression that refers to the filename augmented column.

See Also

open_dataset()

Examples

## Not run: 
open_dataset("nyc-taxi") |>
  mutate(file = add_filename()) |>
  collect()

# Simple expressions on the new column work in a later mutate(), for
# example to recover a partition value from the path
open_dataset("nyc-taxi/year=2015") |>
  mutate(file = add_filename()) |>
  mutate(year_from_path = sub(".*year=([0-9]{4}).*", "\\1", file)) |>
  collect()

# To filter() on the new column, or use functions such as substr() that
# need to know its type, call compute() or collect() first
open_dataset("nyc-taxi") |>
  mutate(file = add_filename()) |>
  compute() |>
  filter(endsWith(file, "part-0.parquet")) |>
  mutate(file_start = substr(file, 1, 10)) |>
  collect()

## End(Not run)

arrow documentation built on Nov. 9, 2026, 1:09 a.m.