dbfs_ls: List the contents of a path on DBFS

In RafiKurlansik/bricksteR: R Package for Making Databricks Easy to Use.

List the contents of a path on DBFS

Description

List the contents of a directory, or the details of a file. If the directory does not exist a RESOURCE_DOES_NOT_EXIST exception will be thrown.

Usage

dbfs_ls(path, workspace, token = NULL, verbose = T, ...)

Arguments

path	A string representing the path to list in DBFS
workspace	A string representing the web workspace of your Databricks instance. E.g., "https://eastus2.azuredatabricks.net" or "https://demo.cloud.databricks.com".
token	A valid authentication token generated via User Settings in Databricks or via the Databricks REST API 2.0. If none is provided, netrc will be used
verbose	If TRUE, will print the API response to the console. Defaults to FALSE.
...	Additional options to be passed

Details

When calling list on a large directory, the list operation will time out after approximately 60s. We strongly recommend using list only on directories containing less than 10K files and discourage using the DBFS REST API for operations that list more than 10k files. Instead, we recommend that you perform such operations in the context of a cluster, using File system utilities, which provides the same functionality without timing out.

The API endpoint for listing a path on DBFS is '2.0/dbfs/list'. For all details on API calls please see the official documentation at https://docs.databricks.com/dev-tools/api/latest/.

Value

The API response

Examples

# No need to include /dbfs/
path <- "/rk/data"

dbfs_ls(path = path, workspace = workspace)

RafiKurlansik/bricksteR documentation built on Oct. 13, 2022, 6:58 a.m.