S3SparkResourceConnector: Apache Spark DBI resource connector for S3

S3SparkResourceConnectorR Documentation

Apache Spark DBI resource connector for S3

Description

Apache Spark DBI resource connector for S3

Apache Spark DBI resource connector for S3

Format

A R6 object of class SparkResourceConnector

Details

Makes a Apache Spark connection object, that is also a DBI connection object, from a S3 resource description.

Super classes

resourcer::DBIResourceConnector -> resourcer::SparkResourceConnector -> S3SparkResourceConnector

Methods

Public methods


Method new()

Create a SparkResourceConnector instance.

Usage
S3SparkResourceConnector$new()
Returns

A SparkResourceConnector object.


Method isFor()

Check if the provided resource applies to a Apache Spark server. The resource URL scheme must be one of "s3+spark", "s3+spark+http" or "s3+spark+https".

Usage
S3SparkResourceConnector$isFor(resource)
Arguments
resource

The resource object to validate.

Returns

A logical.


Method createDBIConnection()

Creates a DBI connection object from a Apache Spark resource.

Usage
S3SparkResourceConnector$createDBIConnection(resource)
Arguments
resource

A valid resource object.

Returns

A DBI connection object.


Method getTableName()

Get the SQL table name from the resource URL.

Usage
S3SparkResourceConnector$getTableName(resource)
Arguments
resource

A valid resource object.

Returns

The SQL table name.


Method readDBTable()

Read a table as a vanilla tibble using DBI connection object.

Usage
S3SparkResourceConnector$readDBTable(conn, resource)
Arguments
conn

A DBI connection object.

resource

A valid resource object.


Method readDBTibble()

Read a table as a SQL tibble using DBI connection object.

Usage
S3SparkResourceConnector$readDBTibble(conn, resource)
Arguments
conn

A DBI connection object.

resource

A valid resource object.


Method closeDBIConnection()

Close the DBI connection to Apache Spark.

Usage
S3SparkResourceConnector$closeDBIConnection(conn)
Arguments
conn

A DBI connection object.


Method clone()

The objects of this class are cloneable with this method.

Usage
S3SparkResourceConnector$clone(deep = FALSE)
Arguments
deep

Whether to make a deep clone.


s3.resourcer documentation built on April 3, 2025, 10:56 p.m.