stream_generate_test: Generate Test Stream

In sparklyr: R Interface to Apache Spark

Generate Test Stream

Description

Generates a local test stream, useful when testing streams locally.

Usage

stream_generate_test(
  df = rep(1:1000),
  path = "source",
  distribution = floor(10 + 1e+05 * stats::dbinom(1:20, 20, 0.5)),
  iterations = 50,
  interval = 1
)

Arguments

df	The data frame used as a source of rows to the stream, will be cast to data frame if needed. Defaults to a sequence of one thousand entries.
path	Path to save stream of files to, defaults to "source".
distribution	The distribution of rows to use over each iteration, defaults to a binomial distribution. The stream will cycle through the distribution if needed.
iterations	Number of iterations to execute before stopping, defaults to fifty.
interval	The inverval in seconds use to write the stream, defaults to one second.

Details

This function requires the callr package to be installed.

sparklyr documentation built on May 29, 2024, 2:58 a.m.