input_fn_builder: Create an 'input_fn' closure to be passed to TPUEstimator

In jonathanbratt/RBERT: R Implementation of BERT

Create an input_fn closure to be passed to TPUEstimator

Description

Creates an input_fn closure to be passed to TPUEstimator. The output of this closure is the (modified) output of tensorflow::tf$data$Dataset$from_tensor_slices (an object of class "tensorflow.python.data.ops.dataset_ops.BatchDataset").

Usage

input_fn_builder(features, seq_length, is_training, drop_remainder)

Arguments

features	A list of features (objects of class InputFeatures).
seq_length	Integer; the maximum length (number of tokens) of each example. (Examples should already be padded to this length by this point.)
is_training	Logical; whether these are training examples.
drop_remainder	Logical; whether to drop the extra if the number of elements in the dataset is not an exact multiple of the batch size,

Value

An input_fn closure to be passed to TPUEstimator.

Examples

## Not run: 
tokenizer <- FullTokenizer("vocab.txt")
seq_len <- 15L
input_ex1 <- InputExample(
  guid = 1L,
  text_a = "Some text to classify.",
  text_b = "More wordy words.",
  label = "good"
)
input_ex2 <- InputExample(
  guid = 2L,
  text_a = "This is another example.",
  text_b = "So many words.",
  label = "bad"
)
feat <- convert_examples_to_features(
  examples = list(input_ex1, input_ex2),
  label_list = c("good", "bad"),
  max_seq_length = seq_len,
  tokenizer = tokenizer
)
input_fn <- input_fn_builder(
  features = feat,
  seq_length = seq_len,
  is_training = TRUE,
  drop_remainder = FALSE
)

## End(Not run)

jonathanbratt/RBERT documentation built on Jan. 26, 2023, 4:15 p.m.