HF_QABeforeBatchTransform: HF_QABeforeBatchTransform

Description Usage Arguments Details Value

View source: R/blurr_hugging_face.R

Description

Handles everything you need to assemble a mini-batch of inputs and targets, as well as decode the dictionary produced

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
HF_QABeforeBatchTransform(
  hf_arch,
  hf_tokenizer,
  max_length = NULL,
  padding = TRUE,
  truncation = TRUE,
  is_split_into_words = FALSE,
  n_tok_inps = 1,
  ...
)

Arguments

hf_arch

architecture

hf_tokenizer

tokenizer

max_length

maximum length

padding

padding or not

truncation

truncation or not

is_split_into_words

into split into words or not

n_tok_inps

number of tok inputs

...

additional arguments

Details

as a byproduct of the tokenization process in the 'encodes' method.

Value

None


fastai documentation built on Oct. 25, 2021, 5:08 p.m.