Tokenizer_from_df: Tokenizer_from_df

View source: R/text_data.R

Tokenizer_from_dfR Documentation

Tokenizer_from_df

Description

Tokenizer_from_df

Usage

Tokenizer_from_df(
  text_cols,
  tok = NULL,
  rules = NULL,
  sep = " ",
  n_workers = 6,
  mark_fields = NULL,
  tok_text_col = "text"
)

Arguments

text_cols

text columns

tok

tokenizer

rules

special rules

sep

separator

n_workers

number of workers

mark_fields

mark fields

tok_text_col

output column name

Value

None


fastai documentation built on June 22, 2024, 11:15 a.m.