tokenize_df: Tokenize_df
In fastai: Interface to 'fastai'

View source: R/text_core.R

tokenize_df

R Documentation

Tokenize_df

Description

Tokenize texts in 'df[text_cols]' in parallel using 'n_workers'

Usage

tokenize_df(
  df,
  text_cols,
  n_workers = 6,
  rules = NULL,
  mark_fields = NULL,
  tok = NULL,
  tok_text_col = "text"
)

Arguments

`df`	data frame
`text_cols`	text columns
`n_workers`	number of workers
`rules`	rules
`mark_fields`	mark_fields
`tok`	tokenizer
`tok_text_col`	tok_text_col

Value

None

fastai documentation built on June 22, 2024, 11:15 a.m.

fastai index

README.md Audio Classification" Basic Image Classification" Basic Tabular" Bayesian Optimisation" Callbacks" Custom Image Classification" Data augmentation" GPT2" Head pose" Low-level ops" Medical image" Migrating from Catalyst" Migrating from Ignite" Migrating from Lightning" Migrating from Pytorch" Multilabel classification" Object detection" Optimizer" Question-Answering" RoBERTa" Speech Recognition" Super-Resolution GAN" Text-summarization" Time-Series"

rdrr.io home R language documentation Run R code online

CRAN packages Bioconductor packages R-Forge packages GitHub packages

Note that we can't provide technical support on individual packages. You should contact the package authors for that.

Tweet to @rdrrHQ

GitHub issue tracker

ian@mutexlabs.com