HF_TokenCategorize: HF_TokenCategorize

View source: R/blurr_hugging_face.R

HF_TokenCategorizeR Documentation

HF_TokenCategorize

Description

Reversible transform of a list of category string to 'vocab' id

Usage

HF_TokenCategorize(vocab = NULL, ignore_token = NULL, ignore_token_id = NULL)

Arguments

vocab

vocabulary

ignore_token

ignore token

ignore_token_id

ignore token id

Value

None


fastai documentation built on June 22, 2024, 11:15 a.m.