tests/testthat/_snaps/tokenize_bpe.md

In tidymodels/textrecipes: Extra 'Recipes' for Text Processing

bad args

Code
  prep(step_tokenize_bpe(recipe(~., data = mtcars), vocabulary_size = -4))
Condition
  Error in `step_tokenize_bpe()`:
  Caused by error in `prep()`:
  ! `vocabulary_size` must be a whole number larger than or equal to 0, not the number -4.

bake method errors when needed non-standard role columns are missing

Code
  bake(trained, new_data = test_data[, -1])
Condition
  Error in `step_tokenize_bpe()`:
  ! The following required column is missing from `new_data`: text1.

empty printing

Code
  rec
Message

  -- Recipe ----------------------------------------------------------------------

  -- Inputs 
  Number of variables by role
  outcome:    1
  predictor: 10

  -- Operations 
  * BPE Tokenization for: <none>

Code
  rec
Message

  -- Recipe ----------------------------------------------------------------------

  -- Inputs 
  Number of variables by role
  outcome:    1
  predictor: 10

  -- Training information 
  Training data contained 32 data points and no incomplete rows.

  -- Operations 
  * BPE Tokenization for: <none> | Trained

printing

Code
  print(rec)
Message

  -- Recipe ----------------------------------------------------------------------

  -- Inputs 
  Number of variables by role
  predictor: 2

  -- Operations 
  * BPE Tokenization for: text1

Code
  prep(rec)
Message

  -- Recipe ----------------------------------------------------------------------

  -- Inputs 
  Number of variables by role
  predictor: 2

  -- Training information 
  Training data contained 4 data points and no incomplete rows.

  -- Operations 
  * BPE Tokenization for: text1 | Trained

tidymodels/textrecipes documentation built on June 12, 2025, 7:24 p.m.