clean_str_2: Cleans a string using after it has been tokenized as a like...

In csps-efpc/TokenLink: Joins two dataframes using tokens or like words

Cleans a string using after it has been tokenized as a like final step. This is the the default string cleaner used after tokenization It can be overriden in tokenizer_basic, tokenize_col, tokenize_df, etc by passing a new function as post_token_clean_str.

Description

Cleans a string using after it has been tokenized as a like final step. This is the the default string cleaner used after tokenization It can be overriden in tokenizer_basic, tokenize_col, tokenize_df, etc by passing a new function as post_token_clean_str.

Usage

clean_str_2(x, ...)

Arguments

x	vector of strings
...	ignored, used to ensure pass by keyword

Examples

c('Z.Y. do things inc', 'z. y. DO things montrèal', 'at&t') |> clean_str_2(token_type = 'company_name')

csps-efpc/TokenLink documentation built on Feb. 10, 2023, 3:30 a.m.