is_whitespace: Check whether 'char' is a whitespace character.

View source: R/tokenization.R

is_whitespaceR Documentation

Check whether 'char' is a whitespace character.

Description

(R implementation of _is_whitespace from BERT: tokenization.py.)

Usage

is_whitespace(char)

Arguments

char

A character scalar, comprising a single unicode character.

Details

"\t", "\n", and "\r" are technically control characters but we treat them as whitespace since they are generally considered as such.

Value

TRUE if char is a whitespace character.


jonathanbratt/RBERT documentation built on Jan. 26, 2023, 4:15 p.m.