remove_control_characters: Remove Non-Character Characters

In piecemaker: Tools for Preparing Text for Tokenizers

Remove Non-Character Characters

Description

Unicode includes several control codes, such as U+0000 (NULL, used in null-terminated strings) and U+000D (carriage return). This function removes all such characters from text.

Usage

remove_control_characters(text)

Arguments

text	A character vector to clean.

Details

Note: We highly recommend that you first condense all space-like characters (including new lines) before removing control codes. You can easily do so with str_squish. We also recommend validating text at the start of any cleaning process using validate_utf8.

Value

The character vector without control characters.

Examples

remove_control_characters("Line 1\nLine2")

piecemaker documentation built on June 7, 2023, 5:55 p.m.