keep_tokens: Filter Tokens

In jonathanbratt/RBERTviz: Visualization Tools for RBERT

Description

Keeps only specified tokens in the given table of embeddings.

Usage

keep_tokens(embedding_df, tokens = "[CLS]")

Arguments

embedding_df	A tbl_df of embedding vectors; the output of extract_features.
tokens	Character vector; which tokens to keep.

Value

The input tbl_df of embedding vectors, with the specified filtering applied.

Examples

## Not run: 
# assuming something like the following has been run:
# feats <- RBERT::extract_features(...) # See RBERT documentation
# Then:
embeddings_layer12_cls <- feats$output %>%
    filter_layer_embeddings(layer_indices = 12L) %>%
    keep_tokens("[CLS]")

## End(Not run)

jonathanbratt/RBERTviz documentation built on June 19, 2021, 6:27 p.m.