unlist_tokens: Create a data.frame from a list of tokens

In udpipe: Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing with the 'UDPipe' 'NLP' Toolkit

Create a data.frame from a list of tokens

Description

Create a data.frame from a list of tokens.

Usage

unlist_tokens(x)

Arguments

x	a list where the list elements are character vectors of tokens

Value

the data of x converted to a data.frame. This data.frame has columns doc_id and token where the doc_id is taken from the list names of x and token contains the data of x

Examples

x <- setNames(c("some text here", "hi  there understand this?"), c("a", "b"))
x <- strsplit(x, split = " ")
x
unlist_tokens(x)

udpipe documentation built on Jan. 6, 2023, 5:06 p.m.