tokenize-methods: Tokenize R Code

Description Usage Arguments Value Note Examples

Description

Tools for tokenizing R code.

Usage

1
2
3
4
5

Arguments

file, path

A file path.

text, string

R code as a character vector of length one.

Value

A data.frame with the following columns:

value The token's contents, as a string.
row The row where the token is located.
column The column where the token is located.
type The token type, as a string.

Note

Line numbers are determined by existence of the \n line feed character, under the assumption that code being tokenized will use either \n to indicate newlines (as on modern Unix systems), or \r\n as on Windows.

Examples

1
tokenize_string("x <- 1 + 2")

Example output

  value row column       type
1     x   1      1     symbol
2         1      2 whitespace
3    <-   1      3   operator
4         1      5 whitespace
5     1   1      6     number
6         1      7 whitespace
7     +   1      8   operator
8         1      9 whitespace
9     2   1     10     number

sourcetools documentation built on May 2, 2019, 4:19 a.m.