tests/testthat/_snaps/vignette-invariants/invariants.md

In tibble: Simple Data Frames

title: "Invariants: Comparing behavior with data frames"

output: rmarkdown::word_document

output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Invariants: Comparing behavior with data frames} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8}

.dftbl { width: 100%; table-layout: fixed; display: inline-table; } .error pre code { color: red; } .warning pre code { color: violet; }

This vignette defines invariants for subsetting and subset-assignment for tibbles, and illustrates where their behaviour differs from data frames. The goal is to define a small set of invariants that consistently define how behaviors interact. Some behaviors are defined using functions of the vctrs package, e.g. vec_slice(), vec_recycle() and vec_as_index(). Refer to their documentation for more details about the invariants that they follow.

The subsetting and subassignment operators for data frames and tibbles are particularly tricky, because they support both row and column indexes, both of which are optionally missing. We resolve this by first defining column access with [[ and $, then column-wise subsetting with [, then row-wise subsetting, then the composition of both.

Conventions

In this article, all behaviors are demonstrated using one example data frame and its tibble equivalent:

library(tibble)
library(vctrs)
new_df <- function() {
  df <- data.frame(n = c(1L, NA, 3L, NA))
  df$c <- letters[5:8]
  df$li <- list(9, 10:11, 12:14, "text")
  df
}
new_tbl <- function() {
  as_tibble(new_df())
}

Results of the same code for data frames and tibbles are presented side by side:

wzxhzdk:1 wzxhzdk:2

If the results are identical (after converting to a data frame if necessary), only the tibble result is shown.

Subsetting operations are read-only. The same objects are reused in all examples:

df <- new_df()
tbl <- new_tbl()

Where needed, we also show examples with hierarchical columns containing a data frame or a matrix:

new_tbl2 <- function() {
  tibble(
    tb = tbl,
    m = diag(4)
  )
}
new_df2 <- function() {
  df2 <- new_tbl2()
  class(df2) <- "data.frame"
  class(df2$tb) <- "data.frame"
  df2
}
df2 <- new_df2()
tbl2 <- new_tbl2()

wzxhzdk:5 wzxhzdk:6

For subset assignment (subassignment, for short), we need a fresh copy of the data for each test. The with_*() functions (omitted here for brevity) allow for a more concise notation. These functions take an assignment expression, execute it on a fresh copy of the data, and return the data for printing. The first example prints what's really executed, further examples omit this output.

wzxhzdk:7 wzxhzdk:8

Column extraction

Definition of x[[j]]

x[[j]] is equal to .subset2(x, j).

wzxhzdk:9 wzxhzdk:10 wzxhzdk:11 wzxhzdk:12 wzxhzdk:13 wzxhzdk:14 wzxhzdk:15 wzxhzdk:16

NB: x[[j]] always returns an object of size nrow(x) if the column exists.

wzxhzdk:17 wzxhzdk:18 wzxhzdk:19 wzxhzdk:20 wzxhzdk:21 wzxhzdk:22 wzxhzdk:23 wzxhzdk:24

j must be a single number or a string, as enforced by .subset2(x, j).

wzxhzdk:25 wzxhzdk:26 wzxhzdk:27 wzxhzdk:28 wzxhzdk:29 wzxhzdk:30 wzxhzdk:31 wzxhzdk:32

NA indexes, numeric out-of-bounds (OOB) values, and non-integers throw an error:

wzxhzdk:33 wzxhzdk:34 wzxhzdk:35 wzxhzdk:36 wzxhzdk:37 wzxhzdk:38 wzxhzdk:39 wzxhzdk:40 wzxhzdk:41 wzxhzdk:42 wzxhzdk:43 wzxhzdk:44 wzxhzdk:45 wzxhzdk:46

Character OOB access is silent because a common package idiom is to check for the absence of a column with is.null(df[[var]]).

wzxhzdk:47 wzxhzdk:48

Definition of x$name

x$name and x$"name" are equal to x[["name"]].

wzxhzdk:49 wzxhzdk:50 wzxhzdk:51 wzxhzdk:52 wzxhzdk:53 wzxhzdk:54 wzxhzdk:55 wzxhzdk:56 wzxhzdk:57 wzxhzdk:58 wzxhzdk:59 wzxhzdk:60

Unlike data frames, tibbles do not partially match names. Because df$x is rarely used in packages, it can raise a warning:

wzxhzdk:61 wzxhzdk:62 wzxhzdk:63 wzxhzdk:64

Column subsetting

Definition of x[j]

j is converted to an integer vector by vec_as_index(j, ncol(x), names = names(x)). Then x[c(j_1, j_2, ..., j_n)] is equivalent to tibble(x[[j_1]], x[[j_2]], ..., x[[j_n]]), keeping the corresponding column names. This implies that j must be a numeric or character vector, or a logical vector with length 1 or ncol(x).[^subset-extract-commute]

[^subset-extract-commute]: x[j][[jj]] is equal to x[[ j[[jj]] ]], in particular x[j][[1]] is equal to x[[j]] for scalar numeric or integer j.

wzxhzdk:65 wzxhzdk:66

When subsetting repeated indexes, the resulting column names are undefined, do not rely on them.

wzxhzdk:67 wzxhzdk:68

For tibbles with repeated column names, subsetting by name uses the first matching column.

nrow(df[j]) equals nrow(df).

wzxhzdk:69 wzxhzdk:70

Tibbles support indexing by a logical matrix, but only if all values in the returned vector are compatible.

wzxhzdk:71 wzxhzdk:72 wzxhzdk:73 wzxhzdk:74

Definition of x[, j]

x[, j] is equal to x[j]. Tibbles do not perform column extraction if x[j] would yield a single column.

wzxhzdk:75 wzxhzdk:76 wzxhzdk:77 wzxhzdk:78 wzxhzdk:79 wzxhzdk:80 wzxhzdk:81 wzxhzdk:82

Definition of x[, j, drop = TRUE]

For backward compatiblity, x[, j, drop = TRUE] performs column extraction, returning x[j][[1]] when ncol(x[j]) is 1.

wzxhzdk:83 wzxhzdk:84 wzxhzdk:85 wzxhzdk:86 wzxhzdk:87 wzxhzdk:88 wzxhzdk:89 wzxhzdk:90

Row subsetting

Definition of x[i, ]

x[i, ] is equal to tibble(vec_slice(x[[1]], i), vec_slice(x[[2]], i), ...).[^row-subset-efficiency]

[^row-subset-efficiency]: Row subsetting x[i, ] is not defined in terms of x[[j]][i] because that definition does not generalise to matrix and data frame columns. For efficiency and backward compatibility, i is converted to an integer vector by vec_as_index(i, nrow(x)) first.

wzxhzdk:91 wzxhzdk:92

This means that i must be a numeric vector, or a logical vector of length nrow(x) or 1. For compatibility, i can also be a character vector containing positive numbers.

wzxhzdk:93 wzxhzdk:94 wzxhzdk:95 wzxhzdk:96 wzxhzdk:97 wzxhzdk:98

Exception: OOB values generate warnings instead of errors:

wzxhzdk:99 wzxhzdk:100 wzxhzdk:101 wzxhzdk:102

Unlike data frames, only logical vectors of length 1 are recycled.

wzxhzdk:103 wzxhzdk:104

NB: scalar logicals are recycled, but scalar numerics are not. That makes the x[NA, ] and x[NA_integer_, ] return different results.

wzxhzdk:105 wzxhzdk:106 wzxhzdk:107 wzxhzdk:108

Definition of x[i, , drop = TRUE]

drop = TRUE has no effect when not selecting a single row:

wzxhzdk:109 wzxhzdk:110

Row and column subsetting

Definition of x[] and x[,]

x[] and x[,] are equivalent to x.[^bracket-comma]

[^bracket-comma]: x[,] is equivalent to x[] because x[, j] is equivalent to x[j].

Definition of x[i, j]

x[i, j] is equal to x[i, ][j].[^bracket-flip]

[^bracket-flip]: A more efficient implementation of x[i, j] would forward to x[j][i, ].

wzxhzdk:111 wzxhzdk:112 wzxhzdk:113 wzxhzdk:114 wzxhzdk:115 wzxhzdk:116 wzxhzdk:117 wzxhzdk:118 wzxhzdk:119 wzxhzdk:120

Definition of x[[i, j]]

i must be a numeric vector of length 1. x[[i, j]] is equal to x[i, ][[j]], or vctrs::vec_slice(x[[j]], i).[^bracket2-flip]

[^bracket2-flip]: Cell subsetting x[[i, j]] is not defined in terms of x[[j]][[i]] because that definition does not generalise to list, matrix and data frame columns. A more efficient implementation of x[[i, j]] would check that j is a scalar and forward to x[i, j][[1]].

df[[1, 1]]
#> [1] 1
df[[1, 3]]
#> [1] 9

This implies that j must be a numeric or character vector of length 1.

NB: vec_size(x[[i, j]]) always equals 1. Unlike x[i, ], x[[i, ]] is not valid.

Column update

Definition of x[[j]] <- a

If a is a vector then x[[j]] <- a replaces the jth column with value a.

wzxhzdk:122 wzxhzdk:123 wzxhzdk:124 wzxhzdk:125 wzxhzdk:126 wzxhzdk:127 wzxhzdk:128 wzxhzdk:129 wzxhzdk:130 wzxhzdk:131 wzxhzdk:132 wzxhzdk:133 wzxhzdk:134 wzxhzdk:135 wzxhzdk:136 wzxhzdk:137 wzxhzdk:138 wzxhzdk:139 wzxhzdk:140 wzxhzdk:141 wzxhzdk:142 wzxhzdk:143 wzxhzdk:144 wzxhzdk:145 wzxhzdk:146 wzxhzdk:147 wzxhzdk:148 wzxhzdk:149

a is recycled to the same size as x so must have size nrow(x) or 1. (The only exception is when a is NULL, as described below.) Recycling also works for list, data frame, and matrix columns.

wzxhzdk:150 wzxhzdk:151 wzxhzdk:152 wzxhzdk:153 wzxhzdk:154 wzxhzdk:155 wzxhzdk:156 wzxhzdk:157 wzxhzdk:158 wzxhzdk:159 wzxhzdk:160 wzxhzdk:161 wzxhzdk:162 wzxhzdk:163

j must be a scalar numeric or a string, and cannot be NA. If j is OOB, a new column is added on the right hand side, with name repair if needed.

wzxhzdk:164 wzxhzdk:165 wzxhzdk:166 wzxhzdk:167 wzxhzdk:168 wzxhzdk:169

df[[j]] <- a replaces the complete column so can change the type.

wzxhzdk:170 wzxhzdk:171 wzxhzdk:172 wzxhzdk:173 wzxhzdk:174 wzxhzdk:175 wzxhzdk:176 wzxhzdk:177 wzxhzdk:178 wzxhzdk:179

[[<- supports removing a column by assigning NULL to it.

wzxhzdk:180 wzxhzdk:181 wzxhzdk:182 wzxhzdk:183

Removing a nonexistent column is a no-op.

wzxhzdk:184 wzxhzdk:185

Definition of x$name <- a

x$name <- a and x$"name" <- a are equivalent to x[["name"]] <- a.[^column-assign-symmetry]

[^column-assign-symmetry]: $ behaves almost completely symmetrically to [[ when comparing subsetting and subassignment.

wzxhzdk:186 wzxhzdk:187 wzxhzdk:188 wzxhzdk:189 wzxhzdk:190 wzxhzdk:191

$<- does not perform partial matching.

wzxhzdk:192 wzxhzdk:193 wzxhzdk:194 wzxhzdk:195

Column subassignment: x[j] <- a

• If j is missing, it's replaced with seq_along(x)
• If j is logical vector, it's converted to numeric with seq_along(x)[j].

a is a list or data frame

If inherits(a, "list") or inherits(a, "data.frame") is TRUE, then x[j] <- a is equivalent to x[[j[[1]]] <- a[[1]], x[[j[[2]]]] <- a[[2]], ...

wzxhzdk:196 wzxhzdk:197 wzxhzdk:198 wzxhzdk:199

If length(a) equals 1, then it is recycled to the same length as j.

wzxhzdk:200 wzxhzdk:201 wzxhzdk:202 wzxhzdk:203 wzxhzdk:204 wzxhzdk:205

An attempt to update the same column twice gives an error.

wzxhzdk:206 wzxhzdk:207

If a contains NULL values, the corresponding columns are removed after updating (i.e. position indexes refer to columns before any modifications).

wzxhzdk:208 wzxhzdk:209

NA indexes are not supported.

wzxhzdk:210 wzxhzdk:211 wzxhzdk:212 wzxhzdk:213 wzxhzdk:214 wzxhzdk:215

Just like column updates, [<- supports changing the type of an existing column.

wzxhzdk:216 wzxhzdk:217 wzxhzdk:218 wzxhzdk:219 wzxhzdk:220 wzxhzdk:221 wzxhzdk:222 wzxhzdk:223 wzxhzdk:224 wzxhzdk:225

Appending columns at the end (without gaps) is supported. The name of new columns is determined by the LHS, the RHS, or by name repair (in that order of precedence).

wzxhzdk:226 wzxhzdk:227 wzxhzdk:228 wzxhzdk:229 wzxhzdk:230 wzxhzdk:231 wzxhzdk:232 wzxhzdk:233

Tibbles support indexing by a logical matrix, but only for a scalar RHS, and if all columns updated are compatible with the value assigned.

wzxhzdk:234 wzxhzdk:235 wzxhzdk:236 wzxhzdk:237 wzxhzdk:238 wzxhzdk:239

a is a matrix or array

If is.matrix(a), then a is coerced to a data frame with as.data.frame() before assigning. If rows are assigned, the matrix type must be compatible with all columns. If is.array(a) and any(dim(a)[-1:-2] != 1), an error is thrown.

wzxhzdk:240 wzxhzdk:241 wzxhzdk:242 wzxhzdk:243 wzxhzdk:244 wzxhzdk:245 wzxhzdk:246 wzxhzdk:247 wzxhzdk:248 wzxhzdk:249 wzxhzdk:250 wzxhzdk:251

a is another type of vector

If vec_is(a), then x[j] <- a is equivalent to x[j] <- list(a). This is primarily provided for backward compatibility.

wzxhzdk:252 wzxhzdk:253 wzxhzdk:254 wzxhzdk:255

Matrices must be wrapped in list() before assignment to create a matrix column.

wzxhzdk:256 wzxhzdk:257 wzxhzdk:258 wzxhzdk:259

a is NULL

Entire columns can be removed. Specifying i is an error.

wzxhzdk:260 wzxhzdk:261 wzxhzdk:262 wzxhzdk:263 wzxhzdk:264 wzxhzdk:265

a is not a vector

Any other type for a is an error. Note that if is.list(a) is TRUE, but inherits(a, "list") is FALSE, then a is considered to be a scalar. See ?vec_is and ?vec_proxy for details.

wzxhzdk:266 wzxhzdk:267 wzxhzdk:268 wzxhzdk:269

Row subassignment: x[i, ] <- list(...)

x[i, ] <- a is the same as vec_slice(x[[j_1]], i) <- a[[1]], vec_slice(x[[j_2]], i) <- a[[2]], ... .[^row-assign-symmetry]

[^row-assign-symmetry]: x[i, ] is symmetrical for subset and subassignment.

wzxhzdk:270 wzxhzdk:271 wzxhzdk:272 wzxhzdk:273 wzxhzdk:274 wzxhzdk:275 wzxhzdk:276 wzxhzdk:277 wzxhzdk:278 wzxhzdk:279 wzxhzdk:280 wzxhzdk:281 wzxhzdk:282 wzxhzdk:283 wzxhzdk:284 wzxhzdk:285 wzxhzdk:286 wzxhzdk:287 wzxhzdk:288 wzxhzdk:289 wzxhzdk:290 wzxhzdk:291

Only values of size one can be recycled.

wzxhzdk:292 wzxhzdk:293 wzxhzdk:294 wzxhzdk:295 wzxhzdk:296 wzxhzdk:297 wzxhzdk:298 wzxhzdk:299 wzxhzdk:300 wzxhzdk:301

For compatibility, only a warning is issued for indexing beyond the number of rows. Appending rows right at the end of the existing data is supported, without warning.

wzxhzdk:302 wzxhzdk:303 wzxhzdk:304 wzxhzdk:305 wzxhzdk:306 wzxhzdk:307 wzxhzdk:308 wzxhzdk:309 wzxhzdk:310 wzxhzdk:311 wzxhzdk:312 wzxhzdk:313

For compatibility, i can also be a character vector containing positive numbers.

wzxhzdk:314 wzxhzdk:315 wzxhzdk:316 wzxhzdk:317 wzxhzdk:318 wzxhzdk:319 wzxhzdk:320 wzxhzdk:321 wzxhzdk:322 wzxhzdk:323

Row and column subassignment

Definition of x[i, j] <- a

x[i, j] <- a is equivalent to x[i, ][j] <- a.[^bracket-assign-flip]

[^bracket-assign-flip]: x[i, j] is symmetrical for subsetting and subassignment. A more efficient implementation of x[i, j] <- a would forward to x[j][i, ] <- a.

Subassignment to x[i, j] is stricter for tibbles than for data frames. x[i, j] <- a can't change the data type of existing columns.

wzxhzdk:324 wzxhzdk:325 wzxhzdk:326 wzxhzdk:327 wzxhzdk:328 wzxhzdk:329 wzxhzdk:330 wzxhzdk:331 wzxhzdk:332 wzxhzdk:333

A notable exception is the population of a column full of NA (which is of type logical), or the use of NA on the right-hand side of the assignment.

wzxhzdk:334 wzxhzdk:335 wzxhzdk:336 wzxhzdk:337

For programming, it is always safer (and faster) to use the correct type of NA to initialize columns.

wzxhzdk:338 wzxhzdk:339

For new columns, x[i, j] <- a fills the unassigned rows with NA.

wzxhzdk:340 wzxhzdk:341 wzxhzdk:342 wzxhzdk:343 wzxhzdk:344 wzxhzdk:345

Likewise, for new rows, x[i, j] <- a fills the unassigned columns with NA.

wzxhzdk:346 wzxhzdk:347

Definition of x[[i, j]] <- a

i must be a numeric vector of length 1. x[[i, j]] <- a is equivalent to x[i, ][[j]] <- a.[^double-bracket-ij-symmetry]

[^double-bracket-ij-symmetry]: x[[i, j]] is symmetrical for subsetting and subassignment. An efficient implementation would check that i and j are scalar and forward to x[i, j][[1]] <- a.

wzxhzdk:348 wzxhzdk:349 wzxhzdk:350 wzxhzdk:351 wzxhzdk:352 wzxhzdk:353 wzxhzdk:354 wzxhzdk:355 wzxhzdk:356 wzxhzdk:357 wzxhzdk:358 wzxhzdk:359 wzxhzdk:360 wzxhzdk:361 wzxhzdk:362 wzxhzdk:363 wzxhzdk:364 wzxhzdk:365 wzxhzdk:366 wzxhzdk:367

NB: vec_size(a) must equal 1. Unlike x[i, ] <-, x[[i, ]] <- is not valid.

tibble documentation built on March 31, 2023, 11 p.m.