In rstudio-education/grader: Automated Feedback for Student Exercises in 'learnr' Tutorials

library(learnr)
library(purrr)

params <- list(
  "n"  = c(1, 2),
  "mu" = c(0, 1),
  "sd" = c(1, 2)
)

Simple exercises

\t

This tutorial relies on several different exercise setup schemes. A solution checker function will need to navigate these schemes to ensure that each solution runs correctly given the setup code that is associated with it.

The exercises in this section have stand alone solutions. They require zero setup code, so a solution checker only needs to check whether or not the code runs.

• Click Run Code. The code should run successfully.

log(1)

log(1)

• Click Run Code. The code should run with a warning message.

log(-1)

log(-1)

• Click Run Code. The code should return an error message.

log(y = 1)

log(y = 1)

Complex exercises

It is more complicated to test solutions that use objects that are created outside of the exercise chunk. Each object used by the solution must be accessible within the scope of the exercise chunk.

The results of three types of chunk fall into this scope:

1. a specific knitr chunk labeled setup (the global setup chunk)
2. setup chunks that are defined through learnr's -setup naming convention
3. setup chunks that are defined with an exercise.setup code attribute

knitr chunks

\t

A solution tester must mark correct chunks that rely on computations in the global setup chunk. For example, solutions in this document can use params, an object created in the setup chunk of this document. Solutions can also use functions in the purrr pacakge, which was loaded in the setup chunk.

• Click Run Code. The code should run successfully.

params %>% map(length)

params %>% map(length)

\t

Exercise chunks will not have access to objects that are created in knitr chunks unless those chunks are specifically designated as a setup chunk for the exercise.

For example, an exercise should not be able to use the object knitr_result that is produced in the normal knitr chunk below.

knitr_result <- 25

• Click Run Code. The code should return an error.

knitr_result

knitr_result

exercise-setup label

\t

An exercise setup chunk is a chunk that is

1. not evaluated when the document is rendered
2. not displayed within the rendered document
3. associated with one or more exercises
4. evaluated each time an associated exercise chunk is run, before the code in the exercise chunk is evaluated

Each exercise can only have on exercise setup chunk.

-setup suffix

One way to create an exercise setup chunk is to create a knitr chunk whose chunk label consists of the chunk label for an exercise chunk followed by -setup.

So, to illustrate, if you give a chunk the label foo-setup, R Markdown will treat the chunk as the exercise setup chunk for the exercise labeled foo. Note: you will need to inspect the .Rmd file of a tutorial to see the labels for each chunk.

Exercise with -setup

This exercise is labelled success-6. It relies on an exercise setup chunk that is labelled success-6-setup, which looks like this (note that this is a manual copy of the exercise setup chunk; the actual chunk will not appear in the rendered document).

rlnorm2 <- function(n, mu, sd) {
  rlnorm(n, meanlog = mu, sdlog = sd)
}

rlnorm2 <- function(n, mu, sd) {
  rlnorm(n, meanlog = mu, sdlog = sd)
}

• Click Run Code. The code should run successfully.

params %>% pmap(rlnorm2)

params %>% pmap(rlnorm2)

Warning in the setup

To be most useful, a solution checker should reveal whether or not a warning is generated in the setup chunk, or a solution chunk.

This exercise is labeled warning-15. It is accompanied by a chunk labeled warning-15-setup.

• Click Run Code. The code should run with a warning message.

three <- 1 + 2
warning("This is a warning message!")

three^2

three^2

Error in the setup

Solution checking code needs to be able to spot errors that occur in setup chunks.

This exercise is labelled failure-8. It is accompanied by a chunk labelled failure-8-setup, which looks like this:

stop("Uh oh. An error!")
two <- 1 + 1

stop("Uh oh. An error!")
two <- 1 + 1

Notice that failure-8-setup returns an error when run.

• Click Run Code. The code should return an error.

sqrt(two)

sqrt(two)

exercise.setup attribute

\t

Alternatively, you can create an exercise setup chunk by adding an exercise.setup chunk attribute to a learnr exercise chunk. To do this, set exercise.setup equal to a character string that contains the label of a knitr code chunk to use as the exercise setup chunk.

The new chunk will now be treated as an exercise setup chunk associated with that exercise (i.e. it will not be evaluated when the document is rendered, and it will not appear in the document). The label of the code chunk does not need to follow any learnr suffix conventions.

This method is useful when you would like to use the same chunk as an exercise setup chunk for two or more exercises.

Exercise with exercise.setup

This exercise is labelled success-9. It relies on an exercise setup chunk that is labelled success-for-9, which looks like this (note that this is a manual copy of the exercise setup chunk; the actual chunk will not appear in the rendered document).

rcauchy2 <- function(n, mu, sd) {
  rcauchy(n, location = mu, scale = sd)
}

rcauchy2 <- function(n, mu, sd) {
  rcauchy(n, location = mu, scale = sd)
}

• Click Run Code. The code should run successfully.

params %>% pmap(rcauchy2)

params %>% pmap(rcauchy2)

\t

This exercise relies on a setup chunk named warning-for-10 that returns a warning message.

• Click Run Code. The code should run with a warning message.

rcauchy3 <- function(n, mu, sd) {
  rcauchy(n, location = mu, scale = sd)
}

warning("Uh oh. A Warning!")

params %>% pmap(rcauchy3)

params %>% pmap(rcauchy3)

\t

This exercise relies on a setup chunk named failure-for-11 that contains an error.

• Click Run Code. The code should return an error.

stop("Uh oh. An error!")

rcauchy4 <- function(n, mu, sd) {
  rcauchy(n, location = mu, scale = sd)
}

params %>% pmap(rcauchy4)

params %>% pmap(rcauchy4)

Shared exercise.setup chunks

\t

One advantage of the exercise.setup attribute is that you can set the exercise.setup attribute of multiple exercises to use the same chunk.

Both of the exercises below rely on the single exercise setup chunk labelled success-for-9.

• Run each exercise. Both should run successfully.

params %>% pmap(rcauchy2) %>% map(abs)

params %>% pmap(rcauchy2) %>% map(abs)

params %>% pmap(rcauchy2) %>% map(round)

params %>% pmap(rcauchy2) %>% map(round)

\t

Both of the exercises below rely on the single exercise setup chunk labelled warning-for-10.

• Run each exercise. Both should run with a warning message.

params %>% pmap(rcauchy3) %>% map(abs)

params %>% pmap(rcauchy3) %>% map(abs)

params %>% pmap(rcauchy3) %>% map(round)

params %>% pmap(rcauchy3) %>% map(round)

\t

Both of the exercises below rely on the single exercise setup chunk labelled failure-for-11.

• Run each exercise. Both should return errors.

params %>% pmap(rcauchy4) %>% map(abs)

params %>% pmap(rcauchy4) %>% map(abs)

params %>% pmap(rcauchy4) %>% map(round)

params %>% pmap(rcauchy4) %>% map(round)

User errors

\t

There are several ways for a tutorial author to generate errors that are unrelated to the code in a solution chunk.

Incorrect label

An author may rely on a setup chunk that is not associated with the exercise. For example, this exercise chunk is labelled usererr-18, but relies on rlnorm2, which is defined in success-6-setup.

• Click Run Code. The code should return an error.

params %>% pmap(rlnorm2)

params %>% pmap(rlnorm2)

Multiple entries

Since each exercise can only have a single exercise setup chunk, you cannot provide multiple chunk labels to the exercise.setup attribute. For example, this exercise contains the attribute exercise.setup = c("success-for-8", "success-6-setup"). As a result, neither chunk will be used as the exercise setup chunk.

• Click Run Code. The code should return an error (object 'rcauchy2' not found).

list(rcauchy2, rlnorm2) %>% invoke_map(transpose(params))

list(rcauchy2, rlnorm2) %>% invoke_map(transpose(params))

Conflicting methods

Likewise, you should not try to create two exercise setup chunks for an exercise by specifing one setup chunk with the exercise.setup attribute and the other with the -setup chunk label.

In this case, R Markdown will use the exercise setup chunk defined by exercise.setup and ignore the exercise setup chunk defined by the -setup chunk label.

To illustrate, the exercise below is labelled usererr-20. It contains the attribute exercise.setup = "success-6-setup. It is also accompanied by a chunk labelled usererr-20-setup, which looks like this:

add_one <- function(x) x + 1

add_one <- function(x) x + 1

• Run the code. The code should return an error.

rlnorm2(1, mu = 1, sd = 1) %>% add_one()

rlnorm2(1, mu = 1, sd = 1) %>% add_one()

rstudio-education/grader documentation built on July 6, 2023, 8:48 a.m.