knitr::opts_chunk$set( collapse = TRUE, message = FALSE, comment = "#>" ) library(gradethis) fibonacci <- c(0, 1, 1, 2, 3, 5, 8, 13, 21, 34) # stable feedback messages set.seed(2022-04-29) grade_html <- function(grade) { htmltools::tags$div( role = "alert", class = if (!length(grade$correct)) { "alert" } else if (isTRUE(grade$correct)) { "alert alert-success" } else { "alert alert-danger" }, htmltools::HTML(commonmark::markdown_html(grade$message)) ) } chunk_code <- function(label) { paste(knitr::knit_code$get(label), collapse = "\n") } grade_example_factory <- function(solution_code, grader) { default_grader <- force(grader) function(label, grader = NULL) { if (is.null(grader)) grader <- default_grader user_code <- chunk_code(label) grade_html(grader(mock_this_exercise(!!user_code, !!solution_code))) } }
```{css echo=FALSE, eval = !identical(Sys.getenv("IN_PKGDOWN", "false"), "true")} .alert { padding: 0.5em 1em; margin: 1em 2em; border: 1px #eee solid; border-radius: 5px; } .alert.alert-success { background-color: #e4f2dc; border-color: #ddedcf; } .alert.alert-success, .alert.alert-success code, .alert.alert-success code a:not(.close):not(.btn) { color: #5c9653; } .alert.alert-danger { background-color: #f2e2e2; border-color: #eed9dc; } .alert.alert-danger, .alert.alert-danger code, .alert.alert-danger code a:not(.close):not(.btn) { color: #ad3532; } .alert code { background-color: #ffffff55; }
```{css echo=FALSE} .alert :last-child { margin-bottom: 0; }
Not all exercises have exactly one right answer: there could be a few different ways to arrive at the same result, or the exercise prompt could be open-ended and ask the student to pick from a selection of possibilities. In either case, and with minimal changes to your learnr tutorial, gradethis can accommodate exercises with multiple solutions!
To signal that your exercise has multiple solutions,
write all of the solutions in the exercise's -solution
chunk,
separating the solutions with code headers —
a comment followed by at least four dashes.
# code header ----
You can think of the code headers as labels for each solution variant.
Here's an example -solution
chunk for an exercise with two possible solutions
(the full exercise is presented below).
```r`r ''` `r chunk_code("average-solution")` ```
There are two types of exercises with multiple solutions:
In most cases, you'll only need to write out the solutions in the -solution
chunk, and gradethis will take care of the rest.
Learn more in Multiple solutions, same result below.
In addition to writing out the solutions in the -solution
chunk,
you may also need to modify your grading code slightly.
Learn more in Multiple solutions, different results below.
In both cases,
multiple solutions are written in the same way in the -solution
chunk,
and minimal changes are needed in your grading code.
For advanced grading requirements,
gradethis adds two objects into the grading environment:
.solution_code_all
and .solution_all
.
We'll cover both straight-forward and advanced uses in the examples that follow.
If all your solutions have the same result, you might not need to do anything special in your grading code, as we'll see in the next example.
In this example,
students are asked to convert a numeric value to a whole number.
The most likely solution is to use the round()
function,
but a creative student might call floor()
or as.integer()
.
Notice that we include all three variations in the round-solution
chunk below,
but we've placed the answer we consider "most correct" as the last solution in the chunk.
The last solution in the chunk will be the default solution used by gradethis
when the student's code is not a good match for any of the solutions.
```r`r ''` library(learnr) library(gradethis) ``` Convert 5.2 to a reasonable whole number. ```r`r ''` ____(5.2) ``` ```r`r ''` # floor ---- floor(5.2) # as.integer ---- as.integer(5.2) # round ---- round(5.2) ``` ```r`r ''` grade_this({ # provide code hints if submission is wrong fail_if_code_feedback() # pass if the value is correct pass_if_equal() # or fallback to a failing grade fail(hint = TRUE) }) ```
solution_code <- " # floor ---- floor(5.2) # as.integer ---- as.integer(5.2) # round ---- round(5.2)" grader <- grade_this({ fail_if_code_feedback() pass_if_equal() fail(hint = TRUE) }) grade_round_example <- grade_example_factory(solution_code, grader)
If a student enters any of the correct answers to arrive at the value 5
,
they receive positive feedback and encouragement.
as.integer(5.2)
grade_round_example("round_correct")
If a student gives an incorrect answer using round()
,
the code feedback hint provided by fail(hint = TRUE)
will nudge them towards the correct round()
solution:
round(5.2, digits = 1)
grade_round_example("round_wrong")
And if a student gives an incorrect answer but their text is closer to one of the other solutions, the code feedback will nudge the student toward the correct solution.
flor(5.2)
grade_round_example("floor_wrong")
as.numeric(5.2)
grade_round_example("as_integer_wrong")
Grading functions that use solution code —
like fail_if_code_feedback()
, code_feedback()
or fail(hint = TRUE)
—
will automatically give feedback based on the closest solution to the student's input.
In some cases,
you may want to accept multiple solutions,
but give different feedback depending on the solution the student gave.
To enable advanced custom grading,
gradethis makes all of the solutions to an exercise available to grading code
as .solution_code_all
,
a named list containing the code for each solution.
You can use code like the following template
to give a special message (<message>
)
for user input that matches a particular solution
identified by the label used for the code heading above the solution (<solution_label>
):
correctly_used_solution <- is.null(code_feedback(.user_code, .solution_code_all[["<solution_label>"]])) pass_if( correctly_used_solution, message = "<message>" )
Some things of note in this template:
You can obtain the code for a specific solution using the label in its code heading:
e.g. .solution_code_all[["floor"]]
in our example would return "floor(5.2)"
.
You need to explicitly ask for code_feedback()
for the solution of interest.
Otherwise, by default, code_feedback()
will first match the user's code
with the most option among the set of solutions.
code_feedback()
returns NULL
when user_code
matches solution_code
.
Testing that the feedback is.null()
is a robust way to test that
the student submitted code that matches a solution
because it ignores minor differences like whitespace and pipe usage.
Using the template above,
we can provide some feedback specifically to students who use the floor()
solution.
```r`r ''` grade_this({ correctly_used_floor <- is.null(code_feedback(.user_code, .solution_code_all[["floor"]])) pass_if( correctly_used_floor, "Correct, but remember that `floor()` always rounds down. If you want to round to the nearest whole number, `round()` is usually safer." ) pass_if_equal() fail(hint = TRUE) }) ```
floor(5.2)
grader_floor_specific <- grade_this({ pass_if( is.null(code_feedback(solution_code = .solution_code_all[["floor"]])), "Correct, but remember that `floor()` always rounds down. If you want to round to the closest whole number, `round()` is usually safer." ) pass_if_equal() fail(hint = TRUE) }) grade_round_example("floor_right", grader_floor_specific)
If your exercise has multiple acceptable results, you'll need to slightly adapt your code to ensure students' inputs are tested against all acceptable results.
To compare the result of student input to each solution,
you should use .solution_all
as the first argument in pass_if_equal()
.
Multiple solution results are saved in an environment called .solution_all
.
(Because .solution_all
is an environment, it requires some special handling.
Check Working with .solution_all
for more details.)
Within pass_if_equal()
messages,
you can use the object .solution_label
,
which corresponds to the code header of the solution that the student's code matched.
Functions that grade students' code can still be used without any changes.
```r`r ''` library(learnr) library(gradethis) fibonacci <- c(0, 1, 1, 2, 3, 5, 8, 13, 21, 34) ``` Use `mean()` or `median()` to find the average of `fibonacci`, a vector containing the first 10 numbers in the Fibonacci sequence. ```r`r ''` ____(fibonacci) ``` ```r`r ''` `r chunk_code("average-solution")` ``` ```r`r ''` `r chunk_code("average-check")` ```
# mean ---- mean(fibonacci) # median ---- median(fibonacci)
grade_this({ pass_if_equal(.solution_all, "You solved it with `{.solution_label}()`!") fail(hint = TRUE) })
grade_average_example <- grade_example_factory( chunk_code("average-solution"), rlang::eval_bare(rlang::parse_expr(chunk_code("average-check"))) )
If a student gives either correct solution,
pass_if_equal()
will give them a passing grade:
mean(fibonacci)
grade_average_example("mean_right")
median(fibonacci)
grade_average_example("median_right")
If the student's input doesn't match any of the solutions, grading will proceed as normal:
sum(fibonacci)
user_code <- paste(knitr::knit_code$get()$sum_wrong, collapse = "\n") grade_html(grader(mock_this_exercise(!!user_code, !!solution_code)))
.solution_all
is an environment, not a list.
This carries certain advantages.
Exercises can be graded faster because each solution can be evaluated as needed rather than all up front.
But it also comes with drawbacks. The most important are:
If you need to access the results of specific solutions, make sure you give a unique label to every solution.
# mean ---- mean(fibonacci) # median ---- median(fibonacci)
To reference a specific solution result in grading code,
use the solution label to access the result in the .solution_all
:
grade_this({ pass_if_equal( .solution_all[["mean"]], "That's right! Do you think the mean is skewed?" ) pass_if_equal(.solution_all) fail_if_code_feedback() fail() })
If you don't give your solutions unique names,
gradethis
will generate unique names for you.
For example, these solutions
# mean ---- mean(fibonacci) # mean ---- mean(fibonacci, na.rm = TRUE) # median ---- median(fibonacci) # median ---- median(fibonacci, na.rm = TRUE)
have these names:
solution_code <- chunk_code("nonunique_names") ex <- mock_this_exercise(.user_code = "", .solution_code = !!solution_code) names(ex$.solution_all[[".solution_labels"]])
This means that if you need to access specific solution results in your grading code,
it's almost always better to give your solutions unique labels.
On the other hand,
if your passing message makes use of .solution_label
,
you may want to duplicate the solution labels.
Recall that the -check
chunk for the average
exercise was:
```r`r ''` `r chunk_code("average-check")` ```
By including the mean()
and median()
solution variants with na.rm = TRUE
we get an appropriate passing message even though the solution labels are duplicated:
median(fibonacci, na.rm = TRUE)
grade_median_2 <- grade_example_factory( solution_code = chunk_code("nonunique_names"), grader = rlang::eval_bare(rlang::parse_expr(chunk_code("average-check"))) ) grade_median_2("median_right_2")
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.