library(learnr) library(gradethis) gradethis_setup() knitr::opts_chunk$set(echo = FALSE)
The gradethis
package provides three ways to autocheck student code in learnr tutorials. You can use:
grade_result()
to check that the student's code returns a correct result.grade_result_strict()
to check that the student's code returns a result that satisfies a complete set of conditions. This is akin to unit testing the result.grade_code()
to check that the student's code exactly matches the solution code.You can also use gradethis to create your own customized checking code.
In each case, you grade an exercise by passing a gradethis
function to the -check
code chunk associated with the exercise. This code chunk will share the same label root as the exercise code chunk, but the root will be suffixed with -check
.
See the learnr documentation to learn more about code chunks in learnr tutorials.
Here is how an example exercise and -check
chunk would appear in a learnr document.
`r ''````r x + 1 ``` `r ''````r grade_result( pass_if(~ identical(.result, 2), "Good job!") ) ```
gradethis
functions are designed to provide formative feedback to the student. When a student clicks "Submit Answer" in his or her learnr tutorial, the gradethis
function immediately performs three tasks. It:
gradethis
does not itself calculate a final grade for the student.
To use gradethis
inside a learnr tutorial, we suggest calling gradethis::gradethis_setup()
inside the tutorial's setup
chunk. This is how it will look in the learnr R Markdown file:
`r ''````r library(gradethis) gradethis_setup() ```
You can also see this in the first code chunk of the .Rmd file associated with this tutorial.
The remainder of this document looks at how to use individual gradethis
functions.
grade_result()
grade_result()
checks whether or not the student's code returns the correct result. It matches the result against one or more conditions and returns the message (and correct/incorrect status) associated with the first matched condition.
Here is an example of grade_result()
in use.
grade_result( pass_if(~ identical(.result, 365.25 / 12 / 7), "There are 4.348214 weeks on average in a month."), fail_if(~ identical(.result, 365 / 7 / 12), "Did you assume the average year has 365 days? Due to leap years, the average year actually has 365.25 days."), fail_if(~ identical(.result, 52 / 12), "Did you assume that the average year has 52 weeks? It is actually has a little more because 52 * 7 is only 364 days."), fail_if(~ identical(.result, 4), "Close, but four is the average number of whole weeks in a month."), fail_if(~ TRUE, "Not quite. Consider that there are 365.25 days in an average year, 12 months in a year, and 7 days in a week.") )
And here is the code behind the example.
`r ''````r ``` `r ''````r grade_result( pass_if(~ identical(.result, 365.25 / 12 / 7), "There are 4.348214 weeks on average in a month."), fail_if(~ identical(.result, 365 / 7 / 12), "Did you assume the average year has 365 days? Due to leap years, the average year actually has 365.25 days."), fail_if(~ identical(.result, 52 / 12), "Did you assume that the average year has 52 weeks? It is actually has a little more because 52 * 7 is only 364 days."), fail_if(~ identical(.result, 4), "Close, but four is the average number of whole weeks in a month."), fail_if(~ TRUE, "Not quite. Consider that there are 365.25 days in an average year, 12 months in a year, and 7 days in a week.") ) ```
grade_result()
should contain a sequence of fail_if()
and pass_if()
functions.
Each fail_if()
and pass_if()
function should contain:
~
Use .result
to refer to the student's answer within the logical tests.
grade_result()
will evaluate the _if
functions in order, replacing .result
with the student's result as it does.
grade_result()
will stop and return the message of the first _if
function whose condition evaluates to true. If that function is:
pass_if()
, the exercise will be marked correct fail_if()
, the exercise will be marked wrongOrder matters! grade_result()
will not continue to evaluate _if()
functions after one returns a message.
grade_result()
will mark a result correct if it passes a single pass_if()
statement (without first triggering a fail_if()
).
If you would like to ensure that a result satisfies every pass_if
statement use grade_result_strict()
.
grade_result()
will not check the students code. Nor will grade_result()
know if the student directly typed the correct result into the exercise box.
If you want to check the code the student used to get a result, use grade_code()
See ?grade_result
for more information.
grade_result_strict()
grade_result_strict()
is similar to grade_result()
, but it requires a result to pass every pass_if()
function contained in its function body. This method is analogous to creating unit tests that all need to pass.
Here is an example of grade_result_strict()
in use:
Please make a function in the exercise space below, but do not assign the function to an object. The function should:
x
argument1
to the x
value.Then click Submit Answer.
function(x) { # solution is x + 1L x + 1 }
grade_result_strict( pass_if(~ .result(3) == 4), pass_if(~ identical(.result(0), 1)), pass_if(~ identical(sapply(1:10, .result), 2:11)), pass_if(~ sapply(1:10, .result) == 2:11), pass_if(~ all.equal(sapply(1:10, .result), 2:11)), pass_if(~ checkmate::test_function(.result, args = c("x"))) )
And here is the code behind the example.
`r ''````r function(x) { # solution is x + 1L x + 1 } ``` `r ''````r grade_result_strict( pass_if(~ .result(3) == 4), pass_if(~ identical(.result(0), 1)), pass_if(~ identical(sapply(1:10, .result), 2:11)), pass_if(~ sapply(1:10, .result) == 2:11), pass_if(~ all.equal(sapply(1:10, .result), 2:11)), pass_if(~ checkmate::test_function(.result, args = c("x"))) ) ```
grade_result_strict()
takes a set of pass_if()
functions. Each pass_if()
function should contain:
~
Use .result
to refer to the student's answer within the logical tests.
grade_result_strict()
will mark a result as correct only if passes every pass_if()
statement. This is especially useful for grading function definitions.
See ?grade_result_strict
for more information.
grade_code()
grade_code()
to checks whether the student code matches the solution code. If the code does not match, grade_code()
will tell the student exactly where their code begins to diverge from the solution and how to get back on track. Here's an example:
sqrt(log(2))
grade_code("Don't worry, things will soon get harder.")
We can also provide a more specific feedback when we expect an assign statement, for example.
a <- sample(1:6, size = 1)
. Enter 123, then click Submit Answer.a <- sample(1:6, size = 1)
grade_code("Don't worry, things will soon get harder.")
You should see a red box with "I expected you to assign something to something else with <- where you wrote 123."
Since gradethis_setup()
sets learnr::tutorial_options(exercise.error.check.code = "gradethis::grade_code()")
, when an exercise submission produces an evaluation error, intelligent feedback is provided via grade_code()
(when a solution is provided). Try it yourself by submitting mt
(R error, incorrect answer), then cars
(No error, incorrect answer), then mtcars
(correct answer) below:
mtcars
grade_result( fail_if(~identical(.result, cars), "This is the cars (not mtcars) dataset."), pass_if(~identical(.result, mtcars)) )
And here is the code behind the example.
`r ''````r ``` `r ''````r mtcars ``` `r ''````r grade_result( fail_if(~identical(.result, cars), "This is the cars (not mtcars) dataset."), pass_if(~identical(.result, mtcars)) ) ```
In the case that you wish to provide custom checking logic for errors, you can also provide a -error-check
chunk. If the student submission generates an error, gradethis will run the checking code in the -error-check
chunk instead of the checking code in the -check
chunk. gradethis will only run the -error-check
code if the student submission generates an error.
mtcars
grade_code(incorrect = "This code produces an error (press 'Run Code' to see it).", glue_incorrect = "{ .message } { .incorrect }")
grade_result( fail_if(~identical(.result, cars), "This is the cars (not mtcars) dataset."), pass_if(~identical(.result, mtcars)) )
And here is the code behind the example.
`r ''````r ``` `r ''````r mtcars ``` `r ''````r grade_code(incorrect = "This code produces an error (press 'Run Code' to see it).", glue_incorrect = "{ .message } { .incorrect }") ``` `r ''````r grade_result( fail_if(~identical(.result, cars), "This is the cars (not mtcars) dataset."), pass_if(~identical(.result, mtcars)) ) ```
grade_code()
requires a model solution to compare student code to. Supply this solution in a learnr -solution
chunk, i.e. a chunk whose label is the label of the exercise it is associated with followed by -solution
:
`r ''````r sqrt(log(2)) ```
grade_code()
will compare the last expression in the student submission to the last expression in the solution chunk.
Teachers will usually pass grade_code()
a character string to display to the student if their code successfully matches the solutiuon, e.g.
grade_code("Good job. Don't worry, things will soon get harder.")
grade_code()
does not check the result of the student's code.
Instead, grade_code()
parses the student code into a call tree. As it does, it standardizes argument names and accounts for the presence of pipes (%>%
).
grade_code()
then does the same for the solution code.
Finally, grade_code()
recursively walks the two call trees.
grade_code()
stops and returns a message if the student code:
match.call()
If none of the above occurs, grade_code()
marks the answer as correct.
grade_code()
attempts to supply helpful feedback that makes sense to the student without giving away the solution. To do this, grade_code()
messages take the form of
"I expected X, where you wrote Y. Please try again."
Here Y may be an argument or function call that appears at the beginning, end, or middle of the student code.
grade_code()
will catch mistakes one at a time in the order that they appear in the students code. This lets the student iteratively improve their code.
See ?grade_code
for more information.
gradethis
can accept any checking method that returns a gradethis::graded
object.
The example below returns a correct/incorrect answer with 50/50 probability.
`r ''````r "Flip a coin" ``` `r ''````r fifty_fifty_checker <- function( correct = "Correct!", incorrect = "May the odds be ever in your favor!", ..., user ) { is_correct <- (runif(1) < 0.5) gradethis::graded( correct = is_correct, message = ifelse(is_correct, correct, incorrect) ) } fifty_fifty_checker() ```
The custom checking function should return the output of gradethis::graded()
which takes two arguments:
correct
- TRUE
is the answer is correct, FALSE
otherwisemessage
- an optional character string to display to the studentTo use the custom checking function, place it in an exercise -check
chunk, as you would place grade_result()
, grade_result_strict()
, or grade_code()
.
Are you feeling lucky?
If you are, click Submit Answer.
"Flip a coin"
fifty_fifty_checker <- function( correct = "Correct!", incorrect = "May the odds be ever in your favor!", ..., user ) { is_correct <- (runif(1) < 0.5) gradethis::graded( correct = is_correct, message = ifelse(is_correct, correct, incorrect) ) } fifty_fifty_checker()
See ?graded
for more information.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.