library(learnr) library(checkr) knitr::opts_chunk$set(echo = FALSE) tutorial_options(exercise.checker = checkr::check_for_learnr) cruel_remarks <- c("Really?", "Think about it!", "Maybe you shouldn't major in this field.", "Perhaps you should review the textbook.", "Look carefully at what you did.", "No!", "{{V}} is not right.") trig_2_check <- function(USER_CODE) { code <- for_checkr(USER_CODE) t1 <- if_empty_submission(code, message = "OK. Let's get you started. Pick a trig function such as sin(), cos(), tan(), giving the angle as an argument. Perhaps something like tan(40). This won't be right, but it will get you on track. ") t1 <- line_calling(t1, sin, cos, tan, message = "You should be using a trigonometric function.") t1 <- line_calling(t1, sin, message = "Make sure to pick the right trig function. cos() does horizontal lengths, sin() does vertical lengths.") if (failed(t1)) return(t1) a1 <- trig_radian_check(t1, 53*pi/180) if (failed(a1)) return(a1) t2 <- line_where(code, insist(F == "*"", "Remember to multiply by the length of the hypotenuse.")) t3 <- arg_calling(t2, `*`) a1 <- arg_number(t3, 1, insist(V == 15, "How long is the hypothenuse? It says right on the diagram.")) a2 <- arg_number(t3, 2, insist(V == 15, "How long is the hypothenuse? It says right on the diagram.")) if (failed(a1 %or% a2)) return(a2) line_where(t2, insist(is.numeric(V)), insist(abs(V - 11.98) < 0.01, "{{V}} is a wrong numerical result. It should be about 11.98."), passif(TRUE, "Good!")) } cruel_trig <- function(USER_CODE) { code <- for_checkr(USER_CODE) line_where(code, insist(abs(V - 11.97) < 0.01, sample(cruel_remarks, 1)), passif(TRUE, "See, it wasn't so hard!")) } cruel_trig("11.98") source(system.file("learnr_examples/internal-examples.R", package = "checkr"))
Write an R statement to compute the numerical value of x.
..trig_function..( ..angle.. )
15 * sin(53 * pi / 180)
Is x a vertical or horizontal difference? Pick the appropriate trig function.
Sine is for vertical, cosine for horizontal.
The angle should be in radians.
sin
part of the answer still has to walk through that hint.You can produce a "Submit" button by adding a -check
chunk.
..trig_function..( ..angle.. )
```r ..trig_function..( ..angle.. ) ``` ```r for_checkr(USER_CODE) ```
for_checkr(USER_CODE)
The positive message for running code is nice, but too generous!
3
res <- pre_check(USER_CODE) if (failed(res)) return(res) # USER_CODE <- quote("15 * sin(53*pi/180) + 2") # USER_CODE <- quote("sin(53*pi/180)") trig_2_check(USER_CODE)
Ordinarily, you would not show students the checkr
statements implementing this behavior. But our purpose here is to introduce checkr
, So here are the statements for the above exercise.
print_function_contents( trig_2_check) # from_file = system.file("learnr_examples/internal-examples.R", package = "checkr"))
Breaking this down, line by line:
learnr
. The submission is always called USER_CODE
. The function for_checkr()
does some pre-processing of the user submission to turn it into evaluated code and format it for use in later checkr
functions.sin()
. So we'll focus on the angle. You can write specialized checkr
functions to handle specialized areas. Degrees vs radians will not be an issue in most tutorials.Depending on the submission, any of the checksmight fail. If a check fails, later checks that use the previous result will short circuit to a failed check. This allows checks to be chained, with the earliest failure determining the outcome.
An instructor with a different pedagogical approach might prefer to structure the checking in an entirely different way. For instance, here are checkr
statements that simply tell the user whether or not the submission did what was requested.
# Get started
cruel_trig(USER_CODE)
print_function_contents(cruel_trig)
Really?
Think about it!
Maybe you shouldn't major in this field.
Perhaps you should review the textbook.
Look carefully at what you did.
No!
{{V}} is not right.
But to give hints without context seems strange, yet this is what
the learnr
-hint
system does.
checkr
is based on the rlang
and redpen
packages. Two basic technologies:
# .(label) refers to an expression # ..(label) refers to a value submission <- quote(15 * sin(53 * pi / 180)) redpen::node_match(submission, 15 * sin(.(ang)) ~ ang) redpen::node_match(submission, 15 * sin(..(ang)) ~ ang)
checkr
adds ...learnr
.You need to be careful about whether an expression can be evaluated.
For instance, here, hp
and mpg
only make sense in the context of mtcars
. And it's select()
that
provides that context.
submission <- quote(mtcars %>% select(hp, mpg)) redpen::node_match(submission, mtcars %>% select(.(v1), .(v2)) ~ c(v1, v2)) redpen::node_match(submission, mtcars %>% select(..(v1), ..(v2)) ~ c(v1, v2))
Checking logic needs to be careful about names versus values when referring to variables.
A good way to start when developing a checking function:
a. Write down examples of the correct and incorrect submissions you anticipate.
- Present an exercise in class to gain experience.
- Deploy a development problem with minimal feedback and harvest the submissions.
b. Create the feedback message for each of these submissions.
c. ... the magic happens here ... Figure out the checking logic to associate (a) and (b).
Some easy cases:
- Did they use a distinctively wrong function? line_calling() %>% misconception()
- Cast as a fill-in-the-blanks
- Are there named arguments? named_arg()
Checkr
can be run independent of learnr
, so regular test-case and debugging tools can be used.
These problems have a predictable structure, making them easier to check.
Exercise 14: Fill in the blanks in the following code to create a
ggplot2
command that will produce the following scatter plot with themtcars
data.
```r library(ggplot2) ggplot(mtcars, aes(x = mpg, y = hp, color = cyl)) + geom_point()
> > There are four blanks. You'll have to replace all of them with the correct contents to generate the plot. ```r library(ggplot2) ggplot(mtcars, aes(x = ____ , y = ____, color = ____)) + ____()
check_exer_14(USER_CODE)
The checking code:
print_function_contents( check_exer_14, from_file = system.file("learnr_examples/internal-examples.R", package = "checkr"), just_the_body = FALSE)
In the ggplot
example, pressing "Submit" on the scaffold produced a native R run-time error as the message.
learnr
system evaluates the submission before handing it to the checker.-code-check
chunk to hand the submission to the checker before evaluating it.library(ggplot2) ggplot(mtcars, aes(x = .... , y = .... , color = .... )) + ....()
3 # chunk must have executable content
res <- pre_check(USER_CODE) if (failed(res)) return(res) check_exer_14(USER_CODE)
s1 <- "library(ggplot2); ggplot(mtcars, aes(x = ...., y = ...., color = ....)) + ....()" pre_check(s1) check_exer_14(s1)
We can only know if a name is valid by evaluating it in the context of earlier computations.
Suggestions:
-run-check
option to let "Run" call a checking function, too.Providing student feedback is largely an empirical problem, not a logical one. We need to know what student misconceptions are in order to know how to give formative feedback.
learnr
tutorial-event-recorder
function: collect submissions.submittr
package (that's another talk!) can handle this with user authentication, etc. install_github("dtkaplan/checkr")
Among other things:
checkr
may lead to better high-level functions for simplifying checking, like check_blanks
. We need to dig the Pit of Success Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.