In dtkaplan/checkr2: Exploring ideas for successor to checkr

library(learnr)
library(checkr2)
knitr::opts_chunk$set(echo = FALSE)
tutorial_options(exercise.checker = checkr2::check_for_learnr)
# tutorial_options(exercise.checker = function(...) cat("Bogus checker\n"))

hello_check <- function(USER_CODE) {
  code <- for_checkr(USER_CODE)
  res <- misconception(code, line_where(code, is.null(F)), 
                       message = paste(
                         "Just typing the string doesn't do it. Note that the output", 
                         "has quotes and is formatted as an output, not a simple message."))
  res <- misconception(res, line_where(res, F == print), 
                       message = paste(
                         "Using print() isn't right.",
                         "True, that's the way character strings are displayed,", 
                         "but the format of the display is as a quoted output and",
                         "not just a simple message."))
  res <- line_where(res, F == cat, message = "Try the cat() function.")
  check(arg_number(res, 1), passif(V == "Hello, World!", "Good job!"),
        failif(tolower(V) == "hello, world!", "Get the capitalization right!"),
        failif(TRUE, "The string should be 'Hello, World!', not '{{V}}'.")) 
}

two_plus_two_check <- function(USER_CODE) {
  res <- for_checkr(USER_CODE)
  res <- line_where(res, V == 4, message = "The result should be 4, not {{V}}.")
  res <- line_where(res, F == `+`, 
                    message = 
                      paste("Think about what function corresponds to 'addition'.", 
                            "It isn't {{F}}."))
  arg_number(res, 1, failif(V != 2, "The first argument should be 2, not {{EX}}."),
             passif(TRUE, "Yes, that's it!"))
}

hello_fun_strict <- function(USER_CODE) {
  code <- for_checkr(USER_CODE)
  line_binding(code, 
               cat("Hello, World!"),  # a pattern with no flexibility.
               passif(TRUE, "That's right."),
               fail = "No. Try aain.")
}

What's this document about?

This is a learnr tutorial to display some examples of code-checking with the checkr2 package.

As you may know, learnr is a system for creating and distributing interactive documents, particularly documents that provide exercise-boxes where the reader can type and execute R code.

Checkr2 is a package that provides feedback about the code that is being executed in a learnr exercise box.

Hello, World!

"Hello, World!" is a famous introductory example in programming. In this task, you will ...

Write the one-line version of "Hello, World!" in R. Your code should cause the message Hello, World! to appear.

Press "Run" to try out your command to make sure it does what you were expecting. When you are satisfied, press "Submit" to get the checkr feedback.

The box below has been instrumented with checkr statements intended to guide the user to a correct answer. Exactly what "guide the user" means is ambiguous. Play around with various possible answers and see if you agree that the checkr statement I choose are helpful.

hello_check(quote("\"Hello, World!\""))
hello_check(quote(cat("Hello, World!")))
hello_check(quote(print("Hello, World")))
hello_check(quote(cat("hello")))
hello_check(quote(cat("Hello, world!")))

hello_check(USER_CODE)

"Hello, World!"

The man behind the curtain

Ordinarily, you would not show students the checkr statements implementing this behavior. But our purpose here is to introduce checkr, So here are the statements for the above exercise.

print_function_contents(
  hello_check,
  from_file = system.file("learnr_examples/internal-examples.R", 
                          package = "checkr2"))

Breaking this down, line by line:

• [1] accepts the user submission from learnr. This is always called USER_CODE. The function for_check() does some pre-processing of the user submission to turn it into evaluated code and format it for use in later checkr functions.
• [2] tests for a particular kind of mistaken answer. The misconception() function will generate a checkr fail message, if the pattern identified in the second argument passes. That pattern, line_where(code, is.null(F)), means, "scan the code looking for a line where no function is being used." This will captures a line that contains only a character string.
• [5] tests for another specific misconception, that the user invokes print() on the string.
• [9] looks whether the function invoked by the user is cat(). If not, the check fails. (Note that [5] already ruled out that print() was being invoked.)
• [10] checks the argument to the cat() function. (We know it's cat(), because [9] has established this.) If that argument is exactly `"Hello, World!" the submission passes. Otherwise, we check for a particular error involving capitalization and, if that's not the case, generate a message to tell the student what's wrong.

Depending on the submission, any of the checks on lines 2, 5, 9, and 10 might fail. If a check fails, later checks that use the previous result will short circuit to a failed check. For instance, if the check on line [2] fails, the remaining checks won't be performed in detail: they will just pass along the failed result from line [2].

An instructor with a different pedagogical approach might prefer to structure the checking in an entirely different way. For instance, here are checkr statements that simply tell the user whether or not the submission did what was requested:

print_function_contents(
  hello_fun_strict,
  from_file = system.file("learnr_examples/internal-examples.R", 
                          package = "checkr2"))

"Hello, World!"

hello_fun_strict(USER_CODE)

Two plus two

Not as famous as "Hello, World!" is the first example in the learnr documentation:

Write the code to add two and two.

Of course, a submission of 2 + 2 will work. See what happens when you try to fool the checking system with a submission like 6 - 4 or 1 + 1 + 1 + 1.

1 # To trigger pre-check, this chunk must have something in it.

two_plus_two_check(USER_CODE)

Here are the checkr statements implementing the above behavior:

print_function_contents(
  two_plus_two_check,
  from_file = system.file("learnr_examples/internal-examples.R", 
                          package = "checkr2"))

dtkaplan/checkr2 documentation built on May 17, 2019, 4:01 p.m.