In debruine/webmorphR: Reproducible Stimuli

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  out.width = "100%",
  fig.path = "images/manip/"
)

starttime <- Sys.time()

This vignette demonstrates the functions you can use to resize, rotate, crop, pad, and align images. Templates are also changed with the images.

# load packages and set maximum plot width
library(webmorphR)
library(webmorphR.stim) # for extra stimulus sets
wm_opts(plot.maxwidth = 850)

Load your images

Load the demo stimuli, or read in your own images with read_stim().

stimuli <- demo_stim()

Resize

Resizing your images at the start of a chain can make everything go faster if you're working with very large images, but can make the results blurry if you go too small.

stimuli |>
  resize(.1)

You can set the resize dimensions by proportions or pixels. Numbers less than 10 are interpreted as proportions, so the code above resized the images to 0.1 of their original size. Numbers larger than 10 are interpreted as pixels. If you set just width or just height, the other dimension is calculated proportionally to keep the aspect ratio.

stimuli |>
  resize(height = 200)

If you set both width and height, you may alter the aspect ratio. Make sure you really want to do this! Use to_size() instead if you want to resize and crop/pad all of your images to the same size without changing the image contents' aspect ratio.

stimuli |>
  resize(width = 200, height = 100)

Rotate

Rotate images by degrees. Set the fill argument to fill in the corners with a specific colour, or use patch(stimuli) to sample the colour from 10x10 pixel square at the top left corner.

stimuli |>
  rotate(degrees = 45, fill = patch(stimuli))

If images aren't centered, you may want to rotate around the center of the image or the center of the face, and may want to keep the original image dimensions or expand to the full rotated image dimensions.

To demonstrate, we'll add some blue padding to the right side of the first demo image to make it off center. The contrasting fills used below are just to help you understand how each rotation option works; you'll usually set fill to the image background colour.

offset <- stimuli[1] |> 
  draw_tem() |>
  pad(0, 500, 0, 0, fill = "dodgerblue") |>
  rep(4)

# set the rotations to show
degrees = seq(0, 90, by = 30)

By default, the rotate happens around the centre of the image. This can be undesirable if the face is off-centre.

rotate(offset, degrees, origin = "image", fill = "pink")

Set origin = "tem" to rotate around the centre of the template.

rotate(offset, degrees, origin = "tem", fill = "pink")

# degrees to rotate
degrees <- seq(0, 360, 10)

# get an image, crop and duplicate it 
img <- demo_stim(1) |> 
  crop(0.5, 0.5) |> 
  pad(0, .5, 0, 0) |> 
  rep(length(degrees))

# rotate around centre of image
image_centered <- img |> 
  mask(fill = "dodgerblue") |>
  rotate(degrees, origin = "image", fill = "dodgerblue")|>
  pad(30, 0, 0, 0) |> 
  label('origin = "image"', size = 20)

# rotate around centre of template
tem_centered <- img |> 
  mask(fill = "hotpink") |>
  rotate(degrees, origin = "tem", fill = "hotpink")|>
  pad(30, 0, 0, 0) |> 
  label('origin = "tem"', size = 20)

# rotate around left eye point
lefteye_centered <- img |>
  mask(fill = "orange") |>
  rotate(degrees, origin = 0, fill = "orange")|>
  pad(30, 0, 0, 0) |> 
  label('origin = 0 (left eye)', size = 20)

# rotate around right eye point
righteye_centered <- img |> 
  mask(fill = "purple") |>
  rotate(degrees, origin = 1, fill = "purple")|>
  pad(30, 0, 0, 0) |> 
  label('origin = 1 (right eye)', size = 20)

# combine and animate
c(image_centered, tem_centered, 
  lefteye_centered, righteye_centered) |> 
  animate(fps = 20)

If you set keep_size = FALSE, the image and template will be rotated and keep the maximum rotated size (so it doesn't matter what you set origin to).

rotate(offset, degrees, keep_size = FALSE, fill = "pink")

Horizontal Eyes

If you want to rotate each image by the amount that makes the eyes horizontal, use horiz_eyes().

unstandard <- demo_unstandard(1:5)
horiz <- horiz_eyes(unstandard, fill = patch(unstandard))

c(unstandard, horiz) |> plot(nrow = 2)

Crop

Crop images to a new width and height. You can specify sizes in proportions or pixels. Values < 2 will be interpreted as proportions.

crop(stimuli, width = .6, height = .8)

The x- and y-offsets default to NULL, which centres the cropping. You can set the offsets to a proportion or pixel value and the cropping will start there. The origin is the upper left corner, so setting x_off = 0 starts the cropping from the left side, and setting y_off = 0 starts the cropping from the top.

crop(stimuli, 
     width = 250, height = 300, 
    x_off = 0, y_off = 0)

Pad

Pad images with a border. Values less than 1 are interpreted as proportions. Set the colour with fill, otherwise its value defaults to the value of wm_opts("fill"), which is white unless you reset it.

pad(stimuli, 0.05, fill = "black")

Pad the top, right, bottom and left borders with different values, and set a different colour for each fill.

pad(stimuli,
    top = 10, right = 20, bottom = 30, left = 40,
    fill = c("dodgerblue", "hotpink"))

Or you can use the patch function to get the median colour from a patch of the image. The patch() function defaults to the top left 10 pixel square of each image.

rb <- load_stim_rainbow(1:8)

rb |>
  pad(50, fill = patch(rb)) |>
  plot(nrow = 2)

Or you can set the boundaries of the patch manually. The code below sets the patch colour to the average colour from to 10% of width and height in the centre of each image.

patch_fills <- patch(rb, 
                     width = .1, 
                     height = .1, 
                     x_off = .45, 
                     y_off = .45)

rb |>
  pad(50, fill = patch_fills) |>
  plot(nrow = 2)

Crop to Template

You can use the crop_tem() function to get the minimum and maximum x- and y-values from all of the image templates, then use that to set the cropping. In the example below, the images are cropped so there is 100 pixels of padding around the boundaries, calculated across all images.

load_stim_lisa() |>
  crop_tem(100) |>
  draw_tem()

Set each to TRUE to calculate the boundaries separately for each template in the list instead of the full set.

load_stim_lisa() |>
  crop_tem(20, each = TRUE) |>
  draw_tem()

To Size

The functionto_size() combines crop and resize to get a batch of images to the same size.

multisize <- demo_unstandard(1:5) |> pad(10, fill = "hotpink")

A 10-pixel pink border has been added to these original images so you can see what has been cropped in the subsequent images.

lab <- paste0(width(multisize), "x", height(multisize))

multisize |> 
  pad(60, 0, 0, 0) |>
  label(lab, size = 40)

The code below resizes each image to fit into a 300 x 400 pixel image and fills in any extra background with blue. You'll probably choose your background colour for a real stimulus set, but this makes the demo clearer.

to_size(multisize, 300, 400, 
        fill = "dodgerblue")

Set keep_rels = TRUE to bring the largest image to fit in the new size, resize the others proportionally, and pad.

to_size(multisize, 300, 400, 
        fill = "dodgerblue", 
        keep_rels = TRUE)

Set crop = TRUE to resize all images until they cover the new size and crop the "overhang". There is no need to set the fill color for padding when crop is TRUE.

to_size(multisize, 400, 400, 
        crop = TRUE)

Set keep_rels = TRUE to bring the smallest image to cover the new size, resize the others proportionally, and crop any overhang.

to_size(multisize, 400, 400, 
        crop = TRUE, 
        keep_rels = TRUE)

Align

You can align a set of images on two points. By default, these are the first two points in the template, which correspond to the left and right eye in the standard FRL template, the 106-point Face++ templates, and the dlib auto_delin() templates. For chaotic historical reasons, the 83-point Face++ template has the eye points as 63 and 81 (pupils) or 57 and 50 (eye centre).

If you don't set the x and y coordinates, the images will align to the average value across the set. The value of fill will fill in any borders from the rotation and cropping so the images stay the same size as the originals.

unstandard_men <- demo_unstandard(6:10)

unstandard_men |>
  align(fill = "dodgerblue")

You can manually set the values for the x and y coordinates, as well as the resulting width and height of the images. These values must be in pixels.

unstandard_men |>
  align(x1 = 350, y1 = 350, x2 = 450, y2 = 450,
        width = 800, height = 800, fill = "dodgerblue")

Instead of aligning to the average of all the images, you may want to align to a reference image. Set ref_img to the index or name of the chosen image.

aligned5 <- align(unstandard_men, ref_img = 5, 
                  fill = "dodgerblue") 

plot_rows(
  "Original images" = unstandard_men,
  "Aligned to image 5" = aligned5,
  top_label = TRUE
)

You can align images to a single point by setting pt1 and pt2 the same. For example, the tip of the nose is point 55 in this template, so the following code moves the tip of the nose of all images to the average position, and alters the image dimensions to the average, but doesn't resize or rotate any of the faces

align(unstandard_men, pt1 = 55, pt2 = 55, 
      fill = "dodgerblue")

Procrustes Align

You can set the procrustes argument to TRUE to use Procrustes analysis to align the images and templates to fit the average shape (or a reference image) as closely as possible.

align(unstandard_men, procrustes = TRUE, fill = "dodgerblue")

Currently, webmorph does not have this capability, but you can download your images and tems, procrustes align them with the R package, save the images and templates, and upload them back to webmorph.

Chaining

You can also chain image manipulation commands.

stimuli |>
  rotate(c(45, -45)) |>
  crop(.5, .5) |>
  pad(0.1, fill = "dodgerblue")

Repeating images

You can use rep() to repeat images in a stimuli. Here, we repeat the faces 3 times each, apply 6 different rotations with different background colours, crop them to the same size, and plot them with 6 different template point colours.

rainbow <- c(pink = "#983E82",
             orange = "#E2A458",
             yellow = "#F5DC70",
             green = "#59935B",
             blue = "#467AAC",
             purple = "#61589C")

stimuli |>
  resize(0.5) |>
  rep(each = 3) |>
  rotate(seq(10, 60, 10), fill = rainbow) |>
  crop(250, 250, fill = rainbow) |>
  draw_tem(pt.color = rainbow, pt.alpha = 1, pt.size = 3) |>
  plot(nrow = 2)

Image functions

You can do so many things with the {magick} package that is installed with {webmorph}. The function image_func() is just a convenient wrapper that allows you to apply {magick} functions (or any other function that takes a magick image as the first argument) to each image in a stimulus list.

stimuli <- demo_stim("m_")
imglist <- c(
  image_func(stimuli, "blur", radius = 10, sigma = 5),
  image_func(stimuli, "charcoal", radius = 5, sigma = 2),
  image_func(stimuli, "oilpaint", radius = 5),
  image_func(stimuli, "implode", factor = 0.25),
  image_func(stimuli, "implode", factor = -0.25),
  image_func(stimuli, "negate"),
  # canny edge detection format is {radius}x{sigma}+{lower%}+{upper%}
  image_func(stimuli, "canny", geometry = "0x1+10%+30%"),
  image_func(stimuli, "edge", radius = 2),
  image_func(stimuli, "modulate", brightness = 80),
  image_func(stimuli, "modulate", brightness = 120),
  image_func(stimuli, "modulate", saturation = 0),
  image_func(stimuli, "modulate", saturation = 110)
)

labs <- c("Blur", "Charcoal", "Oilpaint", "Implode",
          "Explode", "Negate", "Canny", "Edge", 
          "20% Darker", "20% Brighter", "Greyscale", "Saturated")

imglist |>
  pad(70, 0, 0, 0, fill = "dodgerblue3") |>
  label(labs, color = "white") |>
  plot(nrow = 3)

The possible {magick} functions are listed by image_func_types(). Check the magick package help for more info on each (e.g., ?magick::image_oilpaint.

image_func_types()

A potentially useful one is "fill" if your backgrounds are clean enough (the mask() functions are better for this, though). You need to experiment with the fuzz a bit. These functions are vectorised, so you can set a different point to start the flood fill or a different fuzz for each image.

demo_stim(1) |>
  rep(5)  |>
  image_func("fill", "white", point="+10+10", 
             fuzz = seq(5, 25, 5)) |> 
  label(text = paste("fuzz =", seq(5, 25, 5)))

---

elapsed <- (Sys.time() - starttime) |>
  as.numeric(units="mins") |>
  round(1)

This script took r elapsed minutes to render all the included images from scratch.

debruine/webmorphR documentation built on Aug. 15, 2022, 3:51 p.m.