In iholzleitner/facefuns: Convenience Functions Around 2D and 3D Face Shape Analysis

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

options(rgl.useNULL = TRUE) # suppress rgl opening separate windows
library(facefuns)
library(rgl)
library(Morpho)

WORK IN PROGRESS

This tutorial covers basic steps in working with 3D meshes. facefuns includes an example of a 3-D face in OBJ format, as well as 12 3-D faces in PLY format. The latter are composite faces that show average white female and male faces from six different age groups (18-25, 26-35, 36-45, 46-55, 56-65, 66-75). Each composite comprises 10 individual faces that were collected as part of the ERC-funded KINSHIP project (#647910) using a DI3D system[^osf]. 3-D models were delineated and warped to a standard face in MorphAnalyser [@tiddeman2000]. This means that all 3-D faces were were resampled according to a "standard" mesh, i.e. each mesh's tesselations are homologous across the entire sample.

[^osf]: See here for more details on the capturing setup

Importing 3-D data into R

OBJ vs PLY

For shape analyses, all we really need are the vertex coordinates of each mesh. We will discuss a couple of different functions for reading OBJ and PLY files, but all of these will store the results in an object of class "mesh3d", from which we can then extract the vertex data.

If shape analyses are all you are after, and you are conscious of file size, you might want to convert your OBJs to PLYs.[^msl] PLYs come in different flavours: ASCII, binary_little_endian or binary_big_endian. To give you a rough idea, a 12MB OBJ is around 4MB after being converted to an ASCII PLY, and around 2MB after being converted to a binary PLY. They also load much faster; that said, the resulting mesh3d objects are about the same size no matter the initial format.

If you want to render your meshes with textures, specific file formats become more important. It's easy enough to read all the info you need from OBJs, but I have not yet managed to read UV coordinates from PLY files. However, if you have been warping your meshes in MorphAnalyser, not only will all of the 3D models have homologous vertices; they will also have the same UV map - the texture coordinates for any mesh will be the same, which means you can plot any mesh with any (accordingly mapped) texture.

[^msl]: You can batch convert files using MeshLab's command line interface, MeshLabServer. MeshLabServer can export both binary and ASCII PLYs; the corresponding commands on Mac are:
for conversion to PLY/binary_little_endian : for FILE in *.obj; do /Applications/meshlab.app/Contents/MacOS/meshlabserver -i "$FILE" -o "${FILE%.obj}.ply"; done
for conversion to PLY/ASCII: for FILE in *.obj; do /Applications/meshlab.app/Contents/MacOS/meshlabserver -i "$FILE" -o "${FILE%.obj}.ply" -m sa; done

Read OBJ

To load and plot an OBJ[^obj] with texture, you could use

path_to_obj <- system.file("extdata", "obj", package="facefuns")

objexample <- rgl::readOBJ(paste0(path_to_obj, "/example.wavefront"),
                           material = list(color = "white",
                                           texture = paste0(path_to_obj,"/example.png")))

rgl::clear3d(type = c("shapes", "lights")) # clear scene
rgl::shade3d(objexample,
        box = FALSE, axes = FALSE,
        xlab = "", ylab = "", zlab = "",
        col = "white", specular = "black")
rgl::rglwidget() # open widget

rm(objexample)

Click and drag the face to move it. Note that the texture file needs to be a PNG to load successfully. If you get an Instructions "usemtl" ignored warning, you can either ignore it or remove the offending line from the OBJ in a text editor.

readOBJ can be fairly slow - the package readobj [@readobj] promises faster reading of OBJ files!

[^obj]: The example file has the file extension .wavefront because ".obj" triggers a false positive R CMD check NOTE

Read PLY

You can use geomorph::read.ply to import ASCII PLYs, or Morpho::file2mesh [@morpho] to import both ASCII and binary PLYs. The example faces included in facefuns are binary PLYs, so we use Morphoto read them:

# not run to keep file size down
path_to_ply <- system.file("extdata", "ply", package="facefuns")
files <- list.files(path = path_to_ply,
                          pattern = "\\.ply$",
                          full.names = TRUE)

plyexample <- Morpho::file2mesh(files[1])

rgl::clear3d(type = c("shapes", "lights"))
rgl::light3d() # add light source to the scene
rgl::plot3d(plyexample,
       type = "wire",
       box = FALSE, axes = FALSE,
       xlab = "", ylab = "", zlab = "",
       col = "gray", specular = "black")
rgl::aspect3d("iso") # display all axes at same scale
rgl::rglwidget()

As you can see in the environment, the mesh3d object is quite large (>3MB). But, as we said, all we really need at this point are the vertex coordinates.

`read_vertices`

facefuns::read_vertices uses rgl::readOBJ and Morpho::file2mesh to import meshes, extract only vertex coordinates and save them into an array

path_to_ply <- system.file("extdata", "ply", package="facefuns")
data <- read_vertices(path_to_ply)
str(data)

`facefuns3d`

shapedata <- facefuns(data = data,
                      pc_criterion = "broken_stick",
                      quiet = FALSE)

`plot_3dpcs`

plot_3dpcs(input = shapedata$pc_plot,
           which_pcs = 1,
           ref = shapedata$average,
           ref_mesh = paste0(system.file("extdata", "obj", package="facefuns"), "/example.wavefront"))
rgl::rglwidget()