detect_faces: Detect faces in images or a video

View source: R/detect_faces.R

detect_facesR Documentation

Detect faces in images or a video

Description

Finds every face in a set of images or in frames sampled from a video, so you can see which faces the package found, pick the ones to analyse (with select_faces or ordinary subsetting) and score only those with face_scores.

Usage

detect_faces(
  x,
  face_detector = c("yunet", "haar"),
  min_confidence = 0.6,
  identify = TRUE,
  identity_threshold = 0.363,
  nframes = 100,
  start = 0,
  end = -1,
  uniform = FALSE,
  ffreq = 15,
  save_frames = FALSE,
  save_video = FALSE,
  save_dir = tempdir(),
  video_name = "temp",
  pattern = "\\.(jpg|jpeg|png|bmp)$",
  recursive = FALSE
)

Arguments

x

Image paths or URLs, a directory of images, or one video (a file or a YouTube URL).

face_detector

"yunet" (default), OpenCV's YuNet detector, or "haar", the Haar cascade transforEmotion used before version 0.1.8. YuNet finds small, tilted and partly turned faces and reports a confidence score and five landmarks; Haar finds frontal faces only.

min_confidence

YuNet's confidence threshold, between 0 and 1. Ignored for Haar.

identify

Logical. Give faces of the same person the same person_id, using OpenCV's SFace face recognizer. Needs YuNet. The recognition model (10 MB) is downloaded on first use; see Details.

identity_threshold

Cosine similarity between faces (or video tracks) above which they count as the same person. The default, 0.363, is the threshold SFace's authors recommend; raise it if different people share a person_id, lower it if one person gets several.

nframes, start, end, uniform, ffreq

How frames are sampled from a video; see video_scores. Use the same values in both functions to get the same frames.

save_frames

Logical. Keep the extracted video frames in save_dir. Default is FALSE.

save_video

Logical. Keep a downloaded YouTube video in save_dir. Default is FALSE. The video has no sound when YouTube offers no stream with video and audio in one file.

save_dir

Directory for video frames. Default is a temporary directory.

video_name

Name used for the frame files (<video_name>-frame-<i>.jpg).

pattern, recursive

When x is a directory: a regular expression for the image files and whether to search subdirectories.

Details

Faces are numbered from left to right within each image or frame. In a video, a face is followed from frame to frame and keeps its track_id. With identify = TRUE, faces of the same person get the same person_id: across all images, or across all tracks of the video, so a person who leaves the picture and comes back, or who appears in several images, is one person.

Face recognition compares faces through SFace embeddings (128 numbers per face). Faces are grouped by average-linkage clustering on cosine similarity; a video track counts as one face, the average of its frames. Two faces in the same image can be the same person (mirrors, screens, posters). The embeddings stay in memory and are never returned or written to disk, and person_ids only label faces within one call: they do not name anyone and do not match ids from other calls.

Grouping faces by person is processing of biometric data under the EU General Data Protection Regulation (Article 9); make sure your ethics approval covers it, or use identify = FALSE.

The YuNet model ships with the package. The SFace model (face_recognition_sface_2021dec_int8.onnx, from OpenCV's model zoo, Apache 2.0 licence) is downloaded into the package cache on first use, or by setup_modules. To use a copy elsewhere, for example on a computer without internet access, set the environment variable TRANSFOREMOTION_SFACE_MODEL to its path. Without the model, person_id is NA.

Value

A data frame with one row per face and the columns:

source

The image path or URL, or the video.

frame, frame_index, time

For a video: the sample number (as in the frame file name), the frame's index in the video and its time in seconds. NA for images.

face

Face number within the image or frame, from left to right.

track_id

For a video: the face's track. NA for images.

person_id

"p1", "p2", ... numbered by first appearance; NA without recognition.

x, y, width, height

The face box in pixels, from the top-left corner of the image.

confidence

YuNet's score; NA for Haar.

detector

"yunet" or "haar".

image_width, image_height

Size of the image or frame in pixels.

right_eye_x, ..., mouth_left_y

The five YuNet landmarks (NA for Haar).

The attribute "media" holds one row per image or frame, with the number of faces found (n_faces), so images without faces are not lost. The attribute "settings" records the detector and, for a video, how frames were sampled, which face_scores uses to find the same frames again.

Data Privacy

All processing is done locally; images and frames are never sent to a remote server.

See Also

select_faces, face_scores, draw_faces

Examples

## Not run: 
image <- system.file("extdata", "two-people.jpg", package = "transforEmotion")
faces <- detect_faces(image)
faces[, c("face", "person_id", "x", "y", "width", "height", "confidence")]
draw_faces(faces)

# Every face of one person in a video, scored once per sampled frame
video <- system.file("extdata", "two-people.mp4", package = "transforEmotion")
faces <- detect_faces(video, nframes = 20)
table(faces$track_id, faces$person_id)
scores <- face_scores(select_faces(faces, persons = "p1"),
                      classes = c("happy", "sad", "angry", "neutral"))

## End(Not run)


transforEmotion documentation built on Oct. 26, 2026, 5:07 p.m.