mapmate: Make Still Image Sequences For Data Animations

Documented in ffmpeg

#' Make video from still image sequence
#'
#' Make a video file from a sequence of still images using FFmpeg.
#'
#' @details
#' \code{ffmpeg} is a wrapper function around the popular \href{https://www.ffmpeg.org/}{FFmpeg command line multimedia framework}.
#' It translates arguments provided by the user in familiar R syntax into a system call to the \code{ffmpeg} command line tool, which must be installed on the system.
#'
#' \code{ffmpeg} does not provide complete flexibility to allow making every possible valid call to FFmpeg,
#' but users who are that well versed in the use of FFmpeg can use the command line utility directly or pass their custom calls directly to \code{system} from within R.
#' The \code{ffmpeg} R function is primarily useful to users not well versed in the use of the FFmpeg multimedia framework who will be satisfied with the level of flexibility provided.
#' Since this function is provided in the context of the \code{mapmate} package, it is aimed at assisting with converting still image sequences to video.
#' While additional uses may be incorporated into \code{ffmpeg} in future, the FFmpeg multimedia framework itself provides a far broader suite of tools and functionality than is needed here.
#'
#' Keep in mind that the purpose of \code{mapmate} is not to generate animations directly from R. See packages like \code{animation} if that is more the goal.
#' The goal that \code{mapmate} attempts to fulfill is strictly that of animation pre-production and it does so by focusing on the generation of still image sequences.
#' Any animation is expected to be done later by the user via software dedicated to video editing and production.
#' \code{ffmpeg} is provided in \code{mapmate} as an exception to the rule for users who wish to trade the full control and flexibility over video editing and production
#' that \code{mapmate} aims to avoid entangling itself with for the convenience of generating relatively basic video output directly from an R session.
#'
#' Ultimately, if you want an incredibly fancy video, do not rely on \code{ffmpeg} to splice and merge and overlay and juxtapose all your layers together,
#' to crop and pan and rotate, to apply effects and transitions and every other form of video processing to your image sequences; finish the production outside of R, because that is what makes sense.
#' If you are an FFmpeg expert, you don't need to use \code{ffmpeg} at all (but perhaps consider helping to improve this code!).
#' If you are not an FFmpeg expert, use other video editing software.
#'
#' There always comes a point where it makes the most sense to transition from one application to another.
#' When external solutions exist, it does not make sense to port the solution to every problem into R.
#' Future package versions may provide more and more functionality and control over video production directly from R through \code{ffmpeg} or other functions,
#' but at this time this should not be a primary development goal for \code{mapmate}.
#'
#' \subsection{Input Files}{
#' A common way to specify a set of input image files when using FFmpeg directy is to provide something like \code{myimages\%04d.png},
#' which requires specifying the entire, non-changing file name with the only substitution being for the unique, order, consecutive integer file numbering component of the file name.
#' The pattern used indicates how may places are occupied by the file indices, which should be constant. In this example, \code{\%04d} represents the file numbering \code{0000, 0001, 0002, ..., 9999}.
#' If using Windows, you must use this approach. Any image sequences generated by \code{mapmate} will follow this kind of file naming convention.
#' If you want to make videos from image sequences not made by \code{mapmate}, they will also commonly follow this convention, but not always, in which case you will have to rename your files.
#'
#' An alternative and often convenient way to provide a general pattern for matching to a set of input files is with globbing. However, globbing is not available on Windows.
#' Linux users may find this additional option helpful in cases where file naming is not quite as described above or, for example, if there are multiple sequences of files in one directory.
#' If \code{glob=TRUE}, wildcards can be used in the \code{pattern} argument, e.g., \code{pattern="*png"}, \code{pattern="myimages*png"}, or \code{pattern="*images0*.png"}.
#' The default is \code{glob=FALSE} and \code{glob} is simply ignored on Windows.
#'
#' The current package version of \code{ffmpeg} allows merging more than two sequences without error,
#' but testing has not confirmed this is actually working correctly, as all layers do not always appear in the output video.
#' }
#'
#' \subsection{Merging multiple image sequences}{
#'
#' \emph{Merging is experimental and under development. It does not yet work as intended.}
#'
#' \code{pattern} may be a vector referring to multiple image sequences. This is for merging or blending layers into one output video file.
#' The first vector element refers to the top layer among image sequences.
#' All files do not need to be in the same directory; \code{dir} can be vectorized to match with \code{pattern} if sequences are in different locations.
#' Similarly, \code{rate}, \code{delay}, and \code{start} can be vectors. If nothing but \code{pattern} is a vector, the other arguments are duplicated.
#' Vectors should be of equal length.
#'
#' Merging capabilities are limited. An expert in the use of FFmpeg should use it directly and not via this wrapper function.
#' If merging sequences with this function, it is recommended they be the same number of frames, begin from the same starting frame, and proceed at the same frame rate, though this is not strictly required.
#' Also, merging only two sequences at a time is recommended or they may not all display.
#' Sequences must be very similar in a variety of respects. For example, images must be the same dimensions across sequences.
#' For greater control, use FFmpeg directly from the command line and consult official FFmpeg documentation, or help improve this wrapper function via Github issues and pull requests.
#'
#' Remember that \code{mapmate} generates still image sequences that are intended for later use in a dedicated video editing program, one with a GUI, unlike FFmpeg which is a command line application.
#' In such a program, it is assumed the user may be dropping multiple image sequences on different tracks of a project timeline, layering the tracks together,
#' and for this reason the default background png color is transparent.
#' In the default case, using \code{alpha} less than \code{1.0} is generally unnecessary when merging two image sequences into a video with FFmpeg.
#' If not using defaults, \code{alpha} may not provide the flexibility desired.
#' }
#'
#' \subsection{Framerates}{
#' For \code{rate}, non-integer numeric values are rounded. Character options may be a valid abbreviation such as \code{"ntsc"} or a quoted ratio such as \code{"30000/1001"}.
#' Note that this is the familiar "29.97" (or, 29.97003, to be exact) but FFmpeg does not accept values like these.
#' Using \code{delay} instead of \code{rate} is more limiting since \code{delay} is converted back to rate (\eqn{delay=1/rate}), but must then be rounded to an integer.
#' Using \code{rate} is recommended. Arbitrary, non-standard framerates may lead to rendered videos that do not play properly in many media players.
#' For common settings and character abbreviations, see \href{http://ffmpeg.org/ffmpeg-utils.html#Video-rate}{FFmpeg standard video rates}.
#'
#' \code{rate} technically refers to the assumed or intended framerate of the input image file sequence.
#' This is important to mention because of the distinction between input and output framerates in FFmpeg.
#' See the details below on \code{min.rate} and \code{fps.out} to understand the differences and how to avoid some common problems.
#' }
#'
#' \subsection{Output Scaling}{
#' If \code{size} is not set to \code{"source"}, the output video is scaled.
#' \code{size} can be a character string of dimensions in length by height format such as \code{"720x480"} or an abbreviated standard such as \code{"ntsc"}.
#' See \href{http://ffmpeg.org/ffmpeg-utils.html#Video-size}{FFmpeg standard video sizes} for common dimensions and available abbreviations.
#' }
#'
#' \subsection{Presets, Codecs, Pixel Formats, Lossless Encoding, and minimum framerates}{
#' Presets provide a certain encoding speed to compression ratio.
#' Available presets include \code{ultrafast}, \code{superfast}, \code{veryfast}, \code{faster}, \code{fast}, \code{medium}, \code{slow}, \code{slower}, \code{veryslow}.
#' Faster speed corresponds to greater file size. Slower speeds are due to greater compression.
#'
#' \code{codec} is ignored if the file name in \code{pattern} ends with \code{.gif}.
#' For other video output file types a default codec is used depending on the file extension in \code{pattern} when \code{codec="default"}.
#' These can be overridden with options like \code{codec="h264"}, \code{"libx264"}, \code{"libvpx"}, \code{"prores"}, \code{"qtrle"}, etc.,
#' but the user needs to be knowledgeable regarding which codecs can be used for which output types or errors will be thrown.
#'
#' \code{format} is ignored if the file name in \code{pattern} ends with \code{.gif}.
#' The default is \code{"yuv420p"}, which performs 4:2:0 chroma subsampling.
#' This pixel format can reduce video quality, but it is the default because it ensures compatibility with most media players, many of which still cannot play 4:4:4 video.
#' For valid alternatives, run \code{system("ffmpeg -pix_fmts")}.
#'
#' \code{lossless} is ignored except for relevant \code{codec} settings, e.g., \code{h264} or \code{libx264}.
#' If \code{TRUE}, recommended \code{preset} values are \code{ultrafast} or \code{veryslow}. See \code{https://trac.ffmpeg.org/wiki/Encode/H.264} for more information.
#'
#' \code{min.rate} applies only to non-\code{gif} video output. Video files typically have framerates of 25 fps or 30 fps or higher.
#' In the case of creating gifs from an image file sequence, low framerates on the order of 10 fps or lower, even 1 fps, are often desired.
#' If such a low framerate is desired for video file output, many media players may not be able to play, or play properly, such a video file.
#' For example, the popular VLC media player can have difficulties with playback of video files created with a framerate of less than 10 fps, particularly with rates closer to 1.
#'
#' \code{min.rate} sets a lower bound on the framerate of the output file.
#' The intended frame rate given by \code{rate} or derived from \code{delay}, of the input image file sequence specified in \code{pattern},
#' is still preserved in the output playback. However, if \code{rate} is less than \code{min.rate}, the output file will achieve \code{min.rate} fps by duplicating frames.
#' For example, if \code{rate=1} and \code{min.rate=10}, a sequence consisting of 60 images will be converted to a 10 fps video containing 600 frames and taking the intended 60 seconds to play.
#' The tradeoff for compatibility with various media players is increased video file size, but depending on the codec, should not increase file size linearly,
#' e.g., not likely a ten times increase for the given example.
#'
#' Nevertheless, control is given to the user over the video output fps lower bound via \code{min.rate}. Just know that too low a value can cause problems.
#' If \code{rate} is greater than \code{min.rate}, the output file framerate will match the specified \code{rate} of the input image sequence.
#' This also may not be desired if \code{rate} is an atypical number for video framerates.
#' This matching can be overridden by specifying \code{fps.out} as something other than \code{rate}.
#' }
#'
#' @param dir directory containing images, defaults to working directory.
#' @param pattern character, for matching a set of input image files. See details for acceptable and possible alternative patterns.
#' @param output character, output file name.
#' @param output_dir character, output directory. Defaults to working directory.
#' @param rate integer or character, intended framerate of input image sequence in Hertz (Hz) or frames per second (fps). See details.
#' @param delay numeric, time delay between frames in output video. Alternative to \code{rate}. See details.
#' @param start integer, frame to start from in input image sequence. Defaults to \code{start=1}.
#' @param size character, the dimensions of the video output. Defaults to \code{"source"}, which is equal to the dimensions of the input files. Otherwise scaling is performed on the output. See details.
#' @param preset character, encoding presets available in FFmpeg. Defaults to \code{ultrafast}. See details.
#' @param codec character, the video codec used. See details.
#' @param format character, the pixel format. See details.
#' @param lossless logical, use lossless H.264 encoding if applicable. Defaults to \code{FALSE}. See details.
#' Set to zero if your image sequence has file names beginning from zero or a higher number if you want to skip frames.
#' @param min.rate integer, the minimum frame rate for non-\code{gif} video output (\code{mp4}, \code{mov}, \code{mkv}, \code{webm}). Defaults to \code{10}. See details.
#' @param fps.out integer or character, framerate of output video. This can be given in the same ways as \code{rate}. Defaults to \code{rate}. See details.
#' @param alpha numeric, from 0 to 1.0. Only applicable when \code{pattern} is vectorized, referring to layering of multiple image sequences.
#' Defaults to \code{1.0} (non-transparent) since \code{mapmate} produces transparent-background png sequences by default with subsequent layering in mind.
#' @param overwrite logical, overwrite existing output file.
#' @param glob logical, defaults to \code{FALSE}. Globbing is not available on Windows. Linux users, see details on how \code{glob} affects \code{pattern}.
#' @param details logical, whether to show FFmpeg output on the R console.
#'
#' @return returns the system call to FFmpeg as a character string.
#' @export
#'
#' @examples
#' \dontrun{
#' data(borders)
#' library(dplyr)
#' n <- 90
#' borders <- map(1:n, ~mutate(borders, id = .x)) %>% bind_rows()
#' args <- list(width=300, height=300, res=300, bg="black")
#' save_seq(borders, id="id", n.frames=n, col="white",
#'          type="maplines", file="images", png.args=args)
#' ffmpeg(pattern="images_%04d.png", output="video.mp4", rate=10)}
ffmpeg <- function(dir=".", pattern, output, output_dir=".", rate="ntsc", delay=1,
                   start=1, size="source", preset="ultrafast", codec="default",
                   format="yuv420p", lossless=FALSE, min.rate=10, fps.out=rate,
                   alpha=1.0, overwrite=FALSE, glob=FALSE, details=FALSE){
  if (!missing(rate) && !missing(delay)) stop("specify 'rate' or 'delay' but not both")
  if(!missing(delay)) rate <- round(1/delay)

  # input files
  linux <- .Platform$OS.type=="linux"
  iglob <- "-pattern_type glob -i "
  input <- file.path(dir, pattern)
  blend <- if(length(input)==1) FALSE else TRUE
  input <- if(linux & glob) paste0(iglob, "\"", input, "\"") else paste("-i", input)
  inrate <- paste("-framerate", rate)
  start <- paste("-start_number", start)
  input <- paste0(paste(start, inrate, input), collapse=" ")

  #output files
  ext <- strsplit(output, "\\.")[[1]]
  ext_stop <- "'output' must end in '.mp4', '.mov', '.mkv', '.webm', or '.gif'"
  if(length(ext)==1) stop(ext_stop) else ext <- utils::tail(ext, 1)
  if(!ext %in% c("mp4", "mov", "mkv", "webm", "gif")) stop(ext_stop)
  output <- file.path(output_dir, output)

  # video filter chain
  format <- paste0("format=", format)
  if(size == "source"){
    size <- ""
  } else if(ext != "gif"){
    size <- paste0(",scale=", size, ",setsar=1:1")
  } else size <- paste("-s", size)

  if(blend){
    blend_prefix <- "-filter_complex \"blend=all_mode='overlay':all_opacity="
    if(ext=="gif"){
      vf <- paste0(blend_prefix, alpha, "\"")
    } else {
      vf <- paste0(blend_prefix, alpha, ",", format, size, "\"")
    }
  } else if(ext=="gif"){
    vf <- size
  } else vf <- paste0("-vf ", "\"", format, size, "\"")

  output <- paste(vf, output)
  outrate <- paste("-r", max(fps.out, min.rate))
  output <- paste(outrate, output, ifelse(overwrite, "-y", "-n"))

  #video codec
  if(ext=="gif"){
    vc <- " "
  } else {
    if(codec=="default") codec <- switch(ext, mp4="libx264", mov="libx264", mkv="libx264", webm="libvpx")
    vc <- paste0(" -c:v ", codec, " -preset ", preset, " ")
    if(lossless & codec %in% c("h264", "libx264")) vc <- paste0(vc, "-qp 0 ")
  }

  x <- gsub("  ", " ", paste0("ffmpeg ", input, vc, output))
  if(details) system(x) else system(x, show.output.on.console=FALSE)
  x
}