looming_animation: Create a looming animation

In nicholascarey/loomeR: Looming Animations for Use in Behavioural and Neurological Experiments

Create a looming animation

Description

looming_animation creates a movie file (.mp4) of a circle increasing in size to simulate an object (e.g. an attacking predator) coming towards a target. The function input must be an object created in either constant_speed_model, variable_speed_model, or diameter_model which is where the parameters determining the size and speed of the simulation are entered. It requires ffmpeg (http://ffmpeg.org), an external, cross-platform, command line utility for encoding video, to be installed on your system.

Usage

looming_animation(
  x,
  correction = 0.0285,
  width = 1280,
  height = 1024,
  fill = "black",
  background = "white",
  pad = NULL,
  pad_blank = FALSE,
  dots = FALSE,
  dots_interval = 20,
  dots_colour = "grey",
  dots_position = "br",
  dots_size = 0.005,
  dots_inset = 0.05,
  frame_number = FALSE,
  frame_number_tag = NULL,
  frame_number_colour = "grey",
  frame_number_position = "tr",
  frame_number_size = 2,
  frame_number_rotation = 0,
  frame_number_inset = 0.05,
  start_marker = TRUE,
  start_marker_colour = "black",
  start_marker_size = 2,
  loop = 1,
  pause = 0,
  save_data = FALSE,
  save_images = FALSE,
  filename = "animation"
)

Arguments

x	list. A list object of class constant_speed_model, variable_speed_model, or diameter_model.
correction	numeric. Correction factor for the display used to play the animation. Default = 0.0285. Typically falls between 0.01-0.06. Exact value can be determined using looming_animation_calib
width	integer. Width resolution of the display. E.g. for a display set at 1080p resolution (1920x1080), this is width = 1920. Note: this is NOT the native resolution, but the display resolution as set in the operating system settings. Visit https://whatismyscreenresolution.com on the playback display to check.
height	integer. Height resolution of the display. E.g. for a display set at 1080p resolution (1920x1080), this is height = 1080. Note: this is NOT the native resolution, but the display resolution as set in the operating system settings. Visit https://whatismyscreenresolution.com on the playback display to check.
fill	string. Colour of the circle.
background	string. Colour of the background.
pad	numeric. Duration in seconds to pad the start of the video. This replicates the first frame of the animation the required number of times to create this duration. Essentially, it makes the animation static for pad number of seconds before it starts playing (but see pad_blank).
pad_blank	logical. Optionally pad with blank frames rather than the first animation frame.
dots	logical. Controls if frame tracking dots are added to animation frames (see Details).
dots_interval	numeric. Interval in frames from first animation frame when dots are added.
dots_colour	string. Colour of dots
dots_position	string. Corner in which to display dots: tr, tl, br, bl, indicating top right, top left, bottom right, or bottom left.
dots_size	numeric. Size of added dots. Default = 0.005.
dots_inset	numeric. Distance of dots from edges of frame as proportion of entire width/height. Default = 0.05.
frame_number	logical. Controls if frame numbers are added to animation frames (see Details).
frame_number_tag	string. Text string to be added to the start of the frame numbers, e.g. '1-' or 'A-'. Useful for distinguishing between different animations.
frame_number_colour	string. Colour of frame numbers
frame_number_position	string. Corner in which to display frame numbers: tr, tl, br, bl, indicating top right, top left, bottom right, or bottom left.
frame_number_size	numeric. Size of frame numbers as proportion of default plotting text size. Default = 2.
frame_number_rotation	numeric. Value in degrees (0-360) to rotate frame numbers.
frame_number_inset	numeric. Distance of frame numbers from edges of frame as proportion of entire width/height. Increase this if frame numbers start to run off the edge of the screen as they get larger. Default = 0.05.
start_marker	logical. Controls if a marker ("X") is added to the first frame. This is a visual aid: if it disappears it indicates the video is playing.
start_marker_colour	logical. Colour of the start_marker
start_marker_size	logical. Size of the start_marker as proportion of default plotting text size. Default = 2.
loop	integer. Default = 1. This sets the number of times to repeat the animation. Any value of 2 or higher will mean an additional video file is created called animation_loop.mp4 with the original animation (including any padding) repeated this number of times.
pause	numeric. Default = 0. For slow PCs a delay can be added to ensure the filesystem is not overloaded by having to write so many image files. This adds a delay (in seconds) to the generation of each frame. E.g. for a 600 frame video, 'pause = 0.05' will add an additional 30 seconds (600 * 0.05) to the total time taken.
save_data	logical. If =TRUE, exports to the current working directory a .csv file containing the data used to make the animation, including the column of values scaled using the correction value. File name: ANIM_from_**name of R object used**_**frame rate**_**display resolution**.csv
save_images	logical. If =TRUE, does not delete image files after the animation video is created.
filename	string. Desired name of the output file without file extension. Default is animation, and output file is animation.mp4. Also used for looped video output, if the loop operator is used.

Details

IMPORTANT: The function works by saving an image file (loom_img_0000001.png etc.) for every frame of the animation to the current working directory. It then uses ffmpeg to encode these images to an .mp4 file. By default, this is saved as animation.mp4, however the filename operator can be used to specify a custom file name. It then deletes (actually deletes, not just sent to the trash) the .png files from the working directory. The function will overwrite any .png or .mp4 file it encounters which has an identical name. It's recommended you create a new directory (i.e. folder) for each animation, and use setwd() to set this as the current working directory before running the function. If you want to save an animation, you should move or rename it, or change the filename input before running the function again or it will get overwritten. It has not been rigorously tested on older systems with slow read-write speeds to the hard drive, which may cause unknown problems (but see the pause operator, which allows the function to be slowed down) Please provide feedback if you encounter any issues.

The function is capable of controlling precise details of how the object is displayed on screen. For example the correction operator ensures the hypothesised size in the model is displayed at that exact size on a specific display at a specific resolution. These details are important in experiments where the time in the animation at which an escape response occurs (and hence the perceived visual angle (alpha), distance and/or speed of the attacker) are of interest. If you simply want to generate an animation to elicit a response and are unconcerned with these details, you can ignore most of these options. For simple models where these details are unimportant see diameter_model.

Note animations will typically end with a filled screen, because even small objects at viewing distances approaching zero approach infinite perceived size (the exception is if you have used diameter_model and set the end diameter to less than the size of your screen). The final animation may have several completely filled frames at the end, depending on the attacker_diameter as set in constant_speed_model or variable_speed_model. Obviously, an object above a certain diameter cannot be displayed on a screen smaller than that diameter. If your attacker is larger in diameter than the screen, the final stages of the animation cannot physically display the actual perceived size as it gets close to the subject. In these cases, the animation essentially continues past the point where it can display the entirety of the simulation, and shows a filled screen (of the chosen fill colour) for however many frames are left in the simulation beyond that point. In most cases, if using biologically realistic parameters, this will be only a few frames.

Screen display and playback considerations

The output video is of a circle increasing in diameter over time, as specified in the $model$diam_on_screen component of the original constant_speed_model, variable_speed_model or diameter_model object. The function creates a video at the frame_rate specified in this object. The frame rate should be one the playback software handles correctly. Most modern displays have a maximum refresh rate of 60 Hz, so videos at frame rates higher than this may not be displayed correctly. I recommend using either 30 or 60 frames per second (Hz) which is a frame rate most video playback software should honour without problems. (I would be very interested to hear user experiences of using displays which support higher frame rates than this, such as recent iPad Pros with 120Hz refresh rates).

The display resolution of the screen you will use to play the animation should be entered as width and height. NOTE - This is the current DISPLAY resolution, which is not necessarily the native resolution of the screen, but determined in the Displays preferences of your operating system. If you are unsure, visit https://whatismyscreenresolution.com on the device. These settings ensure the animation is in the correct aspect ratio and uses the full screen (although you are free to modify the aspect ratio if, for example, you want your animation to be square). Other resolution values should still produce the correct widths onscreen, however I cannot guarantee all playback software will honour this, so best to follow the above guidelines if these details are important in your experiment.

An object of a hypothetical size may be displayed at a different size on a different screen, due to differences in resolution or the physical size of the pixels that make up the screen. The correction operator is intended to be a display-specific correction factor to ensure the actual, physical size of the circle matches the diameters in the $model$diam_on_screen component of the model. This value can be determined using the looming_animation_calib function. See the documentation for this function for instructions on its use. If creating different animations, the correction value will be the same for a particular screen as long as the display resolution remains the same.

Animation options

The circle colour and background can be specified using fill and background with standard base-R colour syntax as used in graphics functions such as plot() etc.

In your experiment you may want to identify the particular frame of the animation at which an event such as an escape response occurs. There are two ways of marking the animation so the frame can be identified in video recordings of the experiment:

Frame numbers

The animation frame number can be placed in every frame using frame_number = TRUE. The colour, size, inset distance, and corner of the screen to place the number can be specified (see Arguments), and the orientation can be set with frame_number_rotation. In addition, text can be added to the start of the frame numbers using frame_number_tag. For example, with frame_number_tag = "A-", frame numbers will follow the structure A-1, A-2, etc. This is useful if you create multiple animations and want to avoid getting them mixed up.

.

Frame numbers are also added to frames added for padding, although these follow their own numeric sequence and are also appended with "P" (see next section). Animation frame numbers will always refer to the correct row of the original model regardless of padding.

Dots

If dots = TRUE, starting from the first animation frame a small dot is placed in the corner of the frame at the frame interval specified with dots_interval. It is also placed in the last frame, regardless of the dots_interval. Again, colour, size, inset distance and corner can be specified (see Arguments).

These markers are only added to animation frames, not to frames added for padding (see next section). Use frame numbers if you need padding frames marked in some way.

Start Marker

By default (start_marker = TRUE), a marker (an "X") is placed at the bottom centre of the screen in the first frame of the video, regardless of padding. This is a visual aid to allow you to see when the video has started to play: if it is not there, then the video has started playing. This is especially useful for videos that have been padded, and those without frame markers, as there may be no other indication the video is playing. The colour and size of the marker can be specified with start_marker_colour and start_marker_size.

Padding animations to a required duration

The video file can be extended to a required total duration using the pad option. This value is a duration in seconds added to the start of the animation before playback. It duplicates the starting frame of the animation the required number of times to achieve the padding duration. Essentially, this makes the animation static for pad seconds before it starts to play. This means the video may show a static circle until the animation starts. If you do not want this, either modify the model parameters until the initial diameter is negligible, or use the pad_blank = TRUE option, in which case blank frames will be added rather than duplicating the starting frame. Under this option, after pad seconds of blank screen, the animation will suddenly appear and play. Again, how noticable this is depends on the model parameters you have used. Note that frame numbers are also added to padding frames. These follow their own numeric sequence and are also appended with "P". Other tracking markers (i.e. dots) will only be added to animation frames, not to frames added for padding.

NOTE: Be careful with padding options. Padding the video with extra frames will increase the time it takes for the function to run. Finalise your animation options and parameters before padding it. If you need a long duration video, be aware that every frame png file generated is around 40-50KB. Adding 10 minutes (600s) of padding to a video at 60 Hz will mean 600*60 = 36000 extra frames generated. At ~45KB each this would require ~1.6GB of hard drive space, and require considerable time to process! Due to compression, the resulting video file should however be only a few MB in size.

Looping animations

If you want the animation to loop or repeat a set number of times, use the loop = operator. If set with any value greater than the default of 1 this lets you output an additional video of the animation looped the set number of times. In this case there will be two output videos, the original animation.mp4 (or whatever is set with the filename input) plus a looped version appended with _loop.mp4. This process is extremely fast, since it does not recreate the animation for each loop, but just copies the animation.mp4 file the set number of times into a new video file.

Compatibility

The function should work with both Windows and macOS (Linux coming soon), however it requires ffmpeg (http://ffmpeg.org), an external, cross-platform, command line utility for encoding video, to be installed on your system. For installation instructions see http://adaptivesamples.com/how-to-install-ffmpeg-on-windows/ (may need to restart) or https://github.com/fluent-ffmpeg/node-fluent-ffmpeg/wiki/Installing-ffmpeg-on-Mac-OS-oX

On Windows, if you encounter an error after installation (e.g. unable to start png() device), try setting the working directory with setwd() to the current or desired folder. It has not been extensively tested on Windows, so please provide feedback on any other issues you encounter.

Playback in experiments

For triggered playback of the animation on a Mac I recommend Apple Quicktime Player. Playback can be started with the spacebar, the arrow keys allow frame-by-frame movement through the video, and Cmd-Left Arrow makes the video rewind to the start. Other applications such as VLC have quirks, for example automatically playing the video on opening the file, and closing it at the end of the video. However, find the application that works best for your purposes. If you need to loop the video, many video applications have an option for that. However, the loop operator (set with any value greater than 1) also lets you output an additional video of the animation looped the set number of times. In this case there will be two output videos, the original animation.mp4 plus a looped version called animation_loop.mp4.

As a check, it's a good idea to ensure the application you use is correctly identifying the metadata of the video. This depends on the software, but in should be similar to the following: open the video file, pause it, make it fullscreen, and then:

In Quicktime, Cmd-I or Window > Show Movie Inspector. Check 'Format' matches 'Current Size', and that both match your entered screen resolution width and height. Check 'FPS' matches the frame_rate used to create the model in constant_speed_model.

In VLC, Cmd-I or Window > Media Information, the 'Codec Details' tab. Check 'Resolution' and 'Display Resolution' both match your entered screen resolution width and height (there may be small differences, which is ok). Check 'Frame Rate' matches the frame_rate used to create the model in constant_speed_model. Make sure playback speed is at 'Normal' (Menu>Playback).

Cleaning up

The package has been coded to clean up after itself. However, with heavy use occasionally orphaned png and mp4 files may remain in the various working directories, so it's a good idea to go through these periodically and manually delete unwanted files.

Dependencies

The function requires the following packages: glue, plotrix, stringr

Author(s)

Nicholas Carey - [e-mail]

See Also

constant_speed_model, variable_speed_model, diameter_model, looming_animation_calib.

Examples

# uncomment to run
# make a looming model
# loom_model <- constant_speed_model(
#                      screen_distance = 20,
#                      frame_rate = 60,
#                      speed = 500,
#                      attacker_diameter = 50,
#                      start_distance = 1000)

# use it to create an animation with frame numbers in top right corner
# looming_animation(loom_model,
#                   correction = 0.0285,
#                   width = 1920,
#                   height = 1080,
#                   frame_number = TRUE,
#                   frame_number_position = "tr",
#                   frame_number_size = 2)

# pad animation with 5 seconds of the starting frame before playback
# looming_animation(loom_model,
#                    correction = 0.0285,
#                    width = 1920,
#                    height = 1080,
#                    frame_number = TRUE,
#                    frame_number_position = "tr",
#                    frame_number_size = 2,
#                    pad = 5)

nicholascarey/loomeR documentation built on Jan. 13, 2023, 11:31 a.m.