In MilesMcBain/r2vr: Compose and Serve WebVR Scenes

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

R objects for building A-Frame Scenes

The core of r2vr is 3 classes that map to A-Frame entities, assets, and scenes. A scene may contain many child entities which may themselves contain many child entities and assets. Entities may have many components added to define their behaviour.

Objects of these classes are defined using functions: a_entity(), a_asset(), a_scene(). Given a structure of these objects, r2vr does the work of: Rendering them as A-Frame HTML and combining into a selected HTML template. Collecting their assets and required javascript sources and placing them in the appropriate HTML sections. * Serving the HTML scene and asset files.

Entities

In the A-Frame entity-component architecture a scene is composed of entities. These are defined in HTML like:

<a-entity geometry="primitive: box" material="color: red"></a-entity>

Entities are customised using HTML element properties that map configuration to appearance and behaviour. A-Frame provides many 'convenience' tags to simplify the definition of commonly used entity configurations. For example the above entity can also be defined:

<a-box color="red">

r2vr respects these same definition types. It provides the added convenience of defining component configuration as R lists. The above can be defined in R any of:

a_entity(geometry = "primitive: box", material = "colour: red")
a_entity(geometry = list(primitve = "box"), material = list(color = "red"))
a_entity(.tag = "box", color = "red")

Notice that entity components are attached using ... arguments to the a_entity() function. An important convention regarding arguments to a_ functions is now worth discussing:

Arguments prefixed with a ., ie. .tag in the above example, are arguments that will not appear literally in the generated HTML. You can think of them as driving some 'under the hood' behaviour to do with the way the HTML is structured or served. This is discriminates them from arguments without . prefixes like geometry above. These are how you represent components. These will appear literally as attributes of HTML elements in the final output.

Component Naming

For components with multi-word-names in A-Frame, the convention is to separate with -, e.g. wasd-controls, when attaching these in r2vr you swap the - for _ since the dash is not legal as a bare symbol in R. So take this HTML:

<a-camera wasd-controls="fly: true;"></a-camera>

Possible r2vr equivalents are:

a_entity(.tag="camera", wasd_controls = list(fly = TRUE))
a_entity(.tag="camera", wasd_controls = "fly: true;")

One current issue is that you cannot supply an component without configuration neatly. So to use wasd-controls with default configuration we must write either:

a_entity(.tag="camera", wasd_controls="")
## or
a_entity(.tag="camera", wasd_controls=NULL)

To require otherwise would mean cracking open the Non-Standard Evaluation can of worms, a step not to be undertaken lightly.

Nesting entities

In A-Frame entities can have child entities nested within them. This is often extremely convenient to do since nested entities inherit the position and rotation of their parent. So in this HTML:

<a-box position="1 1 1", rotation="45 45 45" color="blue">
  <a-sphere position="1 0 0" color="green"></a-sphere>
</a-box>

The green sphere inherits position and rotation from its blue box parent. It's own position is interpreted as offset from its parent. So the sphere's absolute position is 2 1 1.

In r2vr child entities are nested as a list supplied to the parent's .children argument. So the above pair are defined:

a_entity(.tag = "box", position = c(1, 1, 1), rotation = c(45, 45, 45), 
         color="blue",
         .children = list(
           a_entity(.tag = "sphere", positon = c(2, 1, 1))
         ))

Because the nested a_entity call returns an object it's possible to break up the nesting a bit, if it aids readability:

sphere <- a_entity(.tag = "sphere", positon = c(2, 1, 1))

box <- a_entity(.tag = "box", position = c(1, 1, 1),
                rotation = c(45, 45, 45),
                children = list(sphere))

Attaching Custom Components

To use a custom component with an entity, a list of links to the component's javascript sources can be passed to a_entity using the .js_sources argument. For an example let's crack open the source of r2vr::a_json_model, a convenience entity for defining JSON models, which do not have native A-Frame support:

.extras_model_loader <- "https://cdn.rawgit.com/donmccurdy/aframe-extras/v4.0.2/dist/aframe-extras.loaders.js"


a_json_model <- function(src_asset, 
                         .js_sources = list(.extras_model_loader), ...){
  a_entity(json_model = list(src = src_asset), .js_sources = .js_sources, ...)
}

So every a_json_model() is not much more than an a_entity() with a json_model component attached along with a reference to the javascript that powers the component. You do not have do worry about duplicate .js_sources being added to a scene. r2vr ensures only one reference to each unique javascript file makes it into the final HTML.

Creating your own wrapper entities in this manner is also a way to simplify scene specification.

Scene

A scene is the outer container for all entities. It's also an entity itself. Its special powers are that it understands how to collate and serve a nested structure of entity and asset objects. So just like a regular entity a scene can have component configuration and nested children.

For example:

<a-scene fog stats>
  <a-box position="1 1 1", rotation="45 45 45" color="blue">
    <a-sphere position="1 0 0" color="green"></a-sphere>
  </a-box>
</a-scene>

is defined in r2vr as:

my_scene <- 
  a_scene(fog="", stats="",
          .children = list(
            a_entity(.tag = "box", position = c(1, 1, 1),
                     rotation = c(45, 45, 45),
                     .children = list(
                       a_entity(.tag = "sphere", positon = c(2, 1, 1))))))

Scene templates

In r2vr scenes work with a HTML templating system. This allows you to reduce the complexity of scenes in R, by creating a template for the static parts that are not likely to change, E.g. ground, sky, lighting etc. Templates are selected using the .template argument. Built-in templates at the moment are:

template name | description ---|--- "empty" | An empty scene. Will use default A-Frame lights and camera inserted by A-Frame. Defaults are removed if added to the scene configuration. "basic" | Has a grid ground added. Will use default A-Frame lights and camera. Defaults are removed if added to scene configuration. "basic_map" | Has a large grid ground, high point light source (A Sun) and high camera start position.V

A custom HTML file can be supplied as a .template. The following placeholders are populated by the scene object:

placeholder | function ---|--- ${title} | The title of the HTML page. ${description} | The meta description of the HTML page. ${js_sources} | Location in HTML <head> to place a list of <script> tags linking javascript sources files attached to scene children. ${assets} | Location to place a list of <a-asset-item> tags, one per line, generated from a_assets attached to scene children. ${entities} | Location to place a list of entity tags, one per line, indented as appropriate, generated from child entities. ${scene_compnents} | The space after the start of the <a-scene > tag to inject components attached to a_scene().

Serving and rendering scenes

A scene object can be called upon to render itself to HTML or serve itself to allow viewing in WebVR. Scenes are served using the Fiery webserver framework.

Asumming my_scene is a reference to a scene object returned by a_scene() then the following actions are possible:

call | effect ---|--- my_scene$serve(host = "127.0.0.1", port = 8080) | Serve the Scene HTML as the root, "/", at the supplied IP address and port. my_scene$stop() | Stop serving the scene. my_scene$render() | A test/validaton helper: Return a string containing the complete scene HTML. my_scene$write("file.html") | A test/validation helper: write the complete scene HTML to a file.

Serving Assests and Linking Javascript files

Assests and Javascript sources are attached as arguments to components on entities or the scene itself. The scene collects the id fields and sources of all assets, and the links to all JS files and performs a de-duplication before rendering them in HTML.

It creates routes in the server for all unique assets. All local assets must be below the working directory from which `serve()`` is called.

Assets

Assets are media like models, images, videos, sounds etc associated with A-Frame entities. There are two ways to define assets, depending on whether the scene rendering should be delayed until they are loaded or not.

Assets are created with the a_asset() function. The src argument references the asset file and can point to either a local file or a URI for a remote file. Only local files will be served by r2vr with a scene. The user's browser will fetch the remote files from their URIs.

Assets Required for the Scene to render

In A-Frame HTML Assets that should be fetched before the scene is allowed to render are declared in a special <a-assets> HTML block like this:

<a-scene >
            <a-assets>
                <a-asset-item id="cube" src="./cube.json"></a-asset-item>
                <a-asset-item id="kangaroo" src="./Kangaroo_01.gltf"></a-asset-item>
            </a-assets>
            ...

The id property is referenced in entity component configuration where the asset is to be used like these two entities that render 3D models:

 <a-entity json-model="src: #cube;" position="0 0 -2" scale="0.2 0.2 0"></a-entity>
 <a-gltf-model position="2 2 -3" src="#kangaroo"></a-gltf-model>

When using r2vr you don't have to worry about declaring assets in the a-assets block. That is taken care of automatically. The only concern is passing the r2vr asset object returned by a_asset() to the appropriate entity component, usually as src. For example:

cube_model <- a_asset(id = "cube", src = "./cube.json")

cube <- a_json_model(src = cube_model,
                     position = c(0, 0, -2),
                     scale = c(0.2, 0.2, 0.2))

my_scene <- a_scene(children = list(cube))

Passing cube_model to cube sets up the link between entity and asset in the underlying HTML using the id "#cube", as well as adding the "cube.json" file to the assets block.

Assets needed later

It is possible the you may wish to declare an asset that is not initially assigned in the src of any entity when the scene initially renders. The intention may be for this asset to be used in response to some event. Entities and scenes have a .assets argument for declaring assets of this type. See vignette 'Communicating between R and WebVR sessions' for an example.

Assets not required for the scene to render.

In general is recommended that all assets are loaded before the scene is rendered. This is because when an asset is loaded it can cause the scene to stutter which can contribute to VR sickness symptoms in users.

It is possible to define assets in inline mode that does not delay scene rendering. This can be useful when changing the asset (e.g. texture) associated with some entity in response to a user action. If that action is unlikely, than speculatively fetching the asset may be inappropriate.

Inline mode is just enabled with an argument to a_asset():

cube_model <- a_asset(id = "cube", src = "./cube.json", .inline = TRUE)

cube <- a_json_model(src = cube_model,
                     position = c(0, 0, -2),
                     scale = c(0.2, 0.2, 0.2))

my_scene <- a_scene(.children = list(cube))

Example Asset Usage

Models

To define a scene containing JSON model and a glTF model, you can pass the asset objects to entities with json model and gltf model components:

library(r2vr)
  ## Assest
  cube <- a_asset(id = "cube", src = "./cube.json")
  kangaroo <- a_asset(id = "kangaroo",
                      src = "./Kangaroo_01.gltf",
                      .parts = "./Kangaroo_01.bin")

  ## Scene structure
  my_scene <- a_scene(.template = "empty",
                      title = "A kangaroo and a cube.",
                      .children = list(
                        a_json_model(src = cube,
                                     position = c(-2,1,-5),
                                     scale = c(0.3, 0.3, 0.3)),

                        a_entity(.tag = "gltf-model",
                                 src = kangaroo,
                                 position = c(2, -1, -5),
                                 scale = c(0.3,0.3,0.3),
                                 rotation = c(0, 180, 0))
                          ))

that will allow you to serve HTML that looks like this:

<!DOCTYPE html>
<html>
    <head>
    <meta charset="utf-8">
    <title>A kangaroo and a cube.</title>
    <meta name="description" content= "A kangaroo and a cube.">
    <script crossorigin src="https://aframe.io/releases/0.8.0/aframe.min.js"></script>
    <script crossorigin src="https://cdn.rawgit.com/donmccurdy/aframe-extras/v4.0.2/dist/aframe-extras.loaders.js"></script>
    </head>
    <body>
        <a-scene >
            <a-assets>
                <a-asset-item id="cube" src="./cube.json"></a-asset-item>
                <a-asset-item id="kangaroo" src="./Kangaroo_01.gltf"></a-asset-item>
            </a-assets>

            <!-- Entities added in R -->
            <a-entity json-model="src: #cube;" position="-2 1 -5" scale="0.3 0.3 0.3"></a-entity>
            <a-gltf-model src="#kangaroo" position="2 -1 -5" scale="0.3 0.3 0.3" rotation="0 180 0"></a-gltf-model>


        </a-scene>
    </body>
</html>

And renders like this:

Texture Images

Adding images as textures is another useful application. Image assets use the plain HTML <img> tag instead of <a-asset-item>. Here's an example that builds a VR scene around an R plot image:

## libs
library(r2vr)
library(visdat)
library(ggplot2)

## making an image
## powers of 2 are good since they need less rescaling.
png("VRisdat.png", width = 512, height = 512)
p <- vis_dat(airquality)
p + theme_light(base_size = 13)
dev.off()

## creating a thematic background using 'environment'
## https://github.com/feiss/aframe-environment-component
backdrop <- a_entity(environment = list(preset= "tron", dressingAmount = 40,
                                        playArea = 0), 
                     .js_sources = "https://rawgit.com/feiss/aframe-environment-component/master/dist/aframe-environment-component.min.js")

## rendering the image on a canvas
visdat_plot <- a_asset(.tag = "img", id = "vrisdat", src = "VRisdat.png")

canvas <- a_entity(.tag = "plane", position = c(0, 10, -10), height = 20, width = 20,
          src = visdat_plot)

my_scene <- a_scene(.title = "A VR ggplot",
                    .template = "empty",
                    .children = list(backdrop, canvas))

my_scene$serve()

## my_scene$stop()

Which creates this scene:

Local Variables:

ess-r-package--project-cache: (r2vr . "/home/miles/repos/r2vr/")

End:

MilesMcBain/r2vr documentation built on March 29, 2021, 12:03 p.m.