In WinVector/cdata: Fluid Data Transformations

This is a quick re-work of the Keras record transform example in R. For a Python version please see here.

In the original article we had Keras model performance data, which looked like the following.

library(wrapr)
library(cdata)

df <- wrapr::build_frame(
  "val_loss"  , "val_acc", "loss" , "acc" , "epoch" |
    -0.377    , 0.8722   , -0.5067, 0.7852, 1       |
    -0.2997   , 0.8895   , -0.3002, 0.904 , 2       |
    -0.2964   , 0.8822   , -0.2166, 0.9303, 3       |
    -0.2779   , 0.8899   , -0.1739, 0.9428, 4       |
    -0.2843   , 0.8861   , -0.1411, 0.9545, 5       |
    -0.312    , 0.8817   , -0.1136, 0.9656, 6       )
knitr::kable(df)

But for plotting, it is more convenient to have the data in the following form:

| epoch | measure | training | validation | | ----: | :------------------------- | -------: | ---------: | | 1 | minus binary cross entropy | -0.5067 | -0.3770 | | 1 | accuracy | 0.7852 | 0.8722 | | ... |

The article uses ideas similar to these to visualize the desired record structure and then write down this visualization as a concrete data record example.

The principle is: if you have a visualization of the input and output, it is then trivial to marshal these into a graphical representation of the desired transform. And if you can't work out what the input and output look like, then you really are not quite ready to perform the transform. Knowing what we want is the minimum requirement and with this methodology it is also all that is needed.

shape <- wrapr::build_frame(
  "measure"                     , "training", "validation" |
    "minus binary cross entropy", "loss"    , "val_loss"   |
    "accuracy"                  , "acc"     , "val_acc"    )
knitr::kable(shape)

This description of the desired record shape is easily transformed into a data transformation specification.

xform <- rowrecs_to_blocks_spec(
  controlTable = shape,
  recordKeys = 'epoch') 

print(xform)

(Please see help("rowrecs_to_blocks_spec"), help("blocks_to_rowrecs_spec"), and help("layout_specification") for how to specify the various data transforms.)

We can easily apply this transform to our data.

res <- df %.>% xform

knitr::kable(res)

And it is simple to build an inverse transform.

inverse <- t(xform)

print(inverse)

And equally easy to apply this inverse transform to data.

res %.>% 
  inverse %.>% 
  knitr::kable(.)

Notice how each step can be inspected and checked as we worked. I would definitely recommend re-reading the original article with the new transform notation in mind. In any case, please check out the cdata package and documentation.

WinVector/cdata documentation built on Aug. 29, 2023, 3:56 a.m.