knitr::opts_chunk$set( collapse = TRUE, comment = "#>", eval = identical(Sys.getenv("TORCH_TEST", unset = "0"), "1"), purl = FALSE )
library(torch) torch_manual_seed(1) # setting seed for reproducibility
This vignette showcases the basic functionality of distributions in torch. Currently the distributions modules are considered 'work in progress' and are still experimental features in the torch package. You can see the progress in this link.
The distributions modules in torch are modelled after PyTorch's distributions module which in turn is based on the TensorFlow Distributions package.
This vignette is based in the TensorFlow's distributions tutorial.
Let's start and create a new instance of a normal distribution:
n <- distr_normal(loc = 0, scale = 1) n
We can draw samples from it with:
n$sample()
or, draw multiple samples:
n$sample(3)
We can evaluate the log probability of values:
n$log_prob(0) log(dnorm(0)) # equivalent R code
or, evaluate multiple log probabilities:
n$log_prob(c(0, 2, 4))
A distribution can take a tensor as it's parameters:
b <- distr_bernoulli(probs = torch_tensor(c(0.25, 0.5, 0.75))) b
This object represents 3 independent Bernoulli distributions, one for each element of the tensor.
We can sample a single observation:
b$sample()
or, a batch of n
observations:
b$sample(6)
The log_prob
method of distributions can be differentiated, thus, distributions
can be used to train models in torch.
Let's implement a Gaussian linear model, but first let's simulate some data
x <- torch_randn(100, 1) y <- 2*x + 1 + torch_randn(100, 1)
and plot:
plot(as.numeric(x), as.numeric(y))
We can now define our model:
GaussianLinear <- nn_module( initialize = function() { # this linear predictor will estimate the mean of the normal distribution self$linear <- nn_linear(1, 1) # this parameter will hold the estimate of the variability self$scale <- nn_parameter(torch_ones(1)) }, forward = function(x) { # we estimate the mean loc <- self$linear(x) # return a normal distribution distr_normal(loc, self$scale) } ) model <- GaussianLinear()
We can now train our model with:
opt <- optim_sgd(model$parameters, lr = 0.1) for (i in 1:100) { opt$zero_grad() d <- model(x) loss <- torch_mean(-d$log_prob(y)) loss$backward() opt$step() if (i %% 10 == 0) cat("iter: ", i, " loss: ", loss$item(), "\n") }
We can see the parameter estimates with:
model$parameters
and quickly compare with the glm()
function:
summary(glm(as.numeric(y) ~ as.numeric(x)))
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.