backward: This function traverses the graph back and computes all the...

In krzjoa/aria: Extensible Autodifferentiation

Description

https://towardsdatascience.com/pytorch-autograd-understanding-the-heart-of-pytorchs-magic-2686cd94ec95 https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf http://www.cs.toronto.edu/~rgrosse/courses/csc421_2019/readings/L06 https://j-towns.github.io/2017/06/12/A-new-trick.html See: https://pytorch.org/docs/stable/autograd.html https://github.com/HIPS/autograd/blob/master/autograd/numpy/numpy_vjps.py

Usage

backward(ops, gradient)

Arguments

ops	tensor
gradient	TODO: potentially hard to handle: dimshuffles etc. TODO: transform into S4 method TODO: unify names ops vs ops_ptr TODO: flatten graph(?) We need to propagate information, if gradient is required or not

Examples

library(dlr)
ctx <- get_context()
register_ops(ctx, cars)
register_ops(ctx, data.frame)
x <- cpu_tensor(5, dims = 1)
y <- (x ** 3) / 2
bkw_fun <- get_inputs(y)[[1]]
get_paired_object(bkw_fun)
all.ops <- get_all_ops_ptr(ctx)
backward(y, 1)
TODO: sposób iteracji - obecnie idziemy jak najbardziej wgłąb
TODO: powstają jakieś niezamiezone kopie tensorów - gdzie i kiedy?

krzjoa/aria documentation built on Oct. 1, 2020, 12:48 p.m.