weight_decay: Weight_decay

Description Usage Arguments Value Examples

View source: R/optimizers.R

Description

Weight decay as decaying 'p' with 'lr*wd'

Usage

1
weight_decay(p, lr, wd, do_wd = TRUE, ...)

Arguments

p

p

lr

learning rate

wd

weight decay

do_wd

do_wd

...

additional args to pass

Value

None

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
## Not run: 

tst_param = function(val, grad = NULL) {
  "Create a tensor with `val` and a gradient of `grad` for testing"
  res = tensor(val) %>% float()

  if(is.null(grad)) {
    grad = tensor(val / 10)
  } else {
    grad = tensor(grad)
  }

  res$grad = grad %>% float()
  res
}
p = tst_param(1., 0.1)
weight_decay(p, 1., 0.1)


## End(Not run)

fastai documentation built on July 28, 2021, 5:06 p.m.