WarmUpAndCosineDecay: Warmup and cosine decay scheduler, the schedular used in the...

In gongx030/seatac: ATAC-seq V-plot analysis

Warmup and cosine decay scheduler, the schedular used in the SimCLR paper

Description

Adopted from https://github.com/google-research/simclr/blob/67b562524223e65c0414a15c3bd7ac007d28f629/tf2/model.py#L78

Usage

WarmUpAndCosineDecay(
  base_learning_rate = 0.3,
  num_examples = NULL,
  batch_size = 256L,
  warmup_epochs = 10,
  epochs = 100L
)

Arguments

base_learning_rate	Baseline learning rate (default: 0.3)
num_examples	Number of total training samples
batch_size	Batch size (default : 256)
warmup_epochs	Warmup epochs (default: 10)
epochs	Total training epochs

Value

a WarmUpAndCosineDecay object

gongx030/seatac documentation built on April 15, 2023, 5:53 a.m.