model_bert_pretrained: Construct a Pretrained BERT Model

In macmillancontentscience/torchtransformers: Transformer Models in Torch

Construct a Pretrained BERT Model

Description

Construct a BERT model (using model_bert()) and load pretrained weights.

Usage

model_bert_pretrained(bert_type = "bert_tiny_uncased", redownload = FALSE)

Arguments

bert_type	Character; which flavor of BERT to use. See available_berts() for known models.
redownload	Logical; should the weights be downloaded fresh even if they're cached?

Value

The model with pretrained weights loaded.

Methods

initialize

Initialize this model. This method is called when the model is first created.

forward

Use this model. This method is called during training, and also during prediction. x is a list of torch::torch_tensor() values for token_ids and token_type_ids.

.get_tokenizer_metadata

Look up the tokenizer metadata for this model. This method is called automatically when luz_callback_bert_tokenize() validates that a dataset is tokenized properly for this model.

.load_weights

Load the pretrained weights for this model. This method is called automatically during initialization of this model.

macmillancontentscience/torchtransformers documentation built on Aug. 6, 2023, 5:35 a.m.