TextEmbeddingModel: Text embedding model
In aifeducation: Artificial Intelligence for Education

TextEmbeddingModel

R Documentation

Text embedding model

Description

This R6 class stores a text embedding model which can be used to tokenize, encode, decode, and embed raw texts. The object provides a unique interface for different text processing methods.

Value

Objects of class TextEmbeddingModel transform raw texts into numerical representations which can be used for downstream tasks. For this aim objects of this class allow to tokenize raw texts, to encode tokens to sequences of integers, and to decode sequences of integers back to tokens.

Super classes

aifeducation::AIFEMaster -> aifeducation::AIFEBaseModel -> TextEmbeddingModel

Public fields

BaseModel: ('BaseModelCore')
Object of class BaseModelCore.

Methods

Public methods

TextEmbeddingModel$configure()
TextEmbeddingModel$load_from_disk()
TextEmbeddingModel$save()
TextEmbeddingModel$encode()
TextEmbeddingModel$decode()
TextEmbeddingModel$embed()
TextEmbeddingModel$embed_large()
TextEmbeddingModel$get_n_features()
TextEmbeddingModel$get_pad_value()
TextEmbeddingModel$set_publication_info()
TextEmbeddingModel$get_sustainability_data()
TextEmbeddingModel$estimate_sustainability_inference_embed()
TextEmbeddingModel$clone()

Inherited methods

Method `configure()`

Method for creating a new text embedding model

Usage

TextEmbeddingModel$configure(
  model_name = NULL,
  model_label = NULL,
  model_language = NULL,
  max_length = 512L,
  chunks = 2L,
  overlap = 0L,
  emb_layer_min = 1L,
  emb_layer_max = 2L,
  emb_pool_type = "Average",
  pad_value = -100L,
  base_model = NULL
)

Arguments

model_name: string Name of the new model. Please refer to common name conventions. Free text can be used with parameter label. If set to NULL a unique ID is generated automatically. Allowed values: any
model_label: string Label for the new model. Here you can use free text. Allowed values: any
model_language: string Languages that the models can work with. Allowed values: any
max_length: int Maximal number of token per chunks. Must be equal or lower as the maximal postional embeddings for the model. Allowed values: 20 <= x
chunks: int Maximal number chunks. Allowed values: 2 <= x
overlap: int Number of tokens from the previous chunk that should be added at the beginng of the next chunk. Allowed values: 0 <= x
emb_layer_min: int Minimal layer from which the embeddings should be calculated. Allowed values: 1 <= x
emb_layer_max: int Maximal layer from which the embeddings should be calculated. Allowed values: 1 <= x
emb_pool_type: string Method to summarize the embedding of single tokens into a text embedding. In the case of 'CLS' all cls-tokens between emb_layer_min and emb_layer_max are averaged. In the case of 'Average' the embeddings of all tokens are averaged. Please note that BaseModelFunnel allows only 'CLS'. Allowed values: 'CLS', 'Average'
pad_value: int Value indicating padding. This value should no be in the range of regluar values for computations. Thus it is not recommended to chance this value. Default is -100. Allowed values: x <= -1
base_model: BaseModelCore BaseModels for processing raw texts.
trace: bool TRUE if information about the estimation phase should be printed to the console.

Returns

Does nothing return.

Method `load_from_disk()`

Loads an object from disk and updates the object to the current version of the package.

Usage

TextEmbeddingModel$load_from_disk(dir_path)

Arguments

dir_path: Path where the object set is stored.

Returns

Function does nothin return. It loads an object from disk.

Method `save()`

Method for saving a model on disk.

Usage

TextEmbeddingModel$save(dir_path, folder_name)

Arguments

dir_path: Path to the directory where to save the object.
folder_name: string Name of the folder where the model should be saved. Allowed values: any

Returns

Function does nothing return. It is used to save an object on disk.

Method `encode()`

Method for encoding words of raw texts into integers.

Usage

TextEmbeddingModel$encode(
  raw_text,
  token_encodings_only = FALSE,
  token_to_int = TRUE,
  trace = FALSE
)

Arguments

raw_text

vector Raw text.

token_encodings_only

bool

TRUE: Returns a list containg only the tokens.
FALSE: Returns a list containg a list for the tokens, the number of chunks, and the number potential number of chunks for each document/text.

token_to_int

bool

TRUE: Returns the tokens as int index.
FALSE: Returns the tokens as strings.

trace

bool TRUE if information about the estimation phase should be printed to the console.

Returns

list containing the integer or token sequences of the raw texts with special tokens.

Method `decode()`

Method for decoding a sequence of integers into tokens

Usage

TextEmbeddingModel$decode(int_seqence, to_token = FALSE)

Arguments

int_seqence

list list of integer sequence that should be converted to tokens.

to_token

bool

FALSE: Transforms the integers to plain text.
TRUE: Transforms the integers to a sequence of tokens.

Returns

list of token sequences

Method `embed()`

Method for creating text embeddings from raw texts. This method should only be used if a small number of texts should be transformed into text embeddings. For a large number of texts please use the method embed_large.

Usage

TextEmbeddingModel$embed(
  raw_text = NULL,
  doc_id = NULL,
  batch_size = 8L,
  trace = FALSE,
  return_large_dataset = FALSE
)

Arguments

raw_text: vector Raw text.
doc_id: vector Id for every text.
batch_size: int Size of the batches for training. Allowed values: 1 <= x
trace: bool TRUE if information about the estimation phase should be printed to the console.
return_large_dataset: bool If TRUE a LargeDataSetForTextEmbeddings is returned. If FALSE an object if class EmbeddedText is returned.

Returns

Method returns an object of class EmbeddedText or LargeDataSetForTextEmbeddings. This object contains the embeddings as a data.frame and information about the model creating the embeddings.

Method `embed_large()`

Method for creating text embeddings from raw texts.

Usage

TextEmbeddingModel$embed_large(
  text_dataset,
  batch_size = 32L,
  trace = FALSE,
  log_file = NULL,
  log_write_interval = 2L
)

Arguments

text_dataset: LargeDataSetForText LargeDataSetForText Object storing textual data.
batch_size: int Size of the batches for training. Allowed values: 1 <= x
trace: bool TRUE if information about the estimation phase should be printed to the console.
log_file: string Path to the file where the log files should be saved. If no logging is desired set this argument to NULL. Allowed values: any
log_write_interval: int Time in seconds determining the interval in which the logger should try to update the log files. Only relevant if log_dir is not NULL. Allowed values: 1 <= x

Returns

Method returns an object of class LargeDataSetForTextEmbeddings.

Method `get_n_features()`

Method for requesting the number of features.

Usage

TextEmbeddingModel$get_n_features()

Returns

Returns a double which represents the number of features. This number represents the hidden size of the embeddings for every chunk or time.

Method `get_pad_value()`

Value for indicating padding.

Usage

TextEmbeddingModel$get_pad_value()

Returns

Returns an int describing the value used for padding.

Method `set_publication_info()`

Method for setting the bibliographic information of the model.

Usage

TextEmbeddingModel$set_publication_info(type, authors, citation, url = NULL)

Arguments

type: string Type of information which should be changed/added. developer, and modifier are possible.
authors: List of people.
citation: string Citation in free text.
url: string Corresponding URL if applicable.

Returns

Function does not return a value. It is used to set the private members for publication information of the model.

Method `get_sustainability_data()`

Method for requesting a summary of tracked energy consumption during training and an estimate of the resulting CO2 equivalents in kg.

Usage

TextEmbeddingModel$get_sustainability_data(track_mode = "training")

Arguments

track_mode: string Determines the stept to which the data refer. Allowed values: 'training', 'inference'

Returns

Returns a list containing the tracked energy consumption, CO2 equivalents in kg, information on the tracker used, and technical information on the training infrastructure.

Method `estimate_sustainability_inference_embed()`

Calculates the energy consumption for inference of the given task.

Usage

TextEmbeddingModel$estimate_sustainability_inference_embed(
  text_dataset = NULL,
  batch_size = 32L,
  sustain_iso_code = NULL,
  sustain_region = NULL,
  sustain_interval = 10L,
  sustain_log_level = "warning",
  trace = TRUE
)

Arguments

text_dataset: LargeDataSetForText LargeDataSetForText Object storing textual data.
batch_size: int Size of the batches for training. Allowed values: 1 <= x
sustain_iso_code: string ISO code (Alpha-3-Code) for the country. This variable must be set if sustainability should be tracked. A list can be found on Wikipedia: https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes. Allowed values: any
sustain_region: string Region within a country. Only available for USA and Canada See the documentation of codecarbon for more information. https://mlco2.github.io/codecarbon/parameters.html Allowed values: any
sustain_interval: int Interval in seconds for measuring power usage. Allowed values: 1 <= x
sustain_log_level: string Level for printing information to the console. Allowed values: 'debug', 'info', 'warning', 'error', 'critical'
trace: bool TRUE if information about the estimation phase should be printed to the console.

Returns

Returns nothing. Method saves the statistics internally. The statistics can be accessed with the method get_sustainability_data("inference")

Method `clone()`

The objects of this class are cloneable with this method.

Usage

TextEmbeddingModel$clone(deep = FALSE)

Arguments

deep: Whether to make a deep clone.

aifeducation Artificial Intelligence for Education

TextEmbeddingModel: Text embedding model In aifeducation: Artificial Intelligence for Education