multihead_att: Multi-Head Attention
In quickcode: Quick and Essential 'R' Tricks for Better Scripts

multihead_att

R Documentation

Multi-Head Attention

Description

This function implements a simplified version of multi-head attention for sequence data.

Usage

multihead_att(query, key, value, num_heads)

Arguments

`query`	A numeric matrix representing the query.
`key`	A numeric matrix representing the key.
`value`	A numeric matrix representing the value.
`num_heads`	An integer specifying the number of attention heads.

Value

A numeric matrix representing the attention output.

Examples

query <- matrix(rnorm(12), nrow = 3)
key <- matrix(rnorm(12), nrow = 3)
value <- matrix(rnorm(12), nrow = 3)
multihead_att(query, key, value, num_heads = 2)

quickcode documentation built on April 11, 2025, 5:49 p.m.