Results for "intra-sequence relations"

Attention where queries/keys/values come from the same sequence, enabling token-to-token interactions.

Injects sequence order into Transformers, since attention alone is permutation-invariant.

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Predicting protein 3D structure from sequence.