Masked Language Model

Intermediate

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Full Definition

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Keywords

BERT-style

Domains

Foundations & Theory

Related Terms

Few-Shot Learning related to

Achieving task performance by providing a small number of examples inside the prompt without weight updates.

 Chain-of-Thought related to

Stepwise reasoning patterns that can improve multi-step tasks; often handled implicitly or summarized for safety/priv...

 RAG related to

Architecture that retrieves relevant documents (e.g., from a vector DB) and conditions generation on them to reduce h...

 Semantic Search related to

Retrieval based on embedding similarity rather than keyword overlap, capturing paraphrases and related concepts.

 Positional Encoding related to

Injects sequence order into Transformers, since attention alone is permutation-invariant.

 Tokenization related to

Converting text into discrete units (tokens) for modeling; subword tokenizers balance vocabulary size and coverage.

 Next-Token Prediction related to

Training objective where the model predicts the next token given previous tokens (causal modeling).

 Autoregressive Model related to

Generates sequences one token at a time, conditioning on past tokens.

Concept Map

See how Masked Language Model connects to other concepts.

Open Knowledge Graph