đź“š Common LLM Terminologies You Must Know
Token
A small piece of text processed individually (word, subword, character).
Tokenization
Splitting text into smaller pieces for model input.
Vocabulary
The full set of tokens a model knows and uses.
Embedding
Vector representation of words or tokens capturing their meaning.
Transformer
Architecture based on self-attention mechanisms for sequences.
Attention Mechanism
Focuses on important parts of the input during processing.
Self-Attention
Each word relates to every other word in a sentence for better understanding.
Prompt
The input text you give to an LLM to generate a response.
Prompt Engineering
Designing prompts to guide the model’s output effectively.
Fine-tuning
Further training a pretrained model on a domain-specific dataset.
Pretraining
Initial phase where models learn from huge general datasets.
Next-Token Prediction
Training task where models predict the next word/token.
Loss Function
Measures how wrong the model’s predictions are.
Perplexity
Metric indicating how well a model predicts text (lower = better).
Decoder
Part of the model that generates text output.
Encoder
Part of the model that processes input text.
Alignment
Training models to behave according to human values.
RLHF
Reinforcement Learning from Human Feedback to refine model behavior.
Hallucination
When a model generates plausible but false information.
Parameter
Internal variables (weights) that models learn and adjust.
Context Window
Maximum tokens a model can read/remember at once.
Zero-shot Learning
Solving tasks without seeing examples during training.
Few-shot Learning
Solving tasks after seeing just a few examples.
Chain-of-Thought Prompting
Prompting the model to explain its reasoning step-by-step.
RAG (Retrieval-Augmented Generation)
Combines retrieval with text generation to boost accuracy.
Multimodal Model
Models that handle text, images, audio, or video together.