How a language model learns.

Gather and identify the source material that will form the training corpus.

• Books and articles
• Websites and documents
• Code repositories
• Licensed datasets
• Human-generated content
• Multilingual datasets
• Data source identification
• Data quality assessment

Improve dataset quality before the content is transformed into model-ready sequences.

• Data filtering
• Duplicate removal
• Noise removal
• HTML and formatting removal
• Text normalization
• Language identification
• Spam filtering
• Toxic content filtering
• Personally identifiable information filtering
• Data quality validation

Convert raw documents into token sequences using a tokenizer and vocabulary.

• Text segmentation
• Document splitting
• Sentence splitting
• Tokenization
• Vocabulary construction
• Token ID assignment
• Special token handling
• Training sequence preparation
• Sequence packing

Turn tokenized sequences into training examples, batches and data-loader inputs.

• Training dataset creation
• Training and validation splits
• Data sampling
• Data shuffling
• Batch creation
• Sequence length configuration
• Training example preparation
• Data loading pipeline

Choose the neural network structure and dimensions before initializing its parameters.

• Neural network architecture
• Transformer architecture
• Decoder-only architecture (GPT)
• Number of Transformer layers
• Attention heads
• Hidden dimensions
• Embedding dimensions
• Feed-forward network dimensions
• Context length
• Model parameter configuration

Create the initial parameter state from which optimization begins.

• Weight initialization
• Embedding initialization
• Attention weight initialization
• Feed-forward weight initialization
• Bias initialization (where applicable)
• Parameter configuration
• Initial model state

Train the model on large-scale text with a self-supervised next-token prediction objective.

• Self-supervised learning
• Next-token prediction objective
• Forward propagation
• Token prediction
• Loss calculation
• Cross-entropy loss
• Backpropagation
• Gradient calculation
• Gradient accumulation
• Weight updates
• Optimizers (Adam / AdamW)
• Learning-rate scheduling
• Training epochs and steps

Check whether the model is learning and generalizing beyond its training examples.

• Training loss monitoring
• Validation loss
• Perplexity
• Benchmark evaluation
• Language understanding evaluation
• Reasoning evaluation
• Code generation evaluation
• Generalization assessment
• Overfitting detection

Adapt a pretrained model to follow instructions using curated demonstrations.

• Instruction dataset preparation
• Question-answer datasets
• Human-written demonstrations
• Prompt-response pairs
• Instruction following
• Supervised learning
• Fine-tuning loss
• Parameter updates
• Model evaluation

Use human preferences or preference-derived objectives to improve response behavior.

• Human feedback collection
• Response comparison
• Preference dataset creation
• Reward model training
• Reward scoring
• Reinforcement learning
• PPO (Proximal Policy Optimization)
• Preference optimization
• DPO (Direct Preference Optimization)
• Response quality improvement

Teach and assess safety-related behavior, policy adherence and robustness.

• Safety dataset preparation
• Harmful request identification
• Refusal behavior training
• Bias evaluation
• Toxicity reduction
• Privacy protection
• Instruction hierarchy
• Policy alignment
• Red-team evaluation

Validate the post-trained model across intended capabilities and failure modes.

• Functional testing
• Accuracy evaluation
• Reasoning benchmarks
• Hallucination evaluation
• Safety testing
• Robustness testing
• Bias testing
• Adversarial testing
• Human evaluation

Reduce serving cost and improve memory use, latency and throughput while managing quality trade-offs.

• Quantization
• Pruning
• Knowledge distillation
• Memory optimization
• Inference optimization
• Parallel processing
• Latency optimization
• Throughput optimization

Package the model and make it available to inference infrastructure and applications.

• Model serialization
• Model checkpointing
• Model hosting
• GPU / accelerator deployment
• Inference server setup
• API integration
• Load balancing
• Monitoring
• Version management