🧠 Large Language Models (LLMs)
🌟 Start With a Real-World Scenario
You: "Write a Python function to calculate fibonacci numbers"
ChatGPT: (Writes perfect code in 2 seconds) ✅
You: "Now explain it like I'm 5"
ChatGPT: (Explains with simple analogies) ✅
You: "Translate it to Spanish"
ChatGPT: (Translates perfectly) ✅
You: "Write a poem about it"
ChatGPT: (Writes a creative poem) ✅
Wait... how does ONE model do ALL of this? 🤯
- Write code in 50+ languages 💻
- Translate between 100+ languages 🌍
- Explain complex topics simply 📚
- Write creative content 🎨
- Answer questions about anything 🧠
- Debug code and fix errors 🔧
- Summarize long documents 📄
- Generate ideas and brainstorm 💡
The answer: Large Language Models (LLMs)!
The Journey So Far:
You've learned the building blocks:
- Tokenization - How AI reads text
- Vectorization - How AI understands meaning
- Attention - How AI focuses on relevant information
- Transformer - The architecture that processes everything
- Self-Supervised Learning - How AI learns from unlabeled data
Now, put them ALL together = Large Language Model! 🎉
What Makes an LLM "Large"?
GPT-2 (2019):
- 1.5 billion parameters
- Trained on 40GB of text
- Could write coherent paragraphs
- Cost: ~$50,000 to train
GPT-3 (2020):
- 175 billion parameters (117x larger!)
- Trained on 45TB of text
- Could write essays, code, poems
- Cost: ~$4.6 million to train
GPT-4 (2023):
- ~1.7 trillion parameters (rumored)
- Trained on unknown amount (massive!)
- Can understand images, write complex code, reason
- Cost: ~$100 million to train
The pattern: Bigger = Smarter! 📈
Real-World Impact:
Before LLMs (2019):
- Chatbots: Scripted, limited responses
- Translation: Okay-ish, often wrong
- Code generation: Didn't exist
- Content writing: Humans only
- Customer support: Expensive human agents
After LLMs (2023-2024):
- ChatGPT: 100M users in 2 months (fastest-growing app ever!)
- GitHub Copilot: Writes 40% of code for developers
- Jasper AI: $125M revenue helping businesses write content
- Customer support: 80% automated with AI
- Education: Personalized tutoring for every student
Industries transformed:
- 💼 Business: Automated customer service, content creation
- 💻 Tech: AI pair programming, code review
- 📚 Education: Personalized tutoring, instant feedback
- 🏥 Healthcare: Medical documentation, patient communication
- ⚖️ Legal: Contract analysis, legal research
- 🎨 Creative: Writing, art, music generation
The "Aha!" Moments:
Emergence: As models get larger, they suddenly gain new abilities!
GPT-2 (1.5B params):
- ❌ Can't do math
- ❌ Can't write code
- ❌ Can't reason logically
GPT-3 (175B params):
- ✅ Can do basic math
- ✅ Can write simple code
- ✅ Can reason somewhat
GPT-4 (1.7T params):
- ✅ Can do complex math
- ✅ Can write production code
- ✅ Can reason like a human
- ✅ Can understand images
- ✅ Can pass professional exams
Nobody programmed these abilities! They just emerged from scale! 🤯
How LLMs Actually Work:
Training (One-time, expensive):
1. Collect massive text data (entire internet)
2. Train transformer to predict next word
3. Repeat trillions of times
4. Result: Model that "understands" language
Inference (Every time you use it):
You: "Write a haiku about AI"
↓
Tokenization: ["Write", "a", "haiku", "about", "AI"]
↓
Vectorization: Convert to numbers
↓
Transformer: Process with attention
↓
Generate: Predict next tokens one by one
↓
Output: "Silicon minds think / Patterns in data emerge / Intelligence blooms"
By the end of this guide, you'll understand:
- How LLMs combine all previous concepts (tokenization → transformers)
- Why bigger models are smarter (scaling laws)
- How ChatGPT generates responses token by token
- The difference between GPT (generative) and BERT (understanding)
- How to use LLMs via APIs (OpenAI, Anthropic, Google)
- Why LLMs sometimes "hallucinate" (make up facts)
- How to build applications with LLMs
The revolution: LLMs are the first AI that can do almost anything with language! They're not perfect, but they're good enough to transform every industry! 🚀
Fun facts:
- GPT-3 has more "neurons" than a human brain (175B vs 86B)!
- Training GPT-3 used enough electricity to power 120 US homes for a year!
- ChatGPT reached 1M users in 5 days (Netflix took 3.5 years!)
- GPT-4 can pass the bar exam (90th percentile)!
Let's dive in! 🚀
🎨 Visual Understanding
LLM Architecture Overview
graph TB
Input["User Input:<br/>'Write a poem about AI'"] --> Token[Tokenization]
Token --> Embed[Token Embeddings<br/>+ Positional Encoding]
Embed --> Layer1[Transformer Layer 1<br/>Multi-Head Attention + FFN]
Layer1 --> Layer2[Transformer Layer 2]
Layer2 --> Dots[...]
Dots --> LayerN[Transformer Layer N<br/>(96 layers in GPT-3)]
LayerN --> Logits[Output Logits<br/>Probability for each token]
Logits --> Sample[Sample Next Token<br/>Using temperature]
Sample --> Output["Generated Token:<br/>'Silicon'"]
Output --> Loop["Append & Repeat<br/>Until complete"]
style Input fill:#FF9800,color:#fff
style LayerN fill:#2196F3,color:#fff
style Output fill:#4CAF50,color:#fff
Model Size Comparison
graph LR
subgraph "Small Models (< 1B)"
S1[DistilBERT<br/>66M params] --> S2[BERT-Base<br/>110M params]
S2 --> S3[GPT-2 Small<br/>117M params]
end
subgraph "Medium Models (1B-10B)"
M1[GPT-2 Medium<br/>345M] --> M2[GPT-2 Large<br/>774M]
M2 --> M3[GPT-2 XL<br/>1.5B]
M3 --> M4[LLaMA 7B]
end
subgraph "Large Models (10B-100B)"
L1[GPT-3 Ada<br/>13B] --> L2[GPT-3 Babbage<br/>27B]
L2 --> L3[LLaMA 70B]
end
subgraph "Massive Models (> 100B)"
XL1[GPT-3<br/>175B] --> XL2[GPT-4<br/>~1.7T]
XL2 --> XL3[Future Models<br/>> 10T?]
end
style S1 fill:#4CAF50,color:#fff
style M4 fill:#2196F3,color:#fff
style L3 fill:#FF9800,color:#fff
style XL2 fill:#F44336,color:#fff
Training Pipeline
sequenceDiagram
participant Data as Raw Data<br/>(45TB text)
participant Preprocess as Preprocessing
participant Model as Transformer Model
participant Loss as Loss Function
participant Optimizer as Optimizer
participant Checkpoint as Checkpoints
Data->>Preprocess: Clean & filter
Preprocess->>Model: Batches of sequences
loop Training Loop (weeks/months)
Model->>Model: Forward pass
Model->>Loss: Predictions
Loss->>Optimizer: Gradients
Optimizer->>Model: Update 175B params
alt Every N steps
Model->>Checkpoint: Save model state
end
end
Model->>Model: Pre-trained LLM ready!
GPT vs BERT Architecture
graph TB
subgraph "GPT (Decoder-Only)"
G1[Input Text] --> G2[Causal Attention<br/>Only look left]
G2 --> G3[Generate Next Token]
G3 --> G4[Autoregressive<br/>One token at a time]
G4 --> G5["Use Case:<br/>Text Generation"]
end
subgraph "BERT (Encoder-Only)"
B1[Input Text] --> B2[Bidirectional Attention<br/>Look both ways]
B2 --> B3[Understand Context]
B3 --> B4[Parallel Processing<br/>All tokens together]
B4 --> B5["Use Case:<br/>Classification, Q&A"]
end
subgraph "T5 (Encoder-Decoder)"
T1[Source Text] --> T2[Encoder<br/>Bidirectional]
T2 --> T3[Decoder<br/>Causal]
T3 --> T4["Use Case:<br/>Translation, Summarization"]
end
style G1 fill:#9C27B0,color:#fff
style B1 fill:#2196F3,color:#fff
style T1 fill:#FF9800,color:#fff
Scaling Laws
graph TB
Scale["Model Scale"] --> Params[More Parameters]
Scale --> Data[More Data]
Scale --> Compute[More Compute]
Params --> Perf1[Better Performance]
Data --> Perf1
Compute --> Perf1
Perf1 --> Emerge[Emergent Abilities]
Emerge --> Math[Math Reasoning]
Emerge --> Code[Code Generation]
Emerge --> Logic[Logical Reasoning]
Emerge --> Multi[Multi-step Tasks]
style Scale fill:#FF9800,color:#fff
style Perf1 fill:#2196F3,color:#fff
style Emerge fill:#4CAF50,color:#fff
LLM Evolution Timeline
timeline
title LLM Evolution & Capabilities
2018 : GPT-1 (117M)
: Basic text completion
2019 : GPT-2 (1.5B)
: Coherent paragraphs
: BERT (340M)
: Understanding context
2020 : GPT-3 (175B)
: Few-shot learning
: Code generation
: Creative writing
2022 : ChatGPT
: Conversational AI
: RLHF alignment
: 100M users in 2 months
2023 : GPT-4 (1.7T)
: Multimodal (vision)
: Advanced reasoning
: Professional exams
: Claude 2 (100K context)
2024 : GPT-4 Turbo
: 128K context
: Cheaper & faster
: Gemini 1.5 (1M context)
🗺️ Concept Mind Map
LARGE LANGUAGE MODELS (LLMs)
|
+-------------------+-------------------+
| | |
ARCHITECTURE TRAINING CAPABILITIES
| | |
| | |
Transformer-Based Pre-training Text Generation
- Decoder-only - Self-supervised - Creative writing
(GPT, LLaMA) - Billions of - Code generation
- Encoder-only tokens - Summarization
(BERT) - Weeks/months - Translation
- Encoder-Decoder - $$$$ expensive
(T5, BART) | |
| | |
Scale Matters Fine-tuning Question Answering
- Parameters - Task-specific - Factual queries
(7B to 175B+) - Smaller dataset - Reasoning
- Context Window - Hours/days - Math problems
(2K to 128K+) - Cheaper - Common sense
- Training Data | |
(TB scale) | |
| Alignment Instruction Following
| - RLHF - Follow prompts
Model Families - Human feedback - Multi-step tasks
- GPT (OpenAI) - Safety tuning - Tool usage
- Claude (Anthropic)- Helpful/Harmless - API calls
- LLaMA (Meta) |
- Gemini (Google) |
- Mistral Evaluation
- Benchmarks (MMLU)
- Human eval
- Safety tests
Training Pipeline:
STAGE 1: Pre-training (Foundation)
Raw Text (Internet, Books, Code)
↓
Next Token Prediction
↓
Base Model (General Knowledge)
↓
STAGE 2: Supervised Fine-Tuning (SFT)
Instruction-Response Pairs
↓
Learn to Follow Instructions
↓
Instruction-Tuned Model
↓
STAGE 3: Alignment (RLHF)
Human Feedback on Outputs
↓
Reward Model Training
↓
Policy Optimization (PPO)
↓
Aligned Model (Safe & Helpful)
Model Comparison:
SIZE PARAMETERS USE CASE EXAMPLES
Small < 7B Edge devices, fast Phi-2, Gemma
Medium 7B - 70B General purpose LLaMA-2, Mistral
Large 70B+ Complex reasoning GPT-4, Claude-3
Key Characteristics:
- Emergent Abilities: New capabilities appear at scale
- Few-Shot Learning: Learn from examples in prompt
- Context Window: How much text they can process at once
- Hallucination: Can generate plausible but false information
zations](#🎮-interactive-tools--visualizations) 3. Fun Facts & Hacks 4. What You'll Learn 5. Core Concept 6. Code Examples 7. Hands-On Exercises 8. Assignments & Practice Problems 9. Common Pitfalls & Debugging 10. Related Concepts 11. Resources 12. Key Takeaways 13. Next Steps
🛠️ Local Setup (10 Minutes)
Set up both API access and local LLM running:
Option 1: API Access (Recommended for Beginners)
OpenAI Setup:
# Install OpenAI library
pip install openai==1.3.0
# Set API key (get from https://platform.openai.com/api-keys)
export OPENAI_API_KEY='your-key-here' # macOS/Linux
# set OPENAI_API_KEY=your-key-here # Windows
Anthropic (Claude) Setup:
pip install anthropic==0.7.0
export ANTHROPIC_API_KEY='your-key-here'
Quick Test:
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
print("✅ API working!")
Option 2: Local LLMs (Free, Private)
Install Ollama (Easiest):
# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh
# Windows: Download from https://ollama.com/download
# Pull a model
ollama pull llama2 # 7B model, ~4GB
ollama pull mistral # 7B model, ~4GB
ollama pull codellama # Code-specialized
# Test
ollama run llama2 "Hello, how are you?"
Python Integration:
pip install ollama==0.1.0
# Test
python -c "
import ollama
response = ollama.chat(model='llama2', messages=[
{'role': 'user', 'content': 'Hello!'}
])
print(response['message']['content'])
print('✅ Local LLM working!')
"
Option 3: HuggingFace Transformers
pip install transformers==4.30.0 \
torch==2.0.1 \
accelerate==0.20.0
# Download small model for testing
python -c "
from transformers import pipeline
generator = pipeline('text-generation', model='gpt2')
print(generator('Hello, I am', max_length=20))
print('✅ HuggingFace working!')
"
🚨 Troubleshooting
Issue: "API key not found"
# Set in code instead
import os
os.environ['OPENAI_API_KEY'] = 'your-key-here'
Issue: "Out of memory" with local models
# Use smaller models
ollama pull phi # 2.7B, only 1.6GB
ollama pull tinyllama # 1.1B, only 637MB
Issue: "Slow generation"
# Use GPU if available
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
Compare multiple models
import ollama
models = ['llama2', 'mistral', 'phi'] prompt = "Explain quantum computing in one sentence"
for model in models: response = ollama.chat(model=model, messages=[ {'role': 'user', 'content': prompt} ]) print(f"\n{model}:") print(response['message']['content'])
---
## 💡 The Complete LLM Pipeline
> **A clear mental model for how LLMs work** - contributed by Charan's learning notes
When you understand this pipeline, everything clicks:
TOKENIZATION → EMBEDDING → TRANSFORMER → PROBABILITIES → SAMPLING
**This is the complete flow of how LLMs generate text!**
### Breaking Down Each Step
#### 1. TOKENIZATION
Input: "Hello world" ↓ Tokens: [15496, 995]
Breaking text into pieces the model understands.
#### 2. EMBEDDING
Tokens: [15496, 995] ↓ Vectors: [[0.2, -0.5, 0.8, ...], [0.1, 0.3, -0.2, ...]]
Converting tokens into mathematical representations (vectors).
#### 3. TRANSFORMER
Vectors → Attention Mechanism → Context-Aware Vectors
The model processes and understands relationships between words.
#### 4. PROBABILITIES
Context → Probability Distribution "Hello" → {" world": 0.35, " there": 0.25, " friend": 0.15, ...}
Predicting what comes next with confidence scores.
#### 5. SAMPLING
Probabilities → Selected Token → Output
Choosing the next word based on probabilities (with some randomness).
**This pipeline repeats for EVERY token generated!**
### Visual Flow
┌─────────────────────────────────────────────────────────────┐ │ LLM GENERATION FLOW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ "Hello world" (Input) │ │ ↓ │ │ [15496, 995] (Tokenization) │ │ ↓ │ │ [[0.2, -0.5, ...], [0.1, 0.3, ...]] (Embedding) │ │ ↓ │ │ Transformer Layers (Attention + FFN) │ │ ↓ │ │ {"!": 0.35, ".": 0.25, ...} (Probabilities) │ │ ↓ │ │ "!" (Sampling) │ │ ↓ │ │ "Hello world!" (Output) │ │ │ │ REPEAT for each token until done! │ │ │ └─────────────────────────────────────────────────────────────┘
---
## 🧠 Core Concept
### What is a Large Language Model?
An LLM combines everything we've learned:
```python
LLM = Tokenization + Embeddings + Attention + Transformer + Self-Supervised Learning + Scale
# Key characteristics:
# 1. Large: Billions of parameters (GPT-3: 175B, GPT-4: ~1.7T)
# 2. Pre-trained: On massive text corpora (internet-scale)
# 3. General-purpose: Can handle many tasks without task-specific training
# 4. Few-shot learners: Learn from examples in the prompt
The Training Pipeline
graph LR
A[Raw Text<br/>Trillions of tokens] --> B[Pre-training<br/>Next token prediction]
B --> C[Base Model<br/>Can complete text]
C --> D[Instruction Tuning<br/>Follow instructions]
D --> E[RLHF<br/>Align with humans]
E --> F[Production Model<br/>ChatGPT, Claude]
style A fill:#E3F2FD
style C fill:#FFF9C4
style F fill:#C8E6C9
Stage 1: Pre-training
- Train on massive text corpus (books, web, code)
- Learn language patterns, facts, reasoning
- Cost: Millions of dollars, months of training
Stage 2: Instruction Tuning
- Fine-tune on instruction-following examples
- Learn to follow user commands
- Cost: Thousands of dollars, days of training
Stage 3: RLHF (Reinforcement Learning from Human Feedback)
- Align model with human preferences
- Make outputs helpful, harmless, honest
- Cost: Thousands of dollars, days of training
💻 Code Examples
Example 1: Using OpenAI API
from openai import OpenAI
import os
# Initialize client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
def chat_with_gpt(messages, model="gpt-4", temperature=0.7, max_tokens=500):
"""
Chat with GPT models.
Args:
messages: List of message dicts with 'role' and 'content'
model: Model name (gpt-4, gpt-3.5-turbo, etc.)
temperature: Randomness (0=deterministic, 2=very random)
max_tokens: Maximum response length
Returns:
Response text
"""
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens
)
return response.choices[0].message.content
# Example 1: Simple question
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Explain what a Large Language Model is in simple terms."}
]
response = chat_with_gpt(messages)
print("=== Simple Question ===")
print(response)
# Example 2: Few-shot learning
messages = [
{"role": "system", "content": "You are a sentiment classifier."},
{"role": "user", "content": """Classify the sentiment:
'I love this product!' → Positive
'This is terrible.' → Negative
'It's okay, I guess.' → Neutral
'This is amazing!' → """}
]
response = chat_with_gpt(messages, temperature=0)
print("\n=== Few-shot Learning ===")
print(f"Classification: {response}")
# Example 3: Chain of thought
messages = [
{"role": "user", "content": "Solve this step by step:\n\nIf a train travels 120 miles in 2 hours, what is its average speed?"}
]
response = chat_with_gpt(messages)
print("\n=== Chain of Thought ===")
print(response)
Example 2: Using Anthropic Claude API
from anthropic import Anthropic
import os
# Initialize client
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
def chat_with_claude(prompt, model="claude-3-opus-20240229", max_tokens=1000):
"""
Chat with Claude models.
Args:
prompt: User prompt
model: Model name (claude-3-opus, claude-3-sonnet, etc.)
max_tokens: Maximum response length
Returns:
Response text
"""
message = client.messages.create(
model=model,
max_tokens=max_tokens,
messages=[
{"role": "user", "content": prompt}
]
)
return message.content[0].text
# Example usage
prompt = """Analyze this code and suggest improvements:
def calculate_sum(numbers):
total = 0
for num in numbers:
total = total + num
return total
"""
response = chat_with_claude(prompt)
print("=== Claude Code Review ===")
print(response)
# Streaming example
def stream_claude(prompt):
"""Stream responses from Claude."""
with client.messages.stream(
model="claude-3-opus-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": prompt}]
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
print() # New line at end
print("\n=== Streaming Response ===")
stream_claude("Write a haiku about AI")
Example 3: Running Local Models with Ollama
import requests
import json
class OllamaClient:
"""
Client for running local LLMs with Ollama.
Install Ollama: https://ollama.ai
Then: ollama pull llama2
"""
def __init__(self, base_url="http://localhost:11434"):
self.base_url = base_url
def generate(self, model, prompt, stream=False):
"""
Generate text with local model.
Args:
model: Model name (llama2, mistral, codellama, etc.)
prompt: Input prompt
stream: Whether to stream response
Returns:
Generated text
"""
url = f"{self.base_url}/api/generate"
data = {
"model": model,
"prompt": prompt,
"stream": stream
}
if stream:
response = requests.post(url, json=data, stream=True)
for line in response.iter_lines():
if line:
chunk = json.loads(line)
if not chunk.get("done"):
print(chunk["response"], end="", flush=True)
print()
else:
response = requests.post(url, json=data)
return response.json()["response"]
def chat(self, model, messages):
"""Chat with local model."""
url = f"{self.base_url}/api/chat"
data = {
"model": model,
"messages": messages
}
response = requests.post(url, json=data)
return response.json()["message"]["content"]
def list_models(self):
"""List available local models."""
url = f"{self.base_url}/api/tags"
response = requests.get(url)
return response.json()["models"]
# Example usage
ollama = OllamaClient()
# List available models
print("=== Available Local Models ===")
models = ollama.list_models()
for model in models:
print(f" - {model['name']}")
# Generate text
print("\n=== Text Generation ===")
response = ollama.generate(
model="llama2",
prompt="Explain quantum computing in simple terms:"
)
print(response)
# Chat
print("\n=== Chat ===")
messages = [
{"role": "user", "content": "What is the capital of France?"},
{"role": "assistant", "content": "The capital of France is Paris."},
{"role": "user", "content": "What is its population?"}
]
response = ollama.chat("llama2", messages)
print(response)
# Streaming
print("\n=== Streaming ===")
ollama.generate(
model="llama2",
prompt="Write a short story about a robot:",
stream=True
)
Example 4: Prompt Engineering Patterns
class PromptTemplates:
"""
Collection of effective prompt engineering patterns.
"""
@staticmethod
def zero_shot(task, input_text):
"""Zero-shot: No examples, just instruction."""
return f"""Task: {task}
Input: {input_text}
Output:"""
@staticmethod
def few_shot(task, examples, input_text):
"""Few-shot: Provide examples."""
examples_str = "\n\n".join([
f"Input: {ex['input']}\nOutput: {ex['output']}"
for ex in examples
])
return f"""Task: {task}
{examples_str}
Input: {input_text}
Output:"""
@staticmethod
def chain_of_thought(question):
"""Chain of thought: Ask for step-by-step reasoning."""
return f"""{question}
Let's think step by step:"""
@staticmethod
def role_prompting(role, task):
"""Role prompting: Assign a role to the model."""
return f"""You are a {role}.
{task}"""
@staticmethod
def structured_output(task, format_spec):
"""Request structured output."""
return f"""{task}
Provide your response in the following format:
{format_spec}"""
# Example usage
templates = PromptTemplates()
# Zero-shot
prompt = templates.zero_shot(
task="Classify sentiment as Positive, Negative, or Neutral",
input_text="I absolutely love this product!"
)
print("=== Zero-shot ===")
print(prompt)
# Few-shot
examples = [
{"input": "Great service!", "output": "Positive"},
{"input": "Terrible experience.", "output": "Negative"},
{"input": "It's okay.", "output": "Neutral"}
]
prompt = templates.few_shot(
task="Classify sentiment",
examples=examples,
input_text="This is amazing!"
)
print("\n=== Few-shot ===")
print(prompt)
# Chain of thought
prompt = templates.chain_of_thought(
"If John has 3 apples and gives 1 to Mary, then buys 2 more, how many does he have?"
)
print("\n=== Chain of Thought ===")
print(prompt)
# Role prompting
prompt = templates.role_prompting(
role="expert Python developer",
task="Review this code and suggest improvements:\n\ndef add(a, b):\n return a + b"
)
print("\n=== Role Prompting ===")
print(prompt)
# Structured output
prompt = templates.structured_output(
task="Analyze the sentiment of: 'I love this product!'",
format_spec="""
{
"sentiment": "positive/negative/neutral",
"confidence": 0.0-1.0,
"reasoning": "brief explanation"
}"""
)
print("\n=== Structured Output ===")
print(prompt)
Example 5: Token Counting and Cost Estimation
import tiktoken
class LLMCostCalculator:
"""
Calculate API costs for different LLM providers.
"""
# Pricing per 1K tokens (as of 2024)
PRICING = {
"gpt-4": {"input": 0.03, "output": 0.06},
"gpt-3.5-turbo": {"input": 0.0015, "output": 0.002},
"claude-3-opus": {"input": 0.015, "output": 0.075},
"claude-3-sonnet": {"input": 0.003, "output": 0.015},
}
def __init__(self, model="gpt-4"):
self.model = model
self.encoding = tiktoken.encoding_for_model(model.split("-")[0])
def count_tokens(self, text):
"""Count tokens in text."""
return len(self.encoding.encode(text))
def estimate_cost(self, input_text, output_text):
"""Estimate API cost."""
input_tokens = self.count_tokens(input_text)
output_tokens = self.count_tokens(output_text)
pricing = self.PRICING.get(self.model, self.PRICING["gpt-4"])
input_cost = (input_tokens / 1000) * pricing["input"]
output_cost = (output_tokens / 1000) * pricing["output"]
total_cost = input_cost + output_cost
return {
"input_tokens": input_tokens,
"output_tokens": output_tokens,
"total_tokens": input_tokens + output_tokens,
"input_cost": input_cost,
"output_cost": output_cost,
"total_cost": total_cost
}
# Example usage
calculator = LLMCostCalculator(model="gpt-4")
input_text = """
Analyze this code and provide detailed feedback:
def fibonacci(n):
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
"""
output_text = """
This is a classic recursive implementation of the Fibonacci sequence.
Here's my analysis:
Pros:
- Simple and readable
- Correct implementation
Cons:
- Exponential time complexity O(2^n)
- Redundant calculations
- Stack overflow for large n
Improvements:
1. Use memoization
2. Use iterative approach
3. Use dynamic programming
"""
cost_info = calculator.estimate_cost(input_text, output_text)
print("=== Cost Estimation ===")
print(f"Model: {calculator.model}")
print(f"Input tokens: {cost_info['input_tokens']}")
print(f"Output tokens: {cost_info['output_tokens']}")
print(f"Total tokens: {cost_info['total_tokens']}")
print(f"Input cost: ${cost_info['input_cost']:.4f}")
print(f"Output cost: ${cost_info['output_cost']:.4f}")
print(f"Total cost: ${cost_info['total_cost']:.4f}")
# Compare costs across models
print("\n=== Cost Comparison ===")
for model in ["gpt-4", "gpt-3.5-turbo", "claude-3-opus", "claude-3-sonnet"]:
calc = LLMCostCalculator(model=model)
cost = calc.estimate_cost(input_text, output_text)
print(f"{model:20} ${cost['total_cost']:.4f}")
Test your chatbot
chatbot = build_chatbot("You are a helpful coding assistant.")
chatbot.chat("How do I reverse a string in Python?")
chatbot.chat("Can you show me an example?")
### Exercise 2: Compare Model Outputs
```python
def compare_models(prompt, models):
"""
Compare outputs from different LLMs.
Compare:
1. Response quality
2. Response time
3. Token usage
4. Cost
5. Consistency across runs
"""
# TODO: Implement comparison
pass
❌ Exceeding context window
long_prompt = "..." * 10000 # Too long! response = chat_with_gpt([{"role": "user", "content": long_prompt}])
Error: Context length exceeded
✅ Manage context window
def truncate_to_limit(text, max_tokens=4000): encoding = tiktoken.encoding_for_model("gpt-4") tokens = encoding.encode(text) if len(tokens) > max_tokens: tokens = tokens[:max_tokens] text = encoding.decode(tokens) return text
### Pitfall 2: Inconsistent Outputs
```python
# ❌ High temperature for deterministic tasks
response = chat_with_gpt(messages, temperature=1.5) # Too random!
# ✅ Use low temperature for consistency
response = chat_with_gpt(messages, temperature=0) # Deterministic
Pitfall 3: Not Handling Rate Limits
import time
from openai import RateLimitError
def chat_with_retry(messages, max_retries=3):
---
## 📝 Quick Reference: Personal Learning Notes
> **Charan's Learning Journey** - Additional insights and personal notes
### 🎯 The Pipeline That Makes It All Work
TOKENIZATION → EMBEDDING → TRANSFORMER → PROBABILITIES → SAMPLING
Understanding this pipeline is key to understanding how LLMs actually generate text. Each token goes through this entire process!
### 🔑 Key Insights
**Temperature Controls Creativity:**
- Temperature = 0: Deterministic (always same output)
- Temperature = 1: Balanced creativity
- Temperature = 2: Very creative (sometimes nonsense)
**The Scale Pattern:**
GPT-2 (1.5B params) → GPT-3 (175B params) → GPT-4 (~1.7T params) Bigger = Smarter!
**Training Stages:**
1. Pre-training (millions of $, months)
2. Instruction Tuning (thousands of $, days)
3. RLHF (thousands of $, days)
<!--REF_OPEN:📚 Additional Resources-->
- 🎥 [How LLMs Actually Generate Text](https://www.youtube.com/watch?v=NKnZYvZA7w4) - Excellent video explanation
- 📝 [Charan's Personal Notes](../../../Charan-Experiments/MyReadings/LLM/LLM.md) - Raw learning insights
- 📖 [Enhanced Learning Notes](../../../Charan-Experiments/MyReadings/LLM/LLM-Bob.md) - Comprehensive version
<!--REF_CLOSE-->
### 🎯 Practical Action Items
- [ ] Experiment with different temperature settings
- [ ] Try few-shot learning examples
- [ ] Compare GPT-4 vs Claude vs local models
- [ ] Build a simple chatbot using LLM APIs
- [ ] Understand each pipeline stage deeply
---
> **Learning Tip**: The pipeline visualization above (Tokenization → Embedding → Transformer → Probabilities → Sampling) is the clearest way to understand LLM text generation. Use it as your mental model!
#llm #gpt #claude #chatgpt #foundations #ai-models
"""Chat with automatic retry on rate limit."""
for attempt in range(max_retries):
try:
return chat_with_gpt(messages)
except RateLimitError:
if attempt < max_retries - 1:
wait_time = 2 ** attempt # Exponential backoff
print(f"Rate limited. Waiting {wait_time}s...")
time.sleep(wait_time)
else:
raise
🔗 Related Concepts
- Previous: Self-Supervised Learning - How LLMs are trained
- Next: Intermediate Stage - Working with LLMs
- Related: Context Engineering - Crafting better prompts
- Advanced: Agents - Building autonomous systems with LLMs
📚 Resources
Essential Papers
- 📄 Language Models are Few-Shot Learners (GPT-3)
- 📄 Training language models to follow instructions (InstructGPT)
- 📄 Constitutional AI: Harmlessness from AI Feedback
- 📄 LLaMA: Open and Efficient Foundation Language Models
Video Tutorials
Interactive Tools
- 🔧 OpenAI Playground
- 🔧 Anthropic Console
- 🔧 Ollama - Run models locally
- 🔧 LM Studio - GUI for local models
Code Libraries
<!--REF_CLOSE-->
# Install LLM libraries
pip install openai
pip install anthropic
pip install tiktoken
pip install langchain # For building LLM applications
pip install llama-index # For RAG applications