🎨 Context Engineering
🌟 Start With a Real-World Scenario
You're chatting with ChatGPT about a project:
Message 1: "I'm building a Python web app" Message 5: "How do I add authentication?" Message 10: "What about database migrations?" Message 15: "Can you review my code?" Message 20: "Wait, what framework am I using again?"
ChatGPT: "I don't see any previous mention of a specific framework..."
You: "But I told you in message 1! I'm using Flask!" 😤
The Problem: ChatGPT "forgot"! Why? 🤔
Understanding the Context Window:
Think of AI's memory like a notepad with limited pages:
GPT-3.5: 4,096 tokens (~3,000 words)
- Like a 6-page notepad
- After 6 pages, it starts erasing from the beginning!
GPT-4: 8,192 tokens (~6,000 words)
- Like a 12-page notepad
- Better, but still limited
GPT-4-32k: 32,768 tokens (~24,000 words)
- Like a 50-page notepad
- Expensive! ($0.06 per 1K tokens vs $0.03)
Claude 2: 100,000 tokens (~75,000 words)
- Like a 150-page notepad
- Can remember entire books!
Real Example of Context Limits:
Your conversation:
[Message 1-10]: Discussing Python Flask app (2,000 tokens)
[Message 11-20]: Talking about database design (1,500 tokens)
[Message 21-30]: Debugging authentication (1,000 tokens)
[Message 31]: "Remember that Flask thing from the start?"
What AI sees:
[Message 1-10]: ❌ ERASED (out of context window!)
[Message 11-20]: ✅ Still in memory
[Message 21-30]: ✅ Still in memory
[Message 31]: "What Flask thing?" 🤷
AI literally can't see message 1-10 anymore!
Context Engineering to the Rescue:
Bad approach (wastes context):
You: "Hey! How are you? I hope you're doing well today.
I wanted to ask you something. So, I'm working on this
project, and it's a web application, and I'm using Python,
specifically Flask framework, and I need help with..."
- 50 tokens wasted on pleasantries!
- Verbose, inefficient
Good approach (optimized context):
You: "Python Flask app. Need: authentication setup.
Using: SQLAlchemy, JWT tokens. Show: code example."
- 15 tokens, same information!
- 70% token savings!
Real-World Impact:
Scenario: Customer Support Bot
Bad Context Management:
System: "You are a helpful, friendly, and professional
customer support agent for TechCorp, a leading technology
company that specializes in software solutions. You should
always be polite, courteous, and helpful. Remember to..."
(200 tokens of fluff!)
User: "My order #12345 hasn't arrived"
Bot: "I'm sorry to hear that! Let me help you..."
Good Context Management:
System: "TechCorp support. Be helpful. Check order status,
offer solutions. Tone: professional, concise."
(15 tokens!)
User: "Order #12345 not arrived"
Bot: "Checking #12345... Shipped 3 days ago, expected tomorrow.
Track: [link]. Need help?"
Savings: 185 tokens per conversation! At 1M conversations: Save $370 in API costs! 💰
The Art of Context Engineering:
1. System Prompts (Set the stage):
Bad: "You are an AI assistant that helps people..."
Good: "Expert Python developer. Provide code, not explanations."
2. Few-Shot Examples (Show, don't tell):
Bad: "Please format dates as YYYY-MM-DD"
Good: "2024-01-15 → Jan 15, 2024
2024-12-25 → Dec 25, 2024
2024-06-30 → ?"
3. Retrieved Context (RAG):
Bad: Include entire 50-page document
Good: Include only relevant 3 paragraphs
4. Conversation History:
Bad: Keep all 50 messages
Good: Summarize old messages, keep recent 10
Context Window Strategies:
Strategy 1: Sliding Window
Keep: Last 10 messages
Discard: Older messages
Pro: Simple, fast
Con: Loses old context
Strategy 2: Summarization
Every 10 messages: Summarize into 1 message
Keep: Summary + recent messages
Pro: Retains important info
Con: Loses details
Strategy 3: Semantic Compression
Keep: Most relevant messages (using embeddings)
Discard: Irrelevant messages
Pro: Keeps what matters
Con: More complex
By the end of this guide, you'll understand:
- How to maximize context window efficiency
- When to use system prompts vs. few-shot examples
- How to manage long conversations without losing context
- Token optimization techniques (save 30-50% on costs!)
- How to structure prompts for best results
- Context window limits for different models
- How to build context-aware applications
The secret: Good context engineering is like packing a suitcase - fit the most important things in limited space! 🎒
Fun fact: A single ChatGPT conversation can cost $0.01-0.10 depending on context management. Multiply by 1M users = $10,000-100,000! Good context engineering = massive savings! 💰
Let's dive in! 🚀
🗺️ Concept Mind Map
CONTEXT ENGINEERING
|
+---------------+---------------+
| | |
COMPONENTS TECHNIQUES OPTIMIZATION
| | |
| | |
System Message Prompt Templates Token Management
- Role setting - Reusable - Count tokens
- Behavior patterns - Compress context
- Constraints - Variables - Prioritize info
- Tone - Versioned - Sliding window
| | |
| | |
User Message Context Stuffing Context Window
- The query - Add relevant - GPT-3.5: 4K
- Instructions docs - GPT-4: 8K-128K
- Examples - Background - Claude: 200K
- Data info - Gemini: 1M+
| | |
| | |
Assistant Dynamic Context Cost Control
Message - Conversation - Shorter prompts
- Previous history - Cache system
responses - Retrieve on - Batch requests
- Continuity demand - Model selection
- Memory - Summarize old
| | |
| | |
Few-Shot Prompt Chaining Quality Control
Examples - Break complex - A/B testing
- Show format tasks - Metrics
- Demonstrate - Multi-step - Validation
- Guide style - Sequential - Iteration
Prompt Anatomy:
┌─────────────────────────────────────┐
│ SYSTEM MESSAGE (Role & Rules) │
│ "You are a helpful assistant..." │
├─────────────────────────────────────┤
│ CONTEXT (Background Information) │
│ - Relevant documents │
│ - Previous conversation │
│ - Domain knowledge │
├─────────────────────────────────────┤
│ EXAMPLES (Few-Shot Learning) │
│ Input: "..." → Output: "..." │
│ Input: "..." → Output: "..." │
├─────────────────────────────────────┤
│ INSTRUCTION (What to do) │
│ "Analyze the sentiment and..." │
├─────────────────────────────────────┤
│ INPUT (User's actual query) │
│ "This product is amazing!" │
├─────────────────────────────────────┤
│ OUTPUT FORMAT (How to respond) │
│ "Return JSON: {sentiment: ...}" │
└─────────────────────────────────────┘
Context Window Strategy:
AVAILABLE TOKENS: 4096
├─ System Message: 200 tokens (5%)
├─ Examples: 400 tokens (10%)
├─- Context/Docs: 2000 tokens (50%)
├─ User Query: 200 tokens (5%)
├─ Output Buffer: 1000 tokens (25%)
└─ Safety Margin: 296 tokens (5%)
Key Principles:
- Be Specific: Vague prompts = vague outputs
- Show, Don't Tell: Examples > Instructions
- Manage Tokens: Every token costs money
- Test & Iterate: A/B test your prompts
zations](#🎮-interactive-tools--visualizations) 3. Fun Facts & Hacks 4. What You'll Learn 5. Core Concept 6. Code Examples 7. Hands-On Exercises 8. Assignments & Practice Problems 9. Common Pitfalls & Debugging 10. Related Concepts 11. Resources 12. Key Takeaways 13. Next Steps
🛠️ Local Setup (5 Minutes)
Set up tools for context engineering experiments:
Step 1: Install Dependencies
pip install openai==1.3.0 \
anthropic==0.7.0 \
tiktoken==0.5.1 \
langchain==0.1.0 \
python-dotenv==1.0.0
Step 2: Token Counter Setup
import tiktoken
def count_tokens(text, model="gpt-3.5-turbo"):
"""Count tokens in text for a specific model."""
encoding = tiktoken.encoding_for_model(model)
return len(encoding.encode(text))
# Test
text = "Hello, how are you?"
tokens = count_tokens(text)
print(f"'{text}' = {tokens} tokens")
print("✅ Token counting working!")
Step 3: Create Prompt Template System
from string import Template
class PromptTemplate:
def __init__(self, template):
self.template = Template(template)
def format(self, **kwargs):
return self.template.safe_substitute(**kwargs)
def token_count(self, **kwargs):
prompt = self.format(**kwargs)
return count_tokens(prompt)
# Test
template = PromptTemplate("""
You are a $role.
Task: $task
Context: $context
""")
prompt = template.format(
role="helpful assistant",
task="summarize text",
context="some context here"
)
print(f"Tokens: {template.token_count(role='helpful assistant', task='summarize', context='context')}")
🧠 Core Concept
The Context Window
The context window is the total amount of text an LLM can process at once:
# Context Window Breakdown
context_window = {
"system_message": 100, # Role and instructions
"few_shot_examples": 500, # Example demonstrations
"user_input": 1000, # Actual task
"retrieved_context": 2000, # RAG documents
"response_buffer": 500 # Space for output
}
total = sum(context_window.values()) # 4100 tokens
# Must fit within model's limit:
# GPT-3.5: 4,096 tokens
# GPT-4: 8,192 or 128,000 tokens
# Claude: 200,000 tokens
The Anatomy of a Perfect Prompt
perfect_prompt = """
[ROLE]
You are an expert Python developer with 10 years of experience.
[TASK]
Review the following code and suggest improvements.
[CONSTRAINTS]
- Focus on performance and readability
- Provide specific code examples
- Limit response to 200 words
[FORMAT]
Output as:
1. Issues found
2. Suggested improvements
3. Refactored code
[CONTEXT]
{code_to_review}
"""
Message Types
System Message: Sets behavior and role
system = "You are a helpful assistant that speaks like a pirate."
# Affects all responses
User Message: The actual task
user = "Explain machine learning"
# What you want done
Assistant Message: Previous responses (for conversation)
assistant = "Machine learning is..."
# Maintains conversation context
💻 Code Examples
Example 1: Basic Prompt Template System
from openai import OpenAI
import tiktoken
client = OpenAI()
class PromptEngineer:
"""
Manage and optimize prompts.
"""
def __init__(self, model="gpt-3.5-turbo"):
self.model = model
self.encoding = tiktoken.encoding_for_model(model)
def count_tokens(self, text):
"""Count tokens in text."""
return len(self.encoding.encode(text))
def create_prompt(self, role, task, context, constraints=None):
"""
Create structured prompt.
"""
prompt = f"""You are a {role}.
Task: {task}
Context:
{context}
"""
if constraints:
prompt += f"\nConstraints:\n{constraints}"
return prompt
def optimize_prompt(self, prompt):
"""
Suggest optimizations for prompt.
"""
tokens = self.count_tokens(prompt)
suggestions = []
# Check for redundancy
if "please" in prompt.lower():
suggestions.append("Remove 'please' - LLMs don't need politeness")
# Check for verbosity
if tokens > 500:
suggestions.append(f"Prompt is {tokens} tokens. Consider shortening.")
# Check for structure
if "###" not in prompt and "---" not in prompt:
suggestions.append("Add delimiters (###, ---) for better structure")
return {
"tokens": tokens,
"cost_per_call": tokens * 0.0015 / 1000, # GPT-3.5 pricing
"suggestions": suggestions
}
# Example usage
engineer = PromptEngineer()
prompt = engineer.create_prompt(
role="Python expert",
task="Review this code for bugs",
context="def add(a, b):\n return a + b",
constraints="- Be concise\n- Provide examples"
)
print("Prompt:")
print(prompt)
print("\nAnalysis:")
analysis = engineer.optimize_prompt(prompt)
print(f"Tokens: {analysis['tokens']}")
print(f"Cost: ${analysis['cost_per_call']:.6f}")
print(f"Suggestions: {analysis['suggestions']}")
Example 2: Dynamic Context Management
class ContextManager:
"""
Manage context window efficiently.
"""
def __init__(self, max_tokens=4000):
self.max_tokens = max_tokens
self.encoding = tiktoken.encoding_for_model("gpt-3.5-turbo")
self.messages = []
def add_message(self, role, content):
"""Add message to context."""
self.messages.append({"role": role, "content": content})
self._trim_if_needed()
def _count_tokens(self, messages):
"""Count tokens in message list."""
total = 0
for msg in messages:
total += len(self.encoding.encode(msg["content"]))
total += 4 # Message formatting overhead
return total
def _trim_if_needed(self):
"""
Trim old messages if exceeding limit.
Keep system message and recent messages.
"""
while self._count_tokens(self.messages) > self.max_tokens:
# Keep system message (index 0)
if len(self.messages) > 2:
# Remove oldest user/assistant message
self.messages.pop(1)
else:
break
def get_messages(self):
"""Get current message list."""
return self.messages
def get_stats(self):
"""Get context statistics."""
tokens = self._count_tokens(self.messages)
return {
"messages": len(self.messages),
"tokens": tokens,
"remaining": self.max_tokens - tokens,
"utilization": f"{tokens/self.max_tokens*100:.1f}%"
}
# Example usage
manager = ContextManager(max_tokens=1000)
# Add system message
manager.add_message("system", "You are a helpful assistant.")
# Simulate conversation
manager.add_message("user", "What is Python?")
manager.add_message("assistant", "Python is a programming language...")
manager.add_message("user", "How do I install it?")
manager.add_message("assistant", "You can install Python by...")
# Check stats
stats = manager.get_stats()
print(f"Messages: {stats['messages']}")
print(f"Tokens used: {stats['tokens']}/{manager.max_tokens}")
print(f"Utilization: {stats['utilization']}")
Example 3: Prompt Template Library
class PromptLibrary:
"""
Reusable prompt templates.
"""
@staticmethod
def summarize(text, max_words=100):
return f"""Summarize the following text in {max_words} words or less:
{text}
Summary:"""
@staticmethod
def extract_entities(text):
return f"""Extract named entities from the text below.
Output as JSON with keys: persons, organizations, locations.
Text: {text}
Entities:"""
@staticmethod
def sentiment_analysis(text):
return f"""Analyze the sentiment of this text.
Output: positive, negative, or neutral
Text: {text}
Sentiment:"""
@staticmethod
def code_review(code, language="Python"):
return f"""Review this {language} code for:
1. Bugs
2. Performance issues
3. Best practices
Code:
```{language.lower()}
{code}
Review:"""
@staticmethod
def translate(text, source_lang, target_lang):
return f"""Translate from {source_lang} to {target_lang}:
{source_lang}: {text} {target_lang}:"""
Example usage
library = PromptLibrary()
Summarization
text = "Long article text here..." prompt = library.summarize(text, max_words=50)
Entity extraction
text = "Apple Inc. CEO Tim Cook visited Paris last week." prompt = library.extract_entities(text)
Code review
code = "def add(a, b):\n return a + b" prompt = library.code_review(code)
print(prompt)
---
### Example 4: A/B Testing Prompts
```python
import random
from typing import List, Dict
class PromptTester:
"""
A/B test different prompt variations.
"""
def __init__(self, client):
self.client = client
self.results = []
def test_prompts(self, prompts: List[str], test_inputs: List[str],
model="gpt-3.5-turbo"):
"""
Test multiple prompt variations.
"""
results = {}
for i, prompt_template in enumerate(prompts):
prompt_results = []
for test_input in test_inputs:
# Format prompt with input
prompt = prompt_template.format(input=test_input)
# Get response
response = self.client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
temperature=0
)
output = response.choices[0].message.content
tokens = response.usage.total_tokens
prompt_results.append({
"input": test_input,
"output": output,
"tokens": tokens
})
results[f"Prompt_{i+1}"] = {
"template": prompt_template,
"results": prompt_results,
"avg_tokens": sum(r["tokens"] for r in prompt_results) / len(prompt_results)
}
return results
def compare_results(self, results: Dict):
"""
Compare prompt performance.
"""
print("=== Prompt Comparison ===\n")
for name, data in results.items():
print(f"{name}:")
print(f" Template: {data['template'][:50]}...")
print(f" Avg Tokens: {data['avg_tokens']:.1f}")
print(f" Sample Output: {data['results'][0]['output'][:100]}...")
print()
# Example usage
tester = PromptTester(client)
# Test different prompt styles
prompts = [
"Summarize: {input}",
"Please provide a brief summary of the following text:\n{input}",
"Text: {input}\n\nSummary (max 50 words):"
]
test_inputs = [
"Machine learning is a subset of AI...",
"Python is a popular programming language..."
]
results = tester.test_prompts(prompts, test_inputs)
tester.compare_results(results)
Example 5: Context Window Optimizer
class ContextOptimizer:
"""
Optimize content to fit in context window.
"""
def __init__(self, max_tokens=4000):
self.max_tokens = max_tokens
self.encoding = tiktoken.encoding_for_model("gpt-3.5-turbo")
def count_tokens(self, text):
return len(self.encoding.encode(text))
def truncate_text(self, text, max_tokens):
"""
Truncate text to fit token limit.
"""
tokens = self.encoding.encode(text)
if len(tokens) <= max_tokens:
return text
# Truncate and decode
truncated_tokens = tokens[:max_tokens]
return self.encoding.decode(truncated_tokens)
def smart_truncate(self, text, max_tokens, strategy="middle"):
"""
Intelligently truncate text.
Strategies:
- 'start': Keep beginning
- 'end': Keep ending
- 'middle': Keep start and end, remove middle
"""
tokens = self.encoding.encode(text)
if len(tokens) <= max_tokens:
return text
if strategy == "start":
return self.encoding.decode(tokens[:max_tokens])
elif strategy == "end":
return self.encoding.decode(tokens[-max_tokens:])
elif strategy == "middle":
# Keep first 40% and last 40%, remove middle 20%
keep_start = int(max_tokens * 0.4)
keep_end = int(max_tokens * 0.4)
start_tokens = tokens[:keep_start]
end_tokens = tokens[-keep_end:]
# Add separator
separator = self.encoding.encode("\n...[content truncated]...\n")
combined = start_tokens + separator + end_tokens
return self.encoding.decode(combined)
def optimize_prompt(self, system_msg, user_msg, context,
response_buffer=500):
"""
Optimize all components to fit in context window.
"""
# Calculate available space
system_tokens = self.count_tokens(system_msg)
user_tokens = self.count_tokens(user_msg)
available_for_context = (
self.max_tokens -
system_tokens -
user_tokens -
response_buffer
)
if available_for_context < 0:
raise ValueError("System and user messages too long!")
# Truncate context if needed
context_tokens = self.count_tokens(context)
if context_tokens > available_for_context:
context = self.smart_truncate(
context,
available_for_context,
strategy="middle"
)
return {
"system": system_msg,
"user": user_msg,
"context": context,
"tokens": {
"system": system_tokens,
"user": user_tokens,
"context": self.count_tokens(context),
"response_buffer": response_buffer,
"total": system_tokens + user_tokens + self.count_tokens(context)
}
}
# Example usage
optimizer = ContextOptimizer(max_tokens=4000)
system_msg = "You are a helpful assistant."
user_msg = "Summarize this document:"
context = "Very long document text..." * 1000 # Simulate long text
optimized = optimizer.optimize_prompt(system_msg, user_msg, context)
print("Token Usage:")
for key, value in optimized["tokens"].items():
print(f" {key}: {value}")
print(f"\nContext truncated: {len(context)} → {len(optimized['context'])} chars")
📝 Assignments & Practice Problems
Assignment 1: Prompt Compressor ⭐
Difficulty: Easy | Time: 30 minutes
Build a tool that compresses prompts without losing meaning.
Requirements:
- Remove unnecessary words
- Use abbreviations
- Maintain clarity
- Show token savings
Assignment 2: Multi-Turn Conversation Manager ⭐⭐
Difficulty: Medium | Time: 60 minutes
Create a system that manages long conversations efficiently.
Requirements:
- Sliding context window
- Summarize old messages
- Keep important context
- Handle 100+ turn conversations
Assignment 3: Prompt A/B Testing Framework ⭐⭐⭐
Difficulty: Hard | Time: 90 minutes
Build a framework for testing prompt variations.
Requirements:
- Test multiple prompts
- Measure quality metrics
- Track token usage
- Statistical significance testing
- Generate optimization recommendations
❌ Bad: No token checking
prompt = system + examples + user_input # Might exceed limit!
✅ Good: Check and truncate
total_tokens = count_tokens(prompt) if total_tokens > max_tokens: # Truncate or summarize prompt = optimize_prompt(prompt, max_tokens)
### Pitfall 2: Unclear Instructions
```python
# ❌ Bad: Vague
"Make this better"
# ✅ Good: Specific
"Rewrite this text to be more concise. Target: 50 words. Maintain key points."
Pitfall 3: Ignoring Token Costs
# ❌ Bad: Wasteful
"Please, if you would be so kind, could you possibly help me..."
# ✅ Good: Efficient
"Help me..."
# Same meaning, 90% fewer tokens!
🔗 Related Concepts
- Previous: Few-Shot Prompting - Using examples
- Next: Chain of Thought - Reasoning prompts
- Related: Vector Database - Context retrieval
- Advanced: RAG - Augmenting context
📚 Resources
Papers
Guides
Tools
pip install langchain # Prompt templates
pip install guidance # Structured prompting
pip install tiktoken # Token counting