🔵 Stage 1: Foundations

Goal: Understand how modern AI systems work under the hood

Welcome to the Foundations stage! This is where your AI journey truly begins. These six concepts form the building blocks of all modern AI systems, from ChatGPT to Claude to Gemini.

📚 Concepts in This Stage

graph LR
    A[Tokenization] --> B[Vectorization]
    B --> C[Attention]
    C --> D[Transformer]
    D --> E[Self-Supervised Learning]
    E --> F[Large Language Models]
    
    style A fill:#4CAF50
    style B fill:#4CAF50
    style C fill:#2196F3
    style D fill:#2196F3
    style E fill:#FF9800
    style F fill:#F44336

1. Tokenization 🔤

Time: 2-3 hours | Difficulty: ⭐⭐☆☆☆

Learn how AI breaks down text into processable units. You'll build tokenizers from scratch and use industry-standard tools.

Key Takeaways:

• Text → Tokens conversion
• BPE, WordPiece, SentencePiece algorithms
• Vocabulary management
• Handling unknown words

Code Examples:

• Build a simple tokenizer
• Use HuggingFace tokenizers
• Compare different tokenization strategies

2. Vectorization 📊

Time: 3-4 hours | Difficulty: ⭐⭐⭐☆☆

Discover how tokens become mathematical vectors that capture meaning. This is where the "magic" of semantic understanding begins.

Key Takeaways:

• Embeddings and vector spaces
• Word2Vec, GloVe, FastText
• Contextual embeddings
• Semantic similarity

Code Examples:

• Implement word embeddings
• Use sentence transformers
• Visualize embedding spaces
• Calculate semantic similarity

3. Attention 👁️

Time: 4-5 hours | Difficulty: ⭐⭐⭐⭐☆

Master the mechanism that allows AI to focus on relevant information. This is the breakthrough that made modern AI possible.

Key Takeaways:

• Attention mechanism fundamentals
• Query, Key, Value matrices
• Multi-head attention
• Self-attention vs cross-attention

Code Examples:

• Implement attention from scratch
• Visualize attention weights
• Build multi-head attention
• Understand attention patterns

4. Transformer 🤖

Time: 5-6 hours | Difficulty: ⭐⭐⭐⭐⭐

Build the architecture that powers GPT, BERT, and virtually all modern LLMs. This is the most important concept in modern AI.

Key Takeaways:

• Encoder-decoder architecture
• Positional encoding
• Layer normalization
• Feed-forward networks

Code Examples:

• Build a mini-transformer
• Implement encoder and decoder
• Train on simple tasks
• Understand architecture choices

5. Self-Supervised Learning 🎓

Time: 3-4 hours | Difficulty: ⭐⭐⭐☆☆

Learn how models train themselves on massive amounts of unlabeled data. This is why modern AI can learn from the entire internet.

Key Takeaways:

• Masked language modeling
• Next token prediction
• Contrastive learning
• Pre-training objectives

Code Examples:

• Implement MLM task
• Create training data
• Pre-train a small model
• Understand training dynamics

6. Large Language Models 🧠

Time: 4-5 hours | Difficulty: ⭐⭐⭐⭐☆

Put everything together to understand how ChatGPT, Claude, and other LLMs work. Learn to use them effectively in your applications.

Key Takeaways:

• LLM architecture and scale
• Training process and data
• Inference and generation
• API usage and best practices

Code Examples:

• Use OpenAI/Anthropic APIs
• Run local models with Ollama
• Implement text generation
• Handle streaming responses

🎯 Learning Path

Recommended Order

Follow the concepts sequentially (1 → 6). Each builds on the previous one.

Time Commitment

• Total: 25-30 hours
• Per Day: 2-3 hours recommended
• Duration: 2-3 weeks at steady pace

Prerequisites

• Python programming (intermediate)
• Basic linear algebra (vectors, matrices)
• Understanding of neural networks (helpful but not required)

💻 Hands-On Projects

After completing this stage, build one of these projects:

🎮 Mini-Project 1: Text Classifier

Build a sentiment analysis system from scratch using transformers.

• Time: 4-6 hours
• Skills: Tokenization, Embeddings, Transformers
• Output: Working classifier with 85%+ accuracy

🎮 Mini-Project 2: Simple Chatbot

Create a basic conversational AI using pre-trained models.

• Time: 3-5 hours
• Skills: LLM APIs, Prompt Engineering
• Output: Interactive chatbot

🎮 Mini-Project 3: Text Generator

Build a model that generates coherent text in a specific style.

• Time: 5-7 hours
• Skills: Transformers, Self-Supervised Learning
• Output: Creative text generator

📊 Progress Tracking

• Tokenization - Completed exercises
• Vectorization - Completed exercises
• Attention - Completed exercises
• Transformer - Completed exercises
• Self-Supervised Learning - Completed exercises
• Large Language Models - Completed exercises
• Built Mini-Project 1, 2, or 3

🔗 Quick Navigation

Previous: Getting Started Next: Start with Tokenization Jump to: Intermediate Stage