Traditional Programming

How we've built software for decades

• Rigid set of rules
• Precise specifications
• Deterministic behavior

Example: E-Commerce Platform

Building with traditional rules:

1. User interface in browser/mobile app
2. Database for user state and transactions
3. Business logic mapping circumstances to actions

Programming Business Logic

def add_to_cart(user_id, product_id):
    # Explicit rule for every action
    if cart_exists(user_id):
        cart.add_item(product_id)
    else:
        create_cart(user_id)
        cart.add_item(product_id)

Handle every corner case explicitly

When Traditional Programming Works

• Well-defined rules
• Predictable scenarios
• Limited edge cases
• No adaptation needed

Test Your Understanding

When Traditional Programming Fails

Some problems resist explicit rules

Even the smartest programmers struggle with these...

Challenge Problems

• Weather Prediction: Tomorrow's weather from satellite images
• Question Answering: Understanding free-form text questions
• Person Detection: Identifying people in images
• Recommendations: Suggesting products users haven't seen

Why These Problems Are Hard

• Dynamic patterns: Rules change over time
• Extreme complexity: Millions of interactions
• Unknown principles: We don't know the rules
• Subconscious processing: Beyond conscious understanding

This particular comic aged like milk

Test Your Understanding

Example: Wake Word Detection

How does "Hey Siri" or "OK Google" work?

Why Is Wake Word Detection Hard?

Consider the technical challenge:

• 44,000 samples per second
• Each sample = sound wave amplitude
• Must map raw audio → wake word detection

We don't know how to write such rules from scratch!

How Wake Word Detection Works

1. Continuously listen to audio
2. Extract audio features
3. Check if pattern matches wake word
4. Activate if confidence is high

Traditional Approach Attempt

def detect_wake_word(audio):
    if audio_matches_pattern("ah-lek-sah"):
        return True
    elif audio_matches_pattern("uh-lek-suh"):
        return True
    # ... hundreds more rules?
    return False

Too many variations to code manually!

The ML Alternative

model = train_wake_word_detector(
    positive_examples=["alexa_1.wav", "alexa_2.wav", ...],
    negative_examples=["other_1.wav", "other_2.wav", ...]
)

def detect_wake_word(audio):
    return model.predict(audio) > threshold

Learn patterns from examples instead!

Why This Approach is Powerful

• Handles accent variations
• Works in noisy environments
• Improves with more data
• No need to code every scenario
• Can generalize to new speakers

Test Your Understanding

Key Components of ML

The Four Pillars

1. DataThe examples from which the model learns patterns. Quality and quantity of data often determine the success of ML projects.
2. ModelsMathematical functions that transform inputs to outputs. Models have parameters that are adjusted during training.
3. Objective FunctionsMeasures how well the model is performing. Also called loss functions, they guide the learning process.
4. Optimization AlgorithmsMethods for adjusting model parameters to minimize the objective function. Gradient descent is the most common.

How They Work Together

An iterative process of improvement

Wake Word Example Revisited

Let's see how our four pillars work for "Alexa" detection

Step 1: Define the Problem

• Input: Audio snippet (e.g., 1 second)
• Output: Binary decision (wake word or not)
• Model family: Neural network for audio

Must precisely define inputs and outputs first!

Step 2: Choose Model Family

The model has adjustable "knobs" (parameters)

• One setting → detects "Alexa"
• Different setting → could detect "Apricot"
• Same architecture, different parameters

Step 3: The Training Process

1. Start with random parameters (useless model)
2. Grab labeled audio examples
3. Adjust parameters to improve predictions
4. Repeat until performance is satisfactory

This is "learning" - finding the right parameter settings

Programming with Data

Different Tasks, Different Models

Similar tasks can share model families:

• "Alexa" detection ≈ "Apricot" detection
• Same architecture, different training

But fundamentally different tasks need different models:

• Image captioning needs vision + language models
• Translation needs sequence-to-sequence models

Test Your Understanding

Data

The Foundation of Machine Learning

The Data Revolution in Deep Learning

• More data = more powerful models
• Shift from small to big data transformed deep learning
• Many modern models require large datasets to work
• In small data regime, traditional methods often suffice

Data Quality: Garbage In, Garbage Out

• Having lots of data isn't enough - we need the right data
• Mistakes in data → mistakes in predictions
• Non-predictive features → learning failure

The Danger of Biased Data

Why Data Matters

• Models learn from examples
• Data quality determines model quality
• "Garbage in, garbage out"

Quality vs Quantity

• More data usually helps
• But quality is crucial
• Clean, labeled data is expensive
• Balance is key

Data Representation

How we encode information for models

• Images → Pixel values
• Text → Numerical vectors
• Audio → Frequency features

Example: Images as Numbers

A 3×3 grayscale image:

[[255, 128,   0],
 [128,  64, 128],
 [  0, 128, 255]]

Each number represents pixel brightness

Example: Text as Vectors

Words mapped to numbers:

"deep" → [0.2, -0.5, 0.8, ...]
"learning" → [-0.1, 0.7, 0.3, ...]

Captures semantic relationships

Test Your Understanding

The Mushroom Example

Death cap - highly poisonous!

Why Mushroom Classification?

• Life-or-death importance
• Subtle visual differences
• Expert knowledge required
• Interesting ML use case
• Errors can be very costly

The Cost of Errors

Traditional Approach

Field guides use complex decision trees:

• "If cap is white AND gills are free..."
• "If ring is present AND volva exists..."
• "If spore print is white AND..."

Problem: Too many rules, too many exceptions!

Feature Engineering

What features help identify poisonous mushrooms?

• CapThe top part of the mushroom, varies in color, texture, and shape: color, shape, surface texture
• GillsThe blade-like structures under the cap where spores are produced: attachment, spacing, color
• StemThe stalk supporting the cap, may have rings or bulbs: color, ring, bulb shape
• HabitatWhere the mushroom grows - soil type, nearby trees, season: location, season, substrate

Data Collection Challenges

• Dangerous to collect poisonous samples
• Seasonal availability
• Geographic variation
• Need expert verification
• Imbalanced data (few deadly species)

ML Solution Benefits

• Learns subtle patterns humans miss
• Handles complex feature interactions
  • What is a feature interaction?
• Improves with more data
• Can provide confidence scores

Test Your Understanding

Models

The Learning Structures

What is a Model?

• A mathematical function
• Transforms inputs to outputs
• Has adjustable parameters

Models as Function Approximators

Goal: Learn function f where

Notation: True vs Predicted Values

Parameters and Weights

• Parameters define the model's behavior
• Adjusted during training
• Capture learned patterns

Simple Linear Model

def linear_model(x, w, b):
    return w * x + b

• w: weight (slope)
• b: bias (intercept)
• Both are learned from data

From Simple to Deep Models

Neural Network Preview

Multiple layers of transformations

Model Capacity

• Ability to fit complex patterns
• More parameters = higher capacity
• Balance with overfitting risk

Choosing the Right Model

• Problem complexity
• Available data
• Computational resources
• Interpretability needs

Models: Key Insights

• Functions with learnable parameters
• Transform inputs to predictions
• Capacity must match problem

Test Your Understanding

Objective Functions

Measuring Success

Why We Need Objective Functions

Machine learning = learning from experience

But what constitutes "improvement"?

• Different people might disagree on what's better
• We need formal, mathematical measures
• Enter: Objective Functions

What Are We Optimizing?

• Need to measure model performance
• Quantify "how wrong" predictions are
• Guide parameter updates

Loss Functions Explained

Loss = measure of prediction error

Common Loss Functions

Mean Squared Error

For regression problems:

Penalizes large errors more

Cross-Entropy Loss

For classification problems:

Measures prediction confidence

Can be derived based on the maximum likelihood approach

Loss During Training vs Testing

Loss = function of model parameters

Training vs Validation Loss

• Training loss: on training data
• Validation loss: on held-out data
• Gap indicates overfitting

Overfitting Preview

When overfitting, the model memorizes instead of generalizing

Objective Functions: Summary

• Quantify model performance
• Different losses for different tasks
• Balance training and generalization

Test Your Understanding

Optimization Algorithms

Finding the Best Parameters

Finding the Best Parameters

• Start with random parameters
• Measure performance (loss)
• Adjust to improve
• Repeat until convergence

The Optimization Landscape

Finding the lowest point

Gradient Descent Basics

• Compute gradient (slope)
• Step in opposite direction
• Like walking downhill in fog

The ML Training Loop

Learning Rate Importance

• Too small: slow convergence
• Too large: might overshoot
• Critical hyperparameter

Convergence

• Loss stops decreasing
• Parameters stabilize
• Model has "learned"

Optimization: Key Points

• Iterative improvement process
• Gradient descent is fundamental
• Many variations exist

Test Your Understanding

Types of Machine Learning

Different Learning Paradigms

Three Main Paradigms

1. Supervised LearningLearning from labeled examples where we know the correct answers
2. Unsupervised LearningFinding patterns in data without labels
3. Reinforcement LearningLearning through interaction and rewards

Overview Comparison

Test Your Understanding

Supervised Learning

Learning from Examples

What is Supervised Learning?

• Given: input-output pairs
• Learn: mapping function
• Goal: predict new outputs

Input-Output Pairs

Training data format:

(image of cat) → "cat"
(image of dog) → "dog"
(house features) → $500,000
(email text) → "spam"

Test Your Understanding

Regression

Predicting Continuous Values

What is Regression?

• Output is a continuous number
• Examples: prices, temperatures, scores
• Goal: minimize prediction error

Real-World Regression Tasks

Linear Regression: The Foundation

• w: weight (slope) - relationship strength
  • w is often referred to as the parameters of the linear model
• b: bias (intercept) - baseline value
  • How can b be a part of x and w?
• ŷ: predicted value

Multiple features: $\hat{y} = w_1x_1 + w_2x_2 + ... + b$

Example: House Price Prediction

Each point is a house, line shows learned relationship

Measuring Regression Performance

Mean Squared Error (MSE)

• Squares the errors (penalizes large errors more)
  • Why can it be bad to penalize large errors more? What is the impact of outliers?
• Always positive
• Lower is better

Beyond Linear: Polynomial Regression

When relationships are non-linear:

• Can fit curves, not just lines
• More flexible but risk of overfitting
• Need to choose degree carefully

Test Your Understanding

Classification

Predicting Categories

What is Classification?

• Output is a discrete category/class
• Examples: spam/not spam, digit recognition
• Can be binary or multi-class

Binary Classification

Two possible outcomes:

Output: probability between 0 and 1

Multi-Class Classification

More than two categories:

From Scores to Probabilities

The Softmax Function

Converts raw scores to probabilities:

Scores: [2.0, 1.0, 0.1]
Probabilities: [0.66, 0.24, 0.10]
Prediction: Class 0 (highest probability)

All probabilities sum to 1.0

Decision Boundaries

How classifiers separate classes:

Linear boundaries for simple problems, non-linear for complex ones

Measuring Classification Performance

Confusion Matrix

Visualizing classification errors:

Test Your Understanding

Tagging

Multi-Label Problems

Multi-Label vs Multi-Class

Real-World Tagging Applications

How Multi-Label Works

Independent binary classifiers for each label:

Input: Movie description
                        
Output probabilities:
- Action: 0.85 ✓
- Comedy: 0.72 ✓
- Drama:  0.23 ✗
- Horror: 0.05 ✗
- Sci-Fi: 0.91 ✓

Threshold: 0.5
Tags: [Action, Comedy, Sci-Fi]

Each label has its own probability threshold

Challenges in Multi-Label Learning

• Label Correlation: Some labels often appear together
• Label Imbalance: Some tags are rare
• Threshold Selection: Different thresholds for different labels
• Evaluation Complexity: Partial matches complicate metrics

Test Your Understanding

Search and Ranking

Ordering Results by Relevance

The Ranking Problem

Not just finding results, but ordering them:

• Most relevant at the top
• Personalized to user
• Consider multiple factors

Learning to Rank

ML learns what makes results relevant:

Applications Beyond Web Search

Test Your Understanding

Recommender Systems

Personalized Suggestions

Why Recommendations Matter

• 35% of Amazon purchases from recommendations
• 75% of Netflix viewing from recommendations
• Crucial for user engagement

Two Main Approaches

The Cold Start Problem

How to recommend when you have no data?

• New Users: No history to learn from
• New Items: No ratings or interactions

Matrix Factorization

Core technique for collaborative filtering:

User-Item matrix → User factors × Item factors

Learns latent features automatically

The Danger of Recommendation Rabbit Holes

Test Your Understanding

Sequence Learning

Temporal Dependencies

When Order Matters

Sequential data is everywhere:

• Language: Word order determines meaning
• Time Series: Stock prices, weather patterns
• Audio/Video: Temporal progression
• User Behavior: Click sequences, navigation paths

Challenges of Sequential Data

• Variable Length: Sequences can be any length
• Long-Range Dependencies: Early events affect later ones
• Temporal Patterns: Trends, seasonality, cycles
• Context Window: How much history to consider?

Types of Sequence Problems

Example: Speech Recognition

Recurrent Neural Networks Preview

Networks with memory:

For each time step t:
    hidden_state[t] = f(input[t], hidden_state[t-1])
    output[t] = g(hidden_state[t])

• Maintains state across time steps
• Can handle variable length sequences
• Foundation for LSTMsLong Short-Term Memory: RNN variant with gates (forget, input, output) to control information flow, solving vanishing gradient problem and GRUsGated Recurrent Units: Simplified LSTM with only reset and update gates, fewer parameters but similar performance

Test Your Understanding

Unsupervised Learning

Finding Hidden Patterns

Learning Without Labels

• No correct answers provided
• Find structure in data
• Often used for exploration

Clustering Example

Can you spot the groups?

The Hidden Clusters

Groups emerge naturally — these points were sampled from three different distributions

Try It: K-Means Clustering

The algorithm finds cluster assignments without ever seeing the true labels

Dimensionality Reduction

• Compress high-dimensional data
• Preserve important information
• Enable visualization
• What are examples of dimensionality reduction methods?

Masked Language Modeling

Foundation of modern NLP (BERT, GPT)

• Hide parts of the input text
• Model learns to predict missing words
• No manual labels needed!

Masked Image Modeling

Self-supervised learning for computer vision

• Mask random patches of images
• Model learns visual representations
• Used in MAEMasked Autoencoders (MAE): A self-supervised learning method by Meta AI that masks 75% of image patches and trains a Vision Transformer to reconstruct them, achieving state-of-the-art results on ImageNet., SimMIMSimple Masked Image Modeling (SimMIM): Microsoft's approach that uses a simple random masking strategy with a lightweight prediction head, proving that complex designs aren't necessary for masked image modeling., and others

Self-Supervised Learning

• Create labels from data itself
• Example: predict next word
• Foundation of modern NLP

Unsupervised Applications

• Customer segmentation
• Anomaly detection
• Data compression
• Feature learning

Test Your Understanding

Reinforcement Learning

Learning from Interaction

Beyond Supervised Learning

Not all learning needs labeled data

• Unsupervised Learning: Find patterns without labels
• Self-Supervised: Create labels from data itself
• Reinforcement: Learn from interaction

Unsupervised Learning

"Do some data science with it!"

• ClusteringGrouping similar data points together, like organizing photos by content or grouping users by behavior: Group similar items
• Dimensionality ReductionFinding the most important features that capture the essence of the data, like PCA: Find key features
• Density EstimationLearning the underlying probability distribution of the data to generate new samples: Learn data distribution
• Anomaly DetectionIdentifying unusual patterns that don't conform to expected behavior: Find outliers

Self-Supervised Learning

Creating supervision from the data itself

"The [MASK] jumped over the fence" → "dog"

Offline vs Online Learning

Traditional ML: Collect data → Train → Deploy

But what if our model affects the environment?

Reinforcement Learning

• Agent takes actions
• Environment provides rewards
• Learn through trial and error

Unsupervised and Self-Supervised Learning

Learning without explicit labels

Key Unsupervised Learning Tasks

• ClusteringFinding groups in data: photos → landscapes, dogs, babies, cats
  Group similar data points together
• Dimensionality ReductionLike describing body shape with just a few measurements for tailoring
  Find key parameters (PCA, subspace estimation)
• Representation LearningExample: "Rome" - "Italy" + "France" = "Paris"
  Map objects to meaningful vectors
• Causality DiscoveryUnderstanding relationships between house prices, pollution, crime, education
  Find root causes in data

Self-Supervised Learning

Creating supervision from the data itself

💡 No manual labeling required - the data provides its own supervision!

Deep Generative Models

Learning to generate new data

• Variational Autoencoders (VAEs)Encode data to latent space, then decode back. Can generate new samples by sampling the latent space. - 2014
• Generative Adversarial Networks (GANs)Two networks compete: generator creates fake data, discriminator tries to detect fakes. - 2014
• Normalizing FlowsTransform simple distributions to complex ones through invertible functions. - 2014-2017
• Diffusion ModelsGradually add noise to data, then learn to reverse the process to generate new samples. - 2020-2021

These models learn data distributions and can generate new, realistic samples

Interacting with an Environment

Supervised Learning: Collect data → Train offline → Deploy

This leads us to Reinforcement Learning...

Test Your Understanding

Historical Roots

Where It All Began

The Desire to Predict

Humans have sought to analyze data and predict outcomes for centuries

• Bernoulli distributionNamed after Jacob Bernoulli (1655-1705), models binary outcomes with probability p (1655-1705)
• Gaussian distributionDiscovered by Carl Friedrich Gauss (1777-1855), the normal distribution that appears everywhere in nature (1777-1855)
• Least mean squaresGauss's algorithm still used today for regression and optimization algorithm

Medieval Statistics: Köbel's Algorithm

Jacob Köbel (1460-1533): Early data collection

1. Measure 16 adult men's feet
2. Sum the measurements
3. Divide by 16 for average
4. Improvement: Remove outliers (shortest/longest)

The Rise of Modern Statistics

Ronald Fisher (1890-1962)

• Linear discriminant analysisA method for finding linear combinations of features that characterize or separate classes
• Fisher information matrixMeasures the amount of information that an observable random variable carries about unknown parameters
• Applications in genetics
• The famous Iris dataset (1936)

Information Theory & Computation

Biological Inspiration

Donald Hebb (1904-1985)

"Neurons that fire together, wire together"

• Hebbian learning ruleSynaptic connections strengthen when neurons activate simultaneously - the basis for associative learning
• Positive reinforcement of connections
• Foundation for gradient descent
• Inspired Rosenblatt's perceptron

Core Neural Network Principles

Key ideas that persist today:

• Layers: Alternating linear and nonlinear processing
• Backpropagation: Using chain rule to adjust all parameters
• Networks: Interconnected computational units

But from 1995-2005, neural networks fell out of favor...

The AI Winter (1995-2005)

Why neural networks struggled:

• Computational expense: Training was too slow
• Small datasets: MNIST's 60,000 digits was "huge"
• Better alternatives:
  • Kernel methods (SVMs)
  • Decision trees
  • Graphical models

Test Your Understanding

Neural Network History

• 1943: McCulloch-Pitts neuronFirst mathematical model of artificial neuron, using binary threshold logic
• 1958: PerceptronRosenblatt's linear classifier that could learn from examples
• 1986: BackpropagationAlgorithm to efficiently compute gradients in multi-layer networks
• 2012: Deep learning revolutionAlexNet wins ImageNet by huge margin, starting the modern deep learning era

The Deep Learning Era

• Enabled by GPUs
• Big data availability
• Algorithmic improvements
• Transforming every field

Test Your Understanding

The Road to Deep Learning

Data vs Compute Evolution

Key Algorithmic Breakthroughs

• Dropout (2014): Regularization by noiseRandomly drop neurons during training to prevent overfitting
• Attention (2014): Learnable memory pointersFocus on relevant parts of input, solving the long-sequence problem
• Transformers (2017): Attention-only architectureNo recurrence or convolution, just self-attention layers
• Scaling Laws (2020): Predictable improvements with size

Language Model Revolution

• GPT-3 (2020): 175B parameters, few-shot learningLearn new tasks from just a few examples in the prompt
• ChatGPT (2022): RLHFReinforcement Learning from Human Feedback for alignment
• GPT-4 (2023): Multimodal capabilities
• Open Models: LLaMA, Mistral democratizing AI

Generative Model Revolution

• GANs (2014): Adversarial trainingGenerator vs Discriminator: learning by competition
• Diffusion Models (2020): Denoising approachLearn to reverse noise addition process
• DALL-E 2 (2022): Text-to-image generation
• Stable Diffusion: Open-source creative AI

Distributed Training Breakthroughs

• Training on 1000+ GPUs simultaneously
• Batch sizes: 32 → 64,000 images
• ResNet-50 training: Days → 7 minutes
• Enabling massive model scale

Deep Learning Frameworks Evolution

• Gen 1: Caffe, Torch, Theano
• Gen 2: TensorFlow, Keras, CNTK
• Gen 3: PyTorchImperative, Pythonic, dynamic graphs, JAX
• From PhD homework → 10 lines of code

Test Your Understanding

AI Success Stories

Traditional ML Applications

• OCROptical Character Recognition for mail sorting since 1990s - Source of MNIST dataset
• Check reading & credit scoring
• Fraud detection (PayPal, Stripe, Visa)
• Search, recommendations, ranking

ML has been pervasive, albeit often hidden

Intelligent Assistants

• Siri, Alexa, Google Assistant (before 2022)
• ChatGPT, Claude, Gemini (after 2022)
• Speech recognition: Human parity5.1% word error rate matching professional transcribers (2018)
• From light switches to appointments
• Most visible AI in daily life

Computer Vision Breakthroughs

Also: Birdsong ID, skin cancer diagnosis

Game-Playing AI Milestones

• 1992: TD-Gammon (Backgammon)
• 1997: Deep BlueBeat Kasparov using massive parallelism & special hardware (Chess)
• 2016: AlphaGoDeep learning + Monte Carlo tree search (Go)
• 2017: LibratusHandled partial observability in no-limit poker (Poker)
• 2019: AlphaStar (StarCraft II)

Self-Driving Vehicles

• Tesla Autopilot, Waymo, NVIDIA Drive
• Deep learning for perceptionObject detection, lane detection, pedestrian tracking
• Challenges: Reasoning, rule incorporation
• Partial autonomy achieved, full autonomy pending

Scientific Applications

• Biology: AlphaFoldProtein structure prediction breakthrough
• Physics: Particle detection, gravitational waves
• Astronomy: Galaxy classification, exoplanet discovery
• Medicine: Drug discovery, diagnosis
• Climate: Weather prediction, climate modeling

AI Ethics & Concerns

• Not AGI: Task-specific, engineered systems
• Real concerns:
  • Job automation impact
  • Algorithmic bias
  • Privacy & surveillance
  • Decision transparency
• Need careful, ethical deployment

Test Your Understanding

The Essence of Deep Learning

What Makes It "Deep"?

• Many layersEach layer learns increasingly abstract representations of transformations
• Operations at each layer learned jointly from data
• Not just stacked processing steps
• Depth enables hierarchical feature learning

End-to-End Training Revolution

From Manual to Learned Features

• Before: Domain experts design features
• Problem: Human ingenuity is limited
• Deep Learning: Automatic feature discovery
• Result: Superior accuracy across domains

Millions of automatic choices beat manual design

Unified Tools Across Domains

• Same architectures work for:
  • Computer Vision
  • Natural Language Processing
  • Speech Recognition
  • Medical Imaging
• Eliminated domain-specific boundaries
• Transfer learning across modalities

Parametric → Nonparametric

• Scarce data era: Simplifying assumptions needed
• Big data era: Let data speak for itself
• Like physics: Analytical → Numerical simulations
• Trade-off: Accuracy vs Interpretability

The Empirical Revolution

• Accept suboptimal solutionsGood enough often beats theoretically perfect
• Embrace nonconvex optimizationLocal minima are often good enough in practice
• Try first, prove later
• Rapid experimentation culture

Open Source Culture

• Shared tools across academia & industry
• Released models & datasets
• Collaborative progress
• Lowered barriers to entry

From PhD homework → 10 lines of code

Test Your Understanding