Computer Vision (CNN & Transfer Learning)

Module Completed: December 2021

---

🎯 Module Overview

Convolutional Neural Networks are the dominant architecture for computer vision. This module covers the full CNN anatomy — from convolution and pooling layers through to deep architectures (VGG, ResNet, InceptionNet) — and then Transfer Learning, which lets you leverage millions of pre-trained parameters as a starting point rather than training from scratch. Both are demonstrated on the plant species classification project.

---

📖 Topics Covered

The Limits of Traditional ML on Images

• SVC on images requires: RGB → Grayscale → Flatten 2D → 1D → PCA → model. Each step loses spatial information; accuracy is typically poor (~16% on 12-class plant dataset)
• Fully-connected ANN on images — same problem: spatial relationships are destroyed when flattening. No improvement with more hidden layers; computationally expensive with no benefit
• Conclusion: Image classification requires architectures that preserve and exploit spatial structure → CNNs

Convolutional Layer

The core building block of CNNs. A learnable filter (kernel) slides across the image and computes a dot product at each position:

• Feature maps — output of applying one filter; different filters detect edges, textures, shapes at different scales
• Parameter sharing — same filter weights applied across all spatial positions; dramatically fewer parameters than Dense layers
• Padding — 'valid' (no padding, shrinks output), 'same' (zero-pad to maintain spatial dimensions)
• Stride — step size of the filter; stride=2 halves the spatial dimension

Pooling Layer

Reduces spatial dimensions while retaining dominant features:

• Max Pooling — takes the maximum value in each window; preserves strongest activations; most common
• Average Pooling — takes the mean; smoother downsampling
• Provides translation invariance — small shifts in the image don't change the pooled output significantly
• MaxPooling2D(pool_size=(2,2)) halves height and width

Dropout in CNNs

Randomly drops entire feature map channels (Spatial Dropout) or individual neurons. Applied after Conv or Dense layers to prevent co-adaptation of feature detectors.

Typical CNN Architecture Pattern

Input Image
  → [Conv2D → Activation → MaxPool] × N   (feature extraction)
  → Flatten
  → Dense → Dropout                        (classification head)
  → Dense(num_classes, Softmax)            (output)

Deeper early layers: low-level features (edges, corners)
Deeper later layers: high-level features (shapes, objects)

Deep CNN Architectures

Major architectures used in transfer learning:

Architecture	Key Innovation	Depth	Top-1 ImageNet Acc
AlexNet	First deep CNN; ReLU; Dropout	8 layers	63%
VGG16/19	Small (3×3) filters stacked deep	16/19 layers	74%
ResNet50	Residual/skip connections — solves vanishing gradient in very deep nets	50 layers	80%
InceptionV3	Parallel filters of different sizes in same layer (Inception modules)	48 layers	82%
InceptionResNetV2	Combines Inception modules + Residual connections	572 layers	85%
MobileNet	Depthwise separable convolutions — lightweight; optimised for mobile	—	71%

Residual / Skip Connections (ResNet)

The key insight of ResNet: instead of learning H(x), learn the residual F(x) = H(x) − x. This means gradients can flow directly back through skip connections, enabling training of very deep networks (50–152+ layers) without vanishing gradient.

Transfer Learning

Using a model pre-trained on a large dataset (ImageNet, ~14M images, 1,000 classes) as a starting point:

Two strategies:

1. Feature Extraction — freeze all pre-trained layers; replace only the final classification head; train only the new head. Use when your dataset is small and similar to ImageNet.
2. Fine-tuning — unfreeze some or all pre-trained layers; continue training at a very low learning rate. Use when you have more data or your task differs from ImageNet.

base_model = InceptionResNetV2(weights='imagenet', include_top=False)
base_model.trainable = False   # Feature extraction
x = GlobalAveragePooling2D()(base_model.output)
x = Dense(256, activation='relu')(x)
output = Dense(num_classes, activation='softmax')(x)

ImageDataGenerator & Data Augmentation

Keras utility to load batches of images from disk (avoids loading all into memory) and apply on-the-fly augmentation:

• rotation_range, width_shift_range, height_shift_range
• horizontal_flip, vertical_flip
• zoom_range, shear_range
• rescale=1./255 — normalise pixel values to (0, 1)

Augmentation artificially expands the training set and improves generalisation when data is limited.

---

🏆 Project — Plant Species Classification (CV-1)

Submitted: 05 December 2021 Domain: Botanical Research

Business Problem: University X needs an automated classifier to identify plant species from photos — replacing manual inspection.

Dataset: 12 plant species, 4,767 images total (Kaggle Plant Seedlings Classification)
Classes: Black-grass, Charlock, Cleavers, Common Chickweed, Common Wheat, Fat Hen, Loose Silky-bent, Maize, Scentless Mayweed, Shepherds Purse, Small-flowered Cranesbill, Sugar Beet

Challenge: Images are all different dimensions → resize to median dimensions before training

Approach: Three Models Compared

Model	Preprocessing	Test Accuracy	Verdict
SVC	RGB→Grayscale→Flatten→PCA	~16%	Poor — spatial info lost
Feed-Forward ANN	RGB→Grayscale→Flatten	~14%	Poor — cannot learn image features
CNN (custom)	ImageDataGenerator (rescale + augmentation)	~80%	✅ Selected

CNN Architecture (Final Model)

• 3 × [Conv2D(filters, 3×3, ReLU) → MaxPooling2D(2×2)] blocks
• Flatten
• Dense(256, ReLU) → Dropout
• Dense(12, Softmax)
• Optimiser: Adam | Loss: Categorical Crossentropy

Training: Multiple trials at epochs 5, 10, 15, 20; best result at 20 epochs with augmentation

Why CNN Dominates

• Convolutional layers detect spatial features (leaf edges, vein patterns, textures) that are invisible after flattening
• Pooling retains dominant features while reducing computation
• Augmentation generates variation in rotation, scale, flip — plant images captured at arbitrary angles benefit greatly

Transfer Learning Bonus (InceptionResNetV2)

Applied InceptionResNetV2 pre-trained on ImageNet as feature extractor → fine-tuned classification head → significant accuracy improvement beyond custom CNN (referenced in notebook NOTE section)

Best model saved using Keras model.save() for inference on test image (Predict.png)

---

🔑 Key Learnings

1. Never use SVC/ANN for raw images — flattening destroys spatial relationships; CNN convolution is specifically designed to preserve them
2. Data augmentation is free training data — rotation, flip, zoom generate diversity from a small dataset; mandatory when you have < 10k images
3. ImageDataGenerator for large datasets — loading 4,767 images into RAM at once is wasteful; batch loading from disk is memory-efficient
4. Deeper CNNs ≠ always better — without skip connections (ResNet), very deep networks suffer vanishing gradients; use skip connections or modern architectures
5. Transfer learning requires much less data — a pre-trained VGG16/ResNet already knows edges, textures, shapes; you only need to teach it your specific classes
6. Fine-tuning at low learning rate — when fine-tuning pre-trained layers, use LR ≈ 1e-4 or lower to avoid destroying learned representations

---

🛠️ Tools & Libraries

import numpy as np
import matplotlib.pyplot as plt

import tensorflow as tf
from tensorflow import keras
from keras.models import Sequential, Model, load_model
from keras.layers import (Conv2D, MaxPooling2D, Dense, Dropout,
                           Flatten, GlobalAveragePooling2D, BatchNormalization)
from keras.optimizers import Adam
from keras.preprocessing.image import ImageDataGenerator
from keras.applications import (VGG16, VGG19, ResNet50,
                                  InceptionV3, InceptionResNetV2, MobileNet)
from keras.utils import to_categorical
from sklearn.decomposition import PCA
from sklearn.svm import SVC