AlexNet Architecture (2012)

AlexNet (2012)

#

AlexNet won the ImageNet competition in 2012 and ushered in the deep learning era. Here are its key innovations.

6 Key Innovations

#

1. Deep Architecture

#

5 Convolutional Layers + 3 Fully Connected Layers
Total: 60 million parameters

A groundbreaking network depth for its time.

2. ReLU Activation

#

First major CNN to use ReLU (replacing tanh/sigmoid)

3. Dropout Regularization

#

50% Dropout applied to FC layers

A novel regularization technique for preventing overfitting.

4. GPU Training

#

Trained on 2x GTX 580 GPUs in parallel

Hardware acceleration enabling large-scale network training.

5. Data Augmentation

#

• Image translations
• Horizontal reflections
• PCA color augmentation

Artificially expanding training data for better generalization.

6. Local Response Normalization (LRN)

#

Normalization technique applied after ReLU.

Later replaced by Batch Normalization.

Architecture Summary

#