Machine Learning with Nets

view on github

• 1. Convolutional Neural Networks
  • 1.1 Convolute
  • 1.2 Pooling
  • 1.3 Activation Functions
  • 1.4 Weight Initialization
  • 1.5 Batch Normalization
  • 1.6 Transfer Learning
  • 1.7 Optimizer
  • 1.8 Learning Rate Decay
  • 1.9 Regularization
• 2. CNN Architectures
  • 2.1 AlexNet
  • 2.2 VGGNet
  • 2.3 GoogLeNet
  • 2.4 ResNet
• 3. RNN (Recurrent Neural Networks)
  • 3.1 Genreal Applications
  • 3.2 Recurrence Formula
  • 3.3 Recurrent Model
  • 3.4 Model Applications
  • 3.5 LSTM (Long Short Term Memory)

---

• History Papers
  • A new model learning in graph domains
• Course Material
  • Course CS 231n: Convolutional Neural Networks for Visual Recognition
  • Course Video Link

1. Convolutional Neural Networks

1.1 Convolute

• Dynamic Demo Link for Convolution

• Also, Conv1D and Conv2D have their own application.

Above picture show the application of Conv1D in NLP. By using Conv1D with different kernel sizes, the matrix will be transformed into multiple column vectors.

While Conv2D will keep transform the original matrix to 2D matrices by multiple filters with concatentation.

1.2 Pooling

1.3 Activation Functions

1.4 Weight Initialization

• Xavier Initialization

1.5 Batch Normalization

1.6 Transfer Learning

• Demo Link

1.7 Optimizer

• SGD + Momentum

• Nesterov Momentum

• Adam

1.8 Learning Rate Decay

1.9 Regularization

• Dropout
• Dropout Lecture Link

2. CNN Architectures

2.1 AlexNet

2.2 VGGNet

2.3 GoogLeNet

• Due to huge computation demands, using bottlenect filter can reduce amount of data.

2.4 ResNet

3. RNN (Recurrent Neural Networks)

3.1 Genreal Applications

3.2 Recurrence Formula

[Sorry For Chinese]

3.3 Recurrent Model

3.4 Model Applications

3.5 LSTM (Long Short Term Memory)