Chapter 8: Advanced Convolutional Neural Networks

SqueezeNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

MobileNet and data augmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

How good is the model really?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

Converting to Core ML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Challenges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

Key points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

 

---

SqueezeNet

 

 

The Keras functional API

 

MobileNet and data augmentation

- depthwise convolution

convolution : 기본적으로 convolution layer는 fully connected layer에서 spatial(이미지의 가로세로) 영역을 잘게 쪼개고, 그만큼 weight sharing을 시킨 아키텍쳐이다.

 

- Depth-wise Convolution : 한줄로 말하면 위의 Grouped Convolution에서 group의 개수를 input channel의 개수만큼 만들겠다는 것이다. 즉 1개의 group에 1개의 channel만 들어가 있다.

 

(참고 출처)

https://newsight.tistory.com/310

 

Adding the classifier

 

Data augmentation

You augment the training data through small random transformations

 

Training the classifier layer

It’s also smart to save a model checkpoint every so often. This is a copy of the model’s weights it has learned up to that point.

 

Fine-tuning the feature extractoe

fine-tuning the feature extractor.

With fine-tuning, you can adjust the knowledge inside the feature extractor to make it more relevant to your own data. Now the feature extractor itself already understands more about this specific task.

 

Regularization and dropout

Dropout is a special kind of layer that randomly removes elements from the tensor by setting them to zero.

 

Tune those hyperparameters

• the learning rate

• the dropout probability

• the weight decay factor for L2 regularization

  Choosing appropriate values for these settings — known as hyperparameter tuning — is essential for getting the training process to work optimally.

How good is the model really?

 

Precision, recall, F1-score

Precision means: how many of the images that were classified as being X really are X?

Recall means: how many of the images of class X did the model find? This is in some ways the opposite of precision.

The classification report also includes the F1-score. This is a combination of precision and recall and is useful if you want to get an average of the two.

 

What are the worst predictions?

 

A note on imbalanced classes

 

Converting to Core ML

 

 

* 단어

- augmentation 증가하다

- 합성곱(合成-, convolution, 콘벌루션) : 하나의 함수와 또 다른 함수를 반전 이동한 값을 곱한 다음, 구간에 대해 적분하여 새로운 함수를 구하는 수학 연산자이다.