Bag of Tricks for Image Classification with ResNet50

Inspired by Tong He et. al, “Bag of Tricks for Image Classification with CNNs” paper. This report mostly follows the same tricks and settings.

The ResNet-50 model is used to do all the experiments and the CIFAR-10 &  ImageNet 1000 (mini) dataset are used. Since the original ImageNet dataset capacity is too massive to get hands on, the mini version found on Kaggle will be used instead. 

The tricks that are applied in the experiments are: 

1. ResNet tweaks

   • ResNet B (PyTorch Baseline)

   • ResNet C

   • ResNet D

2. Efficient Training

   • Low-precision training

   • Learning rate warmup

   • Linear Scale LR

3. Refinement

   • Cosine Learning Rate Decay

   • Label Smoothing

4. Knowledge Distillation

5. Experiment Settings & Results