Tiny ImageNet Challenge is the first course project of Introduction to Deep Learning. It runs similar to the [ImageNet challenge] and [Stanford CS231n].
[ImageNet challenge]:http://www.image-net.org/challenges/LSVRC/2014/
[Stanford CS231n]:https://tiny-imagenet.herokuapp.com/

## Dataset
The Tiny ImageNet dataset has 100 classes. Each class has 1,000 training images, 100 validation images. The test set contains 10,000 images in total. We have released the training and validation sets with images and labels. The test set is released without labels. You are asked to predict the class label of each test image.

In detail, all training and validation images are from ImageNet training set. For test set, 5,000 images (id 0~4999) are selected from ImageNet validation set, while another 5,000 images (id 5000-9999) from [ImageNet-A]dataset, which called **Natural Adversarial Examples**. Modern deep CNNs achieve very low accuracy on these natural adversarial examples. ImageNet-A examples cause mistakes due to occlusion, weather, and other complications encountered in the long tail of scene configurations. You are recommended to read the [original paper] for more information.
[ImageNet-A]:https://arxiv.org/pdf/1907.07174.pdf
[original paper]:https://arxiv.org/pdf/1907.07174.pdf

For this course project, you need to consider how to achieve high classification accuracy on both general ImageNet images and natural adversarial examples.

## Submission
We use the test set accuracy to measure the performance. To submit your predictions on the test set, you need to upload a CSV file, which should be a two-column file with 10,001 lines (the first row is a header). Each line contains a pair of test image filename and its predicted class id (0-99). An example might look like:
```
Id,Category
0.jpg,0
1.jpg,1
2.jpg,2
...
9999.jpg,99
```
After the deadline, you are required to submit all codes and a short report (no less than one page). Plagiarism and labeling test images manually will be penalized.

##Evaluation
Competition scoring and ranking are based on two leaderboards : public and private. These two leaderboards are created by randomly splitting the test dataset. Public leaderboard is based on a subset of 40% of observations in test dataset you are submitting while the private leaderboard is based on the remaining 60% of data. The private leaderboard remains secret until the end of the competititon. It will determine the final competition ranking!
 
So far, all your submissions are based on the public leaderboard dataset. Using it, you are receiving a feedback about your performance. It can tell you how good you model is and how it is generalizing on unseen data.
 
However, the final leaderboard (60% of test dataset) will determine final result. Before the end of competition ranking (Submissions dead line : 08 November 2020 23:59, UTC+8), you will have to select your three favorite submissions. By default, Kaggle chooses the three submissions that have performed the best on public leaderboard. However you are free to select the three submissions you want.
 
Finally, these three submissions will result in the private leaderboard and so final ranking. In a Kaggle competition, the purpose of this division is to prevent people from winning by overfitting to the public leaderboard. Participants then are motivated to make sure their models will generalize well to the private leaderboard test set.
 
Therefore, your results on public and private leaderboard could be slightly different. 

## Grading Policy
Total marks for this project is 20:
- 14 marks based on rankings
   - 1st place: 14 marks
   - 2nd - last place: between 7 and 14 marks, in descending order
last place: 7 marks
- 6 marks based on the final report
   - correctness
   - novelty

Members in the same group get the same grade.

## References
Natural Adversarial Examples. https://arxiv.org/abs/1907.07174.
ImageNet LSVRC challenge. http://www.image-net.org/challenges/LSVRC.
Tiny ImageNet Visual Recognition Challenge. https://tiny-imagenet.herokuapp.com/.