# My brief summary of this competition[0.795]

It is an interesting competition,and also a tough one.
I faced the following challenges:
1.Large Dataset with 15+ millions images and 5000+ categories.
2.Some products and 1-4 images
3.CD/BOOK are very hard to classify.
4.I estimated the best overall accuracy will be 0.8 also,there are a lot of methods to choose and there are a large space to improve,so it’s very hard to win.
My solution summary:
1.Making preparations for such a big dataset
2.Finetuning pretrained models.(0.759/0.757,inception-resnet-v2,0.757/0.756 resnet50).
3.Make full use of multi-images of a product. (0.772/0.771 inception-resnet-v2 and 0.769/0.768+ Resnet50 models).
4.Use OCR to add semantics to the models.
Preparing:
At early stage of this competition,I spent 4 days on trying to feed the images to pytorch efficiently,I I found the best way is to buy 512M SSDs.
I also found that if I want to store prediction results into a file,I must use sparse matrix,I spend 1.5 days on writing codes to make sure I can keep probabilities of  prediction in a small file(~40M)
Finetuning pretrained models.
We can not win such a competition only by downloading pretrained models and finetuning them,we must understand the principles behind every models.
After got some statistics of this data set,I searched and read all kinds of papers related:network structures,the optimizers,the train plan．．．．．．
Then I started my experiments on Resnet34,I have following result:
1.The network structure of almost all pretrained model are for imagenet with 1000 labels ,but our dataset have 5270 labels.If we use then directly,there will be some representation bottleneck.
2.I found train with SGD need more epochs to converge than ADAM.
So I modified the head of resnet34,add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270.
And I used Adam with optim.Adam(net.parameters(), lr=lr) and lr plan is:
lr = 0.0003
        if epoch > 7:
            lr = 0.0001
        if epoch > 9:
            lr = 0.00005
        if epoch > 11:
            lr = 0.00001
When I  trained 11.5 epochs with 160x160 patch random cropped from 180*180 images,I predicted and averaged predictions of the images of product,I could get a score >0.72 on public LB.
Then I thought,why not add a FC layer more?So I added a 5270x5270,and I found that it improved more than 0.5
Since I can train 1 epochs in 2.5 hours on a 4x 1080Ti machine,I could do a lot of experiments,I found if the result is not so good on first 1-2 epochs,the result not be good at end,this speed up my experiments.
I tried but not used in the next stage:
1.Multi-level categories as multi-task.
2.Hard Example,Focal Loss,
3.Dilation
4.Dropout
After I got these conclusion,I turned to resnet50,I found resnet50 is better than resnet34: 0.756 on public.
Because I am busy with my everyday job ,I just let my GPUs keep running,and fine-tuned Resnet101,resnet152,and inceptionresnetv2 and inceptionV4.
Inceptionv4 is a little weak at this stage,others could easily get a score >0.755 on public LB.
Make full use of multi-images of a product
Last weeks of this competition,I had more time,and I started analyzing the dataset.
I trained my models using images of a product separately before,but when we have a look at a product,we will treat the images as a whole.So I concatenate the images into a image,and fine-tune the models.
That’s to say,We can split the trainset  into 4 parts,products with 1,2,3,4 images,if we finetune products with 4 images ,we concatenate the 4 images into a image.if we finetune products with 3 images ,we concatenate the 3 images into a image. I also finetune the model with product with only 1 images.
Because I found that there are some invalid or meaningless images for classification in images for multi-images products,that will be noise for product with only on product.It was easy to finetune,the acc improved a lot after 2-3 epochs,it took no so much time.
After this step,I got a cluster of model for resnet50,a model trained with all images,4 models for 1,2,3,4 images products.I predicted the product accordingly and ensembled them,I got a big improvement.Now,Almost all the models can get acc score near 0.77 .
I made several submissions just now,I found that if I ensemble resnet50 models with inception-resnet-v2 models,we can get 0.782/0.781 on  private and public LB.
OCR
I found that CD and BOOK are very hard to classify,because we need know the meaning of the cover!
And I want to evaluate the performance of all kinds of algorithms OCR related for my next project,so I spent more than a week on it,I use CTPN to extract the boxes containing text,the result was quite good.Then I used CRNN to extract the text from the boxes.But I found that the text are not so good,because the images are small sized and CRNN was trained for English and I have no French dataset to retrain it,I can generate it,but I can not found suitable books without copyright limitation to extract French words.Without good words extracted from CD and BOOK cover,I could not use Word2Vec,so I can not provide semantic meaning to the model. As a last resort,I extracted the features of boxes from CRNN and feed them into my multi-input CNN,the first input is Resnet50 FC features and the second is CRNN features.
After got the prediction from this OCR-Network,I ensembled with the predictions of  Resnet50 ,I make a lot of submissions in last week to confirm I did not over-fit.
I found that OCR solution  contributed ~0.14% to my final submissions.
[Edit],I found that if I use OCR on 0.782 model,the result can improve 0.35%,this means that the improvement got from OCR is covered by multi-model ensemble.
Other models
I also tried densenet161,densenet169,dpn92,they are much worse than resnet50,since I have trained them,so I just ensembled them to add some diversities.
After ensembled the models,the public LB can reach 0.79.
To reach my final scored,I fine-tuned the 1-image products models with 224/299 sized images accordingly,to add more diversities,I also change the head to two 4096*4096 FC layer following VGG net and ensemble them.
if you want to reproduce my solution,and if you have 4-1080Ti,you can get 0.782/0.781 on private/public LB in 7-8 days with resnet50 and inception-resnet-v2.