# 8th Place Solution🔺

Congratulations to all the winners and thank the competition hosts for holding such a wonderful competition.
And especially thanks to
[@hengck23](https://www.kaggle.com/hengck23)
for sharing the great ideas and code. I think his post made this competition very active.
This competition was very tough for a solo kaggler like me and I was very inspired by other solo kagglers (
[@fergusoci](https://www.kaggle.com/fergusoci)
,
[@AkiraSosa](https://www.kaggle.com/AkiraSosa)
,
[@nofreewill](https://www.kaggle.com/nofreewill)
) on LB. Finally, I got solo gold and was able to become a Grandmaster at the expense of health and many holidays!
Models
I chose two models to train, SwinTransformer with image size 384x384 and patchwise TNT and I implemented it by referring to
[@hengck23](https://www.kaggle.com/hengck23)
's codes.
I used hyper parameter embed_dim=768, in_dim=48 for TNT and this is larger than TNT-B.
SwinTransformer and TNT both achieved LB~0.8 as single models and their CV were LD=0.80/0.80, CELoss=0.0030/0.0020
Data
I generated about 10M images using all the InChIs in
extra_approved_InChIs.csv
using
[@stainsby](https://www.kaggle.com/stainsby)
's
[notebook](https://www.kaggle.com/stainsby/improved-synthetic-data-for-bms-competition-v3)
. About 13M images are used in training. In the early stage of training, the loss for extra images was bad, but after long training, the loss for extra images and valid score improved. I trained SwinTransformer 15 epochs and TNT 20 epochs. SwinTransformer took 20 hours for 1 epoch and TNT took 12 hours for 1 epoch with 8 x V100.
Ensemble
I ensembled 8 models in each decoder step (weighted average of predict_proba). Models are trained competition data+extra data or finetuned by only competition data after that or different fold. It seems important to set total weight of SwinTransformer and TNT equally. CV score of ensemble model was LD=0.60 and CELoss=0.0013 and LB:0.63
Beam Search
On the last day of the competition, I implemented beam search by referring to fairseq
[code](https://github.com/pytorch/fairseq/blob/master/fairseq/sequence_generator.py)
.
By performing beam search with beam size 16, I could get LB:0.60.
In the later stages of the competition, I found that the prediction discrepancies of the different models of test data were only about 20k. Therefore, beam search or ensemble inference was performed only on 20k images.
PostProcess
I always used the rdkit validation postprocess shared by
[@nofreewill](https://www.kaggle.com/nofreewill)
.
Post-processing improvement diminished with score improvement, but a little improvement remained even at the end.
What didn't work
I trained a generative model by using InChIs from the  competition training data and generated 8M images(molecules), but it did not help improve CV.
I trained the object detection model to detect atoms and bonds like the DACON solution and overlaid bboxes on the images, but it did not help improve CV.
TTA(rotation, flip)