# 3rd place solution: brief summary

First of all, I would like to thank the competition hosts, and all the participants for their hard work. This competition was very challenging, and I learned a lot.
I would also like to thank my teammates
[@lyakaap](https://www.kaggle.com/lyakaap)
[@bamps53](https://www.kaggle.com/bamps53)
. I am convinced that I could not have achieved this result without them.
Here I will try to summarize some of the main points of our solution.
Overview
Our solution consists of three phases:
Phase1: image captioning training
Phase2: InChI candidates generation
Phase3: InChI candidates reranking
![](assets/5173d088efea622ebd8b5f14.png)
Phase1: Image captioning training
One important aspect of this competition was the difference in data trends between the training and test data. At first, I struggled with the difference between CV and LB, but then we noticed that the test data had a higher amount of salt and pepper noise. Therefore, we solved the gap between CV and LB by adding augmentation of the salt and pepper noise during training.
![](assets/165d58101fd2f680706bce49.png)
Phase2: InChI candidates generation
In the phase3, the likelihood for pairs of an image and an InChI candidate are estimated by multiple models, and the InChI candidate having the highest likelihood is used as the final output. Therefore, in this phase, it is necessary to generate a variety of InChI candidates with good quality.
We generated a wide variety of InChI candidates by using various models as shown below and by using beam search:
Swin Transformer + BERT Decoder (by KF & lyakaap)
Transformer in Transformer + BERT Decoder (by KF)
EfficientNet-v2 followed by ViT + BERT Decoder (by lyakaap)
EfficientNet-B4 + Transformer Decoder (image_size:416x736) (by camaro)
EfficientNet-B4 + Transformer Encoder/Decoder(image_size:300x600) (by camaro)
Phase3: InChI candidates reranking
One of the key parts of our solution is reranking. In fact, most of the single model results are around LB0.9~0.8, but we were able to reach the final result (LB0.54) by reranking the generated results of multiple models using the logic shown below.
rdkit.Chem.MolFromInchi function is used to validate for each InChI candidate. (is_valid)
For each InChI candidate, calculate the loss (cross entropy / focal loss) used for training in multiple models, and average it across models. (loss)
Sort the candidates in descending order of “is_valid” → ascending order of “loss”, and the InChI with the highest score is the final output.
What didn't work
Object detection for atoms and bonds w/ yolov5
Reconstruct inchis like Dacon.ai's 1st place solution
Use detection results as additional input channels
MLM pre-training using extra_approved_InChIs.csv
Rank learning of InChI candidates (pointwise)
Perform real/fake classification and levenshtein’s distance prediction using a fake made by beam search as a negative example.