# 47th Place Solution & Code

Summary
First, let me congratulate all the winners and thank the competition hosts - it was a really interesting challenge! It was great to be a part of a competition requiring both CV and NLP skills and having such a large dataset with a very minimal shakeup.
The (not so) funny thing is that I was confident that the competition deadline is actually two days later, 05.06. I mistakingly created a calendar entry for the deadline and never checked if it's actually correct. What a surprise for me was to check out the competition page yesterday and realize that it was just 10 hours before the end! 😅
My solution is an ensemble of seven CNN-LSTM Encoder-Decoder models. All models are implemented in PyTorch and trained on a local machine with Quadro RTX 6000 GPU.
Code
[Kaggle notebook](https://www.kaggle.com/kozodoi/47th-place-solution-bms-ensembling)
reproducing my submission and ensembling pipeline
[GitHub repo](https://github.com/kozodoi/BMS_Molecular_Translation)
with the complete training codes
Data
single 80/20 train/test split stratified by the molecule length
creating 3M extra images from
extra_approved_InChIs
using RDKit (thanks
[@tuckerarrants](https://www.kaggle.com/tuckerarrants)
)
on each epoch, I was taking all train images + random 1M extra images for training
Tokenizer
my tokenizer is very similar to the one proposed by
[@yasufuminakama](https://www.kaggle.com/yasufuminakama)
in his great pipeline
using training + extra data to fit the tokenizer, so it ended up having about 20 more tokens due to more distinct molecules in the data
Image augmentations
making lines and letters thicker with cv2 morphological transformations:
image = cv2.morphologyEx(image, cv2.MORPH_OPEN, np.ones((2, 2)))
image = cv2.erode(image, np.ones((2, 2)))
small image rotations during training:
ShiftScaleRotate(0.01, 0.01, 0.10)
cropping pictures with large empty borders (thanks
[@markwijkhuizen](https://www.kaggle.com/markwijkhuizen)
)
rotating test images with height > width (thanks
[@hengck23](https://www.kaggle.com/hengck23)
)
![images](assets/55b52213fb75a25a212f7c8c.jpg)
Base models
All models used EfficientNet-based CNN encoder and LSTM-based RNN decoder. The table below from Neptune.ai provides the main architecture and training parameters of my top models:
![models](assets/da8e2a6fa9123cbd48b44b8d.jpg)
Ensembling & post-processing
using beam search with k = 5 on most of the base models (thanks
[@tugstugi](https://www.kaggle.com/tugstugi)
)
normalizing each model predictions with RDKit (thanks
[@nofreewill](https://www.kaggle.com/nofreewill)
)
ensembling predictions from base models using majority voting
setting "unconfident" predictions to the ones produced by the lowest-CV model that has a valid prediction according to RDKit (thanks
[@nofreewill](https://www.kaggle.com/nofreewill)
again)
What else I planned to try
adding image meta-data such as pixel resolution and height/width ratio as input features
iteratively predicting different parts of the InChI (formula, /c and /h parts) with separate models that would take image and predictions of the previous molecule part as input
switching to transformers. I only started working on transformer pipeline a few days ago and did not have enough time to train a heavy model due to limited resources
The final solution achieves
1.32
on the public and
1.31
on the private LB (47th place). I hope this summary was useful for some of you. Happy to answer any questions in the comments and see you in the next competitions! 😊