# 70th place solution and code

Hi all!
Thanks to the competition host and for higly useful advices in the discussion! It’s my first competition and I am glad to get a bronze :)
My solution is very simple and based on smile targets. It was hard to make enough experiments due to I have had only a single 2080ti. To some speed up the training, I used torch amp.
Summary of the solution
The main pipeline is similar to
[@yasufuminakama](https://www.kaggle.com/yasufuminakama)
great
[notebook](https://www.kaggle.com/yasufuminakama/inchi-resnet-lstm-with-attention-starter)
, but as encoder I use EfficienNet_B3. The main difference - instead of using InChI-target I use
[Smile](https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system)
notation. On inference I convert Smile back to InChI via
[rdkit](https://github.com/kuelumbus/rdkit_platform_wheels)
and as tokenizer I use
[XinhaoLi74/SmilesPE](https://github.com/XinhaoLi74/SmilesPE)
. Smiles is much more simpler for prediction - shorter notation and less token-classes - and this gave me great improvement in accuracy (from
6.31
to
2.22
).
Data
I use syntheic images and extra_approved_InChIs.csv of course - about 1 million images (thanks for the great
[notebook](https://www.kaggle.com/tuckerarrants/inchi-allowed-external-data)
by
[@tuckerarrants](https://www.kaggle.com/tuckerarrants)
and
[notebook](https://www.kaggle.com/stainsby/improved-synthetic-data-for-bms-competition-v3)
by
[@stainsby](https://www.kaggle.com/stainsby)
)
I have make a lot of experiments - in hyperparameters, batchsampler, backbones, etc.,  but not as many as I wanted because training was soooo long. I ended up with 288x288 images size, as a compromise between accuracy and training speed.
I made an adaptive batchsampler that was quite helpful - it increase probability for a sample to be added in batch by sample's loss during training. So the model was trained on "harder" examples, and samples with almost zero loss wasn’t put in batch often.
Some other notes
There was some little accuracy improvement (about 1.8 -> 1.6) due to freeze encoder and train LSTM-decored only. It significantly speed up training and allowed to increase batch size in final stage of the training.
Also as optimizer I use AdamW with weight_decay 0.01 (as
[article](https://www.fast.ai/2018/07/02/adam-weight-decay/)
suggest) but don't sure that it get any influence at all :) As image augmentations were used only gaussian blur and various rotations (by randomly transpose and vertical/horizontal flipping).
Other experiments I wanted to try
To split InChI up to 8 indepenpent string layers (separated by the "/" notation: "/b", "/t", "/m", "/s", etc.) and than train individual models for each layer. Each layer in the InChI describe different information about a molecule, and several models, trained on separate inchi-layers, presumably can get good results.
To add rotation transoform to any angles (it might help the model to catch some useful feature on images)
To add beam search and some models ensembling.
Trying Transformers instead of LSTM
The source code (pytorch) is available in the github
[repository](https://github.com/skalinin/BMS-competition)
.
Congrat to all who took a part in the competition:)