# 9th Place Solution First of all, I would like to thanks the organizers for organizing such an exciting competition. Second I would like to thank my teammates. We had a great working environment with everybody working hard, and last one week, we had to stay up until 3 am local times. Below I will try to summarize the main idea of our best-performing model. Image Preprocessing One of the challenging things about this competition was that the chemical had very different structures, some were super long, and others were very short. This also meant that images were highly diverse in size and shapes (e.g., super long in 1 axis or super small). Therefore it was essential to preprocess these images correctly. After some initial discussion we decided to try to use two methods: 1) simple rescaling plus - we will get all the information (kind off) minus - feature of the images were become distorted (extreme shrinking across one axis) 2) rescaling with preserving the aspect ratio plus - This method will preserve the feature minus - bigger images, when downscaled, will have extremely small features Both of these methods have some advantages and disadvantages, but we reasoned that it would be best that each of our team members pick one of these methods and stick to it. We also found this an excellent article showing that most modern computer vision libraries employ very buggy resizing methods ( [assets/8768c36d7e350df653b47cec.pdf)](assets/8768c36d7e350df653b47cec.pdf)) . Since we are dealing with a lot of lines when downscaling, it was essential to preserve most of the information (tl dr: use PIL) Image Augmentation. We decided to go with natural image augmentations that were found in the dataset. But to make things a little bit diverse, we decided that everyone should train on a slightly different set of augmentations. Set 1: RandomRotate90 Set 2 RandomBorderCropPixel (cropping 5-20 pixel at random from the border of the image) RandomRotate90 RandomNoiseAugment Set 3 RandomRotate90 RandomBorderCropPixel (cropping 5-20 pixel at random from the border of the image) RandomNoiseAugment RandomLineRemoval RandomPiexlThreshold (some of the chemical lines had different sheds of black pixel values so we perform) thr = random.randint(200, 250) img[img < thr] = 0 img[img >= thr] = 255 Tokenizers [@yasufuminakama](https://www.kaggle.com/yasufuminakama) tokenizers was great, but we were wondering if there is a better way to tokenize. We found a lot of research on different tokenization strategies using SMILES but almost no information about InChI . After some thinking, we decided to try huggingface BPE tokenization. vocab = 128 # can be any tokenizer = Tokenizer(BPE(unk_token="[]", fuse_unk=False)) pre_tokenizer = pre_tokenizers.Sequence([Whitespace(), Digits()]) tokenizer.pre_tokenizer = pre_tokenizer trainer = BpeTrainer(vocab = max_len, special_tokens=["[]", "[]", "[]", "[]"]) We run a simple small-scale experiment using resnet34 and several BPE tokenizers using different vocab sizes. Below are results 42 character tokenzation - 9.7 64 BPE tokenization - 8.9 128 BPE tokenization - 8.01 246 BPE tokenization - 6.9 It clearly shows that more vocab size leads to better-learned embeddings. As per usual, in the end, we decided to use three different tokenizers to bring more diversity. 1) Y.nakama tokenizer 2) BPE - 512 3) BPE - 1024 Models We decided to train several different models with everything mentioned above, with progressive pseudo labeling on test data (take only valid inchis). In total, we had the following modules 1) effnetv2 -> encoder(6) -> decoder(8, 10, 12) 2) effnet3 -> encoder(3) -> decoder(3) 3) vit/cait/swin For vision transformers, we used hengs decoders. For hybrid CNN transformers to increase the diversity of our models, we used both Encoder/Decoder with attention layer with following modifications: 1) gating with GLUE ( [https://arxiv.org/abs/2002.05202)](https://arxiv.org/abs/2002.05202)) . In feed-forward layer, use GLUE in place of the first linear transformation and the activation function. 2) like 4th place solutions, we use relative positional embedding as in T5 Ensembling: We trained our model, everything looks great, but we needed to come with an ensembling strategy because everyone was using different tokenizers , so combining probabilities was not an option. Luckily [@zfturbo](https://www.kaggle.com/zfturbo) wrote a custom ensemble script that operates on string levels. It deserves its own post, which he will share later. Using [@zfturbo](https://www.kaggle.com/zfturbo) script, we were able to get from weakly scoring models (0.74 - 0.90) ~0.67 Classification for m0/m1 We noticed that most of our valid predictions were different only by m0/m1 . To solve this problem, we build a simple resnet50 model classifying images into 3 classes ( no m0/m1 , m0 , m1 ). Using this classifier to post-process our InChI , we gain a boost from 0.67 to 0.65 . Finetuning model only for long InChIs. We observed that our models tend to make a more wrong prediction for long InChIs . As a result last 4 days we decided to finetune our model only on long InChIs (bigger then 180 ). After finetuning, we took the longest 100k InChI in our test and made new predictions. We also applied TTA to images and got those predictions as well. Once we have those predictions, we used a modified version of the zfturbo script to make an ensemble only on the longest InChI . This eventually brought us to our final LB standing 0.65 -> 0.61 . Conclusion I would like to thank my teammates for being a great teamates and working really hard [@zfturbo](https://www.kaggle.com/zfturbo) , [@youhanlee](https://www.kaggle.com/youhanlee) and new GM [@tugstugi](https://www.kaggle.com/tugstugi) =) P.S also a big congratulations to all new GMs =)