First of all thanks to Google and Kaggle for the interesting competition, and to my teammates, @patilaum and @amancframewala for the generous contribution and patience.\nOur approach involved ensembling several BERT-family architectures like Bert, Distil Bert, XLNet trained separately on Questions and Answers to better predict the multi-labels. \nWe used 5 fold GroupKFold CV for validating our models. \nWe also were inspired by some of previous winning approaches of concatenating the hidden states of the models for prediction. \nThe models were averaged using rank-averaging which boosted the score by +0.002\nFinally, we used some basic thresholding for an easy boost for highly imbalanced predictors.
\n\nWhat did'nt work for us(but some worked for others):\nBART /T5 encoder modelling\nSWA\nTraining the head separately\nStackExchange statistical features\nBERT+USE\nGLOVE\nLSTM
\n\nWish we had properly implemented stuff like BART given that the top solutions found a great boost from them. We learned a lot from this competition and thank all the people who posted on the forums with tips and their results. Special thanks to our lab, CoE@ VJTI, for the hardware infra. Looking forward to more such NLP competitions in the year.
", "comments": [], "comment_count": 0, "messages": [ { "id": 743967, "postDate": "2020-02-12T12:52:56.917Z", "content": "First of all thanks to Google and Kaggle for the interesting competition, and to my teammates, @patilaum and @amancframewala for the generous contribution and patience.\nOur approach involved ensembling several BERT-family architectures like Bert, Distil Bert, XLNet trained separately on Questions and Answers to better predict the multi-labels. \nWe used 5 fold GroupKFold CV for validating our models. \nWe also were inspired by some of previous winning approaches of concatenating the hidden states of the models for prediction. \nThe models were averaged using rank-averaging which boosted the score by +0.002\nFinally, we used some basic thresholding for an easy boost for highly imbalanced predictors.
\n\nWhat did'nt work for us(but some worked for others):\nBART /T5 encoder modelling\nSWA\nTraining the head separately\nStackExchange statistical features\nBERT+USE\nGLOVE\nLSTM
\n\nWish we had properly implemented stuff like BART given that the top solutions found a great boost from them. We learned a lot from this competition and thank all the people who posted on the forums with tips and their results. Special thanks to our lab, CoE@ VJTI, for the hardware infra. Looking forward to more such NLP competitions in the year.
", "rawMarkdown": "First of all thanks to Google and Kaggle for the interesting competition, and to my teammates, @patilaum and @amancframewala for the generous contribution and patience.\nOur approach involved ensembling several BERT-family architectures like Bert, Distil Bert, XLNet trained separately on Questions and Answers to better predict the multi-labels. \nWe used 5 fold GroupKFold CV for validating our models. \nWe also were inspired by some of previous winning approaches of concatenating the hidden states of the models for prediction. \nThe models were averaged using rank-averaging which boosted the score by +0.002\nFinally, we used some basic thresholding for an easy boost for highly imbalanced predictors.\n\nWhat did'nt work for us(but some worked for others):\nBART /T5 encoder modelling\nSWA\nTraining the head separately\nStackExchange statistical features\nBERT+USE\nGLOVE\nLSTM\n\nWish we had properly implemented stuff like BART given that the top solutions found a great boost from them. We learned a lot from this competition and thank all the people who posted on the forums with tips and their results. Special thanks to our lab, CoE@ VJTI, for the hardware infra. Looking forward to more such NLP competitions in the year.", "votes": 4 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:09:02Z", "competition_id": 7968, "competition_slug": "google-quest-challenge", "team_id": 4312974, "solution_url": "/competitions/google-quest-challenge/writeups/vjti-coe-team-bronze-medal-solution" }