I was kind of expecting the shakeup because of the saturated leaderboard and no other models were released with a higher LB.
\nBut to my surprise, MuRIL-Large outperformed xlm-roberta-large (Not in the LB). The xlm-roberta was overfitted to the public LB.
\nMuril-Large was introduced lately to the competitors and thanks to @simrankhanuja for introducing the model.
\nSimple blend of 5-fold of MuRIL-Large trained on chaii, mlqa and xquad dataset.
\nThe public LB score was 0.75, did some tweaking, the score decreased.
\nChecked the CV, then lost tracks :)
\nNo-one retrained the MuRIL model so no 0.745 score (my trained model was private, now made public)
\nThe treasure was in-front my eyes, just that it was covered with mud :(
\nAnd as @cdeotte says, 'Trust your CV'
\nSince this was my first competition I will keep that in mind.
\nAnyways these were my kernels and all were public :)
\nYou always be my best partner in anywhere!✊✊✊
", "votes": 1, "replies": [ { "id": 1584096, "author_name": "Kishal Mandal", "author_url": "", "post_date": "2021-11-16T09:13:10.257000", "content": "Thanks a lot man :)
", "votes": 1, "replies": [] } ] }, { "id": 1583780, "author_name": "Vineeth", "author_url": "", "post_date": "2021-11-16T04:04:44.417000", "content": "Hi @kishalmandal, same is the case with me had a MuRIL 5 fold submission with private LB of 0.754 which would have put me in silver zone.It had a public LB 0.769 didn't select it for my final submission. Lessons learnt. But anyways thanks for all your contributions in this competiton.
", "votes": 1, "replies": [ { "id": 1584097, "author_name": "Kishal Mandal", "author_url": "", "post_date": "2021-11-16T09:13:46.923000", "content": ":) sad for us. Best of luck for upcoming competitions
", "votes": 0, "replies": [] } ] }, { "id": 1584129, "author_name": "Luca Massaron", "author_url": "", "post_date": "2021-11-16T09:40:59.753000", "content": "The mantra after a competition is always \"trust your cv\" but when you are in the heat of the competition your eyes always fall for the leaderboard. We need more discipline. But what to do when your cv is unreliable? All top solution claim: \"we didn't watch cv\". It reminds me the meme of the IQ curve XD
", "votes": 2, "replies": [] }, { "id": 1583637, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-11-16T01:17:25.280000", "content": "After reading winners' solutions, it does appear that MuRIL-Large was a powerhouse in this competition. Does it have a better CV than XLM-Roberta-large?
No really. Tried with 2 different seeds. CV was around 0.66-0.68 range and public LB was 0.75 and 0.74 something. So gave up on MuRIL
", "votes": 1, "replies": [] } ] } ], "comment_count": 7, "messages": [ { "id": 1583625, "postDate": "2021-11-16T01:13:43.310Z", "content": "I was kind of expecting the shakeup because of the saturated leaderboard and no other models were released with a higher LB.
\nBut to my surprise, MuRIL-Large outperformed xlm-roberta-large (Not in the LB). The xlm-roberta was overfitted to the public LB.
\nMuril-Large was introduced lately to the competitors and thanks to @simrankhanuja for introducing the model.
\nSimple blend of 5-fold of MuRIL-Large trained on chaii, mlqa and xquad dataset.
\nThe public LB score was 0.75, did some tweaking, the score decreased.
\nChecked the CV, then lost tracks :)
\nNo-one retrained the MuRIL model so no 0.745 score (my trained model was private, now made public)
\nThe treasure was in-front my eyes, just that it was covered with mud :(
\nAnd as @cdeotte says, 'Trust your CV'
\nSince this was my first competition I will keep that in mind.
\nAnyways these were my kernels and all were public :)
\nYou always be my best partner in anywhere!✊✊✊
", "rawMarkdown": "You always be my best partner in anywhere!✊✊✊", "votes": 1, "replies": [ { "id": 1584096, "postDate": "2021-11-16T09:13:10.257Z", "content": "Thanks a lot man :)
", "rawMarkdown": "Thanks a lot man :)", "votes": 1 } ] }, { "id": 1583780, "postDate": "2021-11-16T04:04:44.417Z", "content": "Hi @kishalmandal, same is the case with me had a MuRIL 5 fold submission with private LB of 0.754 which would have put me in silver zone.It had a public LB 0.769 didn't select it for my final submission. Lessons learnt. But anyways thanks for all your contributions in this competiton.
", "rawMarkdown": "Hi @kishalmandal, same is the case with me had a MuRIL 5 fold submission with private LB of 0.754 which would have put me in silver zone.It had a public LB 0.769 didn't select it for my final submission. Lessons learnt. But anyways thanks for all your contributions in this competiton.", "votes": 1, "replies": [ { "id": 1584097, "postDate": "2021-11-16T09:13:46.923Z", "content": ":) sad for us. Best of luck for upcoming competitions
", "rawMarkdown": ":) sad for us. Best of luck for upcoming competitions" } ] }, { "id": 1584129, "postDate": "2021-11-16T09:40:59.753Z", "content": "The mantra after a competition is always \"trust your cv\" but when you are in the heat of the competition your eyes always fall for the leaderboard. We need more discipline. But what to do when your cv is unreliable? All top solution claim: \"we didn't watch cv\". It reminds me the meme of the IQ curve XD
", "rawMarkdown": "The mantra after a competition is always \"trust your cv\" but when you are in the heat of the competition your eyes always fall for the leaderboard. We need more discipline. But what to do when your cv is unreliable? All top solution claim: \"we didn't watch cv\". It reminds me the meme of the IQ curve XD ", "votes": 2 }, { "id": 1583637, "postDate": "2021-11-16T01:17:25.280Z", "content": "After reading winners' solutions, it does appear that MuRIL-Large was a powerhouse in this competition. Does it have a better CV than XLM-Roberta-large?
No really. Tried with 2 different seeds. CV was around 0.66-0.68 range and public LB was 0.75 and 0.74 something. So gave up on MuRIL
", "rawMarkdown": "No really. Tried with 2 different seeds. CV was around 0.66-0.68 range and public LB was 0.75 and 0.74 something. So gave up on MuRIL", "votes": 1 } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T16:19:53Z", "competition_id": 30060, "competition_slug": "chaii-hindi-and-tamil-question-answering", "team_id": 7430689, "solution_url": "/competitions/chaii-hindi-and-tamil-question-answering/writeups/piros-would-have-been-35th-place-lb-0-75-pb-0-745" }