![\"cv/lb](\"https://i.ibb.co/Ntx57Gz/image.png\")
First of all, many thanks to the Kaggle team and Feedback Prize team for hosting this competition, and congrats to all winners ! Thanks to my teammates too, namely @amedprof and @crodoc for their hard work and commitment during all those 3 laborious months.
\nWe struggle a lot during this comp’. We spent almost 2 months far from the bronze zone because our strategy was the exploration one. In fact, a great kernel was published by Abishek at the early beginning of the comp’ and one could just stick to it and keep tuning its hyperparameters to get a decent score (silver zone). But, we didn’t go that way and we kept exploring many ideas.
\nIt seems like the go-to approach here is NER. We didn’t reinvent the wheel, we went that way as well since it seems straightfull and answers the problem quite correctly. In fact, we tested some QA approaches but not only were they too slow (training and much more inference) but they had slightly lower scores than our NER approaches.
\nSetup 1: siblings NER & Span Segmentation
\nWe use a multitask approach to output the segmentation scores (multiclass classification on 3 different values: 0 for background, 1 when inside any entity and 2 for beginning) and the entity scores (15 classes). We still use 15 classes for the NER as well but before computing the class, we convert B-eginning tokens into I-nside.
Backbone: 5 folds Deberta-v1 Large + 5 folds Deberta-v1 xLarge (maxlen = 1024, stride during inference, positinon_biased_inputs=False), trained 5 epochs
\nScheduler: Cosine Annealing with no warmup
\nOptimizer: AdamW
\nLoss: Cross-Entropy with dynamic cosine-based class weights. In fact we overweight rare classes (Rebuttal and Counterclaim) by starting with high values and converge to 1 on final epochs.
Setup 2: Pure NER over 15 classes, alternatively over 10 classes by removing the B target for non-overlapping classes.
\nBackbone: 5 folds Deberta Large + 5 folds Deberta xLarge (maxlen = 1024, stride during inference, positinon_biased_inputs=False), trained 5 epochs
\nScheduler: Polynomial decay with 10% warmup
\nOptimizer: AdamW
\nLoss: Cross-Entropy (no weights over classes)
We used same validation strategy as the one shared by Abishek we also use an enhanced version of the clustesr made by cdeotte to make our MultilabelStratifiedKFold.
\nThese folds were really stable and CV to LB correlation was great.
As cdeotte’s team, we make use of WBF on our final solutions. WBF was very effective and was the main ingredient behind our spectacular jump in the final days of the competition. Take a look on this tab for further details about CV / LB.
\n
Training on 10 labels
\nWe saw in this kernel's markov transition matrix that some discourse_types never overlap so we decide to use B-I-O only on Claim and Evidence and binary target for others.
positinon_biased_inputs = False
\nThe HuggingFace Debrta-v1 version has positinon_biased_inputs parameter set to False by default, so Deberta-v1 is not using global position information by default. We manage to turn global position ON but the perfs are not much better than the vanilla one. Furthermore, turning positinon_biased_inputs OFF allows to use maxlen above 512 without the need to resize the embedding layers.
Thanks for reading our solution.
\nCongrats on 5th place and prize money @kneroma @amedprof and @crodoc. Your team has big jumps in the last days, good job!
", "votes": 7, "replies": [ { "id": 1725863, "author_name": "CroDoc", "author_url": "", "post_date": "2022-03-17T13:40:26.130000", "content": "Thank you @duykhanh99. Did you see @tascj0? This guy is just WOW! I am always amazed when someone gets such a good results with so little submissions. Just awesome.
", "votes": 2, "replies": [] }, { "id": 1725945, "author_name": "KhanhVD", "author_url": "", "post_date": "2022-03-17T14:31:31.777000", "content": "He is outlier :))
", "votes": 6, "replies": [] }, { "id": 1726016, "author_name": "kkiller", "author_url": "", "post_date": "2022-03-17T15:25:38.127000", "content": "Thanks @duykhanh99
", "votes": 1, "replies": [] } ] }, { "id": 1725958, "author_name": "gezi", "author_url": "", "post_date": "2022-03-17T14:39:29.960000", "content": "Thanks for sharing, better decode(GBDT sentence classification, WBF ensemble, Yolo like span detection, beam search) is the key for being on the top!
", "votes": 2, "replies": [ { "id": 1726019, "author_name": "kkiller", "author_url": "", "post_date": "2022-03-17T15:27:03.700000", "content": "Yes ! Second stage modeling has been very important in this comp'. Relying only on the direct output of the transformer model seems to be not enough.
", "votes": 1, "replies": [] }, { "id": 1726048, "author_name": "gezi", "author_url": "", "post_date": "2022-03-17T15:55:47.627000", "content": "en, loss function we try to optimize has big gap with final f1:)
", "votes": 0, "replies": [] } ] } ], "comment_count": 7, "messages": [ { "id": 1725697, "postDate": "2022-03-17T10:47:54.623Z", "content": "First of all, many thanks to the Kaggle team and Feedback Prize team for hosting this competition, and congrats to all winners ! Thanks to my teammates too, namely @amedprof and @crodoc for their hard work and commitment during all those 3 laborious months.
\nWe struggle a lot during this comp’. We spent almost 2 months far from the bronze zone because our strategy was the exploration one. In fact, a great kernel was published by Abishek at the early beginning of the comp’ and one could just stick to it and keep tuning its hyperparameters to get a decent score (silver zone). But, we didn’t go that way and we kept exploring many ideas.
\nIt seems like the go-to approach here is NER. We didn’t reinvent the wheel, we went that way as well since it seems straightfull and answers the problem quite correctly. In fact, we tested some QA approaches but not only were they too slow (training and much more inference) but they had slightly lower scores than our NER approaches.
\nSetup 1: siblings NER & Span Segmentation
\nWe use a multitask approach to output the segmentation scores (multiclass classification on 3 different values: 0 for background, 1 when inside any entity and 2 for beginning) and the entity scores (15 classes). We still use 15 classes for the NER as well but before computing the class, we convert B-eginning tokens into I-nside.
Backbone: 5 folds Deberta-v1 Large + 5 folds Deberta-v1 xLarge (maxlen = 1024, stride during inference, positinon_biased_inputs=False), trained 5 epochs
\nScheduler: Cosine Annealing with no warmup
\nOptimizer: AdamW
\nLoss: Cross-Entropy with dynamic cosine-based class weights. In fact we overweight rare classes (Rebuttal and Counterclaim) by starting with high values and converge to 1 on final epochs.
Setup 2: Pure NER over 15 classes, alternatively over 10 classes by removing the B target for non-overlapping classes.
\nBackbone: 5 folds Deberta Large + 5 folds Deberta xLarge (maxlen = 1024, stride during inference, positinon_biased_inputs=False), trained 5 epochs
\nScheduler: Polynomial decay with 10% warmup
\nOptimizer: AdamW
\nLoss: Cross-Entropy (no weights over classes)
We used same validation strategy as the one shared by Abishek we also use an enhanced version of the clustesr made by cdeotte to make our MultilabelStratifiedKFold.
\nThese folds were really stable and CV to LB correlation was great.
As cdeotte’s team, we make use of WBF on our final solutions. WBF was very effective and was the main ingredient behind our spectacular jump in the final days of the competition. Take a look on this tab for further details about CV / LB.
\n
Training on 10 labels
\nWe saw in this kernel's markov transition matrix that some discourse_types never overlap so we decide to use B-I-O only on Claim and Evidence and binary target for others.
positinon_biased_inputs = False
\nThe HuggingFace Debrta-v1 version has positinon_biased_inputs parameter set to False by default, so Deberta-v1 is not using global position information by default. We manage to turn global position ON but the perfs are not much better than the vanilla one. Furthermore, turning positinon_biased_inputs OFF allows to use maxlen above 512 without the need to resize the embedding layers.
Thanks for reading our solution.
\nCongrats on 5th place and prize money @kneroma @amedprof and @crodoc. Your team has big jumps in the last days, good job!
", "rawMarkdown": "Congrats on 5th place and prize money @kneroma @amedprof and @crodoc. Your team has big jumps in the last days, good job! ", "votes": 7, "replies": [ { "id": 1725863, "postDate": "2022-03-17T13:40:26.130Z", "content": "Thank you @duykhanh99. Did you see @tascj0? This guy is just WOW! I am always amazed when someone gets such a good results with so little submissions. Just awesome.
", "rawMarkdown": "Thank you @duykhanh99. Did you see @tascj0? This guy is just WOW! I am always amazed when someone gets such a good results with so little submissions. Just awesome.", "votes": 2 }, { "id": 1725945, "postDate": "2022-03-17T14:31:31.777Z", "content": "He is outlier :))
", "rawMarkdown": "He is outlier :))", "votes": 6 }, { "id": 1726016, "postDate": "2022-03-17T15:25:38.127Z", "content": "Thanks @duykhanh99
", "rawMarkdown": "Thanks @duykhanh99 ", "votes": 1 } ] }, { "id": 1725958, "postDate": "2022-03-17T14:39:29.960Z", "content": "Thanks for sharing, better decode(GBDT sentence classification, WBF ensemble, Yolo like span detection, beam search) is the key for being on the top!
", "rawMarkdown": "Thanks for sharing, better decode(GBDT sentence classification, WBF ensemble, Yolo like span detection, beam search) is the key for being on the top!", "votes": 2, "replies": [ { "id": 1726019, "postDate": "2022-03-17T15:27:03.700Z", "content": "Yes ! Second stage modeling has been very important in this comp'. Relying only on the direct output of the transformer model seems to be not enough.
", "rawMarkdown": "Yes ! Second stage modeling has been very important in this comp'. Relying only on the direct output of the transformer model seems to be not enough.", "votes": 1 }, { "id": 1726048, "postDate": "2022-03-17T15:55:47.627Z", "content": "en, loss function we try to optimize has big gap with final f1:)
", "rawMarkdown": "en, loss function we try to optimize has big gap with final f1:)" } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T12:37:40Z", "competition_id": 31779, "competition_slug": "feedback-prize-2021", "team_id": 7886310, "solution_url": "/competitions/feedback-prize-2021/writeups/now-you-see-me-5-th-place-simultaneous-span-segmen" }