{ "id": 240948, "title": "Single Model Performance", "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/240948", "author_name": "Chien-Hsiang Hung", "post_date": "2021-05-22T09:15:28.341000", "votes": 18, "comment_count": 5, "views": 0, "content": "

As far as we know that there are 3 main models adopted in this competition:

\n\n

I'm doing MLM right now with LB (w/o matching) scored from 0.17 to 0.25, which is pretty frustrating tbh. We've seen in another discussion, with QA, the LB could be achieved at 0.393 by @vanle73. Most ppl couldn't beat baseline with NER.

\n

Let's talk about your highest LB single model w/o matching 😊

", "messages": [ { "id": 1318370, "postDate": "2021-05-22T09:15:28.340Z", "content": "

As far as we know that there are 3 main models adopted in this competition:

\n\n

I'm doing MLM right now with LB (w/o matching) scored from 0.17 to 0.25, which is pretty frustrating tbh. We've seen in another discussion, with QA, the LB could be achieved at 0.393 by @vanle73. Most ppl couldn't beat baseline with NER.

\n

Let's talk about your highest LB single model w/o matching 😊

", "rawMarkdown": "As far as we know that there are 3 main models adopted in this competition:\n- NER\n- MLM\n- QA\n\nI'm doing MLM right now with LB (w/o matching) scored from **0.17 to 0.25**, which is pretty frustrating tbh. We've seen in another discussion, with QA, the LB could be achieved at **0.393** by @vanle73. Most ppl couldn't beat baseline with NER.\n\n**Let's talk about your highest LB single model w/o matching 😊**", "votes": 18 }, { "id": 1318767, "postDate": "2021-05-22T15:01:36.260Z", "content": "

am using QA and got 0.3 but still not able to beat the baseline after including the str match.
\nnot getting where am going wrong? BTW are you able to filter the predicted noise labels

", "rawMarkdown": "am using QA and got 0.3 but still not able to beat the baseline after including the str match. \nnot getting where am going wrong? BTW are you able to filter the predicted noise labels ", "votes": 1, "replies": [ { "id": 1319146, "postDate": "2021-05-23T00:29:21.990Z", "content": "

Hi @udaygurugubelli, what kind of including do you use? And, what do you mean that is the predicted noise labels?
\nI've been trying QA as well w/ the str match. It downgraded the str match LB. Is that what you meant by the predicted noise labels?

", "rawMarkdown": "Hi @udaygurugubelli, what kind of including do you use? And, what do you mean that is the predicted `noise labels`?\nI've been trying QA as well w/ the str match. It downgraded the str match LB. Is that what you meant by the predicted `noise labels`?", "replies": [ { "id": 1319238, "postDate": "2021-05-23T03:46:31.097Z", "content": "

yes exactly. thats what my point is? predictions downgrads the str match LB
\nthe kind of inclusion is just union of preds and str match. is there any particular strategy in this?
\nregarding noise labels means, the predictions that are not datasets, like some random words. there is no exact pattern to filter them

", "rawMarkdown": "yes exactly. thats what my point is? predictions downgrads the str match LB\nthe kind of inclusion is just union of preds and str match. is there any particular strategy in this?\nregarding noise labels means, the predictions that are not datasets, like some random words. there is no exact pattern to filter them" } ] }, { "id": 1319405, "postDate": "2021-05-23T06:53:11.517Z", "content": "

I've tried several ways to merge predictions, you can find it here. For the best LB right now in the kernel is using the last one though it has a scent of overfitting to the public-test-set to me 😂 given the extra data were generated from training-set according to @mlconsult. Union is more reasonable in my opinion.
\nWell, that's interesting, I agree with you, there must be a decent post-processing method that we haven't thought of.

", "rawMarkdown": "I've tried several ways to merge predictions, you can find it [here](https://www.kaggle.com/chienhsianghung/external-datasets-matching-mlm?rvi=1&scriptVersionId=63605706&cellId=38). For the best LB right now in the kernel is using the last one though it has a scent of overfitting to the public-test-set to me 😂 given the extra data were generated from training-set according to @mlconsult. Union is more reasonable in my opinion.\nWell, that's interesting, I agree with you, there must be a decent post-processing method that we haven't thought of.\n", "votes": 1 } ] }, { "id": 1319017, "postDate": "2021-05-22T19:25:32.590Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1318767, "author_name": "Uday Kumar Gurugubelli", "author_url": "", "post_date": "2021-05-22T15:01:36.260000", "content": "

am using QA and got 0.3 but still not able to beat the baseline after including the str match.
\nnot getting where am going wrong? BTW are you able to filter the predicted noise labels

", "votes": 1, "replies": [ { "id": 1319146, "author_name": "Chien-Hsiang Hung", "author_url": "", "post_date": "2021-05-23T00:29:21.990000", "content": "

Hi @udaygurugubelli, what kind of including do you use? And, what do you mean that is the predicted noise labels?
\nI've been trying QA as well w/ the str match. It downgraded the str match LB. Is that what you meant by the predicted noise labels?

", "votes": 0, "replies": [ { "id": 1319238, "author_name": "Uday Kumar Gurugubelli", "author_url": "", "post_date": "2021-05-23T03:46:31.097000", "content": "

yes exactly. thats what my point is? predictions downgrads the str match LB
\nthe kind of inclusion is just union of preds and str match. is there any particular strategy in this?
\nregarding noise labels means, the predictions that are not datasets, like some random words. there is no exact pattern to filter them

", "votes": 0, "replies": [] } ] }, { "id": 1319405, "author_name": "Chien-Hsiang Hung", "author_url": "", "post_date": "2021-05-23T06:53:11.517000", "content": "

I've tried several ways to merge predictions, you can find it here. For the best LB right now in the kernel is using the last one though it has a scent of overfitting to the public-test-set to me 😂 given the extra data were generated from training-set according to @mlconsult. Union is more reasonable in my opinion.
\nWell, that's interesting, I agree with you, there must be a decent post-processing method that we haven't thought of.

", "votes": 1, "replies": [] } ] }, { "id": 1319017, "author_name": "", "author_url": "", "post_date": "2021-05-22T19:25:32.590000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1318370": "As far as we know that there are 3 main models adopted in this competition:\n- NER\n- MLM\n- QA\n\nI'm doing MLM right now with LB (w/o matching) scored from **0.17 to 0.25**, which is pretty frustrating tbh. We've seen in another discussion, with QA, the LB could be achieved at **0.393** by @vanle73. Most ppl couldn't beat baseline with NER.\n\n**Let's talk about your highest LB single model w/o matching 😊**", "1318767": "am using QA and got 0.3 but still not able to beat the baseline after including the str match. \nnot getting where am going wrong? BTW are you able to filter the predicted noise labels ", "1319017": "" } }