{ "id": 349544, "title": "📊 🔀 Analysis of the updates Public LB->1st rerun ", "url": "/competitions/AI4Code/discussion/349544", "author_name": "", "post_date": "2022-09-01T17:36:24.917172700Z", "votes": 4, "comment_count": 5, "views": 0, "content": "

Now we have got the scores from the first rerun, first out of several reruns coming weeks.
\nCongrats to this update top 5 !
\n@dmytropoplavskiy
\n@yufuin
\n@hydantess
\n@goldenlock
\n@tomishere

\n

It's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.
\nMaybe not much data with only 1 update but with the public leaderboard we can get some view over the latest movement.
\nI'll try continue with updated analysis/plots during the evaluation period.

\n

\"\"
\nTotal LB distribution > .8
\n\"\"
\n\"\"
\n\"\"

", "messages": [ { "id": "1922763", "postDate": "09/01/2022 17:36:24", "content": "

Now we have got the scores from the first rerun, first out of several reruns coming weeks.
\nCongrats to this update top 5 !
\n@dmytropoplavskiy
\n@yufuin
\n@hydantess
\n@goldenlock
\n@tomishere

\n

It's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.
\nMaybe not much data with only 1 update but with the public leaderboard we can get some view over the latest movement.
\nI'll try continue with updated analysis/plots during the evaluation period.

\n

\"\"
\nTotal LB distribution > .8
\n\"\"
\n\"\"
\n\"\"

", "rawMarkdown": "Now we have got the scores from the first rerun, first out of several reruns coming weeks.\nCongrats to this update top 5 !\n@dmytropoplavskiy\n@yufuin \n@hydantess \n@goldenlock \n@tomishere\n\nIt's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.\nMaybe not much data with only 1 update but with the public leaderboard we can get some view over the latest movement.\nI'll try continue with updated analysis/plots during the evaluation period.\n\n\n![](https://i.postimg.cc/59Xj5gkx/upd2-1.png)\nTotal LB distribution > .8\n![](https://i.postimg.cc/Rhkkw4pn/upd2-2.png)\n![](https://i.postimg.cc/2yLpqVGD/upd2-3.png)\n![](https://i.postimg.cc/C1QTLjj7/upd2-4.png)", "votes": null }, { "id": "1922996", "postDate": "09/01/2022 22:44:16", "content": "

Thanks @kirderf, Interesting too see like bestfitting, simple gbdt and match up got even better result on new data, especially for bestfitting, good model generalization!

", "rawMarkdown": "Thanks @kirderf, Interesting too see like bestfitting, simple gbdt and match up got even better result on new data, especially for bestfitting, good model generalization!", "votes": null }, { "id": "1923047", "postDate": "09/02/2022 00:43:29", "content": "

I'm happy for @dmytropoplavskiy that he's up to 1st place with his original and well-thought out model architecture!

", "rawMarkdown": "I'm happy for @dmytropoplavskiy that he's up to 1st place with his original and well-thought out model architecture!", "votes": null }, { "id": "1923128", "postDate": "09/02/2022 02:42:22", "content": "

Yes he has most chance of winning finally as the score diff is only 0.006.

", "rawMarkdown": "Yes he has most chance of winning finally as the score diff is only 0.006.", "votes": null }, { "id": "1924746", "postDate": "09/03/2022 11:06:02", "content": "

Thank you.
\nI'd not trust much the first LB release, with such a small initial set of notebooks I'd still expect quite a bit of shake-up.

", "rawMarkdown": "Thank you.\nI'd not trust much the first LB release, with such a small initial set of notebooks I'd still expect quite a bit of shake-up.", "votes": null }, { "id": "1924755", "postDate": "09/03/2022 11:16:58", "content": "

Yes, but it at least showed that those teams without using external data has smaller score drop(due to original public lb with test set overlap leak).

", "rawMarkdown": "Yes, but it at least showed that those teams without using external data has smaller score drop(due to original public lb with test set overlap leak).", "votes": null } ], "comments": [ { "id": 1922996, "author_name": "goldenlock", "author_url": "", "post_date": "09/01/2022 22:44:16", "content": "

Thanks @kirderf, Interesting too see like bestfitting, simple gbdt and match up got even better result on new data, especially for bestfitting, good model generalization!

", "votes": null, "replies": [] }, { "id": 1923047, "author_name": "cabbage972", "author_url": "", "post_date": "09/02/2022 00:43:29", "content": "

I'm happy for @dmytropoplavskiy that he's up to 1st place with his original and well-thought out model architecture!

", "votes": null, "replies": [ { "id": 1923128, "author_name": "goldenlock", "author_url": "", "post_date": "09/02/2022 02:42:22", "content": "

Yes he has most chance of winning finally as the score diff is only 0.006.

", "votes": null, "replies": [] }, { "id": 1924746, "author_name": "dmytropoplavskiy", "author_url": "", "post_date": "09/03/2022 11:06:02", "content": "

Thank you.
\nI'd not trust much the first LB release, with such a small initial set of notebooks I'd still expect quite a bit of shake-up.

", "votes": null, "replies": [] }, { "id": 1924755, "author_name": "goldenlock", "author_url": "", "post_date": "09/03/2022 11:16:58", "content": "

Yes, but it at least showed that those teams without using external data has smaller score drop(due to original public lb with test set overlap leak).

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1922763": "Now we have got the scores from the first rerun, first out of several reruns coming weeks.\nCongrats to this update top 5 !\n@dmytropoplavskiy\n@yufuin \n@hydantess \n@goldenlock \n@tomishere\n\nIt's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.\nMaybe not much data with only 1 update but with the public leaderboard we can get some view over the latest movement.\nI'll try continue with updated analysis/plots during the evaluation period.\n\n\n![](https://i.postimg.cc/59Xj5gkx/upd2-1.png)\nTotal LB distribution > .8\n![](https://i.postimg.cc/Rhkkw4pn/upd2-2.png)\n![](https://i.postimg.cc/2yLpqVGD/upd2-3.png)\n![](https://i.postimg.cc/C1QTLjj7/upd2-4.png)", "1922996": "Thanks @kirderf, Interesting too see like bestfitting, simple gbdt and match up got even better result on new data, especially for bestfitting, good model generalization!", "1923047": "I'm happy for @dmytropoplavskiy that he's up to 1st place with his original and well-thought out model architecture!", "1923128": "Yes he has most chance of winning finally as the score diff is only 0.006.", "1924746": "Thank you.\nI'd not trust much the first LB release, with such a small initial set of notebooks I'd still expect quite a bit of shake-up.", "1924755": "Yes, but it at least showed that those teams without using external data has smaller score drop(due to original public lb with test set overlap leak)." }, "source": "meta" }