{ "id": 427002, "title": "Question about the high-scored public notebooks", "url": "/competitions/cafa-5-protein-function-prediction/discussion/427002", "author_name": "", "post_date": "2023-07-26T04:27:53.025681100Z", "votes": 4, "comment_count": 6, "views": 0, "content": "

In the high-scored public notebooks, I have a question about the ensemble part. As we see in the code:
\n\"\"
\nThe two results are merged with how='outer'. If there are differential GO terms from one to the other, the outcome will contain nan, and thus we can use this code to ensemble:

\n
submissions = submissions(lambda row: row  not np(row)  row, axis=)\n
\n

But the two submissions used for ensemble are all trained with num_labels = 500. That is, the GO terms in the two submissions are the same, and there wouldn't have nan in the merged output. Is it right?
\nWish to see your views.

", "messages": [ { "id": "2359213", "postDate": "07/26/2023 04:27:53", "content": "

In the high-scored public notebooks, I have a question about the ensemble part. As we see in the code:
\n\"\"
\nThe two results are merged with how='outer'. If there are differential GO terms from one to the other, the outcome will contain nan, and thus we can use this code to ensemble:

\n
submissions = submissions(lambda row: row  not np(row)  row, axis=)\n
\n

But the two submissions used for ensemble are all trained with num_labels = 500. That is, the GO terms in the two submissions are the same, and there wouldn't have nan in the merged output. Is it right?
\nWish to see your views.

", "rawMarkdown": "In the [high-scored public notebooks](https://www.kaggle.com/code/siddhvr/cafa5-using-protbert-embeds/notebook), I have a question about the ensemble part. As we see in the code:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3110858%2Fba1e1f3031c5d89c4fc745c02ab52d91%2FWX20230726-1212072x.png?generation=1690345078224152&alt=media)\nThe two results are merged with `how='outer'`. If there are differential GO terms from one to the other, the outcome will contain `nan`, and thus we can use this code to ensemble:\n```\nsubmissions['confidence_combined'] = submissions.apply(lambda row: row['Confidence2'] if not np.isnan(row['Confidence2']) else row['Confidence'], axis=1)\n```\nBut the two submissions used for ensemble are all trained with `num_labels = 500`. That is, the GO terms in the two submissions are the same, and there wouldn't have `nan` in the merged output. Is it right?\nWish to see your views.", "votes": null }, { "id": "2359243", "postDate": "07/26/2023 04:52:44", "content": "

I think that when it applies the lambda, it always prefers the value in submission2. Therefore, even if you are changing the hyperparameters or even the model, you will still get the same scores.

", "rawMarkdown": "I think that when it applies the lambda, it always prefers the value in submission2. Therefore, even if you are changing the hyperparameters or even the model, you will still get the same scores.", "votes": null }, { "id": "2359255", "postDate": "07/26/2023 05:01:17", "content": "

Yes, I think there is no difference between the unseen submission2 and the ensemble output.

", "rawMarkdown": "Yes, I think there is no difference between the unseen `submission2` and the ensemble output.", "votes": null }, { "id": "2359722", "postDate": "07/26/2023 10:56:05", "content": "

some people use such tricks to earn upvotes on kaggle
\nThe notebook is doing not what we might expect from the first look.
\nBut since not many go into details - upvotes are obtained

", "rawMarkdown": "some people use such tricks to earn upvotes on kaggle\nThe notebook is doing not what we might expect from the first look.\nBut since not many go into details - upvotes are obtained", "votes": null }, { "id": "2359733", "postDate": "07/26/2023 11:04:48", "content": "

Thanks for your explanation.

", "rawMarkdown": "Thanks for your explanation.", "votes": null }, { "id": "2361278", "postDate": "07/27/2023 10:01:21", "content": "

Try submissions.tail(). There is a lot of \"headless\" Confidence 😅
\nAlthough it merge with how='outer', the submission_df unique output is suppressed by submissions.drop 😅
\nI assumed once you add them back, the score will drop 😝

", "rawMarkdown": "Try `submissions.tail()`. There is a lot of \"headless\" Confidence 😅\nAlthough it merge with `how='outer'`, the `submission_df` unique output is suppressed by `submissions.drop` 😅\nI assumed once you add them back, the score will drop 😝", "votes": null }, { "id": "2365124", "postDate": "07/30/2023 02:17:43", "content": "

Yes. I tried to directly submit the \"submission2\" and got the exact same result. lol

", "rawMarkdown": "Yes. I tried to directly submit the \"submission2\" and got the exact same result. lol", "votes": null } ], "comments": [ { "id": 2359243, "author_name": "jeremyxu677", "author_url": "", "post_date": "07/26/2023 04:52:44", "content": "

I think that when it applies the lambda, it always prefers the value in submission2. Therefore, even if you are changing the hyperparameters or even the model, you will still get the same scores.

", "votes": null, "replies": [ { "id": 2359255, "author_name": "takanashihumbert", "author_url": "", "post_date": "07/26/2023 05:01:17", "content": "

Yes, I think there is no difference between the unseen submission2 and the ensemble output.

", "votes": null, "replies": [ { "id": 2359722, "author_name": "alexandervc", "author_url": "", "post_date": "07/26/2023 10:56:05", "content": "

some people use such tricks to earn upvotes on kaggle
\nThe notebook is doing not what we might expect from the first look.
\nBut since not many go into details - upvotes are obtained

", "votes": null, "replies": [ { "id": 2359733, "author_name": "takanashihumbert", "author_url": "", "post_date": "07/26/2023 11:04:48", "content": "

Thanks for your explanation.

", "votes": null, "replies": [] } ] }, { "id": 2365124, "author_name": "yixuansongeason", "author_url": "", "post_date": "07/30/2023 02:17:43", "content": "

Yes. I tried to directly submit the \"submission2\" and got the exact same result. lol

", "votes": null, "replies": [] } ] } ] }, { "id": 2361278, "author_name": "awater1223", "author_url": "", "post_date": "07/27/2023 10:01:21", "content": "

Try submissions.tail(). There is a lot of \"headless\" Confidence 😅
\nAlthough it merge with how='outer', the submission_df unique output is suppressed by submissions.drop 😅
\nI assumed once you add them back, the score will drop 😝

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2359213": "In the [high-scored public notebooks](https://www.kaggle.com/code/siddhvr/cafa5-using-protbert-embeds/notebook), I have a question about the ensemble part. As we see in the code:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3110858%2Fba1e1f3031c5d89c4fc745c02ab52d91%2FWX20230726-1212072x.png?generation=1690345078224152&alt=media)\nThe two results are merged with `how='outer'`. If there are differential GO terms from one to the other, the outcome will contain `nan`, and thus we can use this code to ensemble:\n```\nsubmissions['confidence_combined'] = submissions.apply(lambda row: row['Confidence2'] if not np.isnan(row['Confidence2']) else row['Confidence'], axis=1)\n```\nBut the two submissions used for ensemble are all trained with `num_labels = 500`. That is, the GO terms in the two submissions are the same, and there wouldn't have `nan` in the merged output. Is it right?\nWish to see your views.", "2359243": "I think that when it applies the lambda, it always prefers the value in submission2. Therefore, even if you are changing the hyperparameters or even the model, you will still get the same scores.", "2359255": "Yes, I think there is no difference between the unseen `submission2` and the ensemble output.", "2359722": "some people use such tricks to earn upvotes on kaggle\nThe notebook is doing not what we might expect from the first look.\nBut since not many go into details - upvotes are obtained", "2359733": "Thanks for your explanation.", "2361278": "Try `submissions.tail()`. There is a lot of \"headless\" Confidence 😅\nAlthough it merge with `how='outer'`, the `submission_df` unique output is suppressed by `submissions.drop` 😅\nI assumed once you add them back, the score will drop 😝", "2365124": "Yes. I tried to directly submit the \"submission2\" and got the exact same result. lol" }, "source": "meta" }