Hello - I am new to this competition - on browsing the datasets I noticed that the protein id Q9CQV8 is common to the Training fasta, Test fasta and Train GO terms file and sample submission file (see attached screenshot). Is this overlap expected/frequent? How should we deal with these cases? (e.g. should we remove from Test set?) Apologies if already covered!
", "messages": [ { "id": "2366422", "postDate": "07/31/2023 01:45:46", "content": "Hello - I am new to this competition - on browsing the datasets I noticed that the protein id Q9CQV8 is common to the Training fasta, Test fasta and Train GO terms file and sample submission file (see attached screenshot). Is this overlap expected/frequent? How should we deal with these cases? (e.g. should we remove from Test set?) Apologies if already covered!
", "rawMarkdown": "Hello - I am new to this competition - on browsing the datasets I noticed that the protein id Q9CQV8 is common to the Training fasta, Test fasta and Train GO terms file and sample submission file (see attached screenshot). Is this overlap expected/frequent? How should we deal with these cases? (e.g. should we remove from Test set?) Apologies if already covered!", "votes": null }, { "id": "2367863", "postDate": "07/31/2023 21:47:44", "content": "If you were to go by , 't5embeds'/esm2-cafa-5 datasets (protein embeddings), there are 73653 entries duplicated between train_ids and test_ids.
\nconventional wisdom would be to remove them from test data but leaving them in could possibly reveal if your model is overfitting , perhaps. ie, if the model predicts 100% right for those entries, maybe the model is overfitting?
", "rawMarkdown": "If you were to go by , 't5embeds'/esm2-cafa-5 datasets (protein embeddings), there are 73653 entries duplicated between train_ids and test_ids. \n\nconventional wisdom would be to remove them from test data but leaving them in could possibly reveal if your model is overfitting , perhaps. ie, if the model predicts 100% right for those entries, maybe the model is overfitting?", "votes": null }, { "id": "2368529", "postDate": "08/01/2023 08:13:39", "content": "See that discussion:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403200
I'm gonna go ahead and 'guess' that, technically, the model should have better odds at correctly predicting the labels for the test data points that it previously encountered in the training set, b/c it has already seen their correct labels during the training process. That being said, why would you want to remove the data points that may potentially increase your model's score?
", "rawMarkdown": "I'm gonna go ahead and 'guess' that, technically, the model should have better odds at correctly predicting the labels for the test data points that it previously encountered in the training set, b/c it has already seen their correct labels during the training process. That being said, why would you want to remove the data points that may potentially increase your model's score?", "votes": null } ], "comments": [ { "id": 2367863, "author_name": "sunilkrishnan", "author_url": "", "post_date": "07/31/2023 21:47:44", "content": "If you were to go by , 't5embeds'/esm2-cafa-5 datasets (protein embeddings), there are 73653 entries duplicated between train_ids and test_ids.
\nconventional wisdom would be to remove them from test data but leaving them in could possibly reveal if your model is overfitting , perhaps. ie, if the model predicts 100% right for those entries, maybe the model is overfitting?
", "votes": null, "replies": [] }, { "id": 2368529, "author_name": "alexandervc", "author_url": "", "post_date": "08/01/2023 08:13:39", "content": "See that discussion:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403200
I'm gonna go ahead and 'guess' that, technically, the model should have better odds at correctly predicting the labels for the test data points that it previously encountered in the training set, b/c it has already seen their correct labels during the training process. That being said, why would you want to remove the data points that may potentially increase your model's score?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2366422": "Hello - I am new to this competition - on browsing the datasets I noticed that the protein id Q9CQV8 is common to the Training fasta, Test fasta and Train GO terms file and sample submission file (see attached screenshot). Is this overlap expected/frequent? How should we deal with these cases? (e.g. should we remove from Test set?) Apologies if already covered!", "2367863": "If you were to go by , 't5embeds'/esm2-cafa-5 datasets (protein embeddings), there are 73653 entries duplicated between train_ids and test_ids. \n\nconventional wisdom would be to remove them from test data but leaving them in could possibly reveal if your model is overfitting , perhaps. ie, if the model predicts 100% right for those entries, maybe the model is overfitting?", "2368529": "See that discussion:\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/403200", "2369560": "I'm gonna go ahead and 'guess' that, technically, the model should have better odds at correctly predicting the labels for the test data points that it previously encountered in the training set, b/c it has already seen their correct labels during the training process. That being said, why would you want to remove the data points that may potentially increase your model's score?" }, "source": "meta" }