Is it reasonable to assume that the distribution of cell type abundances in the training set will also hold for the test set?
", "messages": [ { "id": "3168868", "postDate": "04/02/2025 21:31:15", "content": "Is it reasonable to assume that the distribution of cell type abundances in the training set will also hold for the test set?
", "rawMarkdown": "Is it reasonable to assume that the distribution of cell type abundances in the training set will also hold for the test set?", "votes": null }, { "id": "3168963", "postDate": "04/03/2025 01:41:16", "content": "Some people have commented here: https://www.kaggle.com/competitions/el-hackathon-2025/discussion/569715
\nThe distribution of train and test are different. I don't think the LB is reliable
This competition is ranked using a Spearman test. So the absolute values of cell abundance does not matter - only the relative rank.
\nSee https://www.kaggle.com/competitions/el-hackathon-2025/discussion/570035
", "rawMarkdown": "This competition is ranked using a Spearman test. So the absolute values of cell abundance does not matter - only the relative rank.\n\nSee https://www.kaggle.com/competitions/el-hackathon-2025/discussion/570035", "votes": null }, { "id": "3169217", "postDate": "04/03/2025 08:30:16", "content": "so, only the ranking would matters, like 1,2 .. 35!? for every coordinate we need to rank 35 cell types?
", "rawMarkdown": "so, only the ranking would matters, like 1,2 .. 35!? for every coordinate we need to rank 35 cell types?", "votes": null }, { "id": "3169227", "postDate": "04/03/2025 08:41:11", "content": "That's what I have been doing so far - sorting them from 1 to 35. However, it may still be useful to look at the absolute abundance values (which range from 0 to 2), because some cell types have very low abundance overall, and others are always the most abundant.
", "rawMarkdown": "That's what I have been doing so far - sorting them from 1 to 35. However, it may still be useful to look at the absolute abundance values (which range from 0 to 2), because some cell types have very low abundance overall, and others are always the most abundant.", "votes": null }, { "id": "3169252", "postDate": "04/03/2025 09:04:39", "content": "None of the pretrained models for feature extraction is giving good results, do you think some kind of preprocessing for the slides is vital to generate signal?
", "rawMarkdown": "None of the pretrained models for feature extraction is giving good results, do you think some kind of preprocessing for the slides is vital to generate signal?", "votes": null }, { "id": "3170654", "postDate": "04/04/2025 22:00:20", "content": "(you probably want to predict 35.. 1 rather than 1,2.. 35, unless you want a negative score :-))
", "rawMarkdown": "(you probably want to predict 35.. 1 rather than 1,2.. 35, unless you want a negative score :-))", "votes": null } ], "comments": [ { "id": 3168963, "author_name": "bibanh", "author_url": "", "post_date": "04/03/2025 01:41:16", "content": "Some people have commented here: https://www.kaggle.com/competitions/el-hackathon-2025/discussion/569715
\nThe distribution of train and test are different. I don't think the LB is reliable
None of the pretrained models for feature extraction is giving good results, do you think some kind of preprocessing for the slides is vital to generate signal?
", "votes": null, "replies": [] } ] }, { "id": 3169096, "author_name": "dalloliogm", "author_url": "", "post_date": "04/03/2025 05:40:41", "content": "This competition is ranked using a Spearman test. So the absolute values of cell abundance does not matter - only the relative rank.
\nSee https://www.kaggle.com/competitions/el-hackathon-2025/discussion/570035
", "votes": null, "replies": [ { "id": 3169217, "author_name": "jamalsaeedi", "author_url": "", "post_date": "04/03/2025 08:30:16", "content": "so, only the ranking would matters, like 1,2 .. 35!? for every coordinate we need to rank 35 cell types?
", "votes": null, "replies": [ { "id": 3169227, "author_name": "dalloliogm", "author_url": "", "post_date": "04/03/2025 08:41:11", "content": "That's what I have been doing so far - sorting them from 1 to 35. However, it may still be useful to look at the absolute abundance values (which range from 0 to 2), because some cell types have very low abundance overall, and others are always the most abundant.
", "votes": null, "replies": [ { "id": 3170654, "author_name": "dalloliogm", "author_url": "", "post_date": "04/04/2025 22:00:20", "content": "(you probably want to predict 35.. 1 rather than 1,2.. 35, unless you want a negative score :-))
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3168868": "Is it reasonable to assume that the distribution of cell type abundances in the training set will also hold for the test set?", "3168963": "Some people have commented here: https://www.kaggle.com/competitions/el-hackathon-2025/discussion/569715\nThe distribution of train and test are different. I don't think the LB is reliable", "3169096": "This competition is ranked using a Spearman test. So the absolute values of cell abundance does not matter - only the relative rank.\n\nSee https://www.kaggle.com/competitions/el-hackathon-2025/discussion/570035", "3169217": "so, only the ranking would matters, like 1,2 .. 35!? for every coordinate we need to rank 35 cell types?", "3169227": "That's what I have been doing so far - sorting them from 1 to 35. However, it may still be useful to look at the absolute abundance values (which range from 0 to 2), because some cell types have very low abundance overall, and others are always the most abundant.", "3169252": "None of the pretrained models for feature extraction is giving good results, do you think some kind of preprocessing for the slides is vital to generate signal?", "3170654": "(you probably want to predict 35.. 1 rather than 1,2.. 35, unless you want a negative score :-))" }, "source": "meta" }