![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F650%2Fb36b667685305560ea29e7976dfceac4%2FScreenshot%202025-04-04%20094949.png?generation=1743786373890812&alt=media\")
(source: EDA Cell Distribution by Slice )Hi all,
\nI am a novice, and I am here to learn. So far I've been doing a lot of EDA on this dataset, but I am not so experienced at developing the model itself. If anybody could give me some feedback on this, I'd really appreciate it.
I have been wondering about how to make use of the information from the cell abundance in the training data. In principle, this is what we are supposed to predict, so there is no information about it in the Test data. However, I think there are some patterns that can be learned from the data, which may help building a better model.
\nIf we look at the distribution of cell abundance across the slices, some of these patterns become clearer. For example, some cell types tend to be in the same area as others, like C1 and C4, or C3. Another pattern is that some cell types tend to be distributed in specific areas, while others are generally dispersed across the area.
\n(source: EDA Cell Distribution by Slice )
Another patterns is about how often each cell type is the most abundant in a slice, or not. It seems cell C1, C5 and C16 are often the top cell types in the slide, overall. C30 is often the least abundant.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F650%2Fb0d3646b2a6bed2c7ec89816628adab7%2F__results___10_0.png?generation=1743786817459312&alt=media\")
How to make use of this information, when modelling?
\nI think that it may be difficult for a straight model to understand these patterns by itself. Especially the spatial patterns. Here are a few approaches I've tried:
\nDoes this make sense? I'd appreciate any feedback.
", "messages": [ { "id": "3170399", "postDate": "04/04/2025 17:23:03", "content": "Hi all,
\nI am a novice, and I am here to learn. So far I've been doing a lot of EDA on this dataset, but I am not so experienced at developing the model itself. If anybody could give me some feedback on this, I'd really appreciate it.
I have been wondering about how to make use of the information from the cell abundance in the training data. In principle, this is what we are supposed to predict, so there is no information about it in the Test data. However, I think there are some patterns that can be learned from the data, which may help building a better model.
\nIf we look at the distribution of cell abundance across the slices, some of these patterns become clearer. For example, some cell types tend to be in the same area as others, like C1 and C4, or C3. Another pattern is that some cell types tend to be distributed in specific areas, while others are generally dispersed across the area.
\n(source: EDA Cell Distribution by Slice )
Another patterns is about how often each cell type is the most abundant in a slice, or not. It seems cell C1, C5 and C16 are often the top cell types in the slide, overall. C30 is often the least abundant.
\n
How to make use of this information, when modelling?
\nI think that it may be difficult for a straight model to understand these patterns by itself. Especially the spatial patterns. Here are a few approaches I've tried:
\nDoes this make sense? I'd appreciate any feedback.
", "rawMarkdown": "Hi all,\nI am a novice, and I am here to learn. So far I've been doing a lot of EDA on this dataset, but I am not so experienced at developing the model itself. If anybody could give me some feedback on this, I'd really appreciate it.\n\nI have been wondering about how to make use of the information from the cell abundance in the training data. In principle, this is what we are supposed to predict, so there is no information about it in the Test data. However, I think there are some patterns that can be learned from the data, which may help building a better model. \n\nIf we look at the distribution of cell abundance across the slices, some of these patterns become clearer. For example, some cell types tend to be in the same area as others, like C1 and C4, or C3. Another pattern is that some cell types tend to be distributed in specific areas, while others are generally dispersed across the area.\n\n(source: [EDA Cell Distribution by Slice](https://www.kaggle.com/code/dalloliogm/eda-plotting-cell-distribution-by-slice) )\n\nAnother patterns is about how often each cell type is the most abundant in a slice, or not. It seems cell C1, C5 and C16 are often the top cell types in the slide, overall. C30 is often the least abundant.\n\n\nHow to make use of this information, when modelling?\n\nI think that it may be difficult for a straight model to understand these patterns by itself. Especially the spatial patterns. Here are a few approaches I've tried:\n\n- generate a random prediction, based only on the distribution of cell abundancies, without using the image features at all. This is what I've tried [here](https://www.kaggle.com/code/dalloliogm/eda-exploring-cell-type-abundance), and it gets a score of about 0.42. Not bad, after all.\n- train a model to predict only some of the cell type locations, with high precision. Maybe try to reverse-engineer the spots in S_7 used for the public leaderboard. Then train another model that based on the information on cell abundancies from the neighbour spots, predicts the status of cell abundancies. \n- train a sort of foundation model, based on an autoencoder or a transformer architecture, on the cell abundancies. This model should be able to learn the spatial relationships between cell types as above without encoding them explicitly.\n\nDoes this make sense? I'd appreciate any feedback.", "votes": null }, { "id": "3170442", "postDate": "04/04/2025 18:42:35", "content": "since we have limited data training a foundation model is a bit hard I think
", "rawMarkdown": "since we have limited data training a foundation model is a bit hard I think", "votes": null } ], "comments": [ { "id": 3170442, "author_name": "mahdiseddigh", "author_url": "", "post_date": "04/04/2025 18:42:35", "content": "since we have limited data training a foundation model is a bit hard I think
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3170399": "Hi all,\nI am a novice, and I am here to learn. So far I've been doing a lot of EDA on this dataset, but I am not so experienced at developing the model itself. If anybody could give me some feedback on this, I'd really appreciate it.\n\nI have been wondering about how to make use of the information from the cell abundance in the training data. In principle, this is what we are supposed to predict, so there is no information about it in the Test data. However, I think there are some patterns that can be learned from the data, which may help building a better model. \n\nIf we look at the distribution of cell abundance across the slices, some of these patterns become clearer. For example, some cell types tend to be in the same area as others, like C1 and C4, or C3. Another pattern is that some cell types tend to be distributed in specific areas, while others are generally dispersed across the area.\n\n(source: [EDA Cell Distribution by Slice](https://www.kaggle.com/code/dalloliogm/eda-plotting-cell-distribution-by-slice) )\n\nAnother patterns is about how often each cell type is the most abundant in a slice, or not. It seems cell C1, C5 and C16 are often the top cell types in the slide, overall. C30 is often the least abundant.\n\n\nHow to make use of this information, when modelling?\n\nI think that it may be difficult for a straight model to understand these patterns by itself. Especially the spatial patterns. Here are a few approaches I've tried:\n\n- generate a random prediction, based only on the distribution of cell abundancies, without using the image features at all. This is what I've tried [here](https://www.kaggle.com/code/dalloliogm/eda-exploring-cell-type-abundance), and it gets a score of about 0.42. Not bad, after all.\n- train a model to predict only some of the cell type locations, with high precision. Maybe try to reverse-engineer the spots in S_7 used for the public leaderboard. Then train another model that based on the information on cell abundancies from the neighbour spots, predicts the status of cell abundancies. \n- train a sort of foundation model, based on an autoencoder or a transformer architecture, on the cell abundancies. This model should be able to learn the spatial relationships between cell types as above without encoding them explicitly.\n\nDoes this make sense? I'd appreciate any feedback.", "3170442": "since we have limited data training a foundation model is a bit hard I think" }, "source": "meta" }