{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":94147,"databundleVersionId":11390004,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Notes and Thoughts - a primer to this competition","metadata":{}},{"cell_type":"markdown","source":"This post contains my notes and comments about this competition. It can be a good starting point for anyone joining the competition. There is not going to be a lot of code - just notes. Any comment and suggestion is welcome ;-)\n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"markdown","source":"## What do we need to predict?\n\nThis image shows the slice of tissue for sample S_7. The image is divided into spots - the yellow and purple dots you see represented on top of it. The purple spots are the ones used to compute the Leaderboard scores - the one we get when doing a submission. The yellow spots will be used to compute the final score, when the challenge is ended, and it will be used to award the winners.\n\nFor each of these spots, we need to predict the composition of cell types. There are 35 possible cell types, identified by the variables C1 to C35. We do not know the identity of these cell types, although we may attempt to infer them. The objective of the competition is to predict the relative abundance of each cell type. Since the score is computed using a Spearman correlation, we actually need to worry about the rank of abundance per cell type -the actual number is not important.\n\nThe image is the only data made available for this competition. There is no gene expression, antibody labelling, or anything else. The training set is also very limited, as it includes only 6 slices.\n\n![https://www.kaggle.com/code/dalloliogm/eda-plotting-cell-distribution-by-slice](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F650%2F8cf715ca7ea2e742aed67243fddabbd8%2F__results___16_1.png?generation=1743026267320565&alt=media) ","metadata":{}},{"cell_type":"markdown","source":"## How heterogeneous are the 35 cell types?\n\nMy first thought, after reading this competition, was that this is impossible. There is no gene expression data available, nor anything else, apart from the image of the slice. A specialized pathologist, with years of experience, may be able to look at one of these slides and identifies the tissue, and some of the cell types. Yet, here we have to identify 35 different cell types! And we do not even know which cell types they are, or which tissue do the slides come from!\n\nHowever, there may be space for giving it a try. Many cell types seem to be correlated, and colocalize together very often; so we may potentially cluster some of them together, so we do not have to actually predict all the 35 cell types.\n\nSee for example the plots below, taken from my [EDA notebook](https://www.kaggle.com/code/dalloliogm/eda-plotting-cell-distribution-by-slice).  It seems that cell type C1 and C3 tend to cluster together, indicating that they may be similar function. I still need to fully explore this correlation in a notebook, but it is a promising start.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F650%2F0defb7f63ffa22fd1dd24cec74cb769a%2F__results___3_0.png?generation=1743027417213833&alt=media)","metadata":{}},{"cell_type":"markdown","source":"## Some literature on the topic\n\nThere are a few papers on predicting the cell type composition from an histology image. \n\nTraining a model to predict cell type composition from a slice image would likely require a huge training set, and a few million of dollars to spend in compute cost. Here, we only have 6 images in the training set, and we are required to keep computation under 9 hours. Thus, it is evident that to perform well in this competition we would need to find a model trained on a larger dataset, and download its weights, assuming they are publicly available.\n\n- one tool is called [ALBRT](https://arxiv.org/abs/2108.08306). They trained a model to predict cell type composition from a tumour slice - which is very promising, as it is exactly what we need to do here. However, we don't know if our slices are from tumours; and they used a specific staining (Hematoxylin and Eosin) which may be different from what we have here. I've started exploring this option; the [github page](https://github.com/engrodawood/ALBRT) has a couple of notebooks, and points to some zip files containing the weights. Another difficulty using this tool is we need to specify which cell types we want to predict; but we don't even know that, we only have variables named C1 to C35.\n\n- ","metadata":{}}]}