First off: I am more of a ML-newbie and was in this competition to learn and have fun (which I had, by the way). This is to say that the following is really not about any frustration about my scores!
\n\nBut: As a former scientist (in biological research as well), I am a bit disappointed by the competition's selection of data and by its conception.\nI totally agree that it makes a lot of sense to develop models that work on a very broad range of pictures with a large variation in cell types and imaging conditions. And I can also see the point of testing the models by confronting them with different classes of images to check if they are versatile enough. And yes, the aim is of course to detect nuclei without human intervention!\nHowever, the data presented here seems to be 1) too inconsistent and 2) too poor.\nTwo thing that should have been included with every picture are:
\n\n-> label what the channels mean (phase contrast, fluorescence, better also: type of dye etc.)
\n\n-> scale/magnification!!!
\n\nWhy?
\n\nOK, the aim is to do stuff without human intervention. But not to do magic. \nWhoever takes those pictures of cells, tissue etc. will anyway need to add both types of information to his pictures, especially the magnification (if he/she's serious about science anyway). Usually such information is even logged automatically by the microscopy software. It hence does not mean much more human efforts.\nIt's a bit like training a model on smartphone images and than asking the model to find a dog in aerial pictures... seems like a poor strategy.
\n\nSo, not including this extra information, in my opinion ruins any ability to really develop a meaningfull model (given the many contributions, you might still find some by accident).
", "messages": [ { "id": 313535, "postDate": "2018-04-13T12:28:31.307Z", "content": "First off: I am more of a ML-newbie and was in this competition to learn and have fun (which I had, by the way). This is to say that the following is really not about any frustration about my scores!
\n\nBut: As a former scientist (in biological research as well), I am a bit disappointed by the competition's selection of data and by its conception.\nI totally agree that it makes a lot of sense to develop models that work on a very broad range of pictures with a large variation in cell types and imaging conditions. And I can also see the point of testing the models by confronting them with different classes of images to check if they are versatile enough. And yes, the aim is of course to detect nuclei without human intervention!\nHowever, the data presented here seems to be 1) too inconsistent and 2) too poor.\nTwo thing that should have been included with every picture are:
\n\n-> label what the channels mean (phase contrast, fluorescence, better also: type of dye etc.)
\n\n-> scale/magnification!!!
\n\nWhy?
\n\nOK, the aim is to do stuff without human intervention. But not to do magic. \nWhoever takes those pictures of cells, tissue etc. will anyway need to add both types of information to his pictures, especially the magnification (if he/she's serious about science anyway). Usually such information is even logged automatically by the microscopy software. It hence does not mean much more human efforts.\nIt's a bit like training a model on smartphone images and than asking the model to find a dog in aerial pictures... seems like a poor strategy.
\n\nSo, not including this extra information, in my opinion ruins any ability to really develop a meaningfull model (given the many contributions, you might still find some by accident).
", "rawMarkdown": "First off: I am more of a ML-newbie and was in this competition to learn and have fun (which I had, by the way). This is to say that the following is really not about any frustration about my scores!\n\nBut: As a former scientist (in biological research as well), I am a bit disappointed by the competition's selection of data and by its conception.\nI totally agree that it makes a lot of sense to develop models that work on a very broad range of pictures with a large variation in cell types and imaging conditions. And I can also see the point of testing the models by confronting them with different classes of images to check if they are versatile enough. And yes, the aim is of course to detect nuclei without human intervention!\nHowever, the data presented here seems to be 1) too inconsistent and 2) too poor.\nTwo thing that should have been included with every picture are:\n\n-> **label what the channels mean** (phase contrast, fluorescence, better also: type of dye etc.)\n\n-> **scale/magnification**!!! \n\nWhy?\n\nOK, the aim is to do stuff without human intervention. But not to do magic. \nWhoever takes those pictures of cells, tissue etc. will anyway need to add both types of information to his pictures, especially the magnification (if he/she's serious about science anyway). Usually such information is even logged automatically by the microscopy software. It hence does not mean much more human efforts.\nIt's a bit like training a model on smartphone images and than asking the model to find a dog in aerial pictures... seems like a poor strategy.\n\nSo, not including this extra information, in my opinion ruins any ability to really develop a meaningfull model (given the many contributions, you might still find some by accident).", "votes": 11 }, { "id": 314018, "postDate": "2018-04-14T11:49:59.147Z", "content": "Florian, I agree with your point.
\n\nOne question I've been asking myself for the past few days (not being a micro-biologist) is whether the diversity in the nuclei images grows a lot in time or is relatively stagnant. I would believe that the types of nuclei/tissues analyzed is stagnant (evolution looks frozen from our POV) however the techniques to screen may not.
\n\nThe question is how many are known nuclei/tissue types (10s/100s/1000s?) which I'd guess should be frozen and how many screening techniques do exist today and how many are expected to be developed in 2-3 years.
\n\nIf the answer is that the variability of nuclei and screening techniques is expected to be stagnant for the next 2-3 years... I'd say the organizer/sponsor would have done a better job including as much variability in stage 1 test set.
\n\nAnd I say the test set b/c it would have sent a strong message about realistic generation goals and the fact that teams would have to spend time either working on research-type generalization (e.g. capsule nets, etc. ) or develop training set in the same domain as the test set.
", "rawMarkdown": "Florian, I agree with your point.\n\nOne question I've been asking myself for the past few days (not being a micro-biologist) is whether the diversity in the nuclei images grows a lot in time or is relatively stagnant. I would believe that the types of nuclei/tissues analyzed is stagnant (evolution looks frozen from our POV) however the techniques to screen may not.\n\nThe question is how many are known nuclei/tissue types (10s/100s/1000s?) which I'd guess should be frozen and how many screening techniques do exist today and how many are expected to be developed in 2-3 years.\n\nIf the answer is that the variability of nuclei and screening techniques is expected to be stagnant for the next 2-3 years... I'd say the organizer/sponsor would have done a better job including as much variability in stage 1 test set. \n\nAnd I say the test set b/c it would have sent a strong message about realistic generation goals and the fact that teams would have to spend time either working on research-type generalization (e.g. capsule nets, etc. ) or develop training set in the same domain as the test set.", "votes": 2, "replies": [ { "id": 314166, "postDate": "2018-04-14T19:10:02.473Z", "content": "There's quite a lot of techniques to look at cells/tissue, and new ones are frequently invented. During the last decade or so, quite some new fancy optical techniques were added to the repertoire of state-of-the-art labs (most notable many \"superresolution\" techniques such as STED). \nThe type of imaging will not only effect the fluorescent labeling or magnification, but different techniques will reveal vastly different features. For many techniques differences will be so big that you can certainly not expect a model to guess from training on images from one technique how nuclei will look when observed with a completetly different one. So you would at least need severe re-training I'd guess. Or work with a bunch of coupled models each trained on a set of techniques maybe...
\n\nHowever, most images that I saw from the competition where taken with fairly classical optical microscopy techniques: phase contrast and fluorescence microscopy. The main problem I see in those images is that they vary a lot especially in quality and magnification. I totally see that it makes a lot of sense to build a model that can find nuclei from a wide, wide range of cell types nuclei, different fluorescent labels etc. But many images are of such poor quality that without given context (\"magnification, nuclei stained/not stained, tissue/cell tyoe etc.) even researchers would not be able to tell if they see nuclei or just \"fluorescent blobs\". Those could also be \"dirt\", liposomes, protein aggregates, anything...
\n\nNuclei are in every cell, so looking for them makes sense. But unfortunatly if image quality is poor nuclei have very little unique features (there's quite some other \"roundish objects\" in living systems...).
", "rawMarkdown": "There's quite a lot of techniques to look at cells/tissue, and new ones are frequently invented. During the last decade or so, quite some new fancy optical techniques were added to the repertoire of state-of-the-art labs (most notable many \"superresolution\" techniques such as STED). \nThe type of imaging will not only effect the fluorescent labeling or magnification, but different techniques will reveal vastly different features. For many techniques differences will be so big that you can certainly not expect a model to guess from training on images from one technique how nuclei will look when observed with a completetly different one. So you would at least need severe re-training I'd guess. Or work with a bunch of coupled models each trained on a set of techniques maybe...\n\nHowever, most images that I saw from the competition where taken with fairly classical optical microscopy techniques: phase contrast and fluorescence microscopy. The main problem I see in those images is that they vary a lot especially in quality and magnification. I totally see that it makes a lot of sense to build a model that can find nuclei from a wide, wide range of cell types nuclei, different fluorescent labels etc. But many images are of such poor quality that without given context (\"magnification, nuclei stained/not stained, tissue/cell tyoe etc.) even researchers would not be able to tell if they see nuclei or just \"fluorescent blobs\". Those could also be \"dirt\", liposomes, protein aggregates, anything...\n\nNuclei are in every cell, so looking for them makes sense. But unfortunatly if image quality is poor nuclei have very little unique features (there's quite some other \"roundish objects\" in living systems...)." } ] }, { "id": 313715, "postDate": "2018-04-13T18:34:49.343Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 314018, "author_name": "Andrés Miguel Torrubia Sáez", "author_url": "", "post_date": "2018-04-14T11:49:59.147000", "content": "Florian, I agree with your point.
\n\nOne question I've been asking myself for the past few days (not being a micro-biologist) is whether the diversity in the nuclei images grows a lot in time or is relatively stagnant. I would believe that the types of nuclei/tissues analyzed is stagnant (evolution looks frozen from our POV) however the techniques to screen may not.
\n\nThe question is how many are known nuclei/tissue types (10s/100s/1000s?) which I'd guess should be frozen and how many screening techniques do exist today and how many are expected to be developed in 2-3 years.
\n\nIf the answer is that the variability of nuclei and screening techniques is expected to be stagnant for the next 2-3 years... I'd say the organizer/sponsor would have done a better job including as much variability in stage 1 test set.
\n\nAnd I say the test set b/c it would have sent a strong message about realistic generation goals and the fact that teams would have to spend time either working on research-type generalization (e.g. capsule nets, etc. ) or develop training set in the same domain as the test set.
", "votes": 2, "replies": [ { "id": 314166, "author_name": "Florian Huber", "author_url": "", "post_date": "2018-04-14T19:10:02.473000", "content": "There's quite a lot of techniques to look at cells/tissue, and new ones are frequently invented. During the last decade or so, quite some new fancy optical techniques were added to the repertoire of state-of-the-art labs (most notable many \"superresolution\" techniques such as STED). \nThe type of imaging will not only effect the fluorescent labeling or magnification, but different techniques will reveal vastly different features. For many techniques differences will be so big that you can certainly not expect a model to guess from training on images from one technique how nuclei will look when observed with a completetly different one. So you would at least need severe re-training I'd guess. Or work with a bunch of coupled models each trained on a set of techniques maybe...
\n\nHowever, most images that I saw from the competition where taken with fairly classical optical microscopy techniques: phase contrast and fluorescence microscopy. The main problem I see in those images is that they vary a lot especially in quality and magnification. I totally see that it makes a lot of sense to build a model that can find nuclei from a wide, wide range of cell types nuclei, different fluorescent labels etc. But many images are of such poor quality that without given context (\"magnification, nuclei stained/not stained, tissue/cell tyoe etc.) even researchers would not be able to tell if they see nuclei or just \"fluorescent blobs\". Those could also be \"dirt\", liposomes, protein aggregates, anything...
\n\nNuclei are in every cell, so looking for them makes sense. But unfortunatly if image quality is poor nuclei have very little unique features (there's quite some other \"roundish objects\" in living systems...).
", "votes": 0, "replies": [] } ] }, { "id": 313715, "author_name": "", "author_url": "", "post_date": "2018-04-13T18:34:49.343000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "313535": "First off: I am more of a ML-newbie and was in this competition to learn and have fun (which I had, by the way). This is to say that the following is really not about any frustration about my scores!\n\nBut: As a former scientist (in biological research as well), I am a bit disappointed by the competition's selection of data and by its conception.\nI totally agree that it makes a lot of sense to develop models that work on a very broad range of pictures with a large variation in cell types and imaging conditions. And I can also see the point of testing the models by confronting them with different classes of images to check if they are versatile enough. And yes, the aim is of course to detect nuclei without human intervention!\nHowever, the data presented here seems to be 1) too inconsistent and 2) too poor.\nTwo thing that should have been included with every picture are:\n\n-> **label what the channels mean** (phase contrast, fluorescence, better also: type of dye etc.)\n\n-> **scale/magnification**!!! \n\nWhy?\n\nOK, the aim is to do stuff without human intervention. But not to do magic. \nWhoever takes those pictures of cells, tissue etc. will anyway need to add both types of information to his pictures, especially the magnification (if he/she's serious about science anyway). Usually such information is even logged automatically by the microscopy software. It hence does not mean much more human efforts.\nIt's a bit like training a model on smartphone images and than asking the model to find a dog in aerial pictures... seems like a poor strategy.\n\nSo, not including this extra information, in my opinion ruins any ability to really develop a meaningfull model (given the many contributions, you might still find some by accident).", "314018": "Florian, I agree with your point.\n\nOne question I've been asking myself for the past few days (not being a micro-biologist) is whether the diversity in the nuclei images grows a lot in time or is relatively stagnant. I would believe that the types of nuclei/tissues analyzed is stagnant (evolution looks frozen from our POV) however the techniques to screen may not.\n\nThe question is how many are known nuclei/tissue types (10s/100s/1000s?) which I'd guess should be frozen and how many screening techniques do exist today and how many are expected to be developed in 2-3 years.\n\nIf the answer is that the variability of nuclei and screening techniques is expected to be stagnant for the next 2-3 years... I'd say the organizer/sponsor would have done a better job including as much variability in stage 1 test set. \n\nAnd I say the test set b/c it would have sent a strong message about realistic generation goals and the fact that teams would have to spend time either working on research-type generalization (e.g. capsule nets, etc. ) or develop training set in the same domain as the test set.", "313715": "" } }