For this competition, Anne (Anne Carpenter, Director of Imaging Platform @ Broad, has made an excellent, short introductory video posted here : 2018 Data Science Bowl – Technical Overview : https://www.youtube.com/watch?v=Dbiq6l50zO8
\n\nThis must have been linked to in the Description of the Competition or the Overview section, as it succinctly explains the need for such a competition and even addresses fine points like why spot the nuclei alone rather than building it for the whole cell classification directly.
\n\nThanks for the nice video, Anne. All of us must check it out.
", "messages": [ { "id": "271403", "postDate": "01/20/2018 11:39:43", "content": "For this competition, Anne (Anne Carpenter, Director of Imaging Platform @ Broad, has made an excellent, short introductory video posted here : 2018 Data Science Bowl – Technical Overview : https://www.youtube.com/watch?v=Dbiq6l50zO8
\n\nThis must have been linked to in the Description of the Competition or the Overview section, as it succinctly explains the need for such a competition and even addresses fine points like why spot the nuclei alone rather than building it for the whole cell classification directly.
\n\nThanks for the nice video, Anne. All of us must check it out.
", "rawMarkdown": "For this competition, Anne (Anne Carpenter, Director of Imaging Platform @ Broad, has made an excellent, short introductory video posted here : 2018 Data Science Bowl – Technical Overview : https://www.youtube.com/watch?v=Dbiq6l50zO8 \n\nThis must have been linked to in the Description of the Competition or the Overview section, as it succinctly explains the need for such a competition and even addresses fine points like why spot the nuclei alone rather than building it for the whole cell classification directly. \n\nThanks for the nice video, Anne. All of us must check it out.", "votes": null }, { "id": "272877", "postDate": "01/23/2018 22:28:02", "content": "Videos are nice so you capture my enthusiasm, but some of you might absorb information better via text (I know I do!) So for your convenience, here is the script I wrote for making the video. I can't promise it's identical but it is pretty close:
\n\n====
\n\nHello! I am Dr. Anne Carpenter, a cell biologist and computer scientist. I run a research lab at \nthe Broad Institute of Harvard and MIT and we are the nonprofit data partner for the Data Science Bowl presented by Booz Allen Hamilton and Kaggle.
\n\nThe Broad Institute is a technology-focused biomedical research center with strong roots in the Human Genome Project, which provided the foundation for figuring out the underlying genetic causes of various diseases.
\n\nThe Institute’s mission is to translate that knowledge into new medicines, and we are involved in projects to discover the causes and cures of all sorts of diseases, from cancer to metabolic disorders to psychiatric diseases.
\n\nThis year’s Data Science Bowl challenge is driven by a very real need to develop new treatments faster and more accurately. We can look at the numbers:
\n\nIn the tech industry we have Moore’s law which describes the phenomenal pace at which computers get faster and cheaper. Well, in the pharmaceutical industry we have “Eroom’s law” - the inverse of Moore’s law, which describes the decreasing number of new medicines that you get for a billion dollars of research spending. This is a log scale! So in the 1950s you could discover 30 new drugs for a cost that will not even find a single new drug today.
\n\nBut you can also walk into a hospital or doctor’s office and see people suffering with diseases for which we lack the power to cure or even alleviate. One major bottleneck in the search for effective medicines is finding nuclei in microscopy images. Let me explain why: we often want to test thousands of chemicals to find one that might be a good drug.
\n\nScientists prepare batches of cells, add a different chemical to each batch, and take pictures of each batch of cells. They want to see which cells responded in a certain way that indicates that the cells are healthier. Now, healthy might not always be a color change as in this simple example. It might instead be that breast cancer cells look less metastatic or that tuberculosis bacteria are no longer infecting the cells. But whatever it is, we certainly need software to analyze the cell’s health. This is because robots exist to prepare millions of batches of cells, and to add millions of compounds to the cells, and even to capture images of those cells.
\n\nA major bottleneck is analyzing the cell images for whatever characteristics the disease expert is studying. The first step in software analyzing images for a particular disease is often to identify nuclei.
\n\nThese images show the same 6 cells growing on a slide. But in each image we are looking at a different part of the cells using a different stain. In some of the images you can see parts of each cell are touching and overlapping the others. As you can see, it is pretty clear that the stain on the far left is the one where the cells are most readily distinguished from each other. That stain is labeling DNA: that is the genetic material contained within the nucleus, which is usually located towards the center of each cell.
\n\nIn fact, the most effective strategies for a software solution that measures a particular disease state of cells often begins with finding the nuclei, because they are usually the most distinguishable part of the cell. From there, software can more easily measure various properties of the cells to discern their disease state. And that brings us to this year’s challenge of creating a nucleus detector!
\n\nIn order for you to appreciate how transformative this would be, let me first explain what scientists currently do.
\n\nFirst, there are computational algorithms that can do a pretty good job of finding nuclei and then measuring the disease state of cells. But they work best when nuclei are fairly roundish and not too crowded.
\n\nSometimes nuclei have very unusual shapes as shown here.
\n\nAnd sometimes nuclei can be difficult to distinguish, especially in more complicated experiments that involve tissue samples. In experiments like these, classical image processing algorithms tend to fail.
\n\nSometimes the biologist has no choice but to literally look at every image by eye in order to complete their experiment. So we wondered: what if a deep learning model could be trained to just “know” what nuclei look like? … Regardless of the type of stain used, the species or cell type, the microscope, the resolution, and so on. If you could build an accurate model into open-source and commercial software, that could find nuclei in images with no biologist intervention, it would save researchers hundreds of thousands of hours of effort each year. No more configuring classical image processing algorithms and certainly no more looking at images by eye. Pharma companies around the world process millions of images each year and tens of thousands of graduate students do smaller scale experiments that would be accelerated by a successful solution to this year’s challenge.
\n\nNow the key to this challenge is robustness. Any one of these image sets is probably relatively solvable using an off-the-shelf architecture. But what makes this an interesting challenge is that we want a model that detects nuclei across different experimental systems - different stains, species, and microscopes. Biologists generally lack the expertise to train their own models - and we suspect it’s not necessary. We suspect that one of you will come up with a model that will work effectively across a wide variety of types of nuclei and experimental systems.
\n\nSo in the test (oops - I meant data) set you will notice some variety. Many images are very easy, some are very hard. Within a given image, many nuclei might be easy but the challenging part is accurately figuring out the difference between a single lumpy nucleus and two adjacent nuclei. Most importantly, the images to be used in the final test phase on which you will be judged are generally from different experiments or cell types than those included in the training data provided to you. So generalizability is a key goal for your solution.
\n\nMy team annotated more than 25,000 nuclei for this challenge. It was incredibly tedious but our hope is that one of your solutions will pay off and reduce this kind of tedious work for disease researchers. My laboratory’s mission is to bring powerful computational tools to the hands of biologists. This started with the CellProfiler project that is currently used by tens of thousands of biologists around the world, and we are now pioneering interfaces for biologists to use deep learning models such as those developed in this competition.
\n\nSo good luck! Thousands of biologists around the world are counting on you to come up with a good solution. Not to mention millions of patients whose lives could be transformed by new medicines, if we can speed up the process of drug discovery through a successful nucleus detector. I hope you enjoy putting your knowledge and skills to the test.
", "rawMarkdown": "Videos are nice so you capture my enthusiasm, but some of you might absorb information better via text (I know I do!) So for your convenience, here is the script I wrote for making the video. I can't promise it's identical but it is pretty close:\n\n====\n\nHello! I am Dr. Anne Carpenter, a cell biologist and computer scientist. I run a research lab at \nthe Broad Institute of Harvard and MIT and we are the nonprofit data partner for the Data Science Bowl presented by Booz Allen Hamilton and Kaggle.\n\nThe Broad Institute is a technology-focused biomedical research center with strong roots in the Human Genome Project, which provided the foundation for figuring out the underlying genetic causes of various diseases.\n\nThe Institute’s mission is to translate that knowledge into new medicines, and we are involved in projects to discover the causes and cures of all sorts of diseases, from cancer to metabolic disorders to psychiatric diseases. \n\nThis year’s Data Science Bowl challenge is driven by a very real need to develop new treatments faster and more accurately. We can look at the numbers:\n\nIn the tech industry we have Moore’s law which describes the phenomenal pace at which computers get faster and cheaper. Well, in the pharmaceutical industry we have “Eroom’s law” - the inverse of Moore’s law, which describes the decreasing number of new medicines that you get for a billion dollars of research spending. This is a log scale! So in the 1950s you could discover 30 new drugs for a cost that will not even find a single new drug today.\n\nBut you can also walk into a hospital or doctor’s office and see people suffering with diseases for which we lack the power to cure or even alleviate. One major bottleneck in the search for effective medicines is finding nuclei in microscopy images. Let me explain why: we often want to test thousands of chemicals to find one that might be a good drug. \n\nScientists prepare batches of cells, add a different chemical to each batch, and take pictures of each batch of cells. They want to see which cells responded in a certain way that indicates that the cells are healthier. Now, healthy might not always be a color change as in this simple example. It might instead be that breast cancer cells look less metastatic or that tuberculosis bacteria are no longer infecting the cells. But whatever it is, we certainly need software to analyze the cell’s health. This is because robots exist to prepare millions of batches of cells, and to add millions of compounds to the cells, and even to capture images of those cells. \n\nA major bottleneck is analyzing the cell images for whatever characteristics the disease expert is studying. The first step in software analyzing images for a particular disease is often to identify nuclei. \n\nThese images show the same 6 cells growing on a slide. But in each image we are looking at a different part of the cells using a different stain. In some of the images you can see parts of each cell are touching and overlapping the others. As you can see, it is pretty clear that the stain on the far left is the one where the cells are most readily distinguished from each other. That stain is labeling DNA: that is the genetic material contained within the nucleus, which is usually located towards the center of each cell. \n\nIn fact, the most effective strategies for a software solution that measures a particular disease state of cells often begins with finding the nuclei, because they are usually the most distinguishable part of the cell. From there, software can more easily measure various properties of the cells to discern their disease state. And that brings us to this year’s challenge of creating a nucleus detector!\n\nIn order for you to appreciate how transformative this would be, let me first explain what scientists currently do. \n\nFirst, there are computational algorithms that can do a pretty good job of finding nuclei and then measuring the disease state of cells. But they work best when nuclei are fairly roundish and not too crowded. \n\nSometimes nuclei have very unusual shapes as shown here. \n\nAnd sometimes nuclei can be difficult to distinguish, especially in more complicated experiments that involve tissue samples. In experiments like these, classical image processing algorithms tend to fail. \n\nSometimes the biologist has no choice but to literally look at every image by eye in order to complete their experiment. So we wondered: what if a deep learning model could be trained to just “know” what nuclei look like? … Regardless of the type of stain used, the species or cell type, the microscope, the resolution, and so on. If you could build an accurate model into open-source and commercial software, that could find nuclei in images with no biologist intervention, it would save researchers hundreds of thousands of hours of effort each year. No more configuring classical image processing algorithms and certainly no more looking at images by eye. Pharma companies around the world process millions of images each year and tens of thousands of graduate students do smaller scale experiments that would be accelerated by a successful solution to this year’s challenge. \n\nNow the key to this challenge is robustness. Any one of these image sets is probably relatively solvable using an off-the-shelf architecture. But what makes this an interesting challenge is that we want a model that detects nuclei across different experimental systems - different stains, species, and microscopes. Biologists generally lack the expertise to train their own models - and we suspect it’s not necessary. We suspect that one of you will come up with a model that will work effectively across a wide variety of types of nuclei and experimental systems. \n\nSo in the test (oops - I meant data) set you will notice some variety. Many images are very easy, some are very hard. Within a given image, many nuclei might be easy but the challenging part is accurately figuring out the difference between a single lumpy nucleus and two adjacent nuclei. Most importantly, the images to be used in the final test phase on which you will be judged are generally from different experiments or cell types than those included in the training data provided to you. So generalizability is a key goal for your solution. \n\nMy team annotated more than 25,000 nuclei for this challenge. It was incredibly tedious but our hope is that one of your solutions will pay off and reduce this kind of tedious work for disease researchers. My laboratory’s mission is to bring powerful computational tools to the hands of biologists. This started with the CellProfiler project that is currently used by tens of thousands of biologists around the world, and we are now pioneering interfaces for biologists to use deep learning models such as those developed in this competition. \n \nSo good luck! Thousands of biologists around the world are counting on you to come up with a good solution. Not to mention millions of patients whose lives could be transformed by new medicines, if we can speed up the process of drug discovery through a successful nucleus detector. I hope you enjoy putting your knowledge and skills to the test.", "votes": null } ], "comments": [ { "id": 272877, "author_name": "drannecarpenter", "author_url": "", "post_date": "01/23/2018 22:28:02", "content": "Videos are nice so you capture my enthusiasm, but some of you might absorb information better via text (I know I do!) So for your convenience, here is the script I wrote for making the video. I can't promise it's identical but it is pretty close:
\n\n====
\n\nHello! I am Dr. Anne Carpenter, a cell biologist and computer scientist. I run a research lab at \nthe Broad Institute of Harvard and MIT and we are the nonprofit data partner for the Data Science Bowl presented by Booz Allen Hamilton and Kaggle.
\n\nThe Broad Institute is a technology-focused biomedical research center with strong roots in the Human Genome Project, which provided the foundation for figuring out the underlying genetic causes of various diseases.
\n\nThe Institute’s mission is to translate that knowledge into new medicines, and we are involved in projects to discover the causes and cures of all sorts of diseases, from cancer to metabolic disorders to psychiatric diseases.
\n\nThis year’s Data Science Bowl challenge is driven by a very real need to develop new treatments faster and more accurately. We can look at the numbers:
\n\nIn the tech industry we have Moore’s law which describes the phenomenal pace at which computers get faster and cheaper. Well, in the pharmaceutical industry we have “Eroom’s law” - the inverse of Moore’s law, which describes the decreasing number of new medicines that you get for a billion dollars of research spending. This is a log scale! So in the 1950s you could discover 30 new drugs for a cost that will not even find a single new drug today.
\n\nBut you can also walk into a hospital or doctor’s office and see people suffering with diseases for which we lack the power to cure or even alleviate. One major bottleneck in the search for effective medicines is finding nuclei in microscopy images. Let me explain why: we often want to test thousands of chemicals to find one that might be a good drug.
\n\nScientists prepare batches of cells, add a different chemical to each batch, and take pictures of each batch of cells. They want to see which cells responded in a certain way that indicates that the cells are healthier. Now, healthy might not always be a color change as in this simple example. It might instead be that breast cancer cells look less metastatic or that tuberculosis bacteria are no longer infecting the cells. But whatever it is, we certainly need software to analyze the cell’s health. This is because robots exist to prepare millions of batches of cells, and to add millions of compounds to the cells, and even to capture images of those cells.
\n\nA major bottleneck is analyzing the cell images for whatever characteristics the disease expert is studying. The first step in software analyzing images for a particular disease is often to identify nuclei.
\n\nThese images show the same 6 cells growing on a slide. But in each image we are looking at a different part of the cells using a different stain. In some of the images you can see parts of each cell are touching and overlapping the others. As you can see, it is pretty clear that the stain on the far left is the one where the cells are most readily distinguished from each other. That stain is labeling DNA: that is the genetic material contained within the nucleus, which is usually located towards the center of each cell.
\n\nIn fact, the most effective strategies for a software solution that measures a particular disease state of cells often begins with finding the nuclei, because they are usually the most distinguishable part of the cell. From there, software can more easily measure various properties of the cells to discern their disease state. And that brings us to this year’s challenge of creating a nucleus detector!
\n\nIn order for you to appreciate how transformative this would be, let me first explain what scientists currently do.
\n\nFirst, there are computational algorithms that can do a pretty good job of finding nuclei and then measuring the disease state of cells. But they work best when nuclei are fairly roundish and not too crowded.
\n\nSometimes nuclei have very unusual shapes as shown here.
\n\nAnd sometimes nuclei can be difficult to distinguish, especially in more complicated experiments that involve tissue samples. In experiments like these, classical image processing algorithms tend to fail.
\n\nSometimes the biologist has no choice but to literally look at every image by eye in order to complete their experiment. So we wondered: what if a deep learning model could be trained to just “know” what nuclei look like? … Regardless of the type of stain used, the species or cell type, the microscope, the resolution, and so on. If you could build an accurate model into open-source and commercial software, that could find nuclei in images with no biologist intervention, it would save researchers hundreds of thousands of hours of effort each year. No more configuring classical image processing algorithms and certainly no more looking at images by eye. Pharma companies around the world process millions of images each year and tens of thousands of graduate students do smaller scale experiments that would be accelerated by a successful solution to this year’s challenge.
\n\nNow the key to this challenge is robustness. Any one of these image sets is probably relatively solvable using an off-the-shelf architecture. But what makes this an interesting challenge is that we want a model that detects nuclei across different experimental systems - different stains, species, and microscopes. Biologists generally lack the expertise to train their own models - and we suspect it’s not necessary. We suspect that one of you will come up with a model that will work effectively across a wide variety of types of nuclei and experimental systems.
\n\nSo in the test (oops - I meant data) set you will notice some variety. Many images are very easy, some are very hard. Within a given image, many nuclei might be easy but the challenging part is accurately figuring out the difference between a single lumpy nucleus and two adjacent nuclei. Most importantly, the images to be used in the final test phase on which you will be judged are generally from different experiments or cell types than those included in the training data provided to you. So generalizability is a key goal for your solution.
\n\nMy team annotated more than 25,000 nuclei for this challenge. It was incredibly tedious but our hope is that one of your solutions will pay off and reduce this kind of tedious work for disease researchers. My laboratory’s mission is to bring powerful computational tools to the hands of biologists. This started with the CellProfiler project that is currently used by tens of thousands of biologists around the world, and we are now pioneering interfaces for biologists to use deep learning models such as those developed in this competition.
\n\nSo good luck! Thousands of biologists around the world are counting on you to come up with a good solution. Not to mention millions of patients whose lives could be transformed by new medicines, if we can speed up the process of drug discovery through a successful nucleus detector. I hope you enjoy putting your knowledge and skills to the test.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "271403": "For this competition, Anne (Anne Carpenter, Director of Imaging Platform @ Broad, has made an excellent, short introductory video posted here : 2018 Data Science Bowl – Technical Overview : https://www.youtube.com/watch?v=Dbiq6l50zO8 \n\nThis must have been linked to in the Description of the Competition or the Overview section, as it succinctly explains the need for such a competition and even addresses fine points like why spot the nuclei alone rather than building it for the whole cell classification directly. \n\nThanks for the nice video, Anne. All of us must check it out.", "272877": "Videos are nice so you capture my enthusiasm, but some of you might absorb information better via text (I know I do!) So for your convenience, here is the script I wrote for making the video. I can't promise it's identical but it is pretty close:\n\n====\n\nHello! I am Dr. Anne Carpenter, a cell biologist and computer scientist. I run a research lab at \nthe Broad Institute of Harvard and MIT and we are the nonprofit data partner for the Data Science Bowl presented by Booz Allen Hamilton and Kaggle.\n\nThe Broad Institute is a technology-focused biomedical research center with strong roots in the Human Genome Project, which provided the foundation for figuring out the underlying genetic causes of various diseases.\n\nThe Institute’s mission is to translate that knowledge into new medicines, and we are involved in projects to discover the causes and cures of all sorts of diseases, from cancer to metabolic disorders to psychiatric diseases. \n\nThis year’s Data Science Bowl challenge is driven by a very real need to develop new treatments faster and more accurately. We can look at the numbers:\n\nIn the tech industry we have Moore’s law which describes the phenomenal pace at which computers get faster and cheaper. Well, in the pharmaceutical industry we have “Eroom’s law” - the inverse of Moore’s law, which describes the decreasing number of new medicines that you get for a billion dollars of research spending. This is a log scale! So in the 1950s you could discover 30 new drugs for a cost that will not even find a single new drug today.\n\nBut you can also walk into a hospital or doctor’s office and see people suffering with diseases for which we lack the power to cure or even alleviate. One major bottleneck in the search for effective medicines is finding nuclei in microscopy images. Let me explain why: we often want to test thousands of chemicals to find one that might be a good drug. \n\nScientists prepare batches of cells, add a different chemical to each batch, and take pictures of each batch of cells. They want to see which cells responded in a certain way that indicates that the cells are healthier. Now, healthy might not always be a color change as in this simple example. It might instead be that breast cancer cells look less metastatic or that tuberculosis bacteria are no longer infecting the cells. But whatever it is, we certainly need software to analyze the cell’s health. This is because robots exist to prepare millions of batches of cells, and to add millions of compounds to the cells, and even to capture images of those cells. \n\nA major bottleneck is analyzing the cell images for whatever characteristics the disease expert is studying. The first step in software analyzing images for a particular disease is often to identify nuclei. \n\nThese images show the same 6 cells growing on a slide. But in each image we are looking at a different part of the cells using a different stain. In some of the images you can see parts of each cell are touching and overlapping the others. As you can see, it is pretty clear that the stain on the far left is the one where the cells are most readily distinguished from each other. That stain is labeling DNA: that is the genetic material contained within the nucleus, which is usually located towards the center of each cell. \n\nIn fact, the most effective strategies for a software solution that measures a particular disease state of cells often begins with finding the nuclei, because they are usually the most distinguishable part of the cell. From there, software can more easily measure various properties of the cells to discern their disease state. And that brings us to this year’s challenge of creating a nucleus detector!\n\nIn order for you to appreciate how transformative this would be, let me first explain what scientists currently do. \n\nFirst, there are computational algorithms that can do a pretty good job of finding nuclei and then measuring the disease state of cells. But they work best when nuclei are fairly roundish and not too crowded. \n\nSometimes nuclei have very unusual shapes as shown here. \n\nAnd sometimes nuclei can be difficult to distinguish, especially in more complicated experiments that involve tissue samples. In experiments like these, classical image processing algorithms tend to fail. \n\nSometimes the biologist has no choice but to literally look at every image by eye in order to complete their experiment. So we wondered: what if a deep learning model could be trained to just “know” what nuclei look like? … Regardless of the type of stain used, the species or cell type, the microscope, the resolution, and so on. If you could build an accurate model into open-source and commercial software, that could find nuclei in images with no biologist intervention, it would save researchers hundreds of thousands of hours of effort each year. No more configuring classical image processing algorithms and certainly no more looking at images by eye. Pharma companies around the world process millions of images each year and tens of thousands of graduate students do smaller scale experiments that would be accelerated by a successful solution to this year’s challenge. \n\nNow the key to this challenge is robustness. Any one of these image sets is probably relatively solvable using an off-the-shelf architecture. But what makes this an interesting challenge is that we want a model that detects nuclei across different experimental systems - different stains, species, and microscopes. Biologists generally lack the expertise to train their own models - and we suspect it’s not necessary. We suspect that one of you will come up with a model that will work effectively across a wide variety of types of nuclei and experimental systems. \n\nSo in the test (oops - I meant data) set you will notice some variety. Many images are very easy, some are very hard. Within a given image, many nuclei might be easy but the challenging part is accurately figuring out the difference between a single lumpy nucleus and two adjacent nuclei. Most importantly, the images to be used in the final test phase on which you will be judged are generally from different experiments or cell types than those included in the training data provided to you. So generalizability is a key goal for your solution. \n\nMy team annotated more than 25,000 nuclei for this challenge. It was incredibly tedious but our hope is that one of your solutions will pay off and reduce this kind of tedious work for disease researchers. My laboratory’s mission is to bring powerful computational tools to the hands of biologists. This started with the CellProfiler project that is currently used by tens of thousands of biologists around the world, and we are now pioneering interfaces for biologists to use deep learning models such as those developed in this competition. \n \nSo good luck! Thousands of biologists around the world are counting on you to come up with a good solution. Not to mention millions of patients whose lives could be transformed by new medicines, if we can speed up the process of drug discovery through a successful nucleus detector. I hope you enjoy putting your knowledge and skills to the test." }, "source": "meta" }