Evolution of Data Science Bowl prizes.
\n\nEvolution of Data Science Bowl prizes.
\n\nMeh, money was never really a motivator for me in these competitions. There are probably better, more guaranteed means of earning money than competing in a data science contest. A better gauge of the size of data science \"bubble\" is looking at the salaries of data scientists. And those have been steadily increasing over the last few years.
", "rawMarkdown": "Meh, money was never really a motivator for me in these competitions. There are probably better, more guaranteed means of earning money than competing in a data science contest. A better gauge of the size of data science \"bubble\" is looking at the salaries of data scientists. And those have been steadily increasing over the last few years. ", "votes": 20 }, { "id": 269481, "postDate": "2018-01-16T22:50:57.650Z", "content": "$$\n\\uparrow\\mathrm{Ads \\;on\\; Data \\; Science} \\rightarrow\\uparrow\\mathrm{Supply} \\Rightarrow \\mathrm{Supply} >> \\mathrm{Demand} \\Rightarrow \\mathrm{Wages} \\downarrow \\\n$$
", "rawMarkdown": "$$\n\\uparrow\\mathrm{Ads \\;on\\; Data \\; Science} \\rightarrow\\uparrow\\mathrm{Supply} \\Rightarrow \\mathrm{Supply} >> \\mathrm{Demand} \\Rightarrow \\mathrm{Wages} \\downarrow \\\\\n$$", "votes": 12 }, { "id": 269379, "postDate": "2018-01-16T18:08:16.590Z", "content": "There is a lot of hype in the media yet. I think this is more related with supply and demand: the number of kaggle competitors has grown exponentially and they will have figured out that don't need huge prizes anymore to have plenty of data scientists eager to solve the problem.
", "rawMarkdown": "There is a lot of hype in the media yet. I think this is more related with supply and demand: the number of kaggle competitors has grown exponentially and they will have figured out that don't need huge prizes anymore to have plenty of data scientists eager to solve the problem.", "votes": 8, "replies": [ { "id": 269396, "postDate": "2018-01-16T18:44:27.577Z", "content": "Quality of model is directly related to the prize money. For eg: I will be willing to spend $1k in AWS GPUs for a $1MM prize money competition rather than one with 12k.
", "rawMarkdown": "Quality of model is directly related to the prize money. For eg: I will be willing to spend $1k in AWS GPUs for a $1MM prize money competition rather than one with 12k.", "votes": 8 }, { "id": 269415, "postDate": "2018-01-16T19:53:49.267Z", "content": "I think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.
", "rawMarkdown": "I think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.", "votes": 1 }, { "id": 269463, "postDate": "2018-01-16T21:45:44.177Z", "content": "@Mohd, I don't hink the relationship model_quality/prize is linear: I think it is more like the sigmoid, I mean, over a certain prize you don't get further improving in the model: (today) you will have almost the same number of competitors whith a prize of 100k $ and with 1M $, and the difference of the resulting model will be negligible. And for the expenses on AWS GPU, I wouldn't spend 1000 $ either for a prize of 100k or for a prize of 1M, because the probability of winning would be equally very low, it would be a waste of money, but this is only my opinion about how to spend my money. Everyone is free to do with their money whatever they want, of course.
", "rawMarkdown": "@Mohd, I don't hink the relationship model_quality/prize is linear: I think it is more like the sigmoid, I mean, over a certain prize you don't get further improving in the model: (today) you will have almost the same number of competitors whith a prize of 100k $ and with 1M $, and the difference of the resulting model will be negligible. And for the expenses on AWS GPU, I wouldn't spend 1000 $ either for a prize of 100k or for a prize of 1M, because the probability of winning would be equally very low, it would be a waste of money, but this is only my opinion about how to spend my money. Everyone is free to do with their money whatever they want, of course. ", "votes": 1 }, { "id": 269686, "postDate": "2018-01-17T05:23:06.463Z", "content": "\n\n\nSteven wrote
\n \n\n\nI think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.
\n
The teams that scored well on TSA put in a lot of effort, actually. For example, #1 did thousands of photoshoppings to augment the training data. #7 used software to create synthetic images of human bodies. I can say that I spent 3 months on that competition. It wouldn't be very rational to do that if the prize was much lower.
", "rawMarkdown": "\n> **Steven wrote**\n> \n> > I think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.\n\nThe teams that scored well on TSA put in a lot of effort, actually. For example, #1 did thousands of photoshoppings to augment the training data. #7 used software to create synthetic images of human bodies. I can say that I spent 3 months on that competition. It wouldn't be very rational to do that if the prize was much lower.", "votes": 6 }, { "id": 277657, "postDate": "2018-02-03T20:48:09.380Z", "rawMarkdown": "", "isDeleted": true }, { "id": 277667, "postDate": "2018-02-03T21:59:53.183Z", "content": "Consider these scenarios:
\n\nMotivation to do all of the above:\n1. Research interest\n2.Knowledge addition\n3.Prize money
\n\nMost of the challenges are related to prediction, any improvement whether achieved via ensemble or feature engineering matters a lot.
", "rawMarkdown": "Consider these scenarios:\n\n1. To test multiple NN architectures -> need GPUs \n2.Create resource intensive features like page rank or other creative features -> better compute power\n3.Param tuning -> need distributed compute framework\n\nMotivation to do all of the above:\n1. Research interest\n2.Knowledge addition\n3.Prize money\n\n\nMost of the challenges are related to prediction, any improvement whether achieved via ensemble or feature engineering matters a lot. " }, { "id": 277672, "postDate": "2018-02-03T22:10:41.460Z", "content": "I really don’t think anybody will win by hyperparameter optimization since overfitting will be seriously penalized in phase two. I think it’d be a serious mistake spending your time overfitting your leaderscore board rather than generalizing your model. I think it’s probably true for ensemble learning too since the images are not that information rich.
", "rawMarkdown": "I really don’t think anybody will win by hyperparameter optimization since overfitting will be seriously penalized in phase two. I think it’d be a serious mistake spending your time overfitting your leaderscore board rather than generalizing your model. I think it’s probably true for ensemble learning too since the images are not that information rich. ", "votes": 1 }, { "id": 280338, "postDate": "2018-02-09T18:26:55.923Z", "content": "There would have been thousands of participants just from the US if the TSA dataset hadn't been so huge. It was 3 Terabytes.
", "rawMarkdown": "There would have been thousands of participants just from the US if the TSA dataset hadn't been so huge. It was *3 Terabytes*. ", "votes": 1 } ] }, { "id": 269583, "postDate": "2018-01-17T02:31:14.363Z", "content": "Re 2017 the prize had additional funding:\n\"This year, the Data Science Bowl will award $1 million in prizes to those who observe the right patterns, ask the right questions, and in turn, create unprecedented impact around cancer screening care and prevention. The funds for the prize purse will be provided by the Laura and John Arnold Foundation.\"
", "rawMarkdown": "Re 2017 the prize had additional funding:\n\"This year, the Data Science Bowl will award $1 million in prizes to those who observe the right patterns, ask the right questions, and in turn, create unprecedented impact around cancer screening care and prevention. The funds for the prize purse will be provided by the Laura and John Arnold Foundation.\"", "votes": 7 }, { "id": 269424, "postDate": "2018-01-16T20:18:12.600Z", "content": "This is quite interesting indeed. I find the prize very low for such a contest. Those competitions require usually a ton of work (and it's not always interesting research neither - quite often slow, painful experimentation with standard steps, workflows, given competition data particularities etc).
\n\nSo, if you feel confident and think you have say 10% chance of winning the competition (which I think would be a quite optimistic statement for most of us) with the appropriate amount of work put in, you are looking at an expected gain of 5k$ for 3 months of work, not counting the actual hardware / cloud expenses.
", "rawMarkdown": "This is quite interesting indeed. I find the prize very low for such a contest. Those competitions require usually a ton of work (and it's not always interesting research neither - quite often slow, painful experimentation with standard steps, workflows, given competition data particularities etc). \n\nSo, if you feel confident and think you have say 10% chance of winning the competition (which I think would be a quite optimistic statement for most of us) with the appropriate amount of work put in, you are looking at an expected gain of 5k$ for 3 months of work, not counting the actual hardware / cloud expenses.", "votes": 8 }, { "id": 270084, "postDate": "2018-01-17T18:34:00.043Z", "content": "I beg to differ with Ricardus and I'd rather second Bojan.
\n\nI think most kagglers join competitions to have fun, build new skills and/or get (better) job positions thanks to ranking and medals...and certainly not hoping to win the prize.
\n\nEven for those who win the prize, I don't think there are only motivated by the amount of money for their workload.
\n\nMy first competiton on Kaggle was Quora question pairs. I was there to have fun and apply things I learnt so far. But I observed that the competition was pretty intense for many kagglers. Teams at the top of the LB were working incredibly hard until the last minutes, while the prize was \"only 25 000$\".
", "rawMarkdown": "I beg to differ with Ricardus and I'd rather second Bojan. \n\nI think most kagglers join competitions to have fun, build new skills and/or get (better) job positions thanks to ranking and medals...and certainly not hoping to win the prize. \n\nEven for those who win the prize, I don't think there are only motivated by the amount of money for their workload.\n\nMy first competiton on Kaggle was Quora question pairs. I was there to have fun and apply things I learnt so far. But I observed that the competition was pretty intense for many kagglers. Teams at the top of the LB were working incredibly hard until the last minutes, while the prize was \"only 25 000$\". \n\n", "votes": 7, "replies": [ { "id": 280350, "postDate": "2018-02-09T19:12:23.243Z", "content": "Absolutely! The prize amounts don't mean anything. People with the skills to win a competition can make MUCH more money working.
", "rawMarkdown": "Absolutely! The prize amounts don't mean anything. People with the skills to win a competition can make MUCH more money working. " } ] }, { "id": 269536, "postDate": "2018-01-17T01:07:12.060Z", "content": "I did not look specifically at images in this competition, but I have looked at my share of other cell images. Identifying nuclei is not that difficult, which may explain the relatively low prize money. Really, in most animal cells there isn't any kind structure that is even remotely similar to nuclei in terms of size, shape and position.
", "rawMarkdown": "I did not look specifically at images in this competition, but I have looked at my share of other cell images. Identifying nuclei is not that difficult, which may explain the relatively low prize money. Really, in most animal cells there isn't any kind structure that is even remotely similar to nuclei in terms of size, shape and position.", "votes": 3 }, { "id": 270083, "postDate": "2018-01-17T18:32:43.787Z", "content": "There is a possibility that we will NOT be required by 2019 or 2020
\n\nCLOUD AUTOML ALPHA\nTrain high quality custom machine learning models with minimum effort and machine learning expertise
\n\n\"Cloud AutoML is a suite of Machine Learning products that enables developers with limited machine learning expertise to train high quality models by leveraging Google’s state of the art transfer learning, and Neural Architecture Search technology.\"
\n\nNext time, no need human expert to construct CNN network to tune hyper parameter, every thing is \"AI\"
\n\n\n\nhttps://cloud.google.com/automl/
", "rawMarkdown": "There is a possibility that we will NOT be required by 2019 or 2020\n\nCLOUD AUTOML ALPHA\nTrain high quality custom machine learning models with minimum effort and machine learning expertise\n\n\"Cloud AutoML is a suite of Machine Learning products that enables developers with limited machine learning expertise to train high quality models by leveraging Google’s state of the art transfer learning, and Neural Architecture Search technology.\"\n\nNext time, no need human expert to construct CNN network to tune hyper parameter, every thing is \"AI\"\n\n\nhttps://techcrunch.com/2018/01/17/googles-automl-lets-you-train-custom-machine-learning-models-without-having-to-code/\n\nhttps://cloud.google.com/automl/", "votes": 3, "replies": [ { "id": 270090, "postDate": "2018-01-17T18:40:29.797Z", "content": "It makes sense to me, Heng. I posted a similar idea here that automation in data science is going to heavily influence the market. Thanks for sharing this.
", "rawMarkdown": "It makes sense to me, Heng. I posted a similar idea [here][1] that automation in data science is going to heavily influence the market. Thanks for sharing this. \n\n\n[1]: https://www.kaggle.com/general/47293#post267515" }, { "id": 270097, "postDate": "2018-01-17T18:45:18.623Z", "content": "Google bought over kaggle and now he has a database of human performance. He can now use AutoML to train models and compare with human performance and claimed that AutoML is indeed better than \"average data scientist\"
", "rawMarkdown": "Google bought over kaggle and now he has a database of human performance. He can now use AutoML to train models and compare with human performance and claimed that AutoML is indeed better than \"average data scientist\"", "votes": 5 }, { "id": 270107, "postDate": "2018-01-17T18:57:24.977Z", "content": "I’ve been hearing about auto ML ever since I got into ML. A new system that is on par with the best human ML experts pops up every few months. Every once in a while I try one of these systems out, and IF (A BIG IF!!!!!!) data is nicely prepared, plentiful and really predictive, the system scores on par with the best Kaggle human prepared kernel. And that’s just for really artificial world of DS competitions. In real world, and in particular in production, using auto ML solutions is a nonstarter.
", "rawMarkdown": "I’ve been hearing about auto ML ever since I got into ML. A new system that is on par with the best human ML experts pops up every few months. Every once in a while I try one of these systems out, and IF (A BIG IF!!!!!!) data is nicely prepared, plentiful and really predictive, the system scores on par with the best Kaggle human prepared kernel. And that’s just for really artificial world of DS competitions. In real world, and in particular in production, using auto ML solutions is a nonstarter.", "votes": 13 }, { "id": 270221, "postDate": "2018-01-17T23:26:50.747Z", "content": "I don't think the comparison is apt. This looks like an enterprise tool to compete with Azure's offerings and while they may not be able to compete with mega nerds, it can definitely beat a logistic regression classifier in Excel.
", "rawMarkdown": "I don't think the comparison is apt. This looks like an enterprise tool to compete with Azure's offerings and while they may not be able to compete with mega nerds, it can definitely beat a logistic regression classifier in Excel." }, { "id": 270368, "postDate": "2018-01-18T06:17:50.087Z", "content": "I agree with Steven. Creating AI using AI is still a distant place to reach. Yes, 2019 and 2020 will see remarkable changes in the way we use and build AI but surely not take the human AI builders out of the picture.
", "rawMarkdown": "I agree with Steven. Creating AI using AI is still a distant place to reach. Yes, 2019 and 2020 will see remarkable changes in the way we use and build AI but surely not take the human AI builders out of the picture." }, { "id": 270373, "postDate": "2018-01-18T06:23:07.560Z", "content": "Thanks for all the comment.
\n\nI signed up for a alpha tester. If i get to use it, i will try it in this kaggle challenge and let you guys know.
\n\nBut currently, it seems to support only image classification and i need to find out more.
", "rawMarkdown": "Thanks for all the comment.\n\nI signed up for a alpha tester. If i get to use it, i will try it in this kaggle challenge and let you guys know.\n\nBut currently, it seems to support only image classification and i need to find out more.", "votes": 4 }, { "id": 270426, "postDate": "2018-01-18T07:48:19.640Z", "content": "\"Do X without writing code\" has been present since a long time now. But guess what, we still write code.
", "rawMarkdown": "\"Do X without writing code\" has been present since a long time now. But guess what, we still write code.", "votes": 2 }, { "id": 270539, "postDate": "2018-01-18T13:21:23.913Z", "content": "I am amused every time I see such claims. Automation of repetitive programming tasks is nowhere on horizon, ML systems have complication of different order of magnitude...
", "rawMarkdown": "I am amused every time I see such claims. Automation of repetitive programming tasks is nowhere on horizon, ML systems have complication of different order of magnitude..." } ] }, { "id": 271031, "postDate": "2018-01-19T14:45:23.017Z", "content": "There is a DGX system valued at $70k bringing the total competition prize value to ~ $170k. Assuming the other competitions did not offer non cash prizes the 2017 prize amount looks like the exception rather than the norm.
", "rawMarkdown": "There is a DGX system valued at $70k bringing the total competition prize value to ~ $170k. Assuming the other competitions did not offer non cash prizes the 2017 prize amount looks like the exception rather than the norm.", "votes": 1 }, { "id": 270969, "postDate": "2018-01-19T10:43:46.467Z", "content": "I do agree with the poster, the data science bubble will end.\nThe term bubble by definition is referred to some trend that is seen as unsustainable, but unlike other bubbles, this one won't burst.
\n\nAs the poster noticed, the rewards for each Data Science Bowl competitions are going down, but that doesn't mean, as pointed out in the comments by some, that we will be made obsolete by AI - at least not anytime soon.
\n\nOur models now, although increasingly efficient and more generalized, work only for the purpose they are designed for, as long as there are jobs left to automate or things needed to be optimized - Data Scientists will exist - human ones.
\n\nOnce we have automated every job on the planet only then we can automate ours.
\n\nIn my opinion
", "rawMarkdown": "I do agree with the poster, the data science bubble will end.\nThe term bubble by definition is referred to some trend that is seen as unsustainable, but unlike other bubbles, this one won't burst.\n\nAs the poster noticed, the rewards for each Data Science Bowl competitions are going down, but that doesn't mean, as pointed out in the comments by some, that we will be made obsolete by AI - at least not anytime soon.\n\nOur models now, although increasingly efficient and more generalized, work only for the purpose they are designed for, as long as there are jobs left to automate or things needed to be optimized - Data Scientists will exist - human ones.\n\nOnce we have automated every job on the planet only then we can automate ours.\n\nIn my opinion\n", "votes": 1 }, { "id": 270640, "postDate": "2018-01-18T16:49:21.987Z", "content": "Kaggle problems are already greatly simplified in order to fit a one-number-tells-all framework. Optimizing one number is relatively easy. Setting up a problem so that it's solved appropriately by optimizing a single number is much harder.
", "rawMarkdown": "Kaggle problems are already greatly simplified in order to fit a one-number-tells-all framework. Optimizing one number is relatively easy. Setting up a problem so that it's solved appropriately by optimizing a single number is much harder.", "votes": 1 }, { "id": 270543, "postDate": "2018-01-18T13:34:06.593Z", "content": "I don't believe AutoML will be able to beat good data scientists in the next 3 years at least. It is probably trained through reinforcement learning. One problem they will have when diversifying to different things like this competition's segmentation is the intuitive knowledge it will need. In order for it to build this knowledge, it will need a very very large data set. The amount of choices for a good network would be massive. Indeed it can beat an 'average data scientist', but I don't believe people who get 1st place in Kaggle competitions are 'average'.
", "rawMarkdown": "I don't believe AutoML will be able to beat good data scientists in the next 3 years at least. It is probably trained through reinforcement learning. One problem they will have when diversifying to different things like this competition's segmentation is the intuitive knowledge it will need. In order for it to build this knowledge, it will need a very very large data set. The amount of choices for a good network would be massive. Indeed it can beat an 'average data scientist', but I don't believe people who get 1st place in Kaggle competitions are 'average'.", "votes": 1 }, { "id": 272590, "postDate": "2018-01-23T09:51:11.943Z", "content": "Maybe its the fact that its much more accessible and easier to learn, understand and implement DS algorithms today than ever before.
", "rawMarkdown": "Maybe its the fact that its much more accessible and easier to learn, understand and implement DS algorithms today than ever before.", "votes": 2 }, { "id": 269781, "postDate": "2018-01-17T09:47:58.530Z", "content": "In my opinion those who can will provide more prize money. Centers and Institutions tend to have bigger budgets whereas individual labs do not.
", "rawMarkdown": "In my opinion those who can will provide more prize money. Centers and Institutions tend to have bigger budgets whereas individual labs do not. ", "votes": 2 }, { "id": 270930, "postDate": "2018-01-19T09:23:20.043Z", "content": "We are the agents of the reinforcement learning protocol used by Kaggle, the reward function to maximise in our cas is the money D: .. but with AutoML there is no need for money.
", "rawMarkdown": "We are the agents of the reinforcement learning protocol used by Kaggle, the reward function to maximise in our cas is the money D: .. but with AutoML there is no need for money." }, { "id": 269944, "postDate": "2018-01-17T14:57:01.047Z", "content": "YES!
", "rawMarkdown": "YES!" }, { "id": 269616, "postDate": "2018-01-17T03:07:50.033Z", "content": "Wouldn't we need more information on the competitions other than prizes. Time line, nature of the competition. Plus, with the ever growing competition platforms, sponsors have other platforms they can use. Also, what types of companies are investing? I'm just saying going on the above data alone leaves such a bias that the result would tell us little at best.
", "rawMarkdown": "Wouldn't we need more information on the competitions other than prizes. Time line, nature of the competition. Plus, with the ever growing competition platforms, sponsors have other platforms they can use. Also, what types of companies are investing? I'm just saying going on the above data alone leaves such a bias that the result would tell us little at best. " }, { "id": 269539, "postDate": "2018-01-17T01:11:53.420Z", "content": "One possible statistic is calculate #GM/prize-dollar
", "rawMarkdown": "One possible statistic is calculate #GM/prize-dollar" } ], "comments": [ { "id": 269473, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-01-16T22:21:45.690000", "content": "Meh, money was never really a motivator for me in these competitions. There are probably better, more guaranteed means of earning money than competing in a data science contest. A better gauge of the size of data science \"bubble\" is looking at the salaries of data scientists. And those have been steadily increasing over the last few years.
", "votes": 20, "replies": [] }, { "id": 269481, "author_name": "AJ", "author_url": "", "post_date": "2018-01-16T22:50:57.650000", "content": "$$\n\\uparrow\\mathrm{Ads \\;on\\; Data \\; Science} \\rightarrow\\uparrow\\mathrm{Supply} \\Rightarrow \\mathrm{Supply} >> \\mathrm{Demand} \\Rightarrow \\mathrm{Wages} \\downarrow \\\n$$
", "votes": 12, "replies": [] }, { "id": 269379, "author_name": "Ricardo Luján", "author_url": "", "post_date": "2018-01-16T18:08:16.590000", "content": "There is a lot of hype in the media yet. I think this is more related with supply and demand: the number of kaggle competitors has grown exponentially and they will have figured out that don't need huge prizes anymore to have plenty of data scientists eager to solve the problem.
", "votes": 8, "replies": [ { "id": 269396, "author_name": "Mohammad Shadab Alam", "author_url": "", "post_date": "2018-01-16T18:44:27.577000", "content": "Quality of model is directly related to the prize money. For eg: I will be willing to spend $1k in AWS GPUs for a $1MM prize money competition rather than one with 12k.
", "votes": 8, "replies": [] }, { "id": 269415, "author_name": "Alan Khoa Nguyen", "author_url": "", "post_date": "2018-01-16T19:53:49.267000", "content": "I think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.
", "votes": 1, "replies": [] }, { "id": 269463, "author_name": "Ricardo Luján", "author_url": "", "post_date": "2018-01-16T21:45:44.177000", "content": "@Mohd, I don't hink the relationship model_quality/prize is linear: I think it is more like the sigmoid, I mean, over a certain prize you don't get further improving in the model: (today) you will have almost the same number of competitors whith a prize of 100k $ and with 1M $, and the difference of the resulting model will be negligible. And for the expenses on AWS GPU, I wouldn't spend 1000 $ either for a prize of 100k or for a prize of 1M, because the probability of winning would be equally very low, it would be a waste of money, but this is only my opinion about how to spend my money. Everyone is free to do with their money whatever they want, of course.
", "votes": 1, "replies": [] }, { "id": 269686, "author_name": "Oleg Trott", "author_url": "", "post_date": "2018-01-17T05:23:06.463000", "content": "\n\n\nSteven wrote
\n \n\n\nI think the tsa contest proved that more money doesn't necessarily mean more people will participate, even adjusted for geographical differences.
\n
The teams that scored well on TSA put in a lot of effort, actually. For example, #1 did thousands of photoshoppings to augment the training data. #7 used software to create synthetic images of human bodies. I can say that I spent 3 months on that competition. It wouldn't be very rational to do that if the prize was much lower.
", "votes": 6, "replies": [] }, { "id": 277657, "author_name": "", "author_url": "", "post_date": "2018-02-03T20:48:09.380000", "content": "", "votes": 0, "replies": [] }, { "id": 277667, "author_name": "Mohammad Shadab Alam", "author_url": "", "post_date": "2018-02-03T21:59:53.183000", "content": "Consider these scenarios:
\n\nMotivation to do all of the above:\n1. Research interest\n2.Knowledge addition\n3.Prize money
\n\nMost of the challenges are related to prediction, any improvement whether achieved via ensemble or feature engineering matters a lot.
", "votes": 0, "replies": [] }, { "id": 277672, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-03T22:10:41.460000", "content": "I really don’t think anybody will win by hyperparameter optimization since overfitting will be seriously penalized in phase two. I think it’d be a serious mistake spending your time overfitting your leaderscore board rather than generalizing your model. I think it’s probably true for ensemble learning too since the images are not that information rich.
", "votes": 1, "replies": [] }, { "id": 280338, "author_name": "Analog", "author_url": "", "post_date": "2018-02-09T18:26:55.923000", "content": "There would have been thousands of participants just from the US if the TSA dataset hadn't been so huge. It was 3 Terabytes.
", "votes": 1, "replies": [] } ] }, { "id": 269583, "author_name": "something4kag", "author_url": "", "post_date": "2018-01-17T02:31:14.363000", "content": "Re 2017 the prize had additional funding:\n\"This year, the Data Science Bowl will award $1 million in prizes to those who observe the right patterns, ask the right questions, and in turn, create unprecedented impact around cancer screening care and prevention. The funds for the prize purse will be provided by the Laura and John Arnold Foundation.\"
", "votes": 7, "replies": [] }, { "id": 269424, "author_name": "Alchemist", "author_url": "", "post_date": "2018-01-16T20:18:12.600000", "content": "This is quite interesting indeed. I find the prize very low for such a contest. Those competitions require usually a ton of work (and it's not always interesting research neither - quite often slow, painful experimentation with standard steps, workflows, given competition data particularities etc).
\n\nSo, if you feel confident and think you have say 10% chance of winning the competition (which I think would be a quite optimistic statement for most of us) with the appropriate amount of work put in, you are looking at an expected gain of 5k$ for 3 months of work, not counting the actual hardware / cloud expenses.
", "votes": 8, "replies": [] }, { "id": 270084, "author_name": "Serigne ", "author_url": "", "post_date": "2018-01-17T18:34:00.043000", "content": "I beg to differ with Ricardus and I'd rather second Bojan.
\n\nI think most kagglers join competitions to have fun, build new skills and/or get (better) job positions thanks to ranking and medals...and certainly not hoping to win the prize.
\n\nEven for those who win the prize, I don't think there are only motivated by the amount of money for their workload.
\n\nMy first competiton on Kaggle was Quora question pairs. I was there to have fun and apply things I learnt so far. But I observed that the competition was pretty intense for many kagglers. Teams at the top of the LB were working incredibly hard until the last minutes, while the prize was \"only 25 000$\".
", "votes": 7, "replies": [ { "id": 280350, "author_name": "Analog", "author_url": "", "post_date": "2018-02-09T19:12:23.243000", "content": "Absolutely! The prize amounts don't mean anything. People with the skills to win a competition can make MUCH more money working.
", "votes": 0, "replies": [] } ] }, { "id": 269536, "author_name": "Tilii", "author_url": "", "post_date": "2018-01-17T01:07:12.060000", "content": "I did not look specifically at images in this competition, but I have looked at my share of other cell images. Identifying nuclei is not that difficult, which may explain the relatively low prize money. Really, in most animal cells there isn't any kind structure that is even remotely similar to nuclei in terms of size, shape and position.
", "votes": 3, "replies": [] }, { "id": 270083, "author_name": "hengck23", "author_url": "", "post_date": "2018-01-17T18:32:43.787000", "content": "There is a possibility that we will NOT be required by 2019 or 2020
\n\nCLOUD AUTOML ALPHA\nTrain high quality custom machine learning models with minimum effort and machine learning expertise
\n\n\"Cloud AutoML is a suite of Machine Learning products that enables developers with limited machine learning expertise to train high quality models by leveraging Google’s state of the art transfer learning, and Neural Architecture Search technology.\"
\n\nNext time, no need human expert to construct CNN network to tune hyper parameter, every thing is \"AI\"
\n\n\n\nhttps://cloud.google.com/automl/
", "votes": 3, "replies": [ { "id": 270090, "author_name": "AJ", "author_url": "", "post_date": "2018-01-17T18:40:29.797000", "content": "It makes sense to me, Heng. I posted a similar idea here that automation in data science is going to heavily influence the market. Thanks for sharing this.
", "votes": 0, "replies": [] }, { "id": 270097, "author_name": "hengck23", "author_url": "", "post_date": "2018-01-17T18:45:18.623000", "content": "Google bought over kaggle and now he has a database of human performance. He can now use AutoML to train models and compare with human performance and claimed that AutoML is indeed better than \"average data scientist\"
", "votes": 5, "replies": [] }, { "id": 270107, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-01-17T18:57:24.977000", "content": "I’ve been hearing about auto ML ever since I got into ML. A new system that is on par with the best human ML experts pops up every few months. Every once in a while I try one of these systems out, and IF (A BIG IF!!!!!!) data is nicely prepared, plentiful and really predictive, the system scores on par with the best Kaggle human prepared kernel. And that’s just for really artificial world of DS competitions. In real world, and in particular in production, using auto ML solutions is a nonstarter.
", "votes": 13, "replies": [] }, { "id": 270221, "author_name": "Alan Khoa Nguyen", "author_url": "", "post_date": "2018-01-17T23:26:50.747000", "content": "I don't think the comparison is apt. This looks like an enterprise tool to compete with Azure's offerings and while they may not be able to compete with mega nerds, it can definitely beat a logistic regression classifier in Excel.
", "votes": 0, "replies": [] }, { "id": 270368, "author_name": "Somnath Rakshit", "author_url": "", "post_date": "2018-01-18T06:17:50.087000", "content": "I agree with Steven. Creating AI using AI is still a distant place to reach. Yes, 2019 and 2020 will see remarkable changes in the way we use and build AI but surely not take the human AI builders out of the picture.
", "votes": 0, "replies": [] }, { "id": 270373, "author_name": "hengck23", "author_url": "", "post_date": "2018-01-18T06:23:07.560000", "content": "Thanks for all the comment.
\n\nI signed up for a alpha tester. If i get to use it, i will try it in this kaggle challenge and let you guys know.
\n\nBut currently, it seems to support only image classification and i need to find out more.
", "votes": 4, "replies": [] }, { "id": 270426, "author_name": "Somnath Rakshit", "author_url": "", "post_date": "2018-01-18T07:48:19.640000", "content": "\"Do X without writing code\" has been present since a long time now. But guess what, we still write code.
", "votes": 2, "replies": [] }, { "id": 270539, "author_name": "Jarek", "author_url": "", "post_date": "2018-01-18T13:21:23.913000", "content": "I am amused every time I see such claims. Automation of repetitive programming tasks is nowhere on horizon, ML systems have complication of different order of magnitude...
", "votes": 0, "replies": [] } ] }, { "id": 271031, "author_name": "Jack Vial", "author_url": "", "post_date": "2018-01-19T14:45:23.017000", "content": "There is a DGX system valued at $70k bringing the total competition prize value to ~ $170k. Assuming the other competitions did not offer non cash prizes the 2017 prize amount looks like the exception rather than the norm.
", "votes": 1, "replies": [] }, { "id": 270969, "author_name": "vinitmuchhala", "author_url": "", "post_date": "2018-01-19T10:43:46.467000", "content": "I do agree with the poster, the data science bubble will end.\nThe term bubble by definition is referred to some trend that is seen as unsustainable, but unlike other bubbles, this one won't burst.
\n\nAs the poster noticed, the rewards for each Data Science Bowl competitions are going down, but that doesn't mean, as pointed out in the comments by some, that we will be made obsolete by AI - at least not anytime soon.
\n\nOur models now, although increasingly efficient and more generalized, work only for the purpose they are designed for, as long as there are jobs left to automate or things needed to be optimized - Data Scientists will exist - human ones.
\n\nOnce we have automated every job on the planet only then we can automate ours.
\n\nIn my opinion
", "votes": 1, "replies": [] }, { "id": 270640, "author_name": "Dmitriy Guller", "author_url": "", "post_date": "2018-01-18T16:49:21.987000", "content": "Kaggle problems are already greatly simplified in order to fit a one-number-tells-all framework. Optimizing one number is relatively easy. Setting up a problem so that it's solved appropriately by optimizing a single number is much harder.
", "votes": 1, "replies": [] }, { "id": 270543, "author_name": "Arpan Dhatt", "author_url": "", "post_date": "2018-01-18T13:34:06.593000", "content": "I don't believe AutoML will be able to beat good data scientists in the next 3 years at least. It is probably trained through reinforcement learning. One problem they will have when diversifying to different things like this competition's segmentation is the intuitive knowledge it will need. In order for it to build this knowledge, it will need a very very large data set. The amount of choices for a good network would be massive. Indeed it can beat an 'average data scientist', but I don't believe people who get 1st place in Kaggle competitions are 'average'.
", "votes": 1, "replies": [] }, { "id": 272590, "author_name": "Vineet Agarwal", "author_url": "", "post_date": "2018-01-23T09:51:11.943000", "content": "Maybe its the fact that its much more accessible and easier to learn, understand and implement DS algorithms today than ever before.
", "votes": 2, "replies": [] }, { "id": 269781, "author_name": "chhaatra", "author_url": "", "post_date": "2018-01-17T09:47:58.530000", "content": "In my opinion those who can will provide more prize money. Centers and Institutions tend to have bigger budgets whereas individual labs do not.
", "votes": 2, "replies": [] }, { "id": 270930, "author_name": "", "author_url": "", "post_date": "2018-01-19T09:23:20.043000", "content": "We are the agents of the reinforcement learning protocol used by Kaggle, the reward function to maximise in our cas is the money D: .. but with AutoML there is no need for money.
", "votes": 0, "replies": [] }, { "id": 269944, "author_name": "Skip Anderson", "author_url": "", "post_date": "2018-01-17T14:57:01.047000", "content": "YES!
", "votes": 0, "replies": [] }, { "id": 269616, "author_name": "", "author_url": "", "post_date": "2018-01-17T03:07:50.033000", "content": "Wouldn't we need more information on the competitions other than prizes. Time line, nature of the competition. Plus, with the ever growing competition platforms, sponsors have other platforms they can use. Also, what types of companies are investing? I'm just saying going on the above data alone leaves such a bias that the result would tell us little at best.
", "votes": 0, "replies": [] }, { "id": 269539, "author_name": "Bcottman", "author_url": "", "post_date": "2018-01-17T01:11:53.420000", "content": "One possible statistic is calculate #GM/prize-dollar
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "269368": "Evolution of Data Science Bowl prizes.\n\n - 2015: 175k\n - 2016: 200k\n - 2017: 1000k\n - 2018: 100k", "269473": "Meh, money was never really a motivator for me in these competitions. There are probably better, more guaranteed means of earning money than competing in a data science contest. A better gauge of the size of data science \"bubble\" is looking at the salaries of data scientists. And those have been steadily increasing over the last few years. ", "269481": "$$\n\\uparrow\\mathrm{Ads \\;on\\; Data \\; Science} \\rightarrow\\uparrow\\mathrm{Supply} \\Rightarrow \\mathrm{Supply} >> \\mathrm{Demand} \\Rightarrow \\mathrm{Wages} \\downarrow \\\\\n$$", "269379": "There is a lot of hype in the media yet. I think this is more related with supply and demand: the number of kaggle competitors has grown exponentially and they will have figured out that don't need huge prizes anymore to have plenty of data scientists eager to solve the problem.", "269583": "Re 2017 the prize had additional funding:\n\"This year, the Data Science Bowl will award $1 million in prizes to those who observe the right patterns, ask the right questions, and in turn, create unprecedented impact around cancer screening care and prevention. The funds for the prize purse will be provided by the Laura and John Arnold Foundation.\"", "269424": "This is quite interesting indeed. I find the prize very low for such a contest. Those competitions require usually a ton of work (and it's not always interesting research neither - quite often slow, painful experimentation with standard steps, workflows, given competition data particularities etc). \n\nSo, if you feel confident and think you have say 10% chance of winning the competition (which I think would be a quite optimistic statement for most of us) with the appropriate amount of work put in, you are looking at an expected gain of 5k$ for 3 months of work, not counting the actual hardware / cloud expenses.", "270084": "I beg to differ with Ricardus and I'd rather second Bojan. \n\nI think most kagglers join competitions to have fun, build new skills and/or get (better) job positions thanks to ranking and medals...and certainly not hoping to win the prize. \n\nEven for those who win the prize, I don't think there are only motivated by the amount of money for their workload.\n\nMy first competiton on Kaggle was Quora question pairs. I was there to have fun and apply things I learnt so far. But I observed that the competition was pretty intense for many kagglers. Teams at the top of the LB were working incredibly hard until the last minutes, while the prize was \"only 25 000$\". \n\n", "269536": "I did not look specifically at images in this competition, but I have looked at my share of other cell images. Identifying nuclei is not that difficult, which may explain the relatively low prize money. Really, in most animal cells there isn't any kind structure that is even remotely similar to nuclei in terms of size, shape and position.", "270083": "There is a possibility that we will NOT be required by 2019 or 2020\n\nCLOUD AUTOML ALPHA\nTrain high quality custom machine learning models with minimum effort and machine learning expertise\n\n\"Cloud AutoML is a suite of Machine Learning products that enables developers with limited machine learning expertise to train high quality models by leveraging Google’s state of the art transfer learning, and Neural Architecture Search technology.\"\n\nNext time, no need human expert to construct CNN network to tune hyper parameter, every thing is \"AI\"\n\n\nhttps://techcrunch.com/2018/01/17/googles-automl-lets-you-train-custom-machine-learning-models-without-having-to-code/\n\nhttps://cloud.google.com/automl/", "271031": "There is a DGX system valued at $70k bringing the total competition prize value to ~ $170k. Assuming the other competitions did not offer non cash prizes the 2017 prize amount looks like the exception rather than the norm.", "270969": "I do agree with the poster, the data science bubble will end.\nThe term bubble by definition is referred to some trend that is seen as unsustainable, but unlike other bubbles, this one won't burst.\n\nAs the poster noticed, the rewards for each Data Science Bowl competitions are going down, but that doesn't mean, as pointed out in the comments by some, that we will be made obsolete by AI - at least not anytime soon.\n\nOur models now, although increasingly efficient and more generalized, work only for the purpose they are designed for, as long as there are jobs left to automate or things needed to be optimized - Data Scientists will exist - human ones.\n\nOnce we have automated every job on the planet only then we can automate ours.\n\nIn my opinion\n", "270640": "Kaggle problems are already greatly simplified in order to fit a one-number-tells-all framework. Optimizing one number is relatively easy. Setting up a problem so that it's solved appropriately by optimizing a single number is much harder.", "270543": "I don't believe AutoML will be able to beat good data scientists in the next 3 years at least. It is probably trained through reinforcement learning. One problem they will have when diversifying to different things like this competition's segmentation is the intuitive knowledge it will need. In order for it to build this knowledge, it will need a very very large data set. The amount of choices for a good network would be massive. Indeed it can beat an 'average data scientist', but I don't believe people who get 1st place in Kaggle competitions are 'average'.", "272590": "Maybe its the fact that its much more accessible and easier to learn, understand and implement DS algorithms today than ever before.", "269781": "In my opinion those who can will provide more prize money. Centers and Institutions tend to have bigger budgets whereas individual labs do not. ", "270930": "We are the agents of the reinforcement learning protocol used by Kaggle, the reward function to maximise in our cas is the money D: .. but with AutoML there is no need for money.", "269944": "YES!", "269616": "Wouldn't we need more information on the competitions other than prizes. Time line, nature of the competition. Plus, with the ever growing competition platforms, sponsors have other platforms they can use. Also, what types of companies are investing? I'm just saying going on the above data alone leaves such a bias that the result would tell us little at best. ", "269539": "One possible statistic is calculate #GM/prize-dollar" } }