I understand that blending gives a better score. But how physical to do so? What is behind? \nOf course, in different cases the answer is different. But just in the current case study (P-M-P), what is the physicalness of blending, is it possible to control the behavior at some point, and is it appropriate for real science? Never experienced that in my field.
", "messages": [ { "id": "601530", "postDate": "08/17/2019 18:43:25", "content": "I understand that blending gives a better score. But how physical to do so? What is behind? \nOf course, in different cases the answer is different. But just in the current case study (P-M-P), what is the physicalness of blending, is it possible to control the behavior at some point, and is it appropriate for real science? Never experienced that in my field.
", "rawMarkdown": "I understand that blending gives a better score. But how physical to do so? What is behind? \nOf course, in different cases the answer is different. But just in the current case study (P-M-P), what is the physicalness of blending, is it possible to control the behavior at some point, and is it appropriate for real science? Never experienced that in my field.", "votes": null }, { "id": "601585", "postDate": "08/17/2019 20:57:36", "content": "What is easy to predict every model will probably predict correctly. As different models predict in different ways, some may predict higher values while other can predict lower values. However as all models predict values around the true values, the mean is probably closer to true values as positive errors and negative errors negates each other.
", "rawMarkdown": "What is easy to predict every model will probably predict correctly. As different models predict in different ways, some may predict higher values while other can predict lower values. However as all models predict values around the true values, the mean is probably closer to true values as positive errors and negative errors negates each other.", "votes": null }, { "id": "601632", "postDate": "08/17/2019 23:13:47", "content": "Imagine we'd have some money to invest. We can choose between Oil (Feature 1), Lithium (Feature 2) and bicycles (Feature 3). There are 3 scenarios: cars running on gas will be banned in every city (KFold 1), all cars will be banned (KFold 2), all cars will be ok (Kfold 3).\nIf we invest in F1 we will be lucky in Sc3 but we will lose everything in the other scenarios. We have high variance. We overfit to Sc3. \nIf we'd mixed our investments in F1, F2 and F3. We'd be better off in Sc 1 and Sc 2 but would sacrifice a bit in case of Sc3. But the variance would go down. I'm less gambling and the investment seems more stable.\nHmm, the scenarios seem to be to extreme anyway. So let's make new ones and add new features e.g. sports ware ...\nSo now we have two diverse models. I bet we will see some impact on the variance when we blend them.
", "rawMarkdown": "Imagine we'd have some money to invest. We can choose between Oil (Feature 1), Lithium (Feature 2) and bicycles (Feature 3). There are 3 scenarios: cars running on gas will be banned in every city (KFold 1), all cars will be banned (KFold 2), all cars will be ok (Kfold 3).\nIf we invest in F1 we will be lucky in Sc3 but we will lose everything in the other scenarios. We have high variance. We overfit to Sc3. \nIf we'd mixed our investments in F1, F2 and F3. We'd be better off in Sc 1 and Sc 2 but would sacrifice a bit in case of Sc3. But the variance would go down. I'm less gambling and the investment seems more stable.\nHmm, the scenarios seem to be to extreme anyway. So let's make new ones and add new features e.g. sports ware ...\nSo now we have two diverse models. I bet we will see some impact on the variance when we blend them.", "votes": null }, { "id": "602594", "postDate": "08/19/2019 08:49:53", "content": "@somnambwl @joatom \nThank you very much for good examples. Seems to be more clear now.
", "rawMarkdown": "somnambwl @joatom \nThank you very much for good examples. Seems to be more clear now.", "votes": null }, { "id": "605129", "postDate": "08/22/2019 04:58:07", "content": "Its well explained here: https://mlwave.com/kaggle-ensembling-guide/
\n\nA machine learning example
\n\nSuppose we have a test set of 10 samples. The ground truth is all positive (“1”):
\n\n\n1111111111\n
We furthermore have 3 binary classifiers (A,B,C) with a 70% accuracy. You can view these classifiers for now as pseudo-random number generators which output a “1” 70% of the time and a “0” 30% of the time.
\n\nWe will now show how these pseudo-classifiers are able to obtain 78% accuracy through a voting ensemble.
\n\nA pinch of maths
\n\nFor a majority vote with 3 members we can expect 4 outcomes:
\n\n```\nAll three are correct\n 0.7 * 0.7 * 0.7\n= 0.3429
\n\nTwo are correct\n 0.7 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.7\n= 0.4409
\n\nTwo are wrong\n 0.3 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.3\n= 0.189
\n\nAll three are wrong\n 0.3 * 0.3 * 0.3\n= 0.027
\n\n```\nWe see that most of the times (~44%) the majority vote corrects an error. This majority vote ensemble will be correct an average of ~78% (0.3429 + 0.4409 = 0.7838).
", "rawMarkdown": "Its well explained here: https://mlwave.com/kaggle-ensembling-guide/\n\n\nA machine learning example\n\nSuppose we have a test set of 10 samples. The ground truth is all positive (“1”):\n\n```\n1111111111\n```\n\nWe furthermore have 3 binary classifiers (A,B,C) with a 70% accuracy. You can view these classifiers for now as pseudo-random number generators which output a “1” 70% of the time and a “0” 30% of the time.\n\nWe will now show how these pseudo-classifiers are able to obtain 78% accuracy through a voting ensemble.\n\nA pinch of maths\n\nFor a majority vote with 3 members we can expect 4 outcomes:\n\n```\nAll three are correct\n 0.7 * 0.7 * 0.7\n= 0.3429\n\nTwo are correct\n 0.7 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.7\n= 0.4409\n\nTwo are wrong\n 0.3 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.3\n= 0.189\n\nAll three are wrong\n 0.3 * 0.3 * 0.3\n= 0.027\n\n```\nWe see that most of the times (~44%) the majority vote corrects an error. This majority vote ensemble will be correct an average of ~78% (0.3429 + 0.4409 = 0.7838).", "votes": null } ], "comments": [ { "id": 601585, "author_name": "somnambwl", "author_url": "", "post_date": "08/17/2019 20:57:36", "content": "What is easy to predict every model will probably predict correctly. As different models predict in different ways, some may predict higher values while other can predict lower values. However as all models predict values around the true values, the mean is probably closer to true values as positive errors and negative errors negates each other.
", "votes": null, "replies": [] }, { "id": 601632, "author_name": "joatom", "author_url": "", "post_date": "08/17/2019 23:13:47", "content": "Imagine we'd have some money to invest. We can choose between Oil (Feature 1), Lithium (Feature 2) and bicycles (Feature 3). There are 3 scenarios: cars running on gas will be banned in every city (KFold 1), all cars will be banned (KFold 2), all cars will be ok (Kfold 3).\nIf we invest in F1 we will be lucky in Sc3 but we will lose everything in the other scenarios. We have high variance. We overfit to Sc3. \nIf we'd mixed our investments in F1, F2 and F3. We'd be better off in Sc 1 and Sc 2 but would sacrifice a bit in case of Sc3. But the variance would go down. I'm less gambling and the investment seems more stable.\nHmm, the scenarios seem to be to extreme anyway. So let's make new ones and add new features e.g. sports ware ...\nSo now we have two diverse models. I bet we will see some impact on the variance when we blend them.
", "votes": null, "replies": [] }, { "id": 602594, "author_name": "muhakabartay", "author_url": "", "post_date": "08/19/2019 08:49:53", "content": "@somnambwl @joatom \nThank you very much for good examples. Seems to be more clear now.
", "votes": null, "replies": [] }, { "id": 605129, "author_name": "christofhenkel", "author_url": "", "post_date": "08/22/2019 04:58:07", "content": "Its well explained here: https://mlwave.com/kaggle-ensembling-guide/
\n\nA machine learning example
\n\nSuppose we have a test set of 10 samples. The ground truth is all positive (“1”):
\n\n\n1111111111\n
We furthermore have 3 binary classifiers (A,B,C) with a 70% accuracy. You can view these classifiers for now as pseudo-random number generators which output a “1” 70% of the time and a “0” 30% of the time.
\n\nWe will now show how these pseudo-classifiers are able to obtain 78% accuracy through a voting ensemble.
\n\nA pinch of maths
\n\nFor a majority vote with 3 members we can expect 4 outcomes:
\n\n```\nAll three are correct\n 0.7 * 0.7 * 0.7\n= 0.3429
\n\nTwo are correct\n 0.7 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.7\n= 0.4409
\n\nTwo are wrong\n 0.3 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.3\n= 0.189
\n\nAll three are wrong\n 0.3 * 0.3 * 0.3\n= 0.027
\n\n```\nWe see that most of the times (~44%) the majority vote corrects an error. This majority vote ensemble will be correct an average of ~78% (0.3429 + 0.4409 = 0.7838).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "601530": "I understand that blending gives a better score. But how physical to do so? What is behind? \nOf course, in different cases the answer is different. But just in the current case study (P-M-P), what is the physicalness of blending, is it possible to control the behavior at some point, and is it appropriate for real science? Never experienced that in my field.", "601585": "What is easy to predict every model will probably predict correctly. As different models predict in different ways, some may predict higher values while other can predict lower values. However as all models predict values around the true values, the mean is probably closer to true values as positive errors and negative errors negates each other.", "601632": "Imagine we'd have some money to invest. We can choose between Oil (Feature 1), Lithium (Feature 2) and bicycles (Feature 3). There are 3 scenarios: cars running on gas will be banned in every city (KFold 1), all cars will be banned (KFold 2), all cars will be ok (Kfold 3).\nIf we invest in F1 we will be lucky in Sc3 but we will lose everything in the other scenarios. We have high variance. We overfit to Sc3. \nIf we'd mixed our investments in F1, F2 and F3. We'd be better off in Sc 1 and Sc 2 but would sacrifice a bit in case of Sc3. But the variance would go down. I'm less gambling and the investment seems more stable.\nHmm, the scenarios seem to be to extreme anyway. So let's make new ones and add new features e.g. sports ware ...\nSo now we have two diverse models. I bet we will see some impact on the variance when we blend them.", "602594": "somnambwl @joatom \nThank you very much for good examples. Seems to be more clear now.", "605129": "Its well explained here: https://mlwave.com/kaggle-ensembling-guide/\n\n\nA machine learning example\n\nSuppose we have a test set of 10 samples. The ground truth is all positive (“1”):\n\n```\n1111111111\n```\n\nWe furthermore have 3 binary classifiers (A,B,C) with a 70% accuracy. You can view these classifiers for now as pseudo-random number generators which output a “1” 70% of the time and a “0” 30% of the time.\n\nWe will now show how these pseudo-classifiers are able to obtain 78% accuracy through a voting ensemble.\n\nA pinch of maths\n\nFor a majority vote with 3 members we can expect 4 outcomes:\n\n```\nAll three are correct\n 0.7 * 0.7 * 0.7\n= 0.3429\n\nTwo are correct\n 0.7 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.7\n= 0.4409\n\nTwo are wrong\n 0.3 * 0.3 * 0.7\n+ 0.3 * 0.7 * 0.3\n+ 0.7 * 0.3 * 0.3\n= 0.189\n\nAll three are wrong\n 0.3 * 0.3 * 0.3\n= 0.027\n\n```\nWe see that most of the times (~44%) the majority vote corrects an error. This majority vote ensemble will be correct an average of ~78% (0.3429 + 0.4409 = 0.7838)." }, "source": "meta" }