fc feature against the target for each type and we can see that they have a very high correlation:\n![\"\"](\"https://www.kaggleusercontent.com/kf/15387640/eyJhbGciOiJkaXIiLCJlbmMiOiJBMTI4Q0JDLUhTMjU2In0..93_rK1atqGrShOZcUKV7-A.laeSfwYDHB_F2jsq-_I0T45vdHeFb7nwPMPxy14NbtCawK5TJdHBln73uCR5YTIwjB5fdoPPxfTVJ_aj89EIYP0B6NAltIlYhHzknjPGacOZz8UD7sCn_iLwG_Sl2R2ZkSjMk3LgokA6XGvRK2qtlBSjj2XJQLbJqkM1Dz2nvls.MyuBvv8DDvWQ2wOjGBGXig/__results___files/__results___11_0.png\")
In this competition we have several additional files which contain information only for train data.
\n\nI'd like to offer an approach of using this data in my new kernel.
\n\nIn this kernel I use scalar_coupling_contributions.csv file which contains four variables, sum of which is supposed to be equal to the target variable.
I plotted fc feature against the target for each type and we can see that they have a very high correlation:\n
So I do the following:
\n\nfc feature;Scores on leaderboard:\n- single model training with my features: -0.365;\n- single model with my features and meta-feature: -0.508;\n- 8 models (for each type) with my features: -0.595;\n- 8 models (for each type) with my features and meta-feature: -0.720;
\n\nFurther ideas (not sure I'll implement all of them in kernels):\n- generating meta-features for all features in additional data files;\n- tuning 8 separate models separately;
", "messages": [ { "id": 548362, "postDate": "2019-06-09T09:09:20.657Z", "content": "In this competition we have several additional files which contain information only for train data.
\n\nI'd like to offer an approach of using this data in my new kernel.
\n\nIn this kernel I use scalar_coupling_contributions.csv file which contains four variables, sum of which is supposed to be equal to the target variable.
I plotted fc feature against the target for each type and we can see that they have a very high correlation:\n
So I do the following:
\n\nfc feature;Scores on leaderboard:\n- single model training with my features: -0.365;\n- single model with my features and meta-feature: -0.508;\n- 8 models (for each type) with my features: -0.595;\n- 8 models (for each type) with my features and meta-feature: -0.720;
\n\nFurther ideas (not sure I'll implement all of them in kernels):\n- generating meta-features for all features in additional data files;\n- tuning 8 separate models separately;
", "rawMarkdown": "In this competition we have several additional files which contain information only for train data.\n\nI'd like to offer an approach of using this data in my new [kernel](https://www.kaggle.com/artgor/using-meta-features-to-improve-model).\n\nIn this kernel I use `scalar_coupling_contributions.csv` file which contains four variables, sum of which is supposed to be equal to the target variable.\n\nI plotted `fc` feature against the target for each type and we can see that they have a very high correlation:\n\n\nSo I do the following:\n\n- generate features from my previous kernels;\n- train a model on them with 1500 estimators for speed;\n- train a model(on folds) to predict `fc` feature;\n- add out of fold predictions to train and predictions to test;\n- train the same model as on second line to compare performance (it is better);\n- train a better model;\n\nScores on leaderboard:\n- single model training with my features: -0.365;\n- single model with my features and meta-feature: -0.508;\n- 8 models (for each type) with my features: -0.595;\n- 8 models (for each type) with my features and meta-feature: -0.720;\n\nFurther ideas (not sure I'll implement all of them in kernels):\n- generating meta-features for all features in additional data files;\n- tuning 8 separate models separately;\n\n", "votes": 22 }, { "id": 548446, "postDate": "2019-06-09T11:50:04.563Z", "content": "Impressive! I couldn't find a way to use the extra features that are not in the test data. Thanks for the trick!
", "rawMarkdown": "Impressive! I couldn't find a way to use the extra features that are not in the test data. Thanks for the trick!", "votes": 1 }, { "id": 548382, "postDate": "2019-06-09T09:47:42.677Z", "content": "Interesting finding.
\n\nJust a random thought, instead of 8 models (for each type), would it be better to make 3 models because of smaller data prone to overfitting.\nThose could be many variations:\n-Based on the ID : (1JHC 1JHN) & (2JHH 2JHC 2JHN) & (3JHH 3JHC 3JHN) \n-Based on the atom pairs: (1JHC 2JHC 3 JHC) & (2JHH 3JHH) & (1JHN 2JHN 3 JHN) \n-Based on value range : (1JHC) & (1JHN 3JHC) & (2JHH 2JHC 2JHN 3JHH 3JHN)
", "rawMarkdown": "Interesting finding. \n\nJust a random thought, instead of 8 models (for each type), would it be better to make 3 models because of smaller data prone to overfitting.\nThose could be many variations:\n-Based on the ID : (1JHC 1JHN) & (2JHH 2JHC 2JHN) & (3JHH 3JHC 3JHN) \n-Based on the atom pairs: (1JHC 2JHC 3 JHC) & (2JHH 3JHH) & (1JHN 2JHN 3 JHN) \n-Based on value range : (1JHC) & (1JHN 3JHC) & (2JHH 2JHC 2JHN 3JHH 3JHN)", "votes": 2, "replies": [ { "id": 548744, "postDate": "2019-06-09T20:14:52.973Z", "content": "You have an interesting idea - creating 3 or 4 separate discrete models. That might reduce the overfit problem. However, the molecular properties of each type may be sufficiently different to not have much overfitting. We will have to try each approach and measure the overfit.
", "rawMarkdown": "You have an interesting idea - creating 3 or 4 separate discrete models. That might reduce the overfit problem. However, the molecular properties of each type may be sufficiently different to not have much overfitting. We will have to try each approach and measure the overfit.", "votes": 1 }, { "id": 549503, "postDate": "2019-06-10T18:12:40.370Z", "content": "There's plenty of data though, even the least populated type has over 40,000 rows. Eight separate models makes sense to me (but as with everything here, best approach is to try and see what most improves your cv)
", "rawMarkdown": "There's plenty of data though, even the least populated type has over 40,000 rows. Eight separate models makes sense to me (but as with everything here, best approach is to try and see what most improves your cv)", "votes": 1 } ] }, { "id": 548554, "postDate": "2019-06-09T15:09:03.827Z", "content": "Thanks for sharing these insights with us !!
", "rawMarkdown": "Thanks for sharing these insights with us !!" }, { "id": 570882, "postDate": "2019-07-08T22:13:40.063Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 548446, "author_name": "jmtest", "author_url": "", "post_date": "2019-06-09T11:50:04.563000", "content": "Impressive! I couldn't find a way to use the extra features that are not in the test data. Thanks for the trick!
", "votes": 1, "replies": [] }, { "id": 548382, "author_name": "Hervind Philipe", "author_url": "", "post_date": "2019-06-09T09:47:42.677000", "content": "Interesting finding.
\n\nJust a random thought, instead of 8 models (for each type), would it be better to make 3 models because of smaller data prone to overfitting.\nThose could be many variations:\n-Based on the ID : (1JHC 1JHN) & (2JHH 2JHC 2JHN) & (3JHH 3JHC 3JHN) \n-Based on the atom pairs: (1JHC 2JHC 3 JHC) & (2JHH 3JHH) & (1JHN 2JHN 3 JHN) \n-Based on value range : (1JHC) & (1JHN 3JHC) & (2JHH 2JHC 2JHN 3JHH 3JHN)
", "votes": 2, "replies": [ { "id": 548744, "author_name": "Bill Holst", "author_url": "", "post_date": "2019-06-09T20:14:52.973000", "content": "You have an interesting idea - creating 3 or 4 separate discrete models. That might reduce the overfit problem. However, the molecular properties of each type may be sufficiently different to not have much overfitting. We will have to try each approach and measure the overfit.
", "votes": 1, "replies": [] }, { "id": 549503, "author_name": "Marco Gorelli", "author_url": "", "post_date": "2019-06-10T18:12:40.370000", "content": "There's plenty of data though, even the least populated type has over 40,000 rows. Eight separate models makes sense to me (but as with everything here, best approach is to try and see what most improves your cv)
", "votes": 1, "replies": [] } ] }, { "id": 548554, "author_name": "Shweta Goyal", "author_url": "", "post_date": "2019-06-09T15:09:03.827000", "content": "Thanks for sharing these insights with us !!
", "votes": 0, "replies": [] }, { "id": 570882, "author_name": "", "author_url": "", "post_date": "2019-07-08T22:13:40.063000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "548362": "In this competition we have several additional files which contain information only for train data.\n\nI'd like to offer an approach of using this data in my new [kernel](https://www.kaggle.com/artgor/using-meta-features-to-improve-model).\n\nIn this kernel I use `scalar_coupling_contributions.csv` file which contains four variables, sum of which is supposed to be equal to the target variable.\n\nI plotted `fc` feature against the target for each type and we can see that they have a very high correlation:\n\n\nSo I do the following:\n\n- generate features from my previous kernels;\n- train a model on them with 1500 estimators for speed;\n- train a model(on folds) to predict `fc` feature;\n- add out of fold predictions to train and predictions to test;\n- train the same model as on second line to compare performance (it is better);\n- train a better model;\n\nScores on leaderboard:\n- single model training with my features: -0.365;\n- single model with my features and meta-feature: -0.508;\n- 8 models (for each type) with my features: -0.595;\n- 8 models (for each type) with my features and meta-feature: -0.720;\n\nFurther ideas (not sure I'll implement all of them in kernels):\n- generating meta-features for all features in additional data files;\n- tuning 8 separate models separately;\n\n", "548446": "Impressive! I couldn't find a way to use the extra features that are not in the test data. Thanks for the trick!", "548382": "Interesting finding. \n\nJust a random thought, instead of 8 models (for each type), would it be better to make 3 models because of smaller data prone to overfitting.\nThose could be many variations:\n-Based on the ID : (1JHC 1JHN) & (2JHH 2JHC 2JHN) & (3JHH 3JHC 3JHN) \n-Based on the atom pairs: (1JHC 2JHC 3 JHC) & (2JHH 3JHH) & (1JHN 2JHN 3 JHN) \n-Based on value range : (1JHC) & (1JHN 3JHC) & (2JHH 2JHC 2JHN 3JHH 3JHN)", "548554": "Thanks for sharing these insights with us !!", "570882": "" } }