Hey Everybody!
\n\nI am exploring more and more data imputation methods and wanted to know what methods have you all tried so far and what impact those methods had on the model?
\n\nP.S: I got 1.38 LB score after dropping all the columns with NaN values.
", "messages": [ { "id": "674596", "postDate": "11/16/2019 18:48:25", "content": "Hey Everybody!
\n\nI am exploring more and more data imputation methods and wanted to know what methods have you all tried so far and what impact those methods had on the model?
\n\nP.S: I got 1.38 LB score after dropping all the columns with NaN values.
", "rawMarkdown": "Hey Everybody! \n\nI am exploring more and more data imputation methods and wanted to know what methods have you all tried so far and what impact those methods had on the model? \n\nP.S: I got **1.38** LB score after dropping all the columns with NaN values.", "votes": null }, { "id": "693885", "postDate": "12/12/2019 22:11:07", "content": "How did you manage that? I tried using an imputer but it keeps giving me memory errors
", "rawMarkdown": "How did you manage that? I tried using an imputer but it keeps giving me memory errors", "votes": null }, { "id": "694012", "postDate": "12/13/2019 03:47:36", "content": "For avoiding memory errors you can do the data pre-processing and feature engineering part before merging the train and weather files. This will help in low memory consumption. \nI simply dropped all the features with any NaN values for starter kernel and if you wanna try imputation then you can go with Interpolation as it has many methods which can be useful.
\n\nFor reference:\nLink to my Starter code kernel : https://www.kaggle.com/ishaan45/starter-code\nLink to interpolation kernel : https://www.kaggle.com/ishaan45/pandas-profiling-missing-value-imputation
", "rawMarkdown": "For avoiding memory errors you can do the data pre-processing and feature engineering part before merging the train and weather files. This will help in low memory consumption. \nI simply dropped all the features with any NaN values for starter kernel and if you wanna try imputation then you can go with Interpolation as it has many methods which can be useful. \n\nFor reference:\nLink to my Starter code kernel : https://www.kaggle.com/ishaan45/starter-code\nLink to interpolation kernel : https://www.kaggle.com/ishaan45/pandas-profiling-missing-value-imputation", "votes": null }, { "id": "694080", "postDate": "12/13/2019 06:21:30", "content": "there are so many way to handle the missing values one good way is find one Attribute in data set with is linearly related to the Missing values Attribute fit a llinear reg to that and find the missing values ...! :)\ni used to do this :)
", "rawMarkdown": "there are so many way to handle the missing values one good way is find one Attribute in data set with is linearly related to the Missing values Attribute fit a llinear reg to that and find the missing values ...! :)\ni used to do this :)", "votes": null }, { "id": "694101", "postDate": "12/13/2019 07:14:39", "content": "I wanted to know if anybody has used some sophisticated methods for imputing missing values which may be interesting to try. Now I have tried interpolation and estimator based imputation. Thanks for your comment.
", "rawMarkdown": "I wanted to know if anybody has used some sophisticated methods for imputing missing values which may be interesting to try. Now I have tried interpolation and estimator based imputation. Thanks for your comment.", "votes": null } ], "comments": [ { "id": 693885, "author_name": "juanvazquez1992", "author_url": "", "post_date": "12/12/2019 22:11:07", "content": "How did you manage that? I tried using an imputer but it keeps giving me memory errors
", "votes": null, "replies": [ { "id": 694012, "author_name": "ishaan45", "author_url": "", "post_date": "12/13/2019 03:47:36", "content": "For avoiding memory errors you can do the data pre-processing and feature engineering part before merging the train and weather files. This will help in low memory consumption. \nI simply dropped all the features with any NaN values for starter kernel and if you wanna try imputation then you can go with Interpolation as it has many methods which can be useful.
\n\nFor reference:\nLink to my Starter code kernel : https://www.kaggle.com/ishaan45/starter-code\nLink to interpolation kernel : https://www.kaggle.com/ishaan45/pandas-profiling-missing-value-imputation
", "votes": null, "replies": [] } ] }, { "id": 694080, "author_name": "deepakdeepu8978", "author_url": "", "post_date": "12/13/2019 06:21:30", "content": "there are so many way to handle the missing values one good way is find one Attribute in data set with is linearly related to the Missing values Attribute fit a llinear reg to that and find the missing values ...! :)\ni used to do this :)
", "votes": null, "replies": [ { "id": 694101, "author_name": "ishaan45", "author_url": "", "post_date": "12/13/2019 07:14:39", "content": "I wanted to know if anybody has used some sophisticated methods for imputing missing values which may be interesting to try. Now I have tried interpolation and estimator based imputation. Thanks for your comment.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "674596": "Hey Everybody! \n\nI am exploring more and more data imputation methods and wanted to know what methods have you all tried so far and what impact those methods had on the model? \n\nP.S: I got **1.38** LB score after dropping all the columns with NaN values.", "693885": "How did you manage that? I tried using an imputer but it keeps giving me memory errors", "694012": "For avoiding memory errors you can do the data pre-processing and feature engineering part before merging the train and weather files. This will help in low memory consumption. \nI simply dropped all the features with any NaN values for starter kernel and if you wanna try imputation then you can go with Interpolation as it has many methods which can be useful. \n\nFor reference:\nLink to my Starter code kernel : https://www.kaggle.com/ishaan45/starter-code\nLink to interpolation kernel : https://www.kaggle.com/ishaan45/pandas-profiling-missing-value-imputation", "694080": "there are so many way to handle the missing values one good way is find one Attribute in data set with is linearly related to the Missing values Attribute fit a llinear reg to that and find the missing values ...! :)\ni used to do this :)", "694101": "I wanted to know if anybody has used some sophisticated methods for imputing missing values which may be interesting to try. Now I have tried interpolation and estimator based imputation. Thanks for your comment." }, "source": "meta" }