I use Boruta.py and sklearn RandomForest to select feature, but it works really slow, my train_X got 288 features, any idea about how to accelerate it? thanks!
", "messages": [ { "id": 452408, "postDate": "2019-01-08T17:25:59.850Z", "content": "Also sklearn RF is slow. Maybe you can try lightgbm's rf mode.
", "rawMarkdown": "Also sklearn RF is slow. Maybe you can try lightgbm's rf mode. ", "votes": 1, "replies": [ { "id": 456302, "postDate": "2019-01-15T14:17:39.867Z", "content": "I run an error named \"must be str, not TypeError\" with these code, can you show me the correct way？\nrfc = lgb.LGBMRegressor(boosting_type=\"rf\", n_estimators=500, n_jobs=10, max_depth=6)
\nboruta_selector = BorutaPy(rfc, n_estimators='auto', verbose=2)
\nboruta_selector.fit(train[feats].fillna(-999).values, train['target'].values)
To be honest I've not tried it yet by myself but I saw a similar error on github. Also I checked source code and didn't see your error message on ValueErrors.
\n\nps: I found a something like this. Not sure about that but maybe you need to change it to \"feature_importance\" for lightgbm on your local. Tomorrow I'll try to do it and let you know if it works.
", "rawMarkdown": "To be honest I've not tried it yet by myself but I saw a similar error on [github](https://github.com/scikit-learn-contrib/boruta_py/issues/40). Also I checked source code and didn't see your error message on ValueErrors.\n\nps: I found a something like [this](https://github.com/scikit-learn-contrib/boruta_py/blob/master/boruta/boruta_py.py#L391). Not sure about that but maybe you need to change it to \"feature_importance\" for lightgbm on your local. Tomorrow I'll try to do it and let you know if it works. " }, { "id": 456373, "postDate": "2019-01-15T17:26:09.987Z", "content": "Btw this can be a solution for you: https://www.kaggle.com/ogrellier/noise-analysis-of-porto-seguro-s-features#235152
", "rawMarkdown": "Btw this can be a solution for you: https://www.kaggle.com/ogrellier/noise-analysis-of-porto-seguro-s-features#235152" }, { "id": 457880, "postDate": "2019-01-18T08:57:58.873Z", "content": "Just change the line 282 with this one: self.estimator.set_params(random_state=self.random_state.get_state()[2])
credit goes to @ogrellier
", "rawMarkdown": "Just change the line 282 with this one: ```self.estimator.set_params(random_state=self.random_state.get_state()[2])```\n\ncredit goes to @ogrellier ", "votes": 2 }, { "id": 458088, "postDate": "2019-01-18T18:54:05.163Z", "content": "I would not recommend using lightgbm for Boruta. The feature importance measures are inconsistent and yield odd results. It runs fast, though.
", "rawMarkdown": "I would not recommend using lightgbm for Boruta. The feature importance measures are inconsistent and yield odd results. It runs fast, though." }, { "id": 459618, "postDate": "2019-01-22T05:00:32.803Z", "rawMarkdown": "", "isDeleted": true }, { "id": 459723, "postDate": "2019-01-22T08:48:23.867Z", "content": "@lennart\nHi Lennart, I am declear for this, may u explain for me? why lightgbm is bad....
", "rawMarkdown": "@lennart\nHi Lennart, I am declear for this, may u explain for me? why lightgbm is bad.... ", "votes": -1 }, { "id": 459797, "postDate": "2019-01-22T11:39:34.680Z", "content": "@kangzhang yes it works for me. Btw I haven't installed it, changed the line and used whole code in the notebook.
", "rawMarkdown": "@kangzhang yes it works for me. Btw I haven't installed it, changed the line and used whole [code](https://github.com/scikit-learn-contrib/boruta_py/blob/master/boruta/boruta_py.py) in the notebook." }, { "id": 461023, "postDate": "2019-01-25T02:47:46.297Z", "rawMarkdown": "", "isDeleted": true }, { "id": 461087, "postDate": "2019-01-25T07:34:24.463Z", "content": "May i ask you a question? Is your boostingtype of lgb 'rf' and X=train.values, y=train.target.values?
", "rawMarkdown": "May i ask you a question? Is your boostingtype of lgb 'rf' and X=train.values, y=train.target.values?" }, { "id": 461090, "postDate": "2019-01-25T07:41:58.813Z", "content": "@fangwenjun yes
", "rawMarkdown": "@fangwenjun yes" } ] }, { "id": 452298, "postDate": "2019-01-08T14:17:02.623Z", "content": "I have been trying to run Boruta with RFC too, but looks like it can't handle nan values. Is there a way to run it despite the nan values, or do I need to replace nan values with some other values ?
", "rawMarkdown": "I have been trying to run Boruta with RFC too, but looks like it can't handle nan values. Is there a way to run it despite the nan values, or do I need to replace nan values with some other values ? ", "votes": 1, "replies": [ { "id": 452305, "postDate": "2019-01-08T14:29:11.893Z", "content": "Luuhn: I ran Boruta and waited for 17hours plus. Just leave it running overnight and be patient.
\n\nSarang: I have replaced NaNs by standard imputations.
", "rawMarkdown": "Luuhn: I ran Boruta and waited for 17hours plus. Just leave it running overnight and be patient.\n\nSarang: I have replaced NaNs by standard imputations.", "votes": 1 }, { "id": 452329, "postDate": "2019-01-08T15:25:12.703Z", "content": "Thanks for your reply! I'll try it!
", "rawMarkdown": "Thanks for your reply! I'll try it!" }, { "id": 452651, "postDate": "2019-01-09T01:48:32.920Z", "rawMarkdown": "", "isDeleted": true }, { "id": 459726, "postDate": "2019-01-22T08:50:25.670Z", "content": "lennart, may we have a talk ?
", "rawMarkdown": "lennart, may we have a talk ? " }, { "id": 459802, "postDate": "2019-01-22T11:46:31.687Z", "content": "@Neo: Sure. My problem is that using lightgbm the algorithm is too aggressive. It is reducing some 1298 features to only 27. Note that I already cleansed for correlated features with correlation > 0.8. Using RF from sklearn the set is reduced to some more reasonable 108 features. CVs are also significantly different resulting in 3.669 and 3.649 respectively.
", "rawMarkdown": "@Neo: Sure. My problem is that using lightgbm the algorithm is too aggressive. It is reducing some 1298 features to only 27. Note that I already cleansed for correlated features with correlation > 0.8. Using RF from sklearn the set is reduced to some more reasonable 108 features. CVs are also significantly different resulting in 3.669 and 3.649 respectively." }, { "id": 460097, "postDate": "2019-01-23T01:51:11.473Z", "content": "@lennart: I have run boruta for my features(only 160+) but the boruta is broken while it was in the 30th round, and throw a error \"tuple index out of range\", then the result-rank is all in 1 ,such as :\n Feature ranking:\n[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1]
", "rawMarkdown": "@lennart: I have run boruta for my features(only 160+) but the boruta is broken while it was in the 30th round, and throw a error \"tuple index out of range\", then the result-rank is all in 1 ,such as :\n Feature ranking:\n[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1]\n" }, { "id": 460156, "postDate": "2019-01-23T05:18:55.850Z", "content": "Check this out: https://github.com/scikit-learn-contrib/boruta_py/pull/48/files#diff-1733952f437dc5309ab5c7da4bdc0e92R336
", "rawMarkdown": "Check this out: https://github.com/scikit-learn-contrib/boruta_py/pull/48/files#diff-1733952f437dc5309ab5c7da4bdc0e92R336" }, { "id": 460172, "postDate": "2019-01-23T05:49:59.617Z", "content": "Thank u so much and I will check that now
", "rawMarkdown": "Thank u so much and I will check that now" }, { "id": 460216, "postDate": "2019-01-23T08:01:58.310Z", "content": "Hi @sliverstone I had saw it , can the PR's change work for u?
", "rawMarkdown": "Hi @sliverstone I had saw it , can the PR's change work for u?" }, { "id": 460828, "postDate": "2019-01-24T13:48:05.190Z", "content": "@laplaceplanet yes it worked for me, give it a try
", "rawMarkdown": "@laplaceplanet yes it worked for me, give it a try " }, { "id": 461018, "postDate": "2019-01-25T02:18:20.620Z", "content": "yup! it also can work for me but which features can i select? the value of rank is 1,2,3,4,5,6,.......
", "rawMarkdown": "yup! it also can work for me but which features can i select? the value of rank is 1,2,3,4,5,6,......." } ] }, { "id": 452266, "postDate": "2019-01-08T13:26:10.533Z", "content": "You can run it on half the data
", "rawMarkdown": "You can run it on half the data", "votes": 2, "replies": [ { "id": 452328, "postDate": "2019-01-08T15:24:36.253Z", "content": "I'll try it, thanks!
", "rawMarkdown": "I'll try it, thanks!" } ] }, { "id": 452247, "postDate": "2019-01-08T12:38:03.947Z", "content": "I use Boruta.py and sklearn RandomForest to select feature, but it works really slow, my train_X got 288 features, any idea about how to accelerate it? thanks!
", "rawMarkdown": "I use Boruta.py and sklearn RandomForest to select feature, but it works really slow, my train_X got 288 features, any idea about how to accelerate it? thanks!" } ], "comments": [ { "id": 452408, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-08T17:25:59.850000", "content": "Also sklearn RF is slow. Maybe you can try lightgbm's rf mode.
", "votes": 1, "replies": [ { "id": 456302, "author_name": "Ethan", "author_url": "", "post_date": "2019-01-15T14:17:39.867000", "content": "I run an error named \"must be str, not TypeError\" with these code, can you show me the correct way？\nrfc = lgb.LGBMRegressor(boosting_type=\"rf\", n_estimators=500, n_jobs=10, max_depth=6)
\nboruta_selector = BorutaPy(rfc, n_estimators='auto', verbose=2)
\nboruta_selector.fit(train[feats].fillna(-999).values, train['target'].values)
To be honest I've not tried it yet by myself but I saw a similar error on github. Also I checked source code and didn't see your error message on ValueErrors.
\n\nps: I found a something like this. Not sure about that but maybe you need to change it to \"feature_importance\" for lightgbm on your local. Tomorrow I'll try to do it and let you know if it works.
", "votes": 0, "replies": [] }, { "id": 456373, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-15T17:26:09.987000", "content": "Btw this can be a solution for you: https://www.kaggle.com/ogrellier/noise-analysis-of-porto-seguro-s-features#235152
", "votes": 0, "replies": [] }, { "id": 457880, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-18T08:57:58.873000", "content": "Just change the line 282 with this one: self.estimator.set_params(random_state=self.random_state.get_state()[2])
credit goes to @ogrellier
", "votes": 2, "replies": [] }, { "id": 458088, "author_name": "lennart", "author_url": "", "post_date": "2019-01-18T18:54:05.163000", "content": "I would not recommend using lightgbm for Boruta. The feature importance measures are inconsistent and yield odd results. It runs fast, though.
", "votes": 0, "replies": [] }, { "id": 459618, "author_name": "", "author_url": "", "post_date": "2019-01-22T05:00:32.803000", "content": "", "votes": 0, "replies": [] }, { "id": 459723, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-22T08:48:23.867000", "content": "@lennart\nHi Lennart, I am declear for this, may u explain for me? why lightgbm is bad....
", "votes": -1, "replies": [] }, { "id": 459797, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-22T11:39:34.680000", "content": "@kangzhang yes it works for me. Btw I haven't installed it, changed the line and used whole code in the notebook.
", "votes": 0, "replies": [] }, { "id": 461023, "author_name": "", "author_url": "", "post_date": "2019-01-25T02:47:46.297000", "content": "", "votes": 0, "replies": [] }, { "id": 461087, "author_name": "fwj7", "author_url": "", "post_date": "2019-01-25T07:34:24.463000", "content": "May i ask you a question? Is your boostingtype of lgb 'rf' and X=train.values, y=train.target.values?
", "votes": 0, "replies": [] }, { "id": 461090, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-25T07:41:58.813000", "content": "@fangwenjun yes
", "votes": 0, "replies": [] } ] }, { "id": 452298, "author_name": "Sarang Vasishth", "author_url": "", "post_date": "2019-01-08T14:17:02.623000", "content": "I have been trying to run Boruta with RFC too, but looks like it can't handle nan values. Is there a way to run it despite the nan values, or do I need to replace nan values with some other values ?
", "votes": 1, "replies": [ { "id": 452305, "author_name": "lennart", "author_url": "", "post_date": "2019-01-08T14:29:11.893000", "content": "Luuhn: I ran Boruta and waited for 17hours plus. Just leave it running overnight and be patient.
\n\nSarang: I have replaced NaNs by standard imputations.
", "votes": 1, "replies": [] }, { "id": 452329, "author_name": "Luuunh", "author_url": "", "post_date": "2019-01-08T15:25:12.703000", "content": "Thanks for your reply! I'll try it!
", "votes": 0, "replies": [] }, { "id": 452651, "author_name": "", "author_url": "", "post_date": "2019-01-09T01:48:32.920000", "content": "", "votes": 0, "replies": [] }, { "id": 459726, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-22T08:50:25.670000", "content": "lennart, may we have a talk ?
", "votes": 0, "replies": [] }, { "id": 459802, "author_name": "lennart", "author_url": "", "post_date": "2019-01-22T11:46:31.687000", "content": "@Neo: Sure. My problem is that using lightgbm the algorithm is too aggressive. It is reducing some 1298 features to only 27. Note that I already cleansed for correlated features with correlation > 0.8. Using RF from sklearn the set is reduced to some more reasonable 108 features. CVs are also significantly different resulting in 3.669 and 3.649 respectively.
", "votes": 0, "replies": [] }, { "id": 460097, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-23T01:51:11.473000", "content": "@lennart: I have run boruta for my features(only 160+) but the boruta is broken while it was in the 30th round, and throw a error \"tuple index out of range\", then the result-rank is all in 1 ,such as :\n Feature ranking:\n[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1]
", "votes": 0, "replies": [] }, { "id": 460156, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-23T05:18:55.850000", "content": "Check this out: https://github.com/scikit-learn-contrib/boruta_py/pull/48/files#diff-1733952f437dc5309ab5c7da4bdc0e92R336
", "votes": 0, "replies": [] }, { "id": 460172, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-23T05:49:59.617000", "content": "Thank u so much and I will check that now
", "votes": 0, "replies": [] }, { "id": 460216, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-23T08:01:58.310000", "content": "Hi @sliverstone I had saw it , can the PR's change work for u?
", "votes": 0, "replies": [] }, { "id": 460828, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-24T13:48:05.190000", "content": "@laplaceplanet yes it worked for me, give it a try
", "votes": 0, "replies": [] }, { "id": 461018, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-25T02:18:20.620000", "content": "yup! it also can work for me but which features can i select? the value of rank is 1,2,3,4,5,6,.......
", "votes": 0, "replies": [] } ] }, { "id": 452266, "author_name": "AmirH", "author_url": "", "post_date": "2019-01-08T13:26:10.533000", "content": "You can run it on half the data
", "votes": 2, "replies": [ { "id": 452328, "author_name": "Luuunh", "author_url": "", "post_date": "2019-01-08T15:24:36.253000", "content": "I'll try it, thanks!
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "452408": "Also sklearn RF is slow. Maybe you can try lightgbm's rf mode. ", "452298": "I have been trying to run Boruta with RFC too, but looks like it can't handle nan values. Is there a way to run it despite the nan values, or do I need to replace nan values with some other values ? ", "452266": "You can run it on half the data", "452247": "I use Boruta.py and sklearn RandomForest to select feature, but it works really slow, my train_X got 288 features, any idea about how to accelerate it? thanks!" } }