Hi, I was trying to use Boruta do feature elimination with lgb. There are already some code snippets using Boruta on Random Forest (https://www.kaggle.com/tilii7/boruta-feature-elimination). \nSo, I was trying to replace RF with lgb:
\n\nrfc = lgb.LGBMModel(**params)\nBut, sadly, I encounter a problem saying
\n\nTypeError: Unknown type of parameter:random_state, got:RandomState\nEven when I directly set random_state to a fixed value (int/float), it does not work, because in skearn/utils/validation.py, if we set random_state to int/float, it will also return np.random.RandomState
\n\ndef check_random_state(seed):\n \"\"\"Turn seed into a np.random.RandomState instance\n\n Parameters\n ----------\n seed : None | int | instance of RandomState\n If seed is None, return the RandomState singleton used by np.random.\n If seed is an int, return a new RandomState instance seeded with seed.\n If seed is already a RandomState instance, return it.\n Otherwise raise ValueError.\n \"\"\"\n if seed is None or seed is np.random:\n return np.random.mtrand._rand\n if isinstance(seed, (numbers.Integral, np.integer)):\n return np.random.RandomState(seed)\n if isinstance(seed, np.random.RandomState):\n return seed\n raise ValueError('%r cannot be used to seed a numpy.random.RandomState'\n ' instance' % seed)\nI was unable to find this issue on Google, either.(Maybe I was too careless to miss ...).\nAnyway, I guess many people might use Boruta in this competition, it would be nice if you can point out some reference for me. Thanks.
", "messages": [ { "id": 461385, "postDate": "2019-01-25T22:16:45.920Z", "content": "Try the following code.
\n\nfrom lightgbm import LGBMRegressor\n\nlgbmclf = LGBMRegressor(boosting_type='rf', objective='regression', num_iteration=10000, num_leaves=31,\n min_data_in_leaf=27, max_depth=-1, learning_rate=0.015, feature_fraction= 0.9,\n bagging_freq= 1, bagging_fraction= 0.9,bagging_seed= 11, metric= 'rmse', lambda_l1=0.1, verbosity= -1, \n nthread= 4, random_state= 4950)\n\nborutaselector = BorutaPy(lgbmclf, nestimators='auto', verbose=-1) \nborutaselector.fit(train, target)\nRecursive Feature Selection is ready
\n\n", "rawMarkdown": "Recursive Feature Selection is ready\n\nhttps://www.kaggle.com/roydatascience/recursive-feature-selection-using-sklearn-on-elo?scriptVersionId=9917777", "votes": 5 }, { "id": 460471, "postDate": "2019-01-23T19:16:35.127Z", "content": "param = {'num_leaves': 60,\n 'min_data_in_leaf': 60, \n 'objective':'binary',\n 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"rf\",\n \"feature_fraction\": 0.8,\n \"bagging_freq\": 1,\n \"bagging_fraction\": 0.8 ,\n \"bagging_seed\": 11,\n \"metric\": 'auc',\n \"lambda_l1\": 0.1,\n \"random_state\": 133,\n \"verbosity\": -1 }
\n\nd_train = lgb.Dataset(train,label=target)\nnum_round = 5000\nlgbm_clf = lgb.train(param, d_train, num_round,verbose_eval=-1)\nboruta_selector = BorutaPy(lgbm_clf, n_estimators='auto', verbose=-1)\nboruta_selector.fit(train, target)
", "rawMarkdown": "param = {'num_leaves': 60,\n 'min_data_in_leaf': 60, \n 'objective':'binary',\n 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"rf\",\n \"feature_fraction\": 0.8,\n \"bagging_freq\": 1,\n \"bagging_fraction\": 0.8 ,\n \"bagging_seed\": 11,\n \"metric\": 'auc',\n \"lambda_l1\": 0.1,\n \"random_state\": 133,\n \"verbosity\": -1 }\n\nd_train = lgb.Dataset(train,label=target)\nnum_round = 5000\nlgbm_clf = lgb.train(param, d_train, num_round,verbose_eval=-1)\nboruta_selector = BorutaPy(lgbm_clf, n_estimators='auto', verbose=-1)\nboruta_selector.fit(train, target)", "votes": 5, "replies": [ { "id": 460562, "postDate": "2019-01-24T00:51:00.230Z", "content": "Thanks.
", "rawMarkdown": "Thanks." }, { "id": 460640, "postDate": "2019-01-24T05:51:07.223Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 460300, "postDate": "2019-01-23T11:40:34.413Z", "content": "Hi, I was trying to use Boruta do feature elimination with lgb. There are already some code snippets using Boruta on Random Forest (https://www.kaggle.com/tilii7/boruta-feature-elimination). \nSo, I was trying to replace RF with lgb:
\n\nrfc = lgb.LGBMModel(**params)\nBut, sadly, I encounter a problem saying
\n\nTypeError: Unknown type of parameter:random_state, got:RandomState\nEven when I directly set random_state to a fixed value (int/float), it does not work, because in skearn/utils/validation.py, if we set random_state to int/float, it will also return np.random.RandomState
\n\ndef check_random_state(seed):\n \"\"\"Turn seed into a np.random.RandomState instance\n\n Parameters\n ----------\n seed : None | int | instance of RandomState\n If seed is None, return the RandomState singleton used by np.random.\n If seed is an int, return a new RandomState instance seeded with seed.\n If seed is already a RandomState instance, return it.\n Otherwise raise ValueError.\n \"\"\"\n if seed is None or seed is np.random:\n return np.random.mtrand._rand\n if isinstance(seed, (numbers.Integral, np.integer)):\n return np.random.RandomState(seed)\n if isinstance(seed, np.random.RandomState):\n return seed\n raise ValueError('%r cannot be used to seed a numpy.random.RandomState'\n ' instance' % seed)\nI was unable to find this issue on Google, either.(Maybe I was too careless to miss ...).\nAnyway, I guess many people might use Boruta in this competition, it would be nice if you can point out some reference for me. Thanks.
", "rawMarkdown": "Hi, I was trying to use Boruta do feature elimination with lgb. There are already some code snippets using Boruta on Random Forest (https://www.kaggle.com/tilii7/boruta-feature-elimination). \nSo, I was trying to replace RF with lgb:\n\n rfc = lgb.LGBMModel(**params)\n\nBut, sadly, I encounter a problem saying\n\n TypeError: Unknown type of parameter:random_state, got:RandomState\n\nEven when I directly set random_state to a fixed value (int/float), it does not work, because in skearn/utils/validation.py, if we set random_state to int/float, it will also return np.random.RandomState\n\n def check_random_state(seed):\n \"\"\"Turn seed into a np.random.RandomState instance\n \n Parameters\n ----------\n seed : None | int | instance of RandomState\n If seed is None, return the RandomState singleton used by np.random.\n If seed is an int, return a new RandomState instance seeded with seed.\n If seed is already a RandomState instance, return it.\n Otherwise raise ValueError.\n \"\"\"\n if seed is None or seed is np.random:\n return np.random.mtrand._rand\n if isinstance(seed, (numbers.Integral, np.integer)):\n return np.random.RandomState(seed)\n if isinstance(seed, np.random.RandomState):\n return seed\n raise ValueError('%r cannot be used to seed a numpy.random.RandomState'\n ' instance' % seed)\n\nI was unable to find this issue on Google, either.(Maybe I was too careless to miss ...).\nAnyway, I guess many people might use Boruta in this competition, it would be nice if you can point out some reference for me. Thanks.", "votes": 3 }, { "id": 460662, "postDate": "2019-01-24T07:27:59.897Z", "content": "https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/76977#457880
", "rawMarkdown": "https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/76977#457880 ", "votes": 1, "replies": [ { "id": 460717, "postDate": "2019-01-24T09:50:07.277Z", "content": "thx, i will give it a try.
", "rawMarkdown": "thx, i will give it a try." }, { "id": 461245, "postDate": "2019-01-25T16:03:27.840Z", "content": "Dear both, for Boruta with lgb, should we have to pre-process NaN values? Any tricks to avoid the process?\nFor RF, it is a must to process NaN values. For lgb, if we must process NaN, it is not that perfect.
", "rawMarkdown": "Dear both, for Boruta with lgb, should we have to pre-process NaN values? Any tricks to avoid the process?\nFor RF, it is a must to process NaN values. For lgb, if we must process NaN, it is not that perfect." }, { "id": 461246, "postDate": "2019-01-25T16:05:36.243Z", "content": "impute the values before running Boruta. Do you have any suggestions on improving the LB Score?
", "rawMarkdown": "impute the values before running Boruta. Do you have any suggestions on improving the LB Score?", "votes": 3 }, { "id": 462513, "postDate": "2019-01-28T12:12:48.400Z", "content": "@xiaojiu1414,
\nYou can try comment out \"X, y = check_X_y (X, y)\" on line 516 of boruta_py.py.
\n\"sklearn.utils.check_X_y\" checks nan and inf.
@hatunina Thanks! I will try that.
", "rawMarkdown": "@hatunina Thanks! I will try that." } ] }, { "id": 463574, "postDate": "2019-01-30T08:14:20.917Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 461385, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-01-25T22:16:45.920000", "content": "Try the following code.
\n\nfrom lightgbm import LGBMRegressor\n\nlgbmclf = LGBMRegressor(boosting_type='rf', objective='regression', num_iteration=10000, num_leaves=31,\n min_data_in_leaf=27, max_depth=-1, learning_rate=0.015, feature_fraction= 0.9,\n bagging_freq= 1, bagging_fraction= 0.9,bagging_seed= 11, metric= 'rmse', lambda_l1=0.1, verbosity= -1, \n nthread= 4, random_state= 4950)\n\nborutaselector = BorutaPy(lgbmclf, nestimators='auto', verbose=-1) \nborutaselector.fit(train, target)\nRecursive Feature Selection is ready
\n\n", "votes": 5, "replies": [] }, { "id": 460471, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-01-23T19:16:35.127000", "content": "param = {'num_leaves': 60,\n 'min_data_in_leaf': 60, \n 'objective':'binary',\n 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"rf\",\n \"feature_fraction\": 0.8,\n \"bagging_freq\": 1,\n \"bagging_fraction\": 0.8 ,\n \"bagging_seed\": 11,\n \"metric\": 'auc',\n \"lambda_l1\": 0.1,\n \"random_state\": 133,\n \"verbosity\": -1 }
\n\nd_train = lgb.Dataset(train,label=target)\nnum_round = 5000\nlgbm_clf = lgb.train(param, d_train, num_round,verbose_eval=-1)\nboruta_selector = BorutaPy(lgbm_clf, n_estimators='auto', verbose=-1)\nboruta_selector.fit(train, target)
", "votes": 5, "replies": [ { "id": 460562, "author_name": "good good study", "author_url": "", "post_date": "2019-01-24T00:51:00.230000", "content": "Thanks.
", "votes": 0, "replies": [] }, { "id": 460640, "author_name": "", "author_url": "", "post_date": "2019-01-24T05:51:07.223000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 460662, "author_name": "silverstone", "author_url": "", "post_date": "2019-01-24T07:27:59.897000", "content": "https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/76977#457880
", "votes": 1, "replies": [ { "id": 460717, "author_name": "good good study", "author_url": "", "post_date": "2019-01-24T09:50:07.277000", "content": "thx, i will give it a try.
", "votes": 0, "replies": [] }, { "id": 461245, "author_name": "DeepInvolution", "author_url": "", "post_date": "2019-01-25T16:03:27.840000", "content": "Dear both, for Boruta with lgb, should we have to pre-process NaN values? Any tricks to avoid the process?\nFor RF, it is a must to process NaN values. For lgb, if we must process NaN, it is not that perfect.
", "votes": 0, "replies": [] }, { "id": 461246, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2019-01-25T16:05:36.243000", "content": "impute the values before running Boruta. Do you have any suggestions on improving the LB Score?
", "votes": 3, "replies": [] }, { "id": 462513, "author_name": "hatunina", "author_url": "", "post_date": "2019-01-28T12:12:48.400000", "content": "@xiaojiu1414,
\nYou can try comment out \"X, y = check_X_y (X, y)\" on line 516 of boruta_py.py.
\n\"sklearn.utils.check_X_y\" checks nan and inf.
@hatunina Thanks! I will try that.
", "votes": 0, "replies": [] } ] }, { "id": 463574, "author_name": "", "author_url": "", "post_date": "2019-01-30T08:14:20.917000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "461385": "Try the following code.\n\n from lightgbm import LGBMRegressor\n \n lgbmclf = LGBMRegressor(boosting_type='rf', objective='regression', num_iteration=10000, num_leaves=31,\n min_data_in_leaf=27, max_depth=-1, learning_rate=0.015, feature_fraction= 0.9,\n bagging_freq= 1, bagging_fraction= 0.9,bagging_seed= 11, metric= 'rmse', lambda_l1=0.1, verbosity= -1, \n nthread= 4, random_state= 4950)\n \n borutaselector = BorutaPy(lgbmclf, nestimators='auto', verbose=-1) \n borutaselector.fit(train, target)", "461322": "Recursive Feature Selection is ready\n\nhttps://www.kaggle.com/roydatascience/recursive-feature-selection-using-sklearn-on-elo?scriptVersionId=9917777", "460471": "param = {'num_leaves': 60,\n 'min_data_in_leaf': 60, \n 'objective':'binary',\n 'max_depth': -1,\n 'learning_rate': 0.1,\n \"boosting\": \"rf\",\n \"feature_fraction\": 0.8,\n \"bagging_freq\": 1,\n \"bagging_fraction\": 0.8 ,\n \"bagging_seed\": 11,\n \"metric\": 'auc',\n \"lambda_l1\": 0.1,\n \"random_state\": 133,\n \"verbosity\": -1 }\n\nd_train = lgb.Dataset(train,label=target)\nnum_round = 5000\nlgbm_clf = lgb.train(param, d_train, num_round,verbose_eval=-1)\nboruta_selector = BorutaPy(lgbm_clf, n_estimators='auto', verbose=-1)\nboruta_selector.fit(train, target)", "460300": "Hi, I was trying to use Boruta do feature elimination with lgb. There are already some code snippets using Boruta on Random Forest (https://www.kaggle.com/tilii7/boruta-feature-elimination). \nSo, I was trying to replace RF with lgb:\n\n rfc = lgb.LGBMModel(**params)\n\nBut, sadly, I encounter a problem saying\n\n TypeError: Unknown type of parameter:random_state, got:RandomState\n\nEven when I directly set random_state to a fixed value (int/float), it does not work, because in skearn/utils/validation.py, if we set random_state to int/float, it will also return np.random.RandomState\n\n def check_random_state(seed):\n \"\"\"Turn seed into a np.random.RandomState instance\n \n Parameters\n ----------\n seed : None | int | instance of RandomState\n If seed is None, return the RandomState singleton used by np.random.\n If seed is an int, return a new RandomState instance seeded with seed.\n If seed is already a RandomState instance, return it.\n Otherwise raise ValueError.\n \"\"\"\n if seed is None or seed is np.random:\n return np.random.mtrand._rand\n if isinstance(seed, (numbers.Integral, np.integer)):\n return np.random.RandomState(seed)\n if isinstance(seed, np.random.RandomState):\n return seed\n raise ValueError('%r cannot be used to seed a numpy.random.RandomState'\n ' instance' % seed)\n\nI was unable to find this issue on Google, either.(Maybe I was too careless to miss ...).\nAnyway, I guess many people might use Boruta in this competition, it would be nice if you can point out some reference for me. Thanks.", "460662": "https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/76977#457880 ", "463574": "" } }