I am a beginner.\nI have a question.\nThere are a lot of ways to add a feature to another feature , subtract a feature from another feature and so on as feature engineering.\nMay I get a feature(high feature importance) if I add a feature(low feature importance) to another feature(low feature importance)?
", "messages": [ { "id": "575073", "postDate": "07/15/2019 02:46:37", "content": "I am a beginner.\nI have a question.\nThere are a lot of ways to add a feature to another feature , subtract a feature from another feature and so on as feature engineering.\nMay I get a feature(high feature importance) if I add a feature(low feature importance) to another feature(low feature importance)?
", "rawMarkdown": "I am a beginner.\nI have a question.\nThere are a lot of ways to add a feature to another feature , subtract a feature from another feature and so on as feature engineering.\nMay I get a feature(high feature importance) if I add a feature(low feature importance) to another feature(low feature importance)?", "votes": null }, { "id": "575084", "postDate": "07/15/2019 03:19:28", "content": "It is not a hard and fast rule! @yudaiueno \nThis depends on what kind of data one is dealing with... Sometimes, this will work, and other times may not.
\n\nSo, my suggestion is to apply the Hit and Try method, if no intuition is possible.\nTaking Log can also be handy.
\n\nThanks.
", "rawMarkdown": "It is not a hard and fast rule! @yudaiueno \nThis depends on what kind of data one is dealing with... Sometimes, this will work, and other times may not.\n\nSo, my suggestion is to apply the Hit and Try method, if no intuition is possible.\nTaking Log can also be handy.\n\nThanks.", "votes": null }, { "id": "575100", "postDate": "07/15/2019 03:57:23", "content": "Hello @yudaiueno ,
\n\nA) Feature Engineering Materials which you might find helpful .\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature%20Engineering.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Rare%20Feature%20Engineering%20Techniques%20for%20Machine%20Learning%20Competitions.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature-Engineering-Checklist.pdf
\n\nB) Feature Importance Identification \nThere are many ways to identify 'Important Features' ….and this also varies from Algorithm to Algorithm.\nEx : \n Linear Regression Algorithms - These algorithms does return a list of Features and their relative importance ( with P-Value Significance )
\n\n Random Forest Algorithms - These algorithms also returns a list of Important features .\nu
", "rawMarkdown": "Hello @yudaiueno ,\n\nA) Feature Engineering Materials which you might find helpful .\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature%20Engineering.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Rare%20Feature%20Engineering%20Techniques%20for%20Machine%20Learning%20Competitions.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature-Engineering-Checklist.pdf\n\nB) Feature Importance Identification \nThere are many ways to identify 'Important Features' ….and this also varies from Algorithm to Algorithm.\nEx : \n Linear Regression Algorithms - These algorithms does return a list of Features and their relative importance ( with P-Value Significance )\n\n Random Forest Algorithms - These algorithms also returns a list of Important features .\n\n\nu", "votes": null }, { "id": "575112", "postDate": "07/15/2019 04:33:08", "content": "Not that far along, think of myself as a 4th grader. So I still remember 1st grade. One of the things I learned is that tools like xgboost and lightgbm can learn most of the simple features that I thought of - so if I had a and b as my two features those tools were more than smart enough to learn a-b, a*b, etc. I also liked early to take the top two features on my list and make a cross product a*b - again the tools are smart enough. I might still try a*b if I think that means I can remove a and b to save memory - but....
\n\nSo think a decent amount beyond simple math - these tools are wicked smart for that stuff. I don't yet trust them to find non-linear stuff. For example there is a kernel where distance to -3 power (1/r^3) was helpful. I think that starts to be much in the area when given a very large data set and lots of time lightgbm might find it - but don't think our data set big enough for it to find that level.
\n\nAgain - just a 4th graders opinion. But think non linear for features or completely new characteristics is my suggestion for a newbie.
", "rawMarkdown": "Not that far along, think of myself as a 4th grader. So I still remember 1st grade. One of the things I learned is that tools like xgboost and lightgbm can learn most of the simple features that I thought of - so if I had a and b as my two features those tools were more than smart enough to learn a-b, a*b, etc. I also liked early to take the top two features on my list and make a cross product a*b - again the tools are smart enough. I might still try a*b if I think that means I can remove a and b to save memory - but....\n\nSo think a decent amount beyond simple math - these tools are wicked smart for that stuff. I don't yet trust them to find non-linear stuff. For example there is a kernel where distance to -3 power (1/r^3) was helpful. I think that starts to be much in the area when given a very large data set and lots of time lightgbm might find it - but don't think our data set big enough for it to find that level.\n\nAgain - just a 4th graders opinion. But think non linear for features or completely new characteristics is my suggestion for a newbie.", "votes": null } ], "comments": [ { "id": 575084, "author_name": "parakhsrivastava", "author_url": "", "post_date": "07/15/2019 03:19:28", "content": "It is not a hard and fast rule! @yudaiueno \nThis depends on what kind of data one is dealing with... Sometimes, this will work, and other times may not.
\n\nSo, my suggestion is to apply the Hit and Try method, if no intuition is possible.\nTaking Log can also be handy.
\n\nThanks.
", "votes": null, "replies": [] }, { "id": 575100, "author_name": "prmohanty", "author_url": "", "post_date": "07/15/2019 03:57:23", "content": "Hello @yudaiueno ,
\n\nA) Feature Engineering Materials which you might find helpful .\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature%20Engineering.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Rare%20Feature%20Engineering%20Techniques%20for%20Machine%20Learning%20Competitions.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature-Engineering-Checklist.pdf
\n\nB) Feature Importance Identification \nThere are many ways to identify 'Important Features' ….and this also varies from Algorithm to Algorithm.\nEx : \n Linear Regression Algorithms - These algorithms does return a list of Features and their relative importance ( with P-Value Significance )
\n\n Random Forest Algorithms - These algorithms also returns a list of Important features .\nu
", "votes": null, "replies": [] }, { "id": 575112, "author_name": "pcjimmmy", "author_url": "", "post_date": "07/15/2019 04:33:08", "content": "Not that far along, think of myself as a 4th grader. So I still remember 1st grade. One of the things I learned is that tools like xgboost and lightgbm can learn most of the simple features that I thought of - so if I had a and b as my two features those tools were more than smart enough to learn a-b, a*b, etc. I also liked early to take the top two features on my list and make a cross product a*b - again the tools are smart enough. I might still try a*b if I think that means I can remove a and b to save memory - but....
\n\nSo think a decent amount beyond simple math - these tools are wicked smart for that stuff. I don't yet trust them to find non-linear stuff. For example there is a kernel where distance to -3 power (1/r^3) was helpful. I think that starts to be much in the area when given a very large data set and lots of time lightgbm might find it - but don't think our data set big enough for it to find that level.
\n\nAgain - just a 4th graders opinion. But think non linear for features or completely new characteristics is my suggestion for a newbie.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "575073": "I am a beginner.\nI have a question.\nThere are a lot of ways to add a feature to another feature , subtract a feature from another feature and so on as feature engineering.\nMay I get a feature(high feature importance) if I add a feature(low feature importance) to another feature(low feature importance)?", "575084": "It is not a hard and fast rule! @yudaiueno \nThis depends on what kind of data one is dealing with... Sometimes, this will work, and other times may not.\n\nSo, my suggestion is to apply the Hit and Try method, if no intuition is possible.\nTaking Log can also be handy.\n\nThanks.", "575100": "Hello @yudaiueno ,\n\nA) Feature Engineering Materials which you might find helpful .\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature%20Engineering.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Rare%20Feature%20Engineering%20Techniques%20for%20Machine%20Learning%20Competitions.pdf\nhttps://github.com/prmohanty/AI-ML-DL-Resources/blob/master/Feature-Engineering-Checklist.pdf\n\nB) Feature Importance Identification \nThere are many ways to identify 'Important Features' ….and this also varies from Algorithm to Algorithm.\nEx : \n Linear Regression Algorithms - These algorithms does return a list of Features and their relative importance ( with P-Value Significance )\n\n Random Forest Algorithms - These algorithms also returns a list of Important features .\n\n\nu", "575112": "Not that far along, think of myself as a 4th grader. So I still remember 1st grade. One of the things I learned is that tools like xgboost and lightgbm can learn most of the simple features that I thought of - so if I had a and b as my two features those tools were more than smart enough to learn a-b, a*b, etc. I also liked early to take the top two features on my list and make a cross product a*b - again the tools are smart enough. I might still try a*b if I think that means I can remove a and b to save memory - but....\n\nSo think a decent amount beyond simple math - these tools are wicked smart for that stuff. I don't yet trust them to find non-linear stuff. For example there is a kernel where distance to -3 power (1/r^3) was helpful. I think that starts to be much in the area when given a very large data set and lots of time lightgbm might find it - but don't think our data set big enough for it to find that level.\n\nAgain - just a 4th graders opinion. But think non linear for features or completely new characteristics is my suggestion for a newbie." }, "source": "meta" }