{
  "id": 124002,
  "title": "How much does EDA help with feature engineering?",
  "url": "/competitions/ashrae-energy-prediction/discussion/124002",
  "author_name": "lawrence",
  "post_date": "2020-01-01T09:27:51.691000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I'm a beginner, and after looking several notebooks sharing by others, feature engineering is the most time consuming and complex part to me. </p>\n\n<p>However, some notebooks apply EDA to show the data insight, such like data distribution, but when they do feature engineering like create new feature or remove feature, I don't see the relation to their work explaining why they do this way.(Maybe they just don't explain clear, any way it's confusing to me like: Why? Why should I add this feature? Why should I remove this feature? Why should I do this transformation?) </p>\n\n<p>As for this project, why do one drop 'wind_speed' feature?\nSome create 'is_holiday' feature as new feature, how could I validate that the new feature help identify the target? To apply groupby() and check if the mean value of target with or without holiday is identifying?\nHow to detect outliers?</p>\n\n<p>I'm now looking at a discussion here: <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872</a></p>\n\n<p>It's quite long to read, I'm still look for it, and I hope I could found out how could I do the FE with EDA's help.</p>",
  "messages": [
    {
      "id": 707673,
      "postDate": "2020-01-01T09:27:51.690Z",
      "content": "<p>I'm a beginner, and after looking several notebooks sharing by others, feature engineering is the most time consuming and complex part to me. </p>\n\n<p>However, some notebooks apply EDA to show the data insight, such like data distribution, but when they do feature engineering like create new feature or remove feature, I don't see the relation to their work explaining why they do this way.(Maybe they just don't explain clear, any way it's confusing to me like: Why? Why should I add this feature? Why should I remove this feature? Why should I do this transformation?) </p>\n\n<p>As for this project, why do one drop 'wind_speed' feature?\nSome create 'is_holiday' feature as new feature, how could I validate that the new feature help identify the target? To apply groupby() and check if the mean value of target with or without holiday is identifying?\nHow to detect outliers?</p>\n\n<p>I'm now looking at a discussion here: <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872</a></p>\n\n<p>It's quite long to read, I'm still look for it, and I hope I could found out how could I do the FE with EDA's help.</p>",
      "rawMarkdown": "I'm a beginner, and after looking several notebooks sharing by others, feature engineering is the most time consuming and complex part to me. \n\nHowever, some notebooks apply EDA to show the data insight, such like data distribution, but when they do feature engineering like create new feature or remove feature, I don't see the relation to their work explaining why they do this way.(Maybe they just don't explain clear, any way it's confusing to me like: Why? Why should I add this feature? Why should I remove this feature? Why should I do this transformation?) \n\nAs for this project, why do one drop 'wind_speed' feature?\nSome create 'is_holiday' feature as new feature, how could I validate that the new feature help identify the target? To apply groupby() and check if the mean value of target with or without holiday is identifying?\nHow to detect outliers?\n\nI'm now looking at a discussion here: https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872\n\nIt's quite long to read, I'm still look for it, and I hope I could found out how could I do the FE with EDA's help.\n"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "707673": "I'm a beginner, and after looking several notebooks sharing by others, feature engineering is the most time consuming and complex part to me. \n\nHowever, some notebooks apply EDA to show the data insight, such like data distribution, but when they do feature engineering like create new feature or remove feature, I don't see the relation to their work explaining why they do this way.(Maybe they just don't explain clear, any way it's confusing to me like: Why? Why should I add this feature? Why should I remove this feature? Why should I do this transformation?) \n\nAs for this project, why do one drop 'wind_speed' feature?\nSome create 'is_holiday' feature as new feature, how could I validate that the new feature help identify the target? To apply groupby() and check if the mean value of target with or without holiday is identifying?\nHow to detect outliers?\n\nI'm now looking at a discussion here: https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112872\n\nIt's quite long to read, I'm still look for it, and I hope I could found out how could I do the FE with EDA's help.\n"
  }
}