{
  "id": 67818,
  "title": "Insight from Exploratory Data Analysis?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/67818",
  "author_name": "",
  "post_date": "2018-10-05T18:43:06.031749Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Can someone please comment on what insight can be derived from the exploratory data analysis? There are a few kernels with good scores that have little exploratory data analysis. It would be greatly appreciated if someone can use this competition as an example to illustrate how exploratory data analysis can yield better scores.</p>",
  "messages": [
    {
      "id": "399420",
      "postDate": "10/05/2018 18:43:06",
      "content": "<p>Can someone please comment on what insight can be derived from the exploratory data analysis? There are a few kernels with good scores that have little exploratory data analysis. It would be greatly appreciated if someone can use this competition as an example to illustrate how exploratory data analysis can yield better scores.</p>",
      "rawMarkdown": "Can someone please comment on what insight can be derived from the exploratory data analysis? There are a few kernels with good scores that have little exploratory data analysis. It would be greatly appreciated if someone can use this competition as an example to illustrate how exploratory data analysis can yield better scores.",
      "votes": null
    },
    {
      "id": "399428",
      "postDate": "10/05/2018 18:57:04",
      "content": "<p>It's good to see that you are focusing on exploratory data analysis for better score. You can focus on univariate and bivariate analysis and as this is a regression problem, you can check the correlation between the features and select the most useful subset of features and use them with any regression algorithm you find suitable. </p>\n\n<p>Hope this helps also If you want to check in depth  EDA you can refer to : <a href=\"https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis\">https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis</a>.</p>",
      "rawMarkdown": "It's good to see that you are focusing on exploratory data analysis for better score. You can focus on univariate and bivariate analysis and as this is a regression problem, you can check the correlation between the features and select the most useful subset of features and use them with any regression algorithm you find suitable. \n\nHope this helps also If you want to check in depth  EDA you can refer to : https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis.",
      "votes": null
    },
    {
      "id": "399513",
      "postDate": "10/06/2018 00:32:47",
      "content": "<p><a href=\"/vett93\">@vett93</a> Exploratory Data Analysis helps you better understand the quirks of the dataset you are given, which in turn can allow you to construct features to help your model learn better, or perhaps change your validation strategy.</p>",
      "rawMarkdown": "vett93 Exploratory Data Analysis helps you better understand the quirks of the dataset you are given, which in turn can allow you to construct features to help your model learn better, or perhaps change your validation strategy.",
      "votes": null
    },
    {
      "id": "399557",
      "postDate": "10/06/2018 05:03:56",
      "content": "<p>honestly, I have spent a full week on EDA of the training set and most of the insights I got from that were close to useless once turned into features. </p>\n\n<p>This is mostly because\n1. Time based features seem to bring poor results\n2. Ratio features the same\n3. Aggregates of ratios as well...</p>\n\n<p>From discussions on the forum it seems that the test set is a whole different beast to the train set, so perhaps the next step would be to do some EDA on the differences between the train and test, and then gain insights into potential corrective measures, or mistakes to avoid. </p>\n\n<p>But such analysis is more complex and messy than the stuff you see on nice looking EDA kernels, which is why its not posted I suppose </p>",
      "rawMarkdown": "honestly, I have spent a full week on EDA of the training set and most of the insights I got from that were close to useless once turned into features. \n\nThis is mostly because\n1. Time based features seem to bring poor results\n2. Ratio features the same\n3. Aggregates of ratios as well...\n\nFrom discussions on the forum it seems that the test set is a whole different beast to the train set, so perhaps the next step would be to do some EDA on the differences between the train and test, and then gain insights into potential corrective measures, or mistakes to avoid. \n\nBut such analysis is more complex and messy than the stuff you see on nice looking EDA kernels, which is why its not posted I suppose",
      "votes": null
    },
    {
      "id": "400219",
      "postDate": "10/07/2018 23:16:30",
      "content": "<p>@Ishaan Thanks for the pointer. I studied the notebook. It did a good job analyzing the data. However, there is no work linking the extensive EDA to feature/model forming. I was hoping that EDA would lead to a better model.</p>",
      "rawMarkdown": "Ishaan Thanks for the pointer. I studied the notebook. It did a good job analyzing the data. However, there is no work linking the extensive EDA to feature/model forming. I was hoping that EDA would lead to a better model.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 399428,
      "author_name": "ishaan45",
      "author_url": "",
      "post_date": "10/05/2018 18:57:04",
      "content": "<p>It's good to see that you are focusing on exploratory data analysis for better score. You can focus on univariate and bivariate analysis and as this is a regression problem, you can check the correlation between the features and select the most useful subset of features and use them with any regression algorithm you find suitable. </p>\n\n<p>Hope this helps also If you want to check in depth  EDA you can refer to : <a href=\"https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis\">https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis</a>.</p>",
      "votes": null,
      "replies": [
        {
          "id": 400219,
          "author_name": "vett93",
          "author_url": "",
          "post_date": "10/07/2018 23:16:30",
          "content": "<p>@Ishaan Thanks for the pointer. I studied the notebook. It did a good job analyzing the data. However, there is no work linking the extensive EDA to feature/model forming. I was hoping that EDA would lead to a better model.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 399513,
      "author_name": "returnofsputnik",
      "author_url": "",
      "post_date": "10/06/2018 00:32:47",
      "content": "<p><a href=\"/vett93\">@vett93</a> Exploratory Data Analysis helps you better understand the quirks of the dataset you are given, which in turn can allow you to construct features to help your model learn better, or perhaps change your validation strategy.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 399557,
      "author_name": "xavierbourretsicotte",
      "author_url": "",
      "post_date": "10/06/2018 05:03:56",
      "content": "<p>honestly, I have spent a full week on EDA of the training set and most of the insights I got from that were close to useless once turned into features. </p>\n\n<p>This is mostly because\n1. Time based features seem to bring poor results\n2. Ratio features the same\n3. Aggregates of ratios as well...</p>\n\n<p>From discussions on the forum it seems that the test set is a whole different beast to the train set, so perhaps the next step would be to do some EDA on the differences between the train and test, and then gain insights into potential corrective measures, or mistakes to avoid. </p>\n\n<p>But such analysis is more complex and messy than the stuff you see on nice looking EDA kernels, which is why its not posted I suppose </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "399420": "Can someone please comment on what insight can be derived from the exploratory data analysis? There are a few kernels with good scores that have little exploratory data analysis. It would be greatly appreciated if someone can use this competition as an example to illustrate how exploratory data analysis can yield better scores.",
    "399428": "It's good to see that you are focusing on exploratory data analysis for better score. You can focus on univariate and bivariate analysis and as this is a regression problem, you can check the correlation between the features and select the most useful subset of features and use them with any regression algorithm you find suitable. \n\nHope this helps also If you want to check in depth  EDA you can refer to : https://www.kaggle.com/diogomenezesborges/full-depth-eda-analysis.",
    "399513": "vett93 Exploratory Data Analysis helps you better understand the quirks of the dataset you are given, which in turn can allow you to construct features to help your model learn better, or perhaps change your validation strategy.",
    "399557": "honestly, I have spent a full week on EDA of the training set and most of the insights I got from that were close to useless once turned into features. \n\nThis is mostly because\n1. Time based features seem to bring poor results\n2. Ratio features the same\n3. Aggregates of ratios as well...\n\nFrom discussions on the forum it seems that the test set is a whole different beast to the train set, so perhaps the next step would be to do some EDA on the differences between the train and test, and then gain insights into potential corrective measures, or mistakes to avoid. \n\nBut such analysis is more complex and messy than the stuff you see on nice looking EDA kernels, which is why its not posted I suppose",
    "400219": "Ishaan Thanks for the pointer. I studied the notebook. It did a good job analyzing the data. However, there is no work linking the extensive EDA to feature/model forming. I was hoping that EDA would lead to a better model."
  },
  "source": "meta"
}