{
  "id": 74677,
  "title": "Correlation analysis",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/74677",
  "author_name": "",
  "post_date": "2018-12-14T11:31:40.110250600Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello all,\nI did some basic correlation analysis, by summing up the purchase amounts by card_id and trying to correlate with loyalty_score. Dnt see any correlation. \nAlso did pair plots of feature 1 / 2 / 3 and target, visually dnt see any correlation. Can someone suggest where I am going wrong, or is it the same finding by all.\nWe can always do a predictive model and try to fit the data, but i think we need to find meaningful correlation first! Pls suggest.</p>",
  "messages": [
    {
      "id": "438905",
      "postDate": "12/14/2018 11:31:40",
      "content": "<p>Hello all,\nI did some basic correlation analysis, by summing up the purchase amounts by card_id and trying to correlate with loyalty_score. Dnt see any correlation. \nAlso did pair plots of feature 1 / 2 / 3 and target, visually dnt see any correlation. Can someone suggest where I am going wrong, or is it the same finding by all.\nWe can always do a predictive model and try to fit the data, but i think we need to find meaningful correlation first! Pls suggest.</p>",
      "rawMarkdown": "Hello all,\nI did some basic correlation analysis, by summing up the purchase amounts by card_id and trying to correlate with loyalty_score. Dnt see any correlation. \nAlso did pair plots of feature 1 / 2 / 3 and target, visually dnt see any correlation. Can someone suggest where I am going wrong, or is it the same finding by all.\nWe can always do a predictive model and try to fit the data, but i think we need to find meaningful correlation first! Pls suggest.",
      "votes": null
    },
    {
      "id": "439025",
      "postDate": "12/14/2018 16:03:15",
      "content": "<p>It's still possible for there to be a correlation or pattern which is not visible to the human eye but that can be picked up by a machine learning model.  Take a look at some of the kernels, those features will tell you where there is useful information that the model picks up on.  You can then do plots on those features to see them visually (if possible).</p>",
      "rawMarkdown": "It's still possible for there to be a correlation or pattern which is not visible to the human eye but that can be picked up by a machine learning model.  Take a look at some of the kernels, those features will tell you where there is useful information that the model picks up on.  You can then do plots on those features to see them visually (if possible).",
      "votes": null
    },
    {
      "id": "440185",
      "postDate": "12/17/2018 07:30:28",
      "content": "<p>Thats right, i checked some of the other kernels. Looks like i'm right in doing operations like groupby('card_id'), and summing the purchase_amount etc, however, there seems to be correlation with max purchase amount etc (not the sum), which is kind of weird. Will check that. Also I find the transaction volume is high for certain Merchants and for few cities.</p>",
      "rawMarkdown": "Thats right, i checked some of the other kernels. Looks like i'm right in doing operations like groupby('card_id'), and summing the purchase_amount etc, however, there seems to be correlation with max purchase amount etc (not the sum), which is kind of weird. Will check that. Also I find the transaction volume is high for certain Merchants and for few cities.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 439025,
      "author_name": "robertkag",
      "author_url": "",
      "post_date": "12/14/2018 16:03:15",
      "content": "<p>It's still possible for there to be a correlation or pattern which is not visible to the human eye but that can be picked up by a machine learning model.  Take a look at some of the kernels, those features will tell you where there is useful information that the model picks up on.  You can then do plots on those features to see them visually (if possible).</p>",
      "votes": null,
      "replies": [
        {
          "id": 440185,
          "author_name": "anirbank",
          "author_url": "",
          "post_date": "12/17/2018 07:30:28",
          "content": "<p>Thats right, i checked some of the other kernels. Looks like i'm right in doing operations like groupby('card_id'), and summing the purchase_amount etc, however, there seems to be correlation with max purchase amount etc (not the sum), which is kind of weird. Will check that. Also I find the transaction volume is high for certain Merchants and for few cities.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "438905": "Hello all,\nI did some basic correlation analysis, by summing up the purchase amounts by card_id and trying to correlate with loyalty_score. Dnt see any correlation. \nAlso did pair plots of feature 1 / 2 / 3 and target, visually dnt see any correlation. Can someone suggest where I am going wrong, or is it the same finding by all.\nWe can always do a predictive model and try to fit the data, but i think we need to find meaningful correlation first! Pls suggest.",
    "439025": "It's still possible for there to be a correlation or pattern which is not visible to the human eye but that can be picked up by a machine learning model.  Take a look at some of the kernels, those features will tell you where there is useful information that the model picks up on.  You can then do plots on those features to see them visually (if possible).",
    "440185": "Thats right, i checked some of the other kernels. Looks like i'm right in doing operations like groupby('card_id'), and summing the purchase_amount etc, however, there seems to be correlation with max purchase amount etc (not the sum), which is kind of weird. Will check that. Also I find the transaction volume is high for certain Merchants and for few cities."
  },
  "source": "meta"
}