{
  "id": 76037,
  "title": "Ideas for few categories",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/76037",
  "author_name": "Frank",
  "post_date": "2018-12-28T18:05:16.746000",
  "votes": 18,
  "comment_count": 5,
  "views": 0,
  "content": "<p>In Data description for all categories, it only says \"anonymized category\"</p>\n\n<p>After analysis, here is my summary for some categories:\nIn transaction data:\n*Category_1* = Y :  2084029, N : 27028332\ngroup by category_1 and state_id</p>\n\n<pre><code>historical_transactions.groupby(['category_1','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\nhistorical_transactions.groupby(['category_1','city_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n</code></pre>\n\n<p><strong>When category_1 is \"Y\", both city_id and state_id are -1 (Counts 2169546)</strong></p>\n\n<p>*Category_2*</p>\n\n<pre><code>historical_transactions[\"category_2\"].fillna(6, inplace = True) \nhistorical_transactions.groupby(['category_2','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n1    16276518  \n2     1026600 \n3     3912467 \n4     2653629\n5     3726156 \n6     2738410 \n</code></pre>\n\n<p><strong>category_2 groups all 24 states to 5 groups, if category_2 is Null (filled as 6), sate_id is -1 (unknown)</strong></p>\n\n<p>Category_3</p>\n\n<pre><code>historical_transactions.groupby(['installments','category_3']).size()\n</code></pre>\n\n<p><strong>category_3 = A -&gt; 0 installments, B -&gt; 1 installments, C-&gt; more than 1 installments, Nan -&gt; -1 installments (178159).</strong></p>\n\n<p>Welcome to share any other relations you find for any other features, and how would you use those features (combine with other feature, count encoding, onehotencoding etc.)</p>\n\n<p>BTW, it might worth some time to learn about the company's business, check out the website here \n<a href=\"https://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/\">https://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/</a></p>",
  "messages": [
    {
      "id": 446799,
      "postDate": "2018-12-28T18:05:16.747Z",
      "content": "<p>In Data description for all categories, it only says \"anonymized category\"</p>\n\n<p>After analysis, here is my summary for some categories:\nIn transaction data:\n*Category_1* = Y :  2084029, N : 27028332\ngroup by category_1 and state_id</p>\n\n<pre><code>historical_transactions.groupby(['category_1','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\nhistorical_transactions.groupby(['category_1','city_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n</code></pre>\n\n<p><strong>When category_1 is \"Y\", both city_id and state_id are -1 (Counts 2169546)</strong></p>\n\n<p>*Category_2*</p>\n\n<pre><code>historical_transactions[\"category_2\"].fillna(6, inplace = True) \nhistorical_transactions.groupby(['category_2','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n1    16276518  \n2     1026600 \n3     3912467 \n4     2653629\n5     3726156 \n6     2738410 \n</code></pre>\n\n<p><strong>category_2 groups all 24 states to 5 groups, if category_2 is Null (filled as 6), sate_id is -1 (unknown)</strong></p>\n\n<p>Category_3</p>\n\n<pre><code>historical_transactions.groupby(['installments','category_3']).size()\n</code></pre>\n\n<p><strong>category_3 = A -&gt; 0 installments, B -&gt; 1 installments, C-&gt; more than 1 installments, Nan -&gt; -1 installments (178159).</strong></p>\n\n<p>Welcome to share any other relations you find for any other features, and how would you use those features (combine with other feature, count encoding, onehotencoding etc.)</p>\n\n<p>BTW, it might worth some time to learn about the company's business, check out the website here \n<a href=\"https://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/\">https://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/</a></p>",
      "rawMarkdown": "In Data description for all categories, it only says \"anonymized category\"\n\nAfter analysis, here is my summary for some categories:\nIn transaction data:\n*Category_1* = Y :  2084029, N : 27028332\ngroup by category_1 and state_id\n\n    historical_transactions.groupby(['category_1','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n    historical_transactions.groupby(['category_1','city_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n**When category_1 is \"Y\", both city_id and state_id are -1 (Counts 2169546)**\n\n*Category_2*\n\n    historical_transactions[\"category_2\"].fillna(6, inplace = True) \n    historical_transactions.groupby(['category_2','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n    1    16276518  \n    2     1026600 \n    3     3912467 \n    4     2653629\n    5     3726156 \n    6     2738410 \n\n**category_2 groups all 24 states to 5 groups, if category_2 is Null (filled as 6), sate_id is -1 (unknown)**\n\n\nCategory_3\n\n    historical_transactions.groupby(['installments','category_3']).size()\n\n**category_3 = A -&gt; 0 installments, B -&gt; 1 installments, C-&gt; more than 1 installments, Nan -&gt; -1 installments (178159).**\n\nWelcome to share any other relations you find for any other features, and how would you use those features (combine with other feature, count encoding, onehotencoding etc.)\n\nBTW, it might worth some time to learn about the company's business, check out the website here \nhttps://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/",
      "votes": 18
    },
    {
      "id": 446943,
      "postDate": "2018-12-28T22:34:54.883Z",
      "content": "<p>Thanks, Frank \nIt seems that Category 1 is related to International vs Local purchases that is why there is no City and  State Id\nCategory 2 is related to the Regions in Brazil, there is Actually 5 Regions + 1 Federate State = NaN</p>",
      "rawMarkdown": "Thanks, Frank \nIt seems that Category 1 is related to International vs Local purchases that is why there is no City and  State Id\nCategory 2 is related to the Regions in Brazil, there is Actually 5 Regions + 1 Federate State = NaN",
      "votes": 5,
      "replies": [
        {
          "id": 446949,
          "postDate": "2018-12-28T22:57:19.140Z",
          "content": "<p>Cool, now it makes more sense, thanks.</p>",
          "rawMarkdown": "Cool, now it makes more sense, thanks.\n",
          "votes": 1
        }
      ]
    },
    {
      "id": 447113,
      "postDate": "2018-12-29T06:52:50.700Z",
      "content": "<p>Worth taking a look at <a href=\"https://www.kaggle.com/tapioca/category-1-2-3-in-transactions\">https://www.kaggle.com/tapioca/category-1-2-3-in-transactions</a> where I think this was first conclusively laid out</p>",
      "rawMarkdown": "Worth taking a look at https://www.kaggle.com/tapioca/category-1-2-3-in-transactions where I think this was first conclusively laid out",
      "votes": 4,
      "replies": [
        {
          "id": 447401,
          "postDate": "2018-12-29T18:59:56.507Z",
          "content": "<p>You are right, I didn't see that</p>",
          "rawMarkdown": "You are right, I didn't see that"
        }
      ]
    },
    {
      "id": 1036068,
      "postDate": "2020-10-03T11:50:29.637Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 446943,
      "author_name": "C4rl05/V",
      "author_url": "",
      "post_date": "2018-12-28T22:34:54.883000",
      "content": "<p>Thanks, Frank \nIt seems that Category 1 is related to International vs Local purchases that is why there is no City and  State Id\nCategory 2 is related to the Regions in Brazil, there is Actually 5 Regions + 1 Federate State = NaN</p>",
      "votes": 5,
      "replies": [
        {
          "id": 446949,
          "author_name": "Frank",
          "author_url": "",
          "post_date": "2018-12-28T22:57:19.140000",
          "content": "<p>Cool, now it makes more sense, thanks.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 447113,
      "author_name": "Peter Hurford",
      "author_url": "",
      "post_date": "2018-12-29T06:52:50.700000",
      "content": "<p>Worth taking a look at <a href=\"https://www.kaggle.com/tapioca/category-1-2-3-in-transactions\">https://www.kaggle.com/tapioca/category-1-2-3-in-transactions</a> where I think this was first conclusively laid out</p>",
      "votes": 4,
      "replies": [
        {
          "id": 447401,
          "author_name": "Frank",
          "author_url": "",
          "post_date": "2018-12-29T18:59:56.507000",
          "content": "<p>You are right, I didn't see that</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1036068,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-10-03T11:50:29.637000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "446799": "In Data description for all categories, it only says \"anonymized category\"\n\nAfter analysis, here is my summary for some categories:\nIn transaction data:\n*Category_1* = Y :  2084029, N : 27028332\ngroup by category_1 and state_id\n\n    historical_transactions.groupby(['category_1','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n    historical_transactions.groupby(['category_1','city_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n**When category_1 is \"Y\", both city_id and state_id are -1 (Counts 2169546)**\n\n*Category_2*\n\n    historical_transactions[\"category_2\"].fillna(6, inplace = True) \n    historical_transactions.groupby(['category_2','state_id'], as_index=False).agg({'card_id': ['count']}, as_index=False)\n\n    1    16276518  \n    2     1026600 \n    3     3912467 \n    4     2653629\n    5     3726156 \n    6     2738410 \n\n**category_2 groups all 24 states to 5 groups, if category_2 is Null (filled as 6), sate_id is -1 (unknown)**\n\n\nCategory_3\n\n    historical_transactions.groupby(['installments','category_3']).size()\n\n**category_3 = A -&gt; 0 installments, B -&gt; 1 installments, C-&gt; more than 1 installments, Nan -&gt; -1 installments (178159).**\n\nWelcome to share any other relations you find for any other features, and how would you use those features (combine with other feature, count encoding, onehotencoding etc.)\n\nBTW, it might worth some time to learn about the company's business, check out the website here \nhttps://www.cartaoelo.com.br/para-sua-empresa-pequenas-e-medias/",
    "446943": "Thanks, Frank \nIt seems that Category 1 is related to International vs Local purchases that is why there is no City and  State Id\nCategory 2 is related to the Regions in Brazil, there is Actually 5 Regions + 1 Federate State = NaN",
    "447113": "Worth taking a look at https://www.kaggle.com/tapioca/category-1-2-3-in-transactions where I think this was first conclusively laid out",
    "1036068": ""
  }
}