{
  "id": 126433,
  "title": "What can one do of \"day\" and \"month\" !",
  "url": "/competitions/cat-in-the-dat-ii/discussion/126433",
  "author_name": "",
  "post_date": "2020-01-17T13:18:44.580507600Z",
  "votes": 5,
  "comment_count": 11,
  "views": 0,
  "content": "<p>Do \"month\" and \"day\" differ from other categories in any way?</p>",
  "messages": [
    {
      "id": "721539",
      "postDate": "01/17/2020 13:18:44",
      "content": "<p>Do \"month\" and \"day\" differ from other categories in any way?</p>",
      "rawMarkdown": "Do \"month\" and \"day\" differ from other categories in any way?",
      "votes": null
    },
    {
      "id": "721790",
      "postDate": "01/17/2020 17:38:47",
      "content": "<p>Make it cyclical by transforming it to cartesian points using Sin, Cos.</p>",
      "rawMarkdown": "Make it cyclical by transforming it to cartesian points using Sin, Cos.",
      "votes": null
    },
    {
      "id": "721847",
      "postDate": "01/17/2020 19:20:43",
      "content": "<p>They perform better as dummies I think.</p>",
      "rawMarkdown": "They perform better as dummies I think.",
      "votes": null
    },
    {
      "id": "722791",
      "postDate": "01/19/2020 05:42:23",
      "content": "<p>%%time</p>\n\n<p>X_train_cyclic=X.copy()\ncolumns=['day','month']\nfor col in columns:\n    X_train_cyclic[col+'_sin']=np.sin((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\n    X_train_cyclic[col+'_cos']=np.cos((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\nX_train_cyclic=X_train_cyclic.drop(columns,axis=1)</p>\n\n<p>X_train_cyclic[['day_sin','day_cos']].head(3)</p>",
      "rawMarkdown": "%%time\n\nX_train_cyclic=X.copy()\ncolumns=['day','month']\nfor col in columns:\n    X_train_cyclic[col+'_sin']=np.sin((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\n    X_train_cyclic[col+'_cos']=np.cos((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\nX_train_cyclic=X_train_cyclic.drop(columns,axis=1)\n\nX_train_cyclic[['day_sin','day_cos']].head(3)",
      "votes": null
    },
    {
      "id": "722985",
      "postDate": "01/19/2020 11:18:06",
      "content": "<p>in my case i faced that dummies are better in comparison with  sin,cos (0.01 auc score difference on testset). i used logistic regression, and you can also see my public kernel</p>",
      "rawMarkdown": "in my case i faced that dummies are better in comparison with  sin,cos (0.01 auc score difference on testset). i used logistic regression, and you can also see my public kernel",
      "votes": null
    },
    {
      "id": "723328",
      "postDate": "01/19/2020 21:48:46",
      "content": "<p>Has anyone checked to see if day/month are better when combined?</p>",
      "rawMarkdown": "Has anyone checked to see if day/month are better when combined?",
      "votes": null
    },
    {
      "id": "723330",
      "postDate": "01/19/2020 21:49:53",
      "content": "<p>I noticed that day seems to follow a quadratic (\"U-shaped\") function when you look at the % of targets by day #.</p>",
      "rawMarkdown": "I noticed that day seems to follow a quadratic (\"U-shaped\") function when you look at the % of targets by day #.",
      "votes": null
    },
    {
      "id": "730319",
      "postDate": "01/27/2020 10:47:16",
      "content": "<p>Has anyone tried encoding it?</p>",
      "rawMarkdown": "Has anyone tried encoding it?",
      "votes": null
    },
    {
      "id": "730715",
      "postDate": "01/27/2020 20:16:24",
      "content": "<p>Check out this link: </p>\n\n<p><a href=\"https://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction\">https://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction</a></p>",
      "rawMarkdown": "Check out this link: \n\nhttps://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction",
      "votes": null
    },
    {
      "id": "734659",
      "postDate": "02/01/2020 19:48:07",
      "content": "<p>I don’t think they would do anything...</p>\n\n<p>Unless I (we) discover something. I tried a sin/cos transformation but nothing came.</p>\n\n<p>Dig the cat variables and some One Hot Encoding... even if I’am still struggling with them.</p>\n\n<p>But not sure of that neither.</p>",
      "rawMarkdown": "I don’t think they would do anything...\n\nUnless I (we) discover something. I tried a sin/cos transformation but nothing came.\n\nDig the cat variables and some One Hot Encoding... even if I’am still struggling with them.\n\nBut not sure of that neither.",
      "votes": null
    },
    {
      "id": "738586",
      "postDate": "02/06/2020 18:23:20",
      "content": "<p>Nearly every encoding technique performs better than sines and cosines. I don't know any other way to encode cyclic-ness. Projective geometry? And you can't combine them into dates either.\n<code>month</code> performs reasonably well left as-is and considered as an ordinal variable, while <code>day</code> shows less correlation with target than <code>id</code>. I'd go with the usual <code>TargetEncoder</code>, or try <code>WOEEncoder</code> for a thrilling experience.</p>",
      "rawMarkdown": "Nearly every encoding technique performs better than sines and cosines. I don't know any other way to encode cyclic-ness. Projective geometry? And you can't combine them into dates either.\n`month` performs reasonably well left as-is and considered as an ordinal variable, while `day` shows less correlation with target than `id`. I'd go with the usual `TargetEncoder`, or try `WOEEncoder` for a thrilling experience.",
      "votes": null
    },
    {
      "id": "739174",
      "postDate": "02/07/2020 14:08:40",
      "content": "<p>I tried <code>IterativeImputer</code> with different estimators. OHE (I used <code>get_dummies</code> of pandas) did not provide the best result for me. Still digging ... </p>",
      "rawMarkdown": "I tried `IterativeImputer` with different estimators. OHE (I used `get_dummies` of pandas) did not provide the best result for me. Still digging ...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 721790,
      "author_name": "pairwiseseq",
      "author_url": "",
      "post_date": "01/17/2020 17:38:47",
      "content": "<p>Make it cyclical by transforming it to cartesian points using Sin, Cos.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 721847,
      "author_name": "cemsarier",
      "author_url": "",
      "post_date": "01/17/2020 19:20:43",
      "content": "<p>They perform better as dummies I think.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 722791,
      "author_name": "ganeshmundra",
      "author_url": "",
      "post_date": "01/19/2020 05:42:23",
      "content": "<p>%%time</p>\n\n<p>X_train_cyclic=X.copy()\ncolumns=['day','month']\nfor col in columns:\n    X_train_cyclic[col+'_sin']=np.sin((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\n    X_train_cyclic[col+'_cos']=np.cos((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\nX_train_cyclic=X_train_cyclic.drop(columns,axis=1)</p>\n\n<p>X_train_cyclic[['day_sin','day_cos']].head(3)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 722985,
      "author_name": "fkdplc",
      "author_url": "",
      "post_date": "01/19/2020 11:18:06",
      "content": "<p>in my case i faced that dummies are better in comparison with  sin,cos (0.01 auc score difference on testset). i used logistic regression, and you can also see my public kernel</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 723328,
      "author_name": "bluewizard",
      "author_url": "",
      "post_date": "01/19/2020 21:48:46",
      "content": "<p>Has anyone checked to see if day/month are better when combined?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 723330,
      "author_name": "bluewizard",
      "author_url": "",
      "post_date": "01/19/2020 21:49:53",
      "content": "<p>I noticed that day seems to follow a quadratic (\"U-shaped\") function when you look at the % of targets by day #.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 730319,
      "author_name": "brunhs",
      "author_url": "",
      "post_date": "01/27/2020 10:47:16",
      "content": "<p>Has anyone tried encoding it?</p>",
      "votes": null,
      "replies": [
        {
          "id": 739174,
          "author_name": "catadanna",
          "author_url": "",
          "post_date": "02/07/2020 14:08:40",
          "content": "<p>I tried <code>IterativeImputer</code> with different estimators. OHE (I used <code>get_dummies</code> of pandas) did not provide the best result for me. Still digging ... </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 730715,
      "author_name": "frankmollard",
      "author_url": "",
      "post_date": "01/27/2020 20:16:24",
      "content": "<p>Check out this link: </p>\n\n<p><a href=\"https://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction\">https://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 734659,
      "author_name": "pfichou",
      "author_url": "",
      "post_date": "02/01/2020 19:48:07",
      "content": "<p>I don’t think they would do anything...</p>\n\n<p>Unless I (we) discover something. I tried a sin/cos transformation but nothing came.</p>\n\n<p>Dig the cat variables and some One Hot Encoding... even if I’am still struggling with them.</p>\n\n<p>But not sure of that neither.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 738586,
      "author_name": "davidbnn92",
      "author_url": "",
      "post_date": "02/06/2020 18:23:20",
      "content": "<p>Nearly every encoding technique performs better than sines and cosines. I don't know any other way to encode cyclic-ness. Projective geometry? And you can't combine them into dates either.\n<code>month</code> performs reasonably well left as-is and considered as an ordinal variable, while <code>day</code> shows less correlation with target than <code>id</code>. I'd go with the usual <code>TargetEncoder</code>, or try <code>WOEEncoder</code> for a thrilling experience.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "721539": "Do \"month\" and \"day\" differ from other categories in any way?",
    "721790": "Make it cyclical by transforming it to cartesian points using Sin, Cos.",
    "721847": "They perform better as dummies I think.",
    "722791": "%%time\n\nX_train_cyclic=X.copy()\ncolumns=['day','month']\nfor col in columns:\n    X_train_cyclic[col+'_sin']=np.sin((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\n    X_train_cyclic[col+'_cos']=np.cos((2*np.pi*X_train_cyclic[col])/max(X_train_cyclic[col]))\nX_train_cyclic=X_train_cyclic.drop(columns,axis=1)\n\nX_train_cyclic[['day_sin','day_cos']].head(3)",
    "722985": "in my case i faced that dummies are better in comparison with  sin,cos (0.01 auc score difference on testset). i used logistic regression, and you can also see my public kernel",
    "723328": "Has anyone checked to see if day/month are better when combined?",
    "723330": "I noticed that day seems to follow a quadratic (\"U-shaped\") function when you look at the % of targets by day #.",
    "730319": "Has anyone tried encoding it?",
    "730715": "Check out this link: \n\nhttps://www.datasciencecentral.com/profiles/blogs/how-to-make-time-data-cyclical-for-prediction",
    "734659": "I don’t think they would do anything...\n\nUnless I (we) discover something. I tried a sin/cos transformation but nothing came.\n\nDig the cat variables and some One Hot Encoding... even if I’am still struggling with them.\n\nBut not sure of that neither.",
    "738586": "Nearly every encoding technique performs better than sines and cosines. I don't know any other way to encode cyclic-ness. Projective geometry? And you can't combine them into dates either.\n`month` performs reasonably well left as-is and considered as an ordinal variable, while `day` shows less correlation with target than `id`. I'd go with the usual `TargetEncoder`, or try `WOEEncoder` for a thrilling experience.",
    "739174": "I tried `IterativeImputer` with different estimators. OHE (I used `get_dummies` of pandas) did not provide the best result for me. Still digging ..."
  },
  "source": "meta"
}