{
  "id": 114310,
  "title": "A secret duplication in the wind direction values",
  "url": "/competitions/ashrae-energy-prediction/discussion/114310",
  "author_name": "",
  "post_date": "2019-10-25T12:15:58.054528Z",
  "votes": 27,
  "comment_count": 10,
  "views": 0,
  "content": "<p>The wind direction values range from 0 to 360 in the weather data. But 0=360 in polar coordinates. This secret duplication can be avoided by replacing 360 with 0.</p>",
  "messages": [
    {
      "id": "657770",
      "postDate": "10/25/2019 12:15:58",
      "content": "<p>The wind direction values range from 0 to 360 in the weather data. But 0=360 in polar coordinates. This secret duplication can be avoided by replacing 360 with 0.</p>",
      "rawMarkdown": "The wind direction values range from 0 to 360 in the weather data. But 0=360 in polar coordinates. This secret duplication can be avoided by replacing 360 with 0.",
      "votes": null
    },
    {
      "id": "658078",
      "postDate": "10/25/2019 16:35:49",
      "content": "<p>Yeah, I did some <a href=\"https://www.kaggle.com/nroman/eda-for-ashrae\">EDA on it</a></p>\n\n<p><code>\nNumber of observations with 0 degree direction in TRAIN: 2373540 (0.12%)\nNumber of observations with 0 degree direction in TEST: 4796510 (0.12%)\nNumber of observations with 360 degree direction in TRAIN: 421533 (0.02%)\nNumber of observations with 360 degree direction in TEST: 813553 (0.02%)\n</code></p>\n\n<p>Also if wind_speed is 0 then direction does not matter.</p>",
      "rawMarkdown": "Yeah, I did some [EDA on it](https://www.kaggle.com/nroman/eda-for-ashrae)\n\n```\nNumber of observations with 0 degree direction in TRAIN: 2373540 (0.12%)\nNumber of observations with 0 degree direction in TEST: 4796510 (0.12%)\nNumber of observations with 360 degree direction in TRAIN: 421533 (0.02%)\nNumber of observations with 360 degree direction in TEST: 813553 (0.02%)\n```\n\nAlso if wind_speed is 0 then direction does not matter.",
      "votes": null
    },
    {
      "id": "658159",
      "postDate": "10/25/2019 18:11:06",
      "content": "<p>Exactly!</p>",
      "rawMarkdown": "Exactly!",
      "votes": null
    },
    {
      "id": "658882",
      "postDate": "10/26/2019 17:16:12",
      "content": "<p>Very Good indeed !</p>",
      "rawMarkdown": "Very Good indeed !",
      "votes": null
    },
    {
      "id": "659262",
      "postDate": "10/27/2019 11:16:13",
      "content": "<p>Interesting, good to know!</p>",
      "rawMarkdown": "Interesting, good to know!",
      "votes": null
    },
    {
      "id": "660163",
      "postDate": "10/28/2019 19:31:42",
      "content": "<p>I don't believe that they are the same in this case. All measurements with wind direction 0 have wind speed 0 while there are mostly non zero values in wind direction 360. Also 360 seems to be about in the right quantity compared to other wind directions (~2000 to 4000 instances in weather_train), while wind direction 0 is about three times that quantity. So my guess is that wind direction 0 means no wind while 360 means northerly wind. My guess is that the only values to be replaced should be wind direction 360 in case the is wind speed 0.</p>",
      "rawMarkdown": "I don't believe that they are the same in this case. All measurements with wind direction 0 have wind speed 0 while there are mostly non zero values in wind direction 360. Also 360 seems to be about in the right quantity compared to other wind directions (~2000 to 4000 instances in weather_train), while wind direction 0 is about three times that quantity. So my guess is that wind direction 0 means no wind while 360 means northerly wind. My guess is that the only values to be replaced should be wind direction 360 in case the is wind speed 0.",
      "votes": null
    },
    {
      "id": "660182",
      "postDate": "10/28/2019 20:43:49",
      "content": "<p>Thanks for pointing this out!</p>",
      "rawMarkdown": "Thanks for pointing this out!",
      "votes": null
    },
    {
      "id": "660501",
      "postDate": "10/29/2019 08:27:43",
      "content": "<p>I use sin and cos function to transform the wind direction feature.\n<code>df['sin_direction'] = np.sin(df['wind_direction']*np.pi/180)</code>\n<code>df['cos_direction'] = np.cos(df['wind_direction']*np.pi/180)</code>\n<code>del df['wind_direction']</code>\n I think that can help the model find that continuity.</p>",
      "rawMarkdown": "I use sin and cos function to transform the wind direction feature.\n`df['sin_direction'] = np.sin(df['wind_direction']*np.pi/180)`\n`df['cos_direction'] = np.cos(df['wind_direction']*np.pi/180)`\n`del df['wind_direction']`\n I think that can help the model find that continuity.",
      "votes": null
    },
    {
      "id": "660540",
      "postDate": "10/29/2019 09:40:37",
      "content": "<p>I was thinking of categorizing the wind direction values into 4 classes like north, south, west, east. </p>",
      "rawMarkdown": "I was thinking of categorizing the wind direction values into 4 classes like north, south, west, east.",
      "votes": null
    },
    {
      "id": "661149",
      "postDate": "10/30/2019 00:45:11",
      "content": "<p>Great ! Thanks for sharing <a href=\"/uestcblackstar\">@uestcblackstar</a> !</p>",
      "rawMarkdown": "Great ! Thanks for sharing @uestcblackstar !",
      "votes": null
    },
    {
      "id": "661151",
      "postDate": "10/30/2019 00:46:30",
      "content": "<p>Good to know, <a href=\"/nroman\">@nroman</a>  ! Thanks for Sharing !</p>",
      "rawMarkdown": "Good to know, @nroman  ! Thanks for Sharing !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 658078,
      "author_name": "nroman",
      "author_url": "",
      "post_date": "10/25/2019 16:35:49",
      "content": "<p>Yeah, I did some <a href=\"https://www.kaggle.com/nroman/eda-for-ashrae\">EDA on it</a></p>\n\n<p><code>\nNumber of observations with 0 degree direction in TRAIN: 2373540 (0.12%)\nNumber of observations with 0 degree direction in TEST: 4796510 (0.12%)\nNumber of observations with 360 degree direction in TRAIN: 421533 (0.02%)\nNumber of observations with 360 degree direction in TEST: 813553 (0.02%)\n</code></p>\n\n<p>Also if wind_speed is 0 then direction does not matter.</p>",
      "votes": null,
      "replies": [
        {
          "id": 658159,
          "author_name": "tolgadincer",
          "author_url": "",
          "post_date": "10/25/2019 18:11:06",
          "content": "<p>Exactly!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 661151,
          "author_name": "caesarlupum",
          "author_url": "",
          "post_date": "10/30/2019 00:46:30",
          "content": "<p>Good to know, <a href=\"/nroman\">@nroman</a>  ! Thanks for Sharing !</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 658882,
      "author_name": "pfichou",
      "author_url": "",
      "post_date": "10/26/2019 17:16:12",
      "content": "<p>Very Good indeed !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 659262,
      "author_name": "carlolepelaars",
      "author_url": "",
      "post_date": "10/27/2019 11:16:13",
      "content": "<p>Interesting, good to know!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 660163,
      "author_name": "twolfanger",
      "author_url": "",
      "post_date": "10/28/2019 19:31:42",
      "content": "<p>I don't believe that they are the same in this case. All measurements with wind direction 0 have wind speed 0 while there are mostly non zero values in wind direction 360. Also 360 seems to be about in the right quantity compared to other wind directions (~2000 to 4000 instances in weather_train), while wind direction 0 is about three times that quantity. So my guess is that wind direction 0 means no wind while 360 means northerly wind. My guess is that the only values to be replaced should be wind direction 360 in case the is wind speed 0.</p>",
      "votes": null,
      "replies": [
        {
          "id": 660182,
          "author_name": "tolgadincer",
          "author_url": "",
          "post_date": "10/28/2019 20:43:49",
          "content": "<p>Thanks for pointing this out!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 660501,
      "author_name": "uestcblackstar",
      "author_url": "",
      "post_date": "10/29/2019 08:27:43",
      "content": "<p>I use sin and cos function to transform the wind direction feature.\n<code>df['sin_direction'] = np.sin(df['wind_direction']*np.pi/180)</code>\n<code>df['cos_direction'] = np.cos(df['wind_direction']*np.pi/180)</code>\n<code>del df['wind_direction']</code>\n I think that can help the model find that continuity.</p>",
      "votes": null,
      "replies": [
        {
          "id": 660540,
          "author_name": "tolgadincer",
          "author_url": "",
          "post_date": "10/29/2019 09:40:37",
          "content": "<p>I was thinking of categorizing the wind direction values into 4 classes like north, south, west, east. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 661149,
          "author_name": "caesarlupum",
          "author_url": "",
          "post_date": "10/30/2019 00:45:11",
          "content": "<p>Great ! Thanks for sharing <a href=\"/uestcblackstar\">@uestcblackstar</a> !</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "657770": "The wind direction values range from 0 to 360 in the weather data. But 0=360 in polar coordinates. This secret duplication can be avoided by replacing 360 with 0.",
    "658078": "Yeah, I did some [EDA on it](https://www.kaggle.com/nroman/eda-for-ashrae)\n\n```\nNumber of observations with 0 degree direction in TRAIN: 2373540 (0.12%)\nNumber of observations with 0 degree direction in TEST: 4796510 (0.12%)\nNumber of observations with 360 degree direction in TRAIN: 421533 (0.02%)\nNumber of observations with 360 degree direction in TEST: 813553 (0.02%)\n```\n\nAlso if wind_speed is 0 then direction does not matter.",
    "658159": "Exactly!",
    "658882": "Very Good indeed !",
    "659262": "Interesting, good to know!",
    "660163": "I don't believe that they are the same in this case. All measurements with wind direction 0 have wind speed 0 while there are mostly non zero values in wind direction 360. Also 360 seems to be about in the right quantity compared to other wind directions (~2000 to 4000 instances in weather_train), while wind direction 0 is about three times that quantity. So my guess is that wind direction 0 means no wind while 360 means northerly wind. My guess is that the only values to be replaced should be wind direction 360 in case the is wind speed 0.",
    "660182": "Thanks for pointing this out!",
    "660501": "I use sin and cos function to transform the wind direction feature.\n`df['sin_direction'] = np.sin(df['wind_direction']*np.pi/180)`\n`df['cos_direction'] = np.cos(df['wind_direction']*np.pi/180)`\n`del df['wind_direction']`\n I think that can help the model find that continuity.",
    "660540": "I was thinking of categorizing the wind direction values into 4 classes like north, south, west, east.",
    "661149": "Great ! Thanks for sharing @uestcblackstar !",
    "661151": "Good to know, @nroman  ! Thanks for Sharing !"
  },
  "source": "meta"
}