Working on my EDA I compared my data with several notebooks. I live in Germany in I am working with R. For the events file my first timestamp gives:
\n2018-08-15 02:26:00
\nMarlia Prata @mpwolke notebook gives: 2018-08-14 22:26:00
\nAll timestamps have 4 hours difference. I can see the 4 hours difference after the timestamp according to ISO 8601 in Marilia's notebook:
\n2018-08-14T22:26:00-0400,
\nbut not when I read the data with read_parquet from the library(arrow).
\nIt's easy to add 4 hours, but any other solutions?
\nThanks Peter
Working on my EDA I compared my data with several notebooks. I live in Germany in I am working with R. For the events file my first timestamp gives:
\n2018-08-15 02:26:00
\nMarlia Prata @mpwolke notebook gives: 2018-08-14 22:26:00
\nAll timestamps have 4 hours difference. I can see the 4 hours difference after the timestamp according to ISO 8601 in Marilia's notebook:
\n2018-08-14T22:26:00-0400,
\nbut not when I read the data with read_parquet from the library(arrow).
\nIt's easy to add 4 hours, but any other solutions?
\nThanks Peter
Hi Peter,
\n\"There were some series affected by a bug in DST handling that caused events after a time change to be incorrectly offset by an hour. Apparently, this only affected series in which a DST occurred (that is, when the time zone offset changed from \"-400\" to \"-500\"), which is 36 series in the training set (perhaps 6% of all events).\"
\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697
those results are on my Notebook (I Published at the beginning of the competition). However, I copied Chun Fu that was the 1st to deliver the timestamp's parsing:
\nBy Chun Fu https://www.kaggle.com/code/patrick0302/viz-of-sleeping-time-series
\nThose guys below, dealed with timestamps successfully too. I didn't check if there were difference of time on their work.
\nCMI Sleep Visualization - By Connor
\nhttps://www.kaggle.com/code/clane9/cmi-sleep-visualization
Sleeping patterns - EDA - By Robert Kwiatkowski
\nhttps://www.kaggle.com/code/datark1/sleeping-patterns-eda
Xgboost Notebook Updated - By Anmar Ahmadi Qazi
\nhttps://www.kaggle.com/code/ammarahmadqazi/xgboost-notebook-updated
204_Preprocessing_Data - By Alvaro Rodriguez
\nhttps://www.kaggle.com/code/alvrod/204-preprocessing-data
You're welcome Peter,
\nKind Regards,
\nMarília
Hi Marilia,
\nThanks for your help. That make it more complicated for me. Reading the parquet file, I don't see the offset. If their are different offsets as you mentioned, I must figure out how to detect them. On the flip side submission requires only step not time.
\nKind regards
\nPeter
\nP.S. Still searching for the apostroph i on my Mac to get your name correct😘
You can write without the accent ´
\nSometimes (purchases) I avoid to type the accent cause computers don't understand, then it appears \"Marlia\" or something like that. And I'm afraid that the parcel won't be delivered. So even myself, write Marilia without the accent.
", "rawMarkdown": "You can write without the accent ´ \n\nSometimes (purchases) I avoid to type the accent cause computers don't understand, then it appears \"Marlia\" or something like that. And I'm afraid that the parcel won't be delivered. So even myself, write Marilia without the accent.", "votes": null }, { "id": "2442314", "postDate": "09/16/2023 22:18:51", "content": "Hi @pehahn. The ISO 8601 timestamp below is in the time zone \"EDT(-4h)\" and represents the local time 2018-08-14 22:26:00. For this sleep task, I think it would be better to use 22:26 local time.
\n\n", "rawMarkdown": "Hi @pehahn. The ISO 8601 timestamp below is in the time zone \"EDT(-4h)\" and represents the local time 2018-08-14 22:26:00. For this sleep task, I think it would be better to use 22:26 local time.\n\n> 2018-08-14T22:26:00-0400", "votes": null } ], "comments": [ { "id": 2442137, "author_name": "mpwolke", "author_url": "", "post_date": "09/16/2023 18:39:44", "content": "2018-08-14T22:26:00-0400
\n
Hi Peter,
\n\"There were some series affected by a bug in DST handling that caused events after a time change to be incorrectly offset by an hour. Apparently, this only affected series in which a DST occurred (that is, when the time zone offset changed from \"-400\" to \"-500\"), which is 36 series in the training set (perhaps 6% of all events).\"
\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697
those results are on my Notebook (I Published at the beginning of the competition). However, I copied Chun Fu that was the 1st to deliver the timestamp's parsing:
\nBy Chun Fu https://www.kaggle.com/code/patrick0302/viz-of-sleeping-time-series
\nThose guys below, dealed with timestamps successfully too. I didn't check if there were difference of time on their work.
\nCMI Sleep Visualization - By Connor
\nhttps://www.kaggle.com/code/clane9/cmi-sleep-visualization
Sleeping patterns - EDA - By Robert Kwiatkowski
\nhttps://www.kaggle.com/code/datark1/sleeping-patterns-eda
Xgboost Notebook Updated - By Anmar Ahmadi Qazi
\nhttps://www.kaggle.com/code/ammarahmadqazi/xgboost-notebook-updated
204_Preprocessing_Data - By Alvaro Rodriguez
\nhttps://www.kaggle.com/code/alvrod/204-preprocessing-data
You're welcome Peter,
\nKind Regards,
\nMarília
Hi Marilia,
\nThanks for your help. That make it more complicated for me. Reading the parquet file, I don't see the offset. If their are different offsets as you mentioned, I must figure out how to detect them. On the flip side submission requires only step not time.
\nKind regards
\nPeter
\nP.S. Still searching for the apostroph i on my Mac to get your name correct😘
You can write without the accent ´
\nSometimes (purchases) I avoid to type the accent cause computers don't understand, then it appears \"Marlia\" or something like that. And I'm afraid that the parcel won't be delivered. So even myself, write Marilia without the accent.
", "votes": null, "replies": [] } ] } ] }, { "id": 2442314, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "09/16/2023 22:18:51", "content": "Hi @pehahn. The ISO 8601 timestamp below is in the time zone \"EDT(-4h)\" and represents the local time 2018-08-14 22:26:00. For this sleep task, I think it would be better to use 22:26 local time.
\n\n", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2441886": "Working on my EDA I compared my data with several notebooks. I live in Germany in I am working with R. For the events file my first timestamp gives: \n2018-08-15 02:26:00\t\nMarlia Prata @mpwolke [notebook](https://www.kaggle.com/code/mpwolke/zzzs-wake-up-kagglers) gives: 2018-08-14 22:26:00 \nAll timestamps have 4 hours difference. I can see the 4 hours difference after the timestamp according to ISO 8601 in Marilia's notebook:\n2018-08-14T22:26:00**-0400**,\nbut not when I read the data with read_parquet from the library(arrow).\nIt's easy to add 4 hours, but any other solutions?\nThanks Peter", "2442137": "Hi Peter,\n\n#Ryan Holbrook posted that 1 day ago: (INCORRECTLY OFFSET BY AN HOUR.)\n\n\"There were some series affected by a bug in DST handling that caused events after a time change to be incorrectly offset by an hour. Apparently, this only affected series in which a DST occurred (that is, when the time zone offset changed from \"-400\" to \"-500\"), which is 36 series in the training set (perhaps 6% of all events).\"\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697\n\nthose results are on my Notebook (I Published at the beginning of the competition). However, I copied Chun Fu that was the 1st to deliver the timestamp's parsing:\n\nBy Chun Fu https://www.kaggle.com/code/patrick0302/viz-of-sleeping-time-series \n\nThose guys below, dealed with timestamps successfully too. I didn't check if there were difference of time on their work.\n\nCMI Sleep Visualization - By Connor\nhttps://www.kaggle.com/code/clane9/cmi-sleep-visualization\n\nSleeping patterns - EDA - By Robert Kwiatkowski\nhttps://www.kaggle.com/code/datark1/sleeping-patterns-eda\n\nXgboost Notebook Updated - By Anmar Ahmadi Qazi\nhttps://www.kaggle.com/code/ammarahmadqazi/xgboost-notebook-updated\n\n204_Preprocessing_Data - By Alvaro Rodriguez\nhttps://www.kaggle.com/code/alvrod/204-preprocessing-data\n\nYou're welcome Peter,\n\nKind Regards,\nMarília", "2442148": "Hi Marilia,\nThanks for your help. That make it more complicated for me. Reading the parquet file, I don't see the offset. If their are different offsets as you mentioned, I must figure out how to detect them. On the flip side submission requires only step not time. \nKind regards\nPeter\nP.S. Still searching for the apostroph i on my Mac to get your name correct😘", "2442158": "You can write without the accent ´ \n\nSometimes (purchases) I avoid to type the accent cause computers don't understand, then it appears \"Marlia\" or something like that. And I'm afraid that the parcel won't be delivered. So even myself, write Marilia without the accent.", "2442314": "Hi @pehahn. The ISO 8601 timestamp below is in the time zone \"EDT(-4h)\" and represents the local time 2018-08-14 22:26:00. For this sleep task, I think it would be better to use 22:26 local time.\n\n> 2018-08-14T22:26:00-0400" }, "source": "meta" }2018-08-14T22:26:00-0400
\n