![\"enter](\"https://www.kaggle.io/svf/373973/d40122fa65101a0b4887caa0aaca4ac4/train_time_auto_correlation.png\")
Hi, \nim currently trying to engineer some features with the train_date. I suspect that the data is from different days of production, did anyone find a way to get to the days? I plotted the timestamps against the id, but the data seems to be shuffled. Could be a nice way to get a better understanding of the data, when looking at the days individually!
", "messages": [ { "id": "135146", "postDate": "09/11/2016 16:38:51", "content": "Hi, \nim currently trying to engineer some features with the train_date. I suspect that the data is from different days of production, did anyone find a way to get to the days? I plotted the timestamps against the id, but the data seems to be shuffled. Could be a nice way to get a better understanding of the data, when looking at the days individually!
", "rawMarkdown": "Hi, \r\nim currently trying to engineer some features with the train_date. I suspect that the data is from different days of production, did anyone find a way to get to the days? I plotted the timestamps against the id, but the data seems to be shuffled. Could be a nice way to get a better understanding of the data, when looking at the days individually!", "votes": null }, { "id": "135196", "postDate": "09/12/2016 03:05:11", "content": "@Malte, why do you think the time data is shuffled? The data appears continuous to me. If you look at any one station, there are breaks. You have to look at the overall facility. This appears to be a 24 hour operation. For example look at parameter L3_S30_F3519.
", "rawMarkdown": "Malte, why do you think the time data is shuffled? The data appears continuous to me. If you look at any one station, there are breaks. You have to look at the overall facility. This appears to be a 24 hour operation. For example look at parameter L3_S30_F3519.", "votes": null }, { "id": "135336", "postDate": "09/13/2016 07:15:13", "content": "Thanks Jeff, makes sense. I came to the suspicion because i observed, that when i create time features for lets say the first 300 000 observations, those work nicely in cross validation in this group, however generalize not well to the rest of the observations, much worse than would be expected due to cross validation variance. My first intuition was, that the data might be from different days and that leads to the concept drift.\n But given, that they do not turn off the machines at the same time every day, but more due to disfunctioning as your plots suggest, then it really clearly indicates that is a 24 hour operation. Thanks!
\n\nDid anyone else observe a change of pattern in different splits of the data?
", "rawMarkdown": "Thanks Jeff, makes sense. I came to the suspicion because i observed, that when i create time features for lets say the first 300 000 observations, those work nicely in cross validation in this group, however generalize not well to the rest of the observations, much worse than would be expected due to cross validation variance. My first intuition was, that the data might be from different days and that leads to the concept drift.\r\n But given, that they do not turn off the machines at the same time every day, but more due to disfunctioning as your plots suggest, then it really clearly indicates that is a 24 hour operation. Thanks!\r\n\r\nDid anyone else observe a change of pattern in different splits of the data?", "votes": null }, { "id": "135481", "postDate": "09/14/2016 13:08:44", "content": "If you assume tick size to be 1 second, and maximum is 1728, then it would seem like a 48 hour run. Note the gap at 864.
", "rawMarkdown": "If you assume tick size to be 1 second, and maximum is 1728, then it would seem like a 48 hour run. Note the gap at 864.", "votes": null }, { "id": "135502", "postDate": "09/14/2016 15:27:03", "content": "Glimmung, could you walk me through your math? There're 86,400 seconds in a day, and 1800 seconds in half an hour. Could this actually be production data from just a half hour run?
\n\nThe number after the decimal ranges from 1-99 -- maybe these are units of 0.01 s?
", "rawMarkdown": "Glimmung, could you walk me through your math? There're 86,400 seconds in a day, and 1800 seconds in half an hour. Could this actually be production data from just a half hour run? \r\n\r\nThe number after the decimal ranges from 1-99 -- maybe these are units of 0.01 s?", "votes": null }, { "id": "135535", "postDate": "09/14/2016 20:03:02", "content": "Nothing special. If the tick size is 1 second, then "time" = proper time / 100, and thus maximum would be 48 hours. Some sort of "resonance" analysis gives 10s tick size seems quiet likely yielding a 20 days run. Neither of these gave any improvement in my trials. Any tick size smaller than 1s would be of no interest, I reckon.
", "rawMarkdown": "Nothing special. If the tick size is 1 second, then \"time\" = proper time / 100, and thus maximum would be 48 hours. Some sort of \"resonance\" analysis gives 10s tick size seems quiet likely yielding a 20 days run. Neither of these gave any improvement in my trials. Any tick size smaller than 1s would be of no interest, I reckon.", "votes": null }, { "id": "136720", "postDate": "09/24/2016 10:23:27", "content": "Does this looks like two shifts of a day with each shift of approx. 8+ hrs? The gap at 864 been the change of shift ?
", "rawMarkdown": "Does this looks like two shifts of a day with each shift of approx. 8+ hrs? The gap at 864 been the change of shift ?", "votes": null }, { "id": "136757", "postDate": "09/24/2016 20:48:44", "content": "I would rather say different years. 0.01 tick is probably 6 minutes.
\n\nCheck my script for further explanations
\n\n
Feels like Christmas in here...:-)
", "rawMarkdown": "Feels like Christmas in here...:-)", "votes": null } ], "comments": [ { "id": 135196, "author_name": "jeffhebert", "author_url": "", "post_date": "09/12/2016 03:05:11", "content": "@Malte, why do you think the time data is shuffled? The data appears continuous to me. If you look at any one station, there are breaks. You have to look at the overall facility. This appears to be a 24 hour operation. For example look at parameter L3_S30_F3519.
", "votes": null, "replies": [] }, { "id": 135336, "author_name": "malten", "author_url": "", "post_date": "09/13/2016 07:15:13", "content": "Thanks Jeff, makes sense. I came to the suspicion because i observed, that when i create time features for lets say the first 300 000 observations, those work nicely in cross validation in this group, however generalize not well to the rest of the observations, much worse than would be expected due to cross validation variance. My first intuition was, that the data might be from different days and that leads to the concept drift.\n But given, that they do not turn off the machines at the same time every day, but more due to disfunctioning as your plots suggest, then it really clearly indicates that is a 24 hour operation. Thanks!
\n\nDid anyone else observe a change of pattern in different splits of the data?
", "votes": null, "replies": [] }, { "id": 135481, "author_name": "glimmung", "author_url": "", "post_date": "09/14/2016 13:08:44", "content": "If you assume tick size to be 1 second, and maximum is 1728, then it would seem like a 48 hour run. Note the gap at 864.
", "votes": null, "replies": [] }, { "id": 135502, "author_name": "fuzzylogic", "author_url": "", "post_date": "09/14/2016 15:27:03", "content": "Glimmung, could you walk me through your math? There're 86,400 seconds in a day, and 1800 seconds in half an hour. Could this actually be production data from just a half hour run?
\n\nThe number after the decimal ranges from 1-99 -- maybe these are units of 0.01 s?
", "votes": null, "replies": [] }, { "id": 135535, "author_name": "glimmung", "author_url": "", "post_date": "09/14/2016 20:03:02", "content": "Nothing special. If the tick size is 1 second, then "time" = proper time / 100, and thus maximum would be 48 hours. Some sort of "resonance" analysis gives 10s tick size seems quiet likely yielding a 20 days run. Neither of these gave any improvement in my trials. Any tick size smaller than 1s would be of no interest, I reckon.
", "votes": null, "replies": [] }, { "id": 136720, "author_name": "azacharia", "author_url": "", "post_date": "09/24/2016 10:23:27", "content": "Does this looks like two shifts of a day with each shift of approx. 8+ hrs? The gap at 864 been the change of shift ?
", "votes": null, "replies": [] }, { "id": 136757, "author_name": "gaborfodor", "author_url": "", "post_date": "09/24/2016 20:48:44", "content": "I would rather say different years. 0.01 tick is probably 6 minutes.
\n\nCheck my script for further explanations
\n\n
Feels like Christmas in here...:-)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "135146": "Hi, \r\nim currently trying to engineer some features with the train_date. I suspect that the data is from different days of production, did anyone find a way to get to the days? I plotted the timestamps against the id, but the data seems to be shuffled. Could be a nice way to get a better understanding of the data, when looking at the days individually!", "135196": "Malte, why do you think the time data is shuffled? The data appears continuous to me. If you look at any one station, there are breaks. You have to look at the overall facility. This appears to be a 24 hour operation. For example look at parameter L3_S30_F3519.", "135336": "Thanks Jeff, makes sense. I came to the suspicion because i observed, that when i create time features for lets say the first 300 000 observations, those work nicely in cross validation in this group, however generalize not well to the rest of the observations, much worse than would be expected due to cross validation variance. My first intuition was, that the data might be from different days and that leads to the concept drift.\r\n But given, that they do not turn off the machines at the same time every day, but more due to disfunctioning as your plots suggest, then it really clearly indicates that is a 24 hour operation. Thanks!\r\n\r\nDid anyone else observe a change of pattern in different splits of the data?", "135481": "If you assume tick size to be 1 second, and maximum is 1728, then it would seem like a 48 hour run. Note the gap at 864.", "135502": "Glimmung, could you walk me through your math? There're 86,400 seconds in a day, and 1800 seconds in half an hour. Could this actually be production data from just a half hour run? \r\n\r\nThe number after the decimal ranges from 1-99 -- maybe these are units of 0.01 s?", "135535": "Nothing special. If the tick size is 1 second, then \"time\" = proper time / 100, and thus maximum would be 48 hours. Some sort of \"resonance\" analysis gives 10s tick size seems quiet likely yielding a 20 days run. Neither of these gave any improvement in my trials. Any tick size smaller than 1s would be of no interest, I reckon.", "136720": "Does this looks like two shifts of a day with each shift of approx. 8+ hrs? The gap at 864 been the change of shift ?", "136757": "I would rather say different years. 0.01 tick is probably 6 minutes.\r\n\r\nCheck my [script][1] for further explanations \r\n\r\n![enter image description here][2]\r\n\r\n\r\n [1]: https://www.kaggle.com/gaborfodor/bosch-production-line-performance/notebookd19d11e4f2\r\n [2]: https://www.kaggle.io/svf/373973/d40122fa65101a0b4887caa0aaca4ac4/train_time_auto_correlation.png", "139165": "", "139378": "Feels like Christmas in here...:-)" }, "source": "meta" }