{
  "id": 5212,
  "title": "Nature of time stamps",
  "url": "/competitions/accelerometer-biometric-competition/discussion/5212",
  "author_name": "",
  "post_date": "2013-07-27T02:57:21.807Z",
  "votes": null,
  "comment_count": 18,
  "views": 6703,
  "content": "<p>Are the time stamps all GMT? &nbsp;That is, not adjusted for local time or daylight savings?</p>",
  "messages": [
    {
      "id": "27711",
      "postDate": "07/27/2013 02:57:21",
      "content": "<p>Are the time stamps all GMT? &nbsp;That is, not adjusted for local time or daylight savings?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "27715",
      "postDate": "07/27/2013 06:02:10",
      "content": "<p>No. Timestamps were derived by the collecting app using the function: java.lang.System.currentTimeMillis(). This function returns the current local system time in milliseconds since January 1, 1970 00:00:00 UTC.</p>\n<p>Note however that since our objective is to differentiate devices by X,Y,Z accelerometer readings, raw timestamps were adjusted so that collection times don't give away devices in the test set. </p>\n<p>The following steps were applied to make the adjustment:<br>1. Evaluate earliest sample in the train set &amp; calculate a day offset (/86,400,000 )<br>2. Adjust the start time of each device in the train set so that all start on the same day (without impacting time of day)<br>3. Adjust the T values of all samples in the train set relative to adjusted device start times<br>4. Evaluate oldest sample in the adjusted train set &amp; calculate it's day offset<br>5. Adjust the start time of each device in the test set so that all start one day after the oldest train sample<br>6. Adjust the T values of all samples in the test set relative to the adjusted device start times</p>\n<p>So, while the date part of each timestamp has been adjusted, time of day is not impacted, leaving contestants with the capability to investigate a time based behavioral monitoring approach if they desire.</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "27804",
      "postDate": "07/30/2013 02:41:07",
      "content": "<p>That is a spectacular solution. &nbsp;I was going for exactly what you noted in that last sentence. &nbsp;Time of day behaviors seem a very legitimate biometric, yet the date part would allow extreme gaming of the contest here. &nbsp;Well designed.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "27821",
      "postDate": "07/30/2013 07:54:57",
      "content": "<p>Then, is that true that a day of the week information being lost with these timestamps adjustments?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "27835",
      "postDate": "07/30/2013 14:31:30",
      "content": "<p>[quote=Pavel Senin;27821]</p>\n<p>Then, is that true that a day of the week information being lost with these timestamps adjustments?</p>\n<p>[/quote]</p>\n<p>Yes, that is true.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28091",
      "postDate": "08/02/2013 21:47:38",
      "content": "<p>I note some time stamps have been converted to scientific form and thus lost resolution, eg:&nbsp;</p>\n<p>1.336733e+12,-10.460427,1.1537236,-0.19228727,698</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28145",
      "postDate": "08/03/2013 18:25:19",
      "content": "<p>Are features based off of the device polling/sampling rates allowed? It's already come up once as something that a user (adveboy) consider to be cheating and I agree it isn't in the spirit of the bio-metric part of the challenge.<br><br>Official clarification would be great!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28149",
      "postDate": "08/03/2013 20:36:56",
      "content": "<p>Totally agree with Alec's concern! &nbsp;One can very much game this particular competition because the update rate signatures of the devices themselves are non-identical. &nbsp;Not sure how one stops that in such a competition. &nbsp;Can the publisher of the particular contest outlaw such behaviors? &nbsp;Or is anything one can get from the data fair game? &nbsp;Can the publisher put out such a rule--which they cannot detect the violation of until the contest is over, then disqualify the here-to-fore money winners? &nbsp; I'm new to Kaggle, so forgive the newbie ignorance.</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28150",
      "postDate": "08/03/2013 20:43:04",
      "content": "<p>Actually, I took it upon myself to only compete using outer data (meaning, ignoring the accelerometer readings completely). Makes this competition a bit more interesting, and I want to prove a point :)</p>\n<p>&nbsp;</p>\n<p>[so far, so good - you can get to at least 0.69 ignoring the actual data]</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28151",
      "postDate": "08/03/2013 20:50:22",
      "content": "<p>Wow. &nbsp;Good rigor. &nbsp;That's pretty significant since that number roughly splits the random benchmark and the leader. &nbsp;One question though--ignoring accelerometer still takes into account time-of-day rhythms, which per the thread above is allowed. &nbsp;So, how much of the 0.20 of that delta from the benchmark to your outer data test is due to time of day? &nbsp;Which, I believe no one is disputing is legitimate data.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28152",
      "postDate": "08/03/2013 20:54:59",
      "content": "<p>[quote=James Madison;28151]</p>\n<p>&nbsp;So, how much of the 0.20 of that delta from the benchmark to your outer data test is due to time of day? &nbsp;</p>\n<p>[/quote]</p>\n<p>&nbsp;</p>\n<p>you are going to like it - none.</p>\n<p>(for now, I plan to add it later). Currently I believe there's even a more significant target-leak in the data, but I'm still trying to understand how to exploit it :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28154",
      "postDate": "08/03/2013 20:59:27",
      "content": "<p>Thank you Alec for this question.</p>\n<p>We didn't officially ban using features derived from sampling rates as we decided this may be too cumbersome to enforce. We are obviously looking for models that can differentiate users from the accelerometer data &amp; have attempted to prepare the data so that sampling rates don't leak the device. If anyone does discover useful features derived solely from the time variable, please share these leaks so that we have a fair competition based on honest methods with everyone starting at the same point.</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28155",
      "postDate": "08/03/2013 21:01:11",
      "content": "<p>Alright, good to know, thanks for clarifying!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28165",
      "postDate": "08/04/2013 03:35:12",
      "content": "<p>Actually there's quite significant leak from the timestamp data. I experimented with the sampling rate feature: without it my model produced 0.86575 in the leaderboard score, and with it, my score is 0.90172. A difference of about 0.036.</p>\n<p>As suggested by Geoff, I'd like to share my feature variable. My sampling rate feature is just the mean of differences of consecutive timestamp values after discarding top &amp; bottom 10%.</p>\n<p>In R, it's</p>\n<p>mean(diff(dataset$T), trim=0.1)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28166",
      "postDate": "08/04/2013 03:58:31",
      "content": "<p>[quote=Alec Radford;28145]</p>\n<p>Are features based off of the device polling/sampling rates allowed? It's already come up once as something that a user (adveboy) consider to be cheating and I agree it isn't in the spirit of the bio-metric part of the challenge.<br><br>Official clarification would be great!</p>\n<p>[/quote]</p>\n<p>It was even simpler than sampling rate usage. The training and testing data from the same device seem to be recorded at the same time (the statement is rough, that's just for understanding the idea that we can get some significant information from absolute time records). Evidently that this information is much less&nbsp;relevant than sampling rate for the goal...</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28211",
      "postDate": "08/05/2013 15:24:10",
      "content": "<p>[quote=mlearn;28091]</p>\n<p>I note some time stamps have been converted to scientific form and thus lost resolution, eg:&nbsp;</p>\n<p>1.336733e+12,-10.460427,1.1537236,-0.19228727,698</p>\n<p>[/quote]</p>\n<p>I see them now. Given there are so few of them, I don't think it warrants a data re-release. Consider it noise and sorry for any parsing headaches they create.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28888",
      "postDate": "08/18/2013 05:57:27",
      "content": "<p>[quote=Geoff;27715]</p>\n<p>The following steps were applied to make the adjustment:<br>1. Evaluate earliest sample in the train set &amp; calculate a day offset (/86,400,000 )<br>2. Adjust the start time of each device in the train set so that all start on the same day (without impacting time of day)<br>3. Adjust the T values of all samples in the train set relative to adjusted device start times<br><strong>4. Evaluate oldest sample in the adjusted train set &amp; calculate it's day offset</strong><br>5. Adjust the start time of each device in the test set so that all start one day after the oldest train sample<br>6. Adjust the T values of all samples in the test set relative to the adjusted device start times</p>\n<p>So, while the date part of each timestamp has been adjusted, time of day is not impacted, leaving contestants with the capability to investigate a time based behavioral monitoring approach if they desire.[/quote]</p>\n<p>Hi Geoff - about this, so if I'm interpreting correctly there shouldn't be any 'T' in the test afterword the oldest sample in the training set, am I wrong?</p>\n<p>After parsing the files in Python using pandas I do have these two T values in my set:</p>\n<ul>\n<li><span style=\"line-height: 1.4\"><strong>1362485512294</strong> - at row 11932599 in the training set, which should correspond to Tue, 05 Mar 2013 12:11:52</span></li>\n<li><span style=\"line-height: 1.4\"><strong>1362441727675</strong> - at row 16280100&nbsp;in the test set, which should correspond to Tue, 05 Mar 2013 00:02:07 GMT</span></li>\n</ul>\n<p><span style=\"line-height: 1.4\">...which is almost 12 hours later. Am I the only one? I'm asking because I'm looking at patterns at different times of the day, and I do have some improvements in my CV, but at the time of testing on the public leaderboard I have an opposite trend :|</span></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28912",
      "postDate": "08/18/2013 20:38:54",
      "content": "<p>Alessandro, you are correct. There are&nbsp;2452097 samples in the test set that have T value earlier than the oldest sample in the train set. Maximum overlap is 12 hours.</p>\n<p>There appears to have been an error in the offset used for the date adjustment at step 5. Time of day should not be impacted.</p>\n<p>I apologize for the incorrect information provided.</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "28914",
      "postDate": "08/18/2013 21:06:02",
      "content": "<p>as long as time of day is not impacted, is not really a big deal!! Usually, when there are this sort of errors is a major indication that something else went wrong, so asking is a must - thanks for checking! xD</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 27715,
      "author_name": "geoff3",
      "author_url": "",
      "post_date": "07/27/2013 06:02:10",
      "content": "<p>No. Timestamps were derived by the collecting app using the function: java.lang.System.currentTimeMillis(). This function returns the current local system time in milliseconds since January 1, 1970 00:00:00 UTC.</p>\n<p>Note however that since our objective is to differentiate devices by X,Y,Z accelerometer readings, raw timestamps were adjusted so that collection times don't give away devices in the test set. </p>\n<p>The following steps were applied to make the adjustment:<br>1. Evaluate earliest sample in the train set &amp; calculate a day offset (/86,400,000 )<br>2. Adjust the start time of each device in the train set so that all start on the same day (without impacting time of day)<br>3. Adjust the T values of all samples in the train set relative to adjusted device start times<br>4. Evaluate oldest sample in the adjusted train set &amp; calculate it's day offset<br>5. Adjust the start time of each device in the test set so that all start one day after the oldest train sample<br>6. Adjust the T values of all samples in the test set relative to the adjusted device start times</p>\n<p>So, while the date part of each timestamp has been adjusted, time of day is not impacted, leaving contestants with the capability to investigate a time based behavioral monitoring approach if they desire.</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 27804,
      "author_name": "jamesmadison",
      "author_url": "",
      "post_date": "07/30/2013 02:41:07",
      "content": "<p>That is a spectacular solution. &nbsp;I was going for exactly what you noted in that last sentence. &nbsp;Time of day behaviors seem a very legitimate biometric, yet the date part would allow extreme gaming of the contest here. &nbsp;Well designed.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 27821,
      "author_name": "seninp",
      "author_url": "",
      "post_date": "07/30/2013 07:54:57",
      "content": "<p>Then, is that true that a day of the week information being lost with these timestamps adjustments?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 27835,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "07/30/2013 14:31:30",
      "content": "<p>[quote=Pavel Senin;27821]</p>\n<p>Then, is that true that a day of the week information being lost with these timestamps adjustments?</p>\n<p>[/quote]</p>\n<p>Yes, that is true.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28091,
      "author_name": "mlearn",
      "author_url": "",
      "post_date": "08/02/2013 21:47:38",
      "content": "<p>I note some time stamps have been converted to scientific form and thus lost resolution, eg:&nbsp;</p>\n<p>1.336733e+12,-10.460427,1.1537236,-0.19228727,698</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28145,
      "author_name": "alecradford",
      "author_url": "",
      "post_date": "08/03/2013 18:25:19",
      "content": "<p>Are features based off of the device polling/sampling rates allowed? It's already come up once as something that a user (adveboy) consider to be cheating and I agree it isn't in the spirit of the bio-metric part of the challenge.<br><br>Official clarification would be great!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28149,
      "author_name": "jamesmadison",
      "author_url": "",
      "post_date": "08/03/2013 20:36:56",
      "content": "<p>Totally agree with Alec's concern! &nbsp;One can very much game this particular competition because the update rate signatures of the devices themselves are non-identical. &nbsp;Not sure how one stops that in such a competition. &nbsp;Can the publisher of the particular contest outlaw such behaviors? &nbsp;Or is anything one can get from the data fair game? &nbsp;Can the publisher put out such a rule--which they cannot detect the violation of until the contest is over, then disqualify the here-to-fore money winners? &nbsp; I'm new to Kaggle, so forgive the newbie ignorance.</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28150,
      "author_name": "roulinir",
      "author_url": "",
      "post_date": "08/03/2013 20:43:04",
      "content": "<p>Actually, I took it upon myself to only compete using outer data (meaning, ignoring the accelerometer readings completely). Makes this competition a bit more interesting, and I want to prove a point :)</p>\n<p>&nbsp;</p>\n<p>[so far, so good - you can get to at least 0.69 ignoring the actual data]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28151,
      "author_name": "jamesmadison",
      "author_url": "",
      "post_date": "08/03/2013 20:50:22",
      "content": "<p>Wow. &nbsp;Good rigor. &nbsp;That's pretty significant since that number roughly splits the random benchmark and the leader. &nbsp;One question though--ignoring accelerometer still takes into account time-of-day rhythms, which per the thread above is allowed. &nbsp;So, how much of the 0.20 of that delta from the benchmark to your outer data test is due to time of day? &nbsp;Which, I believe no one is disputing is legitimate data.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28152,
      "author_name": "roulinir",
      "author_url": "",
      "post_date": "08/03/2013 20:54:59",
      "content": "<p>[quote=James Madison;28151]</p>\n<p>&nbsp;So, how much of the 0.20 of that delta from the benchmark to your outer data test is due to time of day? &nbsp;</p>\n<p>[/quote]</p>\n<p>&nbsp;</p>\n<p>you are going to like it - none.</p>\n<p>(for now, I plan to add it later). Currently I believe there's even a more significant target-leak in the data, but I'm still trying to understand how to exploit it :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28154,
      "author_name": "geoff3",
      "author_url": "",
      "post_date": "08/03/2013 20:59:27",
      "content": "<p>Thank you Alec for this question.</p>\n<p>We didn't officially ban using features derived from sampling rates as we decided this may be too cumbersome to enforce. We are obviously looking for models that can differentiate users from the accelerometer data &amp; have attempted to prepare the data so that sampling rates don't leak the device. If anyone does discover useful features derived solely from the time variable, please share these leaks so that we have a fair competition based on honest methods with everyone starting at the same point.</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28155,
      "author_name": "alecradford",
      "author_url": "",
      "post_date": "08/03/2013 21:01:11",
      "content": "<p>Alright, good to know, thanks for clarifying!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28165,
      "author_name": "tyi2000",
      "author_url": "",
      "post_date": "08/04/2013 03:35:12",
      "content": "<p>Actually there's quite significant leak from the timestamp data. I experimented with the sampling rate feature: without it my model produced 0.86575 in the leaderboard score, and with it, my score is 0.90172. A difference of about 0.036.</p>\n<p>As suggested by Geoff, I'd like to share my feature variable. My sampling rate feature is just the mean of differences of consecutive timestamp values after discarding top &amp; bottom 10%.</p>\n<p>In R, it's</p>\n<p>mean(diff(dataset$T), trim=0.1)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28166,
      "author_name": "",
      "author_url": "",
      "post_date": "08/04/2013 03:58:31",
      "content": "<p>[quote=Alec Radford;28145]</p>\n<p>Are features based off of the device polling/sampling rates allowed? It's already come up once as something that a user (adveboy) consider to be cheating and I agree it isn't in the spirit of the bio-metric part of the challenge.<br><br>Official clarification would be great!</p>\n<p>[/quote]</p>\n<p>It was even simpler than sampling rate usage. The training and testing data from the same device seem to be recorded at the same time (the statement is rough, that's just for understanding the idea that we can get some significant information from absolute time records). Evidently that this information is much less&nbsp;relevant than sampling rate for the goal...</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28211,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "08/05/2013 15:24:10",
      "content": "<p>[quote=mlearn;28091]</p>\n<p>I note some time stamps have been converted to scientific form and thus lost resolution, eg:&nbsp;</p>\n<p>1.336733e+12,-10.460427,1.1537236,-0.19228727,698</p>\n<p>[/quote]</p>\n<p>I see them now. Given there are so few of them, I don't think it warrants a data re-release. Consider it noise and sorry for any parsing headaches they create.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28888,
      "author_name": "alzmcr",
      "author_url": "",
      "post_date": "08/18/2013 05:57:27",
      "content": "<p>[quote=Geoff;27715]</p>\n<p>The following steps were applied to make the adjustment:<br>1. Evaluate earliest sample in the train set &amp; calculate a day offset (/86,400,000 )<br>2. Adjust the start time of each device in the train set so that all start on the same day (without impacting time of day)<br>3. Adjust the T values of all samples in the train set relative to adjusted device start times<br><strong>4. Evaluate oldest sample in the adjusted train set &amp; calculate it's day offset</strong><br>5. Adjust the start time of each device in the test set so that all start one day after the oldest train sample<br>6. Adjust the T values of all samples in the test set relative to the adjusted device start times</p>\n<p>So, while the date part of each timestamp has been adjusted, time of day is not impacted, leaving contestants with the capability to investigate a time based behavioral monitoring approach if they desire.[/quote]</p>\n<p>Hi Geoff - about this, so if I'm interpreting correctly there shouldn't be any 'T' in the test afterword the oldest sample in the training set, am I wrong?</p>\n<p>After parsing the files in Python using pandas I do have these two T values in my set:</p>\n<ul>\n<li><span style=\"line-height: 1.4\"><strong>1362485512294</strong> - at row 11932599 in the training set, which should correspond to Tue, 05 Mar 2013 12:11:52</span></li>\n<li><span style=\"line-height: 1.4\"><strong>1362441727675</strong> - at row 16280100&nbsp;in the test set, which should correspond to Tue, 05 Mar 2013 00:02:07 GMT</span></li>\n</ul>\n<p><span style=\"line-height: 1.4\">...which is almost 12 hours later. Am I the only one? I'm asking because I'm looking at patterns at different times of the day, and I do have some improvements in my CV, but at the time of testing on the public leaderboard I have an opposite trend :|</span></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28912,
      "author_name": "geoff3",
      "author_url": "",
      "post_date": "08/18/2013 20:38:54",
      "content": "<p>Alessandro, you are correct. There are&nbsp;2452097 samples in the test set that have T value earlier than the oldest sample in the train set. Maximum overlap is 12 hours.</p>\n<p>There appears to have been an error in the offset used for the date adjustment at step 5. Time of day should not be impacted.</p>\n<p>I apologize for the incorrect information provided.</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 28914,
      "author_name": "alzmcr",
      "author_url": "",
      "post_date": "08/18/2013 21:06:02",
      "content": "<p>as long as time of day is not impacted, is not really a big deal!! Usually, when there are this sort of errors is a major indication that something else went wrong, so asking is a must - thanks for checking! xD</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "27711": "",
    "27715": "",
    "27804": "",
    "27821": "",
    "27835": "",
    "28091": "",
    "28145": "",
    "28149": "",
    "28150": "",
    "28151": "",
    "28152": "",
    "28154": "",
    "28155": "",
    "28165": "",
    "28166": "",
    "28211": "",
    "28888": "",
    "28912": "",
    "28914": ""
  },
  "source": "meta"
}