{
  "id": 13818,
  "title": "Time Obfuscation",
  "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/13818",
  "author_name": "",
  "post_date": "2015-04-27T22:08:14.503Z",
  "votes": 7,
  "comment_count": 20,
  "views": 6486,
  "content": "<p>Does time-obfuscation maintain monotonicity?</p>\n<p>That is, if original times t1 and t2 had relation t1 &lt; t2, then do their obfuscated counterparts T1 and T2 also satisfy T1 &lt; T2?</p>",
  "messages": [
    {
      "id": "75133",
      "postDate": "04/27/2015 22:08:14",
      "content": "<p>Does time-obfuscation maintain monotonicity?</p>\n<p>That is, if original times t1 and t2 had relation t1 &lt; t2, then do their obfuscated counterparts T1 and T2 also satisfy T1 &lt; T2?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75181",
      "postDate": "04/28/2015 04:54:47",
      "content": "<p>I'm not entirely sure but from some simple playing around i've done im willing to guess so since there seems to be a few specific instances with multiple bids placed together (same time) with no bids following, presumably the last second bids before auction closes.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75191",
      "postDate": "04/28/2015 06:08:18",
      "content": "<p>Yes, the order of time points were kept the same during&nbsp;obfuscation.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75196",
      "postDate": "04/28/2015 06:53:53",
      "content": "<p>Is it possible to get the resolution of the feature? i.e., a difference in one(1) unit is equal to&nbsp;something (1 second?) in real life?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75215",
      "postDate": "04/28/2015 10:33:32",
      "content": "<p>[quote=Sudeep Juvekar;75133]</p>\n<p>Does time-obfuscation maintain monotonicity?</p>\n<p>That is, if original times t1 and t2 had relation t1 &lt; t2, then do their obfuscated counterparts T1 and T2 also satisfy T1 &lt; T2?</p>\n<p>[/quote]</p>\n<p>Is comparing time according to lexicographical order correct?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75257",
      "postDate": "04/28/2015 14:47:11",
      "content": "<p>are times across different bidders comparable? i.e. if i compare two obfuscated times belonging to different users, can I order them in the natural way, or it does not make any sense?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75289",
      "postDate": "04/28/2015 17:34:43",
      "content": "<p>@Antonio Massaro&nbsp;Yes you can compare them across different bidders.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75729",
      "postDate": "04/30/2015 19:55:57",
      "content": "<p>How reliable are the deltas between the transformed time? That is to say, does (transformed time y)&nbsp;- (transformed time x) match&nbsp;(original time y) - (original time x) with&nbsp;any degree of accuracy?&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75768",
      "postDate": "04/30/2015 21:23:38",
      "content": "<p>Or rather if the transformation was not an isometry, has it at least been an isometry with additional scale factor?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75800",
      "postDate": "04/30/2015 23:24:03",
      "content": "<p>Not to be a pessimist here but... I think we will have to find out all these (and future ones) questions by our own.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75926",
      "postDate": "05/01/2015 15:44:58",
      "content": "<p>I would just make the easyest assumptions, that is an affine transformation applied&nbsp; all the time stamps. Any other transformation would modify the dependencies between data in some complicated way..</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "76481",
      "postDate": "05/04/2015 01:35:58",
      "content": "<p>I am skeptical of the point of obfuscating this field. affine transformation just fucked any possibility of looking at the time of day of bids.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77181",
      "postDate": "05/05/2015 23:54:45",
      "content": "<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77368",
      "postDate": "05/06/2015 17:56:16",
      "content": "<p>[quote=JLStamper;77181]</p>\n<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>\n<p>[/quote]</p>\n<p>Well, they did include the address and payment_account fields, which turned out to be completely redundant.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77386",
      "postDate": "05/06/2015 19:37:26",
      "content": "<p>[quote=cfh;77368]</p>\n<p>[quote=JLStamper;77181]</p>\n<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>\n<p>[/quote]</p>\n<p>Well, they did include the address and payment_account fields, which turned out to be completely redundant.</p>\n<p>[/quote]</p>\n\n<p>I'm guessing before they got hashed they were actually a bit useful (email domains and zipcode). &nbsp;I'm also guessing if anyone figures out any pattern in their obfuscation of these features though they won't be posting about it on the forums.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80460",
      "postDate": "05/30/2015 15:17:52",
      "content": "<p>[quote=Wendy Kan;75191]</p>\n<p>Yes, the order of time points were kept the same during&nbsp;obfuscation.&nbsp;</p>\n<p>[/quote]</p>\n<p>Hello Wendy,</p>\n<p>Sorry to bring back the old question over again but when I plot time again bid_id, it isn't monotonous (cf attached image). So if the time column reflect order of the bid then the bid_id column is not in the chronological order or vice versa?Otherwise there are 3 datasets with distinct time and id and distinct auction/bidders?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80492",
      "postDate": "05/31/2015 01:27:34",
      "content": "<p>@Lam Dang,</p>\n<p>Good observation. It is indeed aggregated from 3 datasets with 3 different starting dates. In the plot you just provided&nbsp;you can see that the dates aren't overlapping. So that tells us:</p>\n<p>1. the time column reflects the order of the bid</p>\n<p>2. the bid id doesn't reflect the order of the bid</p>\n<p>3. there are 3 datasets but they all have unique id and unique timepoint that are not overlapping</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80543",
      "postDate": "05/31/2015 21:02:22",
      "content": "<p>Thanks for your answer. Now it is finally clear.</p>\n<p>Based on my observation, the data consists of 3 different sets. Each set contains all the bids in a selected period define by a start date-time and a end date-time. Actually for one of the three dataset, it looks like there are 2 different end date, so it could even be 4 sets.</p>\n<p>This is truncated data (the auctions with bids before or after the selected periods are not complete) &nbsp;which make global variables (auction length, total number of bids) irrelevant..</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80603",
      "postDate": "06/01/2015 19:12:01",
      "content": "<p>@ Lam, why do you assume the auction data is truncated?&nbsp; Wouldn't they just end when someone wins?&nbsp; Couldn't the data obfuscation just make multiple auctions all have the same &quot;end&quot; time for each batch of auction?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80623",
      "postDate": "06/01/2015 22:17:24",
      "content": "<p>@Micheal: When I plot the time of the first bid and the last bid of the auction, I have a lot of border effect around the min(time) and max(time) of each data set (cf. attached).</p>\n<ul>\n<li>Naturally on the penny auction websites, there are auctions starting and ending every day, so we should expect the distribution to be quite homogenous</li>\n<li>One can always argue that the data is transformed/chosen on purpose but I strongly feel that the actual situation is quite simple: the data are extracted using a start and end limit on date of the bids.</li>\n</ul>\n<p>The consequence is that any feature using auction duration or auction start/end time as such would be highly random. If anyone has any kind of features like that proven significant then maybe I am wrong :)</p>\n<p>P.S : I also think that this kind of property of the data set should be stated in the data description to avoid ppl losing time on irrelevant features</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80626",
      "postDate": "06/02/2015 00:15:01",
      "content": "<p>Ah, got it.&nbsp; Thanks Lam!</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 75181,
      "author_name": "hassiktir",
      "author_url": "",
      "post_date": "04/28/2015 04:54:47",
      "content": "<p>I'm not entirely sure but from some simple playing around i've done im willing to guess so since there seems to be a few specific instances with multiple bids placed together (same time) with no bids following, presumably the last second bids before auction closes.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75191,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "04/28/2015 06:08:18",
      "content": "<p>Yes, the order of time points were kept the same during&nbsp;obfuscation.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75196,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "04/28/2015 06:53:53",
      "content": "<p>Is it possible to get the resolution of the feature? i.e., a difference in one(1) unit is equal to&nbsp;something (1 second?) in real life?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75215,
      "author_name": "",
      "author_url": "",
      "post_date": "04/28/2015 10:33:32",
      "content": "<p>[quote=Sudeep Juvekar;75133]</p>\n<p>Does time-obfuscation maintain monotonicity?</p>\n<p>That is, if original times t1 and t2 had relation t1 &lt; t2, then do their obfuscated counterparts T1 and T2 also satisfy T1 &lt; T2?</p>\n<p>[/quote]</p>\n<p>Is comparing time according to lexicographical order correct?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75257,
      "author_name": "antoniomassaro",
      "author_url": "",
      "post_date": "04/28/2015 14:47:11",
      "content": "<p>are times across different bidders comparable? i.e. if i compare two obfuscated times belonging to different users, can I order them in the natural way, or it does not make any sense?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75289,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "04/28/2015 17:34:43",
      "content": "<p>@Antonio Massaro&nbsp;Yes you can compare them across different bidders.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75729,
      "author_name": "jlstamper",
      "author_url": "",
      "post_date": "04/30/2015 19:55:57",
      "content": "<p>How reliable are the deltas between the transformed time? That is to say, does (transformed time y)&nbsp;- (transformed time x) match&nbsp;(original time y) - (original time x) with&nbsp;any degree of accuracy?&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75768,
      "author_name": "mpekalski",
      "author_url": "",
      "post_date": "04/30/2015 21:23:38",
      "content": "<p>Or rather if the transformation was not an isometry, has it at least been an isometry with additional scale factor?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75800,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "04/30/2015 23:24:03",
      "content": "<p>Not to be a pessimist here but... I think we will have to find out all these (and future ones) questions by our own.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75926,
      "author_name": "antoniomassaro",
      "author_url": "",
      "post_date": "05/01/2015 15:44:58",
      "content": "<p>I would just make the easyest assumptions, that is an affine transformation applied&nbsp; all the time stamps. Any other transformation would modify the dependencies between data in some complicated way..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 76481,
      "author_name": "g1685786",
      "author_url": "",
      "post_date": "05/04/2015 01:35:58",
      "content": "<p>I am skeptical of the point of obfuscating this field. affine transformation just fucked any possibility of looking at the time of day of bids.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77181,
      "author_name": "jlstamper",
      "author_url": "",
      "post_date": "05/05/2015 23:54:45",
      "content": "<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77368,
      "author_name": "cfhlearns",
      "author_url": "",
      "post_date": "05/06/2015 17:56:16",
      "content": "<p>[quote=JLStamper;77181]</p>\n<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>\n<p>[/quote]</p>\n<p>Well, they did include the address and payment_account fields, which turned out to be completely redundant.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77386,
      "author_name": "hassiktir",
      "author_url": "",
      "post_date": "05/06/2015 19:37:26",
      "content": "<p>[quote=cfh;77368]</p>\n<p>[quote=JLStamper;77181]</p>\n<p>There must be something more than order being preserved - otherwise why bother? There is already a bid_id which appears to indicate the order. If there isn't something more than that, it wouldn't be there. They wouldn't toss it in as a red herring, would they?</p>\n<p>[/quote]</p>\n<p>Well, they did include the address and payment_account fields, which turned out to be completely redundant.</p>\n<p>[/quote]</p>\n\n<p>I'm guessing before they got hashed they were actually a bit useful (email domains and zipcode). &nbsp;I'm also guessing if anyone figures out any pattern in their obfuscation of these features though they won't be posting about it on the forums.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80460,
      "author_name": "lamdang",
      "author_url": "",
      "post_date": "05/30/2015 15:17:52",
      "content": "<p>[quote=Wendy Kan;75191]</p>\n<p>Yes, the order of time points were kept the same during&nbsp;obfuscation.&nbsp;</p>\n<p>[/quote]</p>\n<p>Hello Wendy,</p>\n<p>Sorry to bring back the old question over again but when I plot time again bid_id, it isn't monotonous (cf attached image). So if the time column reflect order of the bid then the bid_id column is not in the chronological order or vice versa?Otherwise there are 3 datasets with distinct time and id and distinct auction/bidders?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80492,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "05/31/2015 01:27:34",
      "content": "<p>@Lam Dang,</p>\n<p>Good observation. It is indeed aggregated from 3 datasets with 3 different starting dates. In the plot you just provided&nbsp;you can see that the dates aren't overlapping. So that tells us:</p>\n<p>1. the time column reflects the order of the bid</p>\n<p>2. the bid id doesn't reflect the order of the bid</p>\n<p>3. there are 3 datasets but they all have unique id and unique timepoint that are not overlapping</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80543,
      "author_name": "lamdang",
      "author_url": "",
      "post_date": "05/31/2015 21:02:22",
      "content": "<p>Thanks for your answer. Now it is finally clear.</p>\n<p>Based on my observation, the data consists of 3 different sets. Each set contains all the bids in a selected period define by a start date-time and a end date-time. Actually for one of the three dataset, it looks like there are 2 different end date, so it could even be 4 sets.</p>\n<p>This is truncated data (the auctions with bids before or after the selected periods are not complete) &nbsp;which make global variables (auction length, total number of bids) irrelevant..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80603,
      "author_name": "thawatt",
      "author_url": "",
      "post_date": "06/01/2015 19:12:01",
      "content": "<p>@ Lam, why do you assume the auction data is truncated?&nbsp; Wouldn't they just end when someone wins?&nbsp; Couldn't the data obfuscation just make multiple auctions all have the same &quot;end&quot; time for each batch of auction?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80623,
      "author_name": "lamdang",
      "author_url": "",
      "post_date": "06/01/2015 22:17:24",
      "content": "<p>@Micheal: When I plot the time of the first bid and the last bid of the auction, I have a lot of border effect around the min(time) and max(time) of each data set (cf. attached).</p>\n<ul>\n<li>Naturally on the penny auction websites, there are auctions starting and ending every day, so we should expect the distribution to be quite homogenous</li>\n<li>One can always argue that the data is transformed/chosen on purpose but I strongly feel that the actual situation is quite simple: the data are extracted using a start and end limit on date of the bids.</li>\n</ul>\n<p>The consequence is that any feature using auction duration or auction start/end time as such would be highly random. If anyone has any kind of features like that proven significant then maybe I am wrong :)</p>\n<p>P.S : I also think that this kind of property of the data set should be stated in the data description to avoid ppl losing time on irrelevant features</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80626,
      "author_name": "thawatt",
      "author_url": "",
      "post_date": "06/02/2015 00:15:01",
      "content": "<p>Ah, got it.&nbsp; Thanks Lam!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "75133": "",
    "75181": "",
    "75191": "",
    "75196": "",
    "75215": "",
    "75257": "",
    "75289": "",
    "75729": "",
    "75768": "",
    "75800": "",
    "75926": "",
    "76481": "",
    "77181": "",
    "77368": "",
    "77386": "",
    "80460": "",
    "80492": "",
    "80543": "",
    "80603": "",
    "80623": "",
    "80626": ""
  },
  "source": "meta"
}