{
  "id": 13874,
  "title": "Obfuscation and Similarity",
  "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/13874",
  "author_name": "",
  "post_date": "2015-04-30T03:36:58.337Z",
  "votes": 3,
  "comment_count": 5,
  "views": 1636,
  "content": "<p>Hi everyone. Obfuscation is not that familiar to me, therefore I was wondering whether a string preserves the lexical similarity after obfuscation. For instance, is it possible that applying Hamming distance to get the most similar word between two original strings gives the same result with an obfuscated version of these strings?</p>",
  "messages": [
    {
      "id": "75560",
      "postDate": "04/30/2015 03:36:58",
      "content": "<p>Hi everyone. Obfuscation is not that familiar to me, therefore I was wondering whether a string preserves the lexical similarity after obfuscation. For instance, is it possible that applying Hamming distance to get the most similar word between two original strings gives the same result with an obfuscated version of these strings?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75588",
      "postDate": "04/30/2015 09:10:14",
      "content": "<p>I don't think so...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77419",
      "postDate": "05/06/2015 23:01:55",
      "content": "<p>I'm pretty sure they're hashes of some sort, though I doubt they're SHA1 or anything...</p>\n\n<p>More information about how the IPs were obfuscated would be nice - are they transformed keeping subnets the same, or is it a useless field?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77449",
      "postDate": "05/07/2015 05:58:24",
      "content": "<p>If the obfuscation is implemented by locality-sensitive hashing,two similar strings will hold the similarity after they're hashed</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77467",
      "postDate": "05/07/2015 10:43:09",
      "content": "<p>My original thought on the address and payment_account field was that they were a hash of some sort, however that didn't sit well:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">38 characters is an odd length for a hash</span></li>\n<li><span style=\"line-height: 1.4\">If all values are unique and a true hash function was used the organisers would know that there is no predictive value to be had and not provided those fields.</span></li>\n</ul>\n<p>Browsing though the data during lunch I noticed an oddity, several values are almost exact except for the last 6 characters. &nbsp;This occurs several times with different different prefixes in both columns, &nbsp;such behavior would be highly suspect in a cypto-hash function.</p>\n<p>Seperating out the first 32 characters and the last 6 looks like an interesting avenue to pursue, which is what I'm doing now.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77484",
      "postDate": "05/07/2015 15:15:28",
      "content": "<p>I think that in a competition like that, when no code should be shared, you also should restrain from sharing insights like that - at least during the competition time. If you browsed through forums of previous competition it is quite easy to guess that the data is there for a reason and as this is can be messy as it is data mining/modelling/learning competition it is about discovering knowledge in the data.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 75588,
      "author_name": "bluesilence",
      "author_url": "",
      "post_date": "04/30/2015 09:10:14",
      "content": "<p>I don't think so...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77419,
      "author_name": "g1685786",
      "author_url": "",
      "post_date": "05/06/2015 23:01:55",
      "content": "<p>I'm pretty sure they're hashes of some sort, though I doubt they're SHA1 or anything...</p>\n\n<p>More information about how the IPs were obfuscated would be nice - are they transformed keeping subnets the same, or is it a useless field?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77449,
      "author_name": "ab19026",
      "author_url": "",
      "post_date": "05/07/2015 05:58:24",
      "content": "<p>If the obfuscation is implemented by locality-sensitive hashing,two similar strings will hold the similarity after they're hashed</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77467,
      "author_name": "danielthomas",
      "author_url": "",
      "post_date": "05/07/2015 10:43:09",
      "content": "<p>My original thought on the address and payment_account field was that they were a hash of some sort, however that didn't sit well:</p>\n<ul>\n<li><span style=\"line-height: 1.4\">38 characters is an odd length for a hash</span></li>\n<li><span style=\"line-height: 1.4\">If all values are unique and a true hash function was used the organisers would know that there is no predictive value to be had and not provided those fields.</span></li>\n</ul>\n<p>Browsing though the data during lunch I noticed an oddity, several values are almost exact except for the last 6 characters. &nbsp;This occurs several times with different different prefixes in both columns, &nbsp;such behavior would be highly suspect in a cypto-hash function.</p>\n<p>Seperating out the first 32 characters and the last 6 looks like an interesting avenue to pursue, which is what I'm doing now.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77484,
      "author_name": "mpekalski",
      "author_url": "",
      "post_date": "05/07/2015 15:15:28",
      "content": "<p>I think that in a competition like that, when no code should be shared, you also should restrain from sharing insights like that - at least during the competition time. If you browsed through forums of previous competition it is quite easy to guess that the data is there for a reason and as this is can be messy as it is data mining/modelling/learning competition it is about discovering knowledge in the data.&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "75560": "",
    "75588": "",
    "77419": "",
    "77449": "",
    "77467": "",
    "77484": ""
  },
  "source": "meta"
}