{
  "id": 13826,
  "title": "A human bidder with different device,country and ip",
  "url": "/competitions/facebook-recruiting-iv-human-or-bot/discussion/13826",
  "author_name": "",
  "post_date": "2015-04-28T08:00:31.087Z",
  "votes": null,
  "comment_count": 16,
  "views": 4946,
  "content": "<p>I can see that a single human bidder in the <strong>same auction</strong>, has different devices, countries and ip adds. In what case is this possible?</p>",
  "messages": [
    {
      "id": "75203",
      "postDate": "04/28/2015 08:00:31",
      "content": "<p>I can see that a single human bidder in the <strong>same auction</strong>, has different devices, countries and ip adds. In what case is this possible?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75205",
      "postDate": "04/28/2015 08:12:31",
      "content": "<p>I guess, when many people share an account</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75281",
      "postDate": "04/28/2015 16:26:55",
      "content": "<p>Using a different device when traveling is not uncommon either.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75457",
      "postDate": "04/29/2015 17:44:55",
      "content": "<p>Also manipulating UserAgentString which probably is used to identify a device is not a problem. Unless they use some more sophisticated methods.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75491",
      "postDate": "04/29/2015 20:16:21",
      "content": "<p>Also, when bidders' connection switches to use proxy, or VPN, could change the country and IP address (could be also device) instantly without moving/travelling. There are lots of free (ads-based) or cheap proxy/VPN offered with multi-countries servers. </p>\n<p>Another..., many ISPs use dynamic IP (in opposite to single-static IP addr, usually dial-up or mobile or wireless connection), so everytime a user connects to the network, she/he gets different IP address (but not different country). Though different country might be also possible, e.g. a multinational ISP covers Singapore, Malaysia and Thailand, so dynamic IP scheme might attach different IP + different country to a user over time. (but not different device.)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "75921",
      "postDate": "05/01/2015 15:19:31",
      "content": "<p>From my point of view, we have been asked to identify bots, disregarding the fact that human bidders can also be organized to try and game the system, as opposed to a 'legit' user who has a genuine interest on a particular item. Several human (outcome = 0.0) identifiers on the training dataset may very well look like multiple users sharing a single account.</p>\n<p>I also agree with the fact that some people may be travelling and / or getting different IPs from their provider, but in some cases the variety of different countries is so huge that these explanations are not enough.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "76011",
      "postDate": "05/02/2015 01:25:16",
      "content": "<p>I wonder how did they identify bots in the first place. Is the &quot;outcome&quot; we are trying to predict simply a prediction of some other automatic bot identification system?</p>\n<p>Look at the bidder_id d6517684989560cbb0da1fb2f5bbba9b9y2st. 58195 bids, 1396 devices, 137 countries,&nbsp;12978 ips,&nbsp;1621 auction, merchandise = jewelry, outcome = 0. A thousand jewelry-obsessed travelers sharing an account? This doesn't make any sense. And there are many records like that. As well as some bots with a single bid.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "76040",
      "postDate": "05/02/2015 07:33:41",
      "content": "<p>There could be many explanations for many of those numbers:</p>\n<p>1) I would guess that they identify devices by browser's UserAgentString, which is easy to change. But you do not have to be a bot to do that. UserAgentString also often may contain a firmware version of the phone, or browser version. So 1396 devices does not have to mean so many people.</p>\n<p>2) You do not know the time span, if you get your ip is dynamic and gets changed every day, or you use mobile device and just wonder around the city (each BTS usually has a range of ip to assign) you may get many different ip numbers.&nbsp;</p>\n<p>3) Number of countries, might come simply from inaccuracy of ip range mapping. There is no official database (at least I do not know about it) of ip-country mapping. So if you use 13K ips some of them may get wrongly mapped. Maybe it is worth looking at frequencies.&nbsp;</p>\n<p>For me that is a data mining competition, so pretty much they could name the features x1-x15, and you still would be able to say something about it - mine the information form the data without prior knowledge.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "76337",
      "postDate": "05/03/2015 15:26:53",
      "content": "<p>Those could be bids from mechanical turks. If a seller, or group of sellers, wanted to increase their margins, they could pay people to place slightly higher bids if they were confident&nbsp;the user or users who want the item will&nbsp;outbid the turk.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77541",
      "postDate": "05/07/2015 22:20:56",
      "content": "<p>[quote=mpekalski;76040]</p>\n<p>2) You do not know the time span, if you get your ip is dynamic and gets changed every day, or you use mobile device and just wonder around the city (each BTS usually has a range of ip to assign) you may get many different ip numbers.&nbsp;</p>\n<p>[/quote]</p>\n<p>IP Country mappings companies are using are not that bad. I've been working in multiple companies and from my experience you can expect accuracy &gt;95% for IP Country mapping that you can get from open source resources. Dynamic IPs you get from you provider are pulled from same range that provider assigned and it is the same country. Same for wondering around the city. Probably only requests from web page or phone malware can explain that many IPs for bidder account.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77662",
      "postDate": "05/08/2015 23:17:47",
      "content": "<p>Also, isn't is possible that some labels could be wrong, as the data has been said to be noisy ?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "79556",
      "postDate": "05/21/2015 01:46:34",
      "content": "<p>People move more than servers do.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80084",
      "postDate": "05/27/2015 02:12:56",
      "content": "<p>As aldente mentioned it is fairly common for people&nbsp;to use VPN's for privacy and security reasons. When you're not on a VPN, your ISP can see and/or control / throttle all of your activity as they see fit. If you're on a VPN, they cannot. VPN's also add protection when using open WiFi networks.</p>\n<p>My (paid) VPN service has access points all over the world (see attached image). I can switch to any one of these at the click of a button, changing my IP and location (as far as visited websites are concerned). Once the secure connection is made I use the internet normally just like any other non-bot :)</p>\n<p>That said, I wouldn't be surprised if bot developers don't use VPN's and other privacy tools extensively.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80190",
      "postDate": "05/27/2015 20:25:26",
      "content": "<p>[quote=mpekalski;76040]</p>\n\n<p>For me that is a data mining competition, so pretty much they could name the features x1-x15, and you still would be able to say something about it - mine the information form the data without prior knowledge.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>In my opinion, you're absolutely right. For the purpose of the competition, this discussion is not so important.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80213",
      "postDate": "05/28/2015 03:57:10",
      "content": "<p>[quote=Jos&#233; Mar&#237;a Mateos;75921]</p>\n<p>I also agree with the fact that some people may be travelling and / or getting different IPs from their provider, but in some cases the variety of different countries is so huge that these explanations are not enough.</p>\n<p>[/quote]</p>\n\n<p>VPN perhaps? Some of us work for different companies who only allow VPN server access.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80214",
      "postDate": "05/28/2015 03:58:56",
      "content": "<p>[quote=Brach;80084]</p>\n<p>As aldente mentioned it is fairly common for people&nbsp;to use VPN's for privacy and security reasons. When you're not on a VPN, your ISP can see and/or control / throttle all of your activity as they see fit. If you're on a VPN, they cannot. VPN's also add protection when using open WiFi networks.</p>\n<p>My (paid) VPN service has access points all over the world (see attached image). I can switch to any one of these at the click of a button, changing my IP and location (as far as visited websites are concerned). Once the secure connection is made I use the internet normally just like any other non-bot :)</p>\n<p>That said, I wouldn't be surprised if bot developers don't use VPN's and other privacy tools extensively.</p>\n<p>[/quote]</p>\n\n<p>Sorry Brach, I replied before seeing your post. By the way, I totally agree with the points you raised.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "80309",
      "postDate": "05/28/2015 21:17:39",
      "content": "<p>[quote=Darin;76337]</p>\n<p>Those could be bids from mechanical turks. If a seller, or group of sellers, wanted to increase their margins, they could pay people to place slightly higher bids if they were confident&nbsp;the user or users who want the item will&nbsp;outbid the turk.</p>\n<p>[/quote]</p>\n<p>If this is a penny auction the shills&nbsp;wouldn't be trying to drive up the price but instead be trying to increase the number of bids (by keeping the auction open longer), right?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 75205,
      "author_name": "poppyqueen",
      "author_url": "",
      "post_date": "04/28/2015 08:12:31",
      "content": "<p>I guess, when many people share an account</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75281,
      "author_name": "jasontam",
      "author_url": "",
      "post_date": "04/28/2015 16:26:55",
      "content": "<p>Using a different device when traveling is not uncommon either.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75457,
      "author_name": "mpekalski",
      "author_url": "",
      "post_date": "04/29/2015 17:44:55",
      "content": "<p>Also manipulating UserAgentString which probably is used to identify a device is not a problem. Unless they use some more sophisticated methods.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75491,
      "author_name": "aldente",
      "author_url": "",
      "post_date": "04/29/2015 20:16:21",
      "content": "<p>Also, when bidders' connection switches to use proxy, or VPN, could change the country and IP address (could be also device) instantly without moving/travelling. There are lots of free (ads-based) or cheap proxy/VPN offered with multi-countries servers. </p>\n<p>Another..., many ISPs use dynamic IP (in opposite to single-static IP addr, usually dial-up or mobile or wireless connection), so everytime a user connects to the network, she/he gets different IP address (but not different country). Though different country might be also possible, e.g. a multinational ISP covers Singapore, Malaysia and Thailand, so dynamic IP scheme might attach different IP + different country to a user over time. (but not different device.)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 75921,
      "author_name": "rinzewind",
      "author_url": "",
      "post_date": "05/01/2015 15:19:31",
      "content": "<p>From my point of view, we have been asked to identify bots, disregarding the fact that human bidders can also be organized to try and game the system, as opposed to a 'legit' user who has a genuine interest on a particular item. Several human (outcome = 0.0) identifiers on the training dataset may very well look like multiple users sharing a single account.</p>\n<p>I also agree with the fact that some people may be travelling and / or getting different IPs from their provider, but in some cases the variety of different countries is so huge that these explanations are not enough.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 76011,
      "author_name": "redstr",
      "author_url": "",
      "post_date": "05/02/2015 01:25:16",
      "content": "<p>I wonder how did they identify bots in the first place. Is the &quot;outcome&quot; we are trying to predict simply a prediction of some other automatic bot identification system?</p>\n<p>Look at the bidder_id d6517684989560cbb0da1fb2f5bbba9b9y2st. 58195 bids, 1396 devices, 137 countries,&nbsp;12978 ips,&nbsp;1621 auction, merchandise = jewelry, outcome = 0. A thousand jewelry-obsessed travelers sharing an account? This doesn't make any sense. And there are many records like that. As well as some bots with a single bid.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 76040,
      "author_name": "mpekalski",
      "author_url": "",
      "post_date": "05/02/2015 07:33:41",
      "content": "<p>There could be many explanations for many of those numbers:</p>\n<p>1) I would guess that they identify devices by browser's UserAgentString, which is easy to change. But you do not have to be a bot to do that. UserAgentString also often may contain a firmware version of the phone, or browser version. So 1396 devices does not have to mean so many people.</p>\n<p>2) You do not know the time span, if you get your ip is dynamic and gets changed every day, or you use mobile device and just wonder around the city (each BTS usually has a range of ip to assign) you may get many different ip numbers.&nbsp;</p>\n<p>3) Number of countries, might come simply from inaccuracy of ip range mapping. There is no official database (at least I do not know about it) of ip-country mapping. So if you use 13K ips some of them may get wrongly mapped. Maybe it is worth looking at frequencies.&nbsp;</p>\n<p>For me that is a data mining competition, so pretty much they could name the features x1-x15, and you still would be able to say something about it - mine the information form the data without prior knowledge.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 76337,
      "author_name": "darin157216",
      "author_url": "",
      "post_date": "05/03/2015 15:26:53",
      "content": "<p>Those could be bids from mechanical turks. If a seller, or group of sellers, wanted to increase their margins, they could pay people to place slightly higher bids if they were confident&nbsp;the user or users who want the item will&nbsp;outbid the turk.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77541,
      "author_name": "sanityio",
      "author_url": "",
      "post_date": "05/07/2015 22:20:56",
      "content": "<p>[quote=mpekalski;76040]</p>\n<p>2) You do not know the time span, if you get your ip is dynamic and gets changed every day, or you use mobile device and just wonder around the city (each BTS usually has a range of ip to assign) you may get many different ip numbers.&nbsp;</p>\n<p>[/quote]</p>\n<p>IP Country mappings companies are using are not that bad. I've been working in multiple companies and from my experience you can expect accuracy &gt;95% for IP Country mapping that you can get from open source resources. Dynamic IPs you get from you provider are pulled from same range that provider assigned and it is the same country. Same for wondering around the city. Probably only requests from web page or phone malware can explain that many IPs for bidder account.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77662,
      "author_name": "abhowmick22",
      "author_url": "",
      "post_date": "05/08/2015 23:17:47",
      "content": "<p>Also, isn't is possible that some labels could be wrong, as the data has been said to be noisy ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 79556,
      "author_name": "g1685786",
      "author_url": "",
      "post_date": "05/21/2015 01:46:34",
      "content": "<p>People move more than servers do.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80084,
      "author_name": "datamagic",
      "author_url": "",
      "post_date": "05/27/2015 02:12:56",
      "content": "<p>As aldente mentioned it is fairly common for people&nbsp;to use VPN's for privacy and security reasons. When you're not on a VPN, your ISP can see and/or control / throttle all of your activity as they see fit. If you're on a VPN, they cannot. VPN's also add protection when using open WiFi networks.</p>\n<p>My (paid) VPN service has access points all over the world (see attached image). I can switch to any one of these at the click of a button, changing my IP and location (as far as visited websites are concerned). Once the secure connection is made I use the internet normally just like any other non-bot :)</p>\n<p>That said, I wouldn't be surprised if bot developers don't use VPN's and other privacy tools extensively.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80190,
      "author_name": "intelligence",
      "author_url": "",
      "post_date": "05/27/2015 20:25:26",
      "content": "<p>[quote=mpekalski;76040]</p>\n\n<p>For me that is a data mining competition, so pretty much they could name the features x1-x15, and you still would be able to say something about it - mine the information form the data without prior knowledge.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>In my opinion, you're absolutely right. For the purpose of the competition, this discussion is not so important.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80213,
      "author_name": "sushanttripathy",
      "author_url": "",
      "post_date": "05/28/2015 03:57:10",
      "content": "<p>[quote=Jos&#233; Mar&#237;a Mateos;75921]</p>\n<p>I also agree with the fact that some people may be travelling and / or getting different IPs from their provider, but in some cases the variety of different countries is so huge that these explanations are not enough.</p>\n<p>[/quote]</p>\n\n<p>VPN perhaps? Some of us work for different companies who only allow VPN server access.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80214,
      "author_name": "sushanttripathy",
      "author_url": "",
      "post_date": "05/28/2015 03:58:56",
      "content": "<p>[quote=Brach;80084]</p>\n<p>As aldente mentioned it is fairly common for people&nbsp;to use VPN's for privacy and security reasons. When you're not on a VPN, your ISP can see and/or control / throttle all of your activity as they see fit. If you're on a VPN, they cannot. VPN's also add protection when using open WiFi networks.</p>\n<p>My (paid) VPN service has access points all over the world (see attached image). I can switch to any one of these at the click of a button, changing my IP and location (as far as visited websites are concerned). Once the secure connection is made I use the internet normally just like any other non-bot :)</p>\n<p>That said, I wouldn't be surprised if bot developers don't use VPN's and other privacy tools extensively.</p>\n<p>[/quote]</p>\n\n<p>Sorry Brach, I replied before seeing your post. By the way, I totally agree with the points you raised.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 80309,
      "author_name": "dodger",
      "author_url": "",
      "post_date": "05/28/2015 21:17:39",
      "content": "<p>[quote=Darin;76337]</p>\n<p>Those could be bids from mechanical turks. If a seller, or group of sellers, wanted to increase their margins, they could pay people to place slightly higher bids if they were confident&nbsp;the user or users who want the item will&nbsp;outbid the turk.</p>\n<p>[/quote]</p>\n<p>If this is a penny auction the shills&nbsp;wouldn't be trying to drive up the price but instead be trying to increase the number of bids (by keeping the auction open longer), right?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "75203": "",
    "75205": "",
    "75281": "",
    "75457": "",
    "75491": "",
    "75921": "",
    "76011": "",
    "76040": "",
    "76337": "",
    "77541": "",
    "77662": "",
    "79556": "",
    "80084": "",
    "80190": "",
    "80213": "",
    "80214": "",
    "80309": ""
  },
  "source": "meta"
}