{
  "id": 21812,
  "title": "Feature engineering thoughts",
  "url": "/competitions/facebook-v-predicting-check-ins/discussion/21812",
  "author_name": "Dan Hilton",
  "post_date": "2016-06-20T23:01:19.050000",
  "votes": 1,
  "comment_count": 0,
  "views": 363,
  "content": "<p>Assuming the data has been generated off of real world location data a couple of thoughts came to me that I'd thought I'd sense check with the group:</p>\n\n<ol>\n<li><p>Cellphone location most probably is reported as <a href=\"https://en.wikipedia.org/wiki/Circular_error_probable\" title=\"CER\">CER</a> ( at least <a href=\"http://gis.stackexchange.com/questions/25115/probability-of-ios-android-device-location-accuracy\">Android</a> which, assuming this data is again modelled on the real world, should be a significant subsection of the devices used to checkin). However, the figures do look like meters - especially if it is modelled on US cellphone location having to have a ceiling of 300 meters accuracy (FCC 94&#8211;102).</p></li>\n<li><p>The accuracy jumps could be related to software updates (<a href=\"http://www.gps.gov/systems/gps/modernization/\">GPS upgrade</a>?) Or mobile base station upgrades (most cell towers use software defined radios, so I could imagine a network wide upgrade or a rolling upgrade program over time - that could fit well with some of the observed improvements in accuracy in time.) The other realworld issue with cellular networks (well GSM at least) is that in a 3G network the physical cells grow and shrink dependent on density of subscribers - so coverage will change dependent on how many people are in a cell. </p></li>\n</ol>\n\n<p>Given these sorts of hypothesis, I'd expect to see a strongish correlation (and hopefully causation) between location and accuracy, as the network coverage (either wifi or Cellular) shouldn't differ massively between device  (again assuming the time column maps to a minute time unit, and thus unlikely to see a massive update / refresh cycle of devices in play). Indeed, it might be possible to derive / build some sort of coverage map based on accuracy - i.e. heat map of accuracy by location. </p>\n\n<p>If I get time before the end of the contest, I'll have a go at that.</p>",
  "messages": [
    {
      "id": 124657,
      "postDate": "2016-06-20T23:01:19.050Z",
      "content": "<p>Assuming the data has been generated off of real world location data a couple of thoughts came to me that I'd thought I'd sense check with the group:</p>\n\n<ol>\n<li><p>Cellphone location most probably is reported as <a href=\"https://en.wikipedia.org/wiki/Circular_error_probable\" title=\"CER\">CER</a> ( at least <a href=\"http://gis.stackexchange.com/questions/25115/probability-of-ios-android-device-location-accuracy\">Android</a> which, assuming this data is again modelled on the real world, should be a significant subsection of the devices used to checkin). However, the figures do look like meters - especially if it is modelled on US cellphone location having to have a ceiling of 300 meters accuracy (FCC 94&#8211;102).</p></li>\n<li><p>The accuracy jumps could be related to software updates (<a href=\"http://www.gps.gov/systems/gps/modernization/\">GPS upgrade</a>?) Or mobile base station upgrades (most cell towers use software defined radios, so I could imagine a network wide upgrade or a rolling upgrade program over time - that could fit well with some of the observed improvements in accuracy in time.) The other realworld issue with cellular networks (well GSM at least) is that in a 3G network the physical cells grow and shrink dependent on density of subscribers - so coverage will change dependent on how many people are in a cell. </p></li>\n</ol>\n\n<p>Given these sorts of hypothesis, I'd expect to see a strongish correlation (and hopefully causation) between location and accuracy, as the network coverage (either wifi or Cellular) shouldn't differ massively between device  (again assuming the time column maps to a minute time unit, and thus unlikely to see a massive update / refresh cycle of devices in play). Indeed, it might be possible to derive / build some sort of coverage map based on accuracy - i.e. heat map of accuracy by location. </p>\n\n<p>If I get time before the end of the contest, I'll have a go at that.</p>",
      "rawMarkdown": "Assuming the data has been generated off of real world location data a couple of thoughts came to me that I'd thought I'd sense check with the group:\r\n\r\n1. Cellphone location most probably is reported as [CER][1] ( at least [Android][2] which, assuming this data is again modelled on the real world, should be a significant subsection of the devices used to checkin). However, the figures do look like meters - especially if it is modelled on US cellphone location having to have a ceiling of 300 meters accuracy (FCC 94–102).\r\n\r\n2. The accuracy jumps could be related to software updates ([GPS upgrade][3]?) Or mobile base station upgrades (most cell towers use software defined radios, so I could imagine a network wide upgrade or a rolling upgrade program over time - that could fit well with some of the observed improvements in accuracy in time.) The other realworld issue with cellular networks (well GSM at least) is that in a 3G network the physical cells grow and shrink dependent on density of subscribers - so coverage will change dependent on how many people are in a cell. \r\n\r\n\r\nGiven these sorts of hypothesis, I'd expect to see a strongish correlation (and hopefully causation) between location and accuracy, as the network coverage (either wifi or Cellular) shouldn't differ massively between device  (again assuming the time column maps to a minute time unit, and thus unlikely to see a massive update / refresh cycle of devices in play). Indeed, it might be possible to derive / build some sort of coverage map based on accuracy - i.e. heat map of accuracy by location. \r\n\r\nIf I get time before the end of the contest, I'll have a go at that.\r\n\r\n\r\n  [1]: https://en.wikipedia.org/wiki/Circular_error_probable \"CER\"\r\n  [2]: http://gis.stackexchange.com/questions/25115/probability-of-ios-android-device-location-accuracy\r\n  [3]: http://www.gps.gov/systems/gps/modernization/",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "124657": "Assuming the data has been generated off of real world location data a couple of thoughts came to me that I'd thought I'd sense check with the group:\r\n\r\n1. Cellphone location most probably is reported as [CER][1] ( at least [Android][2] which, assuming this data is again modelled on the real world, should be a significant subsection of the devices used to checkin). However, the figures do look like meters - especially if it is modelled on US cellphone location having to have a ceiling of 300 meters accuracy (FCC 94–102).\r\n\r\n2. The accuracy jumps could be related to software updates ([GPS upgrade][3]?) Or mobile base station upgrades (most cell towers use software defined radios, so I could imagine a network wide upgrade or a rolling upgrade program over time - that could fit well with some of the observed improvements in accuracy in time.) The other realworld issue with cellular networks (well GSM at least) is that in a 3G network the physical cells grow and shrink dependent on density of subscribers - so coverage will change dependent on how many people are in a cell. \r\n\r\n\r\nGiven these sorts of hypothesis, I'd expect to see a strongish correlation (and hopefully causation) between location and accuracy, as the network coverage (either wifi or Cellular) shouldn't differ massively between device  (again assuming the time column maps to a minute time unit, and thus unlikely to see a massive update / refresh cycle of devices in play). Indeed, it might be possible to derive / build some sort of coverage map based on accuracy - i.e. heat map of accuracy by location. \r\n\r\nIf I get time before the end of the contest, I'll have a go at that.\r\n\r\n\r\n  [1]: https://en.wikipedia.org/wiki/Circular_error_probable \"CER\"\r\n  [2]: http://gis.stackexchange.com/questions/25115/probability-of-ios-android-device-location-accuracy\r\n  [3]: http://www.gps.gov/systems/gps/modernization/"
  }
}