Just wondering, what you guys think of this:
\n\nmerged_df[merged_df['geoNetwork.city'] == 'Toronto']['geoNetwork.country'].value_counts()\ngives:
\n\nCanada 10074\nUnited States 95\nUnited Kingdom 6\nChina 4\nIndonesia 4\nDenmark 3\nJapan 2\nNew Zealand 2\nIndia 2\nTaiwan 1\nIran 1\nItaly 1\nArgentina 1\nThere are many examples like this in the data.
\n\nJust wondering, what you guys think?
", "messages": [ { "id": "392387", "postDate": "09/23/2018 17:18:10", "content": "Just wondering, what you guys think of this:
\n\nmerged_df[merged_df['geoNetwork.city'] == 'Toronto']['geoNetwork.country'].value_counts()\ngives:
\n\nCanada 10074\nUnited States 95\nUnited Kingdom 6\nChina 4\nIndonesia 4\nDenmark 3\nJapan 2\nNew Zealand 2\nIndia 2\nTaiwan 1\nIran 1\nItaly 1\nArgentina 1\nThere are many examples like this in the data.
\n\nJust wondering, what you guys think?
", "rawMarkdown": "Just wondering, what you guys think of this: \n\n merged_df[merged_df['geoNetwork.city'] == 'Toronto']['geoNetwork.country'].value_counts()\n\ngives: \n\n Canada 10074\n United States 95\n United Kingdom 6\n China 4\n Indonesia 4\n Denmark 3\n Japan 2\n New Zealand 2\n India 2\n Taiwan 1\n Iran 1\n Italy 1\n Argentina 1\n\nThere are many examples like this in the data. \n\nJust wondering, what you guys think?", "votes": null }, { "id": "392414", "postDate": "09/23/2018 18:29:37", "content": "Geolocation databases have a significant error base, especially at the city level. For instance, one of those databases, Maxmind, is 99.8% accurate on a country level, 90% accurate on a state level in the US, and 86% accurate for cities in the US within a 50 kilometer radius.\nhttps://support.maxmind.com/geoip-faq/geoip2-and-geoip-legacy-databases/how-accurate-are-your-geoip2-and-geoip-legacy-databases/
", "rawMarkdown": "Geolocation databases have a significant error base, especially at the city level. For instance, one of those databases, Maxmind, is 99.8% accurate on a country level, 90% accurate on a state level in the US, and 86% accurate for cities in the US within a 50 kilometer radius.\nhttps://support.maxmind.com/geoip-faq/geoip2-and-geoip-legacy-databases/how-accurate-are-your-geoip2-and-geoip-legacy-databases/", "votes": null }, { "id": "392425", "postDate": "09/23/2018 18:48:24", "content": "Thank you for your reply. Good to know. \nSo, what do you think, should we ignore it? \nI quickly checked and it seems that just in the 'Toronto' example, there are 4.5% of the total 'Toronto' revenue coming from 'not Canadian Toronto'.
", "rawMarkdown": "Thank you for your reply. Good to know. \nSo, what do you think, should we ignore it? \nI quickly checked and it seems that just in the 'Toronto' example, there are 4.5% of the total 'Toronto' revenue coming from 'not Canadian Toronto'.", "votes": null }, { "id": "392463", "postDate": "09/23/2018 20:07:06", "content": "6.8% of Toronto+USA rows have sales, while 1.2% of Toronto+Canada have sales. US vs Canada sales % overall are 3% vs 0.7%.
\n\nBetween that and the idea that country data is more accurate than city data, I would guess your 95 Toronto+US rows are in the US and not in Toronto.
", "rawMarkdown": "6.8% of Toronto+USA rows have sales, while 1.2% of Toronto+Canada have sales. US vs Canada sales % overall are 3% vs 0.7%.\n\nBetween that and the idea that country data is more accurate than city data, I would guess your 95 Toronto+US rows are in the US and not in Toronto.", "votes": null }, { "id": "392612", "postDate": "09/24/2018 04:48:06", "content": "Well, it depends on how the geo information is detected. I've noticed that the for this particular example for all Toronto+USA cases, the 'geoNetwork.networkDomain' is not Canadian. Is it possible that the city information is extracted from the IP address, but the country info is coming from the provider (Domain)?
Also I was wondering, how about vpn services?
", "rawMarkdown": "Well, it depends on how the geo information is detected. I've noticed that the for this particular example for all Toronto+USA cases, the `'geoNetwork.networkDomain'` is not Canadian. Is it possible that the city information is extracted from the IP address, but the country info is coming from the provider (Domain)? \n\nAlso I was wondering, how about vpn services?", "votes": null }, { "id": "392647", "postDate": "09/24/2018 06:24:57", "content": "There can be people using VPN or even Tor. But VPN won't make the contradictory country & city pairs, right?
", "rawMarkdown": "There can be people using VPN or even Tor. But VPN won't make the contradictory country & city pairs, right?", "votes": null }, { "id": "392798", "postDate": "09/24/2018 11:40:29", "content": "Yes I don't think it would make for contradictory city/countries. It might make for different geo info between sessions however.
", "rawMarkdown": "Yes I don't think it would make for contradictory city/countries. It might make for different geo info between sessions however.", "votes": null }, { "id": "392800", "postDate": "09/24/2018 11:42:05", "content": "Wouldn't 'geoNetwork.networkDomain' also be extracted from the IP address? I believe so.
", "rawMarkdown": "Wouldn't 'geoNetwork.networkDomain' also be extracted from the IP address? I believe so.", "votes": null }, { "id": "392863", "postDate": "09/24/2018 14:02:22", "content": "It makes sense! Yes! \nBut then if all the geo info is extracted from the same source (say IP), then why not all 'Toronto's are in Canada?
", "rawMarkdown": "It makes sense! Yes! \nBut then if all the geo info is extracted from the same source (say IP), then why not all 'Toronto's are in Canada?", "votes": null }, { "id": "392870", "postDate": "09/24/2018 14:22:49", "content": "Here are a few examples:
\n\nToronto Americas Canada NaN bell.ca Ontario\n\nToronto Asia Japan NaN access-internet.ne.jp Ontario\n\nToronto Asia China NaN NaN Ontario\nSo which one is more accurate?
", "rawMarkdown": "Here are a few examples: \n\n Toronto\tAmericas\tCanada\tNaN\tbell.ca\t Ontario\n\n Toronto\tAsia\tJapan\tNaN\taccess-internet.ne.jp\tOntario\n\n Toronto\tAsia\tChina\tNaN\tNaN\t Ontario\n\nSo which one is more accurate?", "votes": null }, { "id": "392891", "postDate": "09/24/2018 14:52:16", "content": "For the Japan example, the domain corresponds to the country. Goes in the direction of country being more accurate than city...
", "rawMarkdown": "For the Japan example, the domain corresponds to the country. Goes in the direction of country being more accurate than city...", "votes": null }, { "id": "392897", "postDate": "09/24/2018 14:58:30", "content": "Yep. That's what I thought as well, but the 'China' example killed the theory!
\n\nI guess I'm missing somthing here...
", "rawMarkdown": "Yep. That's what I thought as well, but the 'China' example killed the theory! \n\nI guess I'm missing somthing here...", "votes": null }, { "id": "392905", "postDate": "09/24/2018 15:11:24", "content": "They didn't detect the domain for the China example, but that doesn't mean it's not China.
", "rawMarkdown": "They didn't detect the domain for the China example, but that doesn't mean it's not China.", "votes": null }, { "id": "392911", "postDate": "09/24/2018 15:21:44", "content": "But then there is no way to tell if it's 'Toronto' or 'China', right?!
", "rawMarkdown": "But then there is no way to tell if it's 'Toronto' or 'China', right?!", "votes": null }, { "id": "392916", "postDate": "09/24/2018 15:31:28", "content": "Toronto is a village in County Durham, in England.
", "rawMarkdown": "Toronto is a village in County Durham, in England.", "votes": null }, { "id": "392923", "postDate": "09/24/2018 15:37:39", "content": "Not with certainty, but chances are it is China since generally country is more accurate than city. Also a missing domain name leads me to believe it is China more than Toronto.
", "rawMarkdown": "Not with certainty, but chances are it is China since generally country is more accurate than city. Also a missing domain name leads me to believe it is China more than Toronto.", "votes": null }, { "id": "392935", "postDate": "09/24/2018 15:50:40", "content": "Good to know! Thanks!
", "rawMarkdown": "Good to know! Thanks!", "votes": null }, { "id": "392938", "postDate": "09/24/2018 15:51:44", "content": "I see. OK. \nThank you for the clarification.
", "rawMarkdown": "I see. OK. \nThank you for the clarification.", "votes": null }, { "id": "1267083", "postDate": "04/08/2021 10:03:58", "content": "In Italy they could be confused with Taranto
", "rawMarkdown": "In Italy they could be confused with Taranto", "votes": null }, { "id": "2181443", "postDate": "03/14/2023 14:21:27", "content": "There must be an error in the data-set because in Denmark we do not have tree city's name Toronto.
", "rawMarkdown": "There must be an error in the data-set because in Denmark we do not have tree city's name Toronto.", "votes": null } ], "comments": [ { "id": 1267083, "author_name": "paoloantonuccio", "author_url": "", "post_date": "04/08/2021 10:03:58", "content": "In Italy they could be confused with Taranto
", "votes": null, "replies": [] }, { "id": 2181443, "author_name": "hansenmike", "author_url": "", "post_date": "03/14/2023 14:21:27", "content": "There must be an error in the data-set because in Denmark we do not have tree city's name Toronto.
", "votes": null, "replies": [] }, { "id": 392414, "author_name": "sdoria", "author_url": "", "post_date": "09/23/2018 18:29:37", "content": "Geolocation databases have a significant error base, especially at the city level. For instance, one of those databases, Maxmind, is 99.8% accurate on a country level, 90% accurate on a state level in the US, and 86% accurate for cities in the US within a 50 kilometer radius.\nhttps://support.maxmind.com/geoip-faq/geoip2-and-geoip-legacy-databases/how-accurate-are-your-geoip2-and-geoip-legacy-databases/
", "votes": null, "replies": [ { "id": 392425, "author_name": "leonidg", "author_url": "", "post_date": "09/23/2018 18:48:24", "content": "Thank you for your reply. Good to know. \nSo, what do you think, should we ignore it? \nI quickly checked and it seems that just in the 'Toronto' example, there are 4.5% of the total 'Toronto' revenue coming from 'not Canadian Toronto'.
", "votes": null, "replies": [] }, { "id": 392463, "author_name": "sdoria", "author_url": "", "post_date": "09/23/2018 20:07:06", "content": "6.8% of Toronto+USA rows have sales, while 1.2% of Toronto+Canada have sales. US vs Canada sales % overall are 3% vs 0.7%.
\n\nBetween that and the idea that country data is more accurate than city data, I would guess your 95 Toronto+US rows are in the US and not in Toronto.
", "votes": null, "replies": [] }, { "id": 392612, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 04:48:06", "content": "Well, it depends on how the geo information is detected. I've noticed that the for this particular example for all Toronto+USA cases, the 'geoNetwork.networkDomain' is not Canadian. Is it possible that the city information is extracted from the IP address, but the country info is coming from the provider (Domain)?
Also I was wondering, how about vpn services?
", "votes": null, "replies": [] }, { "id": 392647, "author_name": "kingychiu", "author_url": "", "post_date": "09/24/2018 06:24:57", "content": "There can be people using VPN or even Tor. But VPN won't make the contradictory country & city pairs, right?
", "votes": null, "replies": [] }, { "id": 392798, "author_name": "sdoria", "author_url": "", "post_date": "09/24/2018 11:40:29", "content": "Yes I don't think it would make for contradictory city/countries. It might make for different geo info between sessions however.
", "votes": null, "replies": [] }, { "id": 392800, "author_name": "sdoria", "author_url": "", "post_date": "09/24/2018 11:42:05", "content": "Wouldn't 'geoNetwork.networkDomain' also be extracted from the IP address? I believe so.
", "votes": null, "replies": [] }, { "id": 392863, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 14:02:22", "content": "It makes sense! Yes! \nBut then if all the geo info is extracted from the same source (say IP), then why not all 'Toronto's are in Canada?
", "votes": null, "replies": [] }, { "id": 392870, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 14:22:49", "content": "Here are a few examples:
\n\nToronto Americas Canada NaN bell.ca Ontario\n\nToronto Asia Japan NaN access-internet.ne.jp Ontario\n\nToronto Asia China NaN NaN Ontario\nSo which one is more accurate?
", "votes": null, "replies": [] }, { "id": 392891, "author_name": "sdoria", "author_url": "", "post_date": "09/24/2018 14:52:16", "content": "For the Japan example, the domain corresponds to the country. Goes in the direction of country being more accurate than city...
", "votes": null, "replies": [] }, { "id": 392897, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 14:58:30", "content": "Yep. That's what I thought as well, but the 'China' example killed the theory!
\n\nI guess I'm missing somthing here...
", "votes": null, "replies": [] }, { "id": 392905, "author_name": "sdoria", "author_url": "", "post_date": "09/24/2018 15:11:24", "content": "They didn't detect the domain for the China example, but that doesn't mean it's not China.
", "votes": null, "replies": [] }, { "id": 392911, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 15:21:44", "content": "But then there is no way to tell if it's 'Toronto' or 'China', right?!
", "votes": null, "replies": [] }, { "id": 392923, "author_name": "sdoria", "author_url": "", "post_date": "09/24/2018 15:37:39", "content": "Not with certainty, but chances are it is China since generally country is more accurate than city. Also a missing domain name leads me to believe it is China more than Toronto.
", "votes": null, "replies": [] }, { "id": 392938, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 15:51:44", "content": "I see. OK. \nThank you for the clarification.
", "votes": null, "replies": [] } ] }, { "id": 392916, "author_name": "scirpus", "author_url": "", "post_date": "09/24/2018 15:31:28", "content": "Toronto is a village in County Durham, in England.
", "votes": null, "replies": [ { "id": 392935, "author_name": "leonidg", "author_url": "", "post_date": "09/24/2018 15:50:40", "content": "Good to know! Thanks!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "392387": "Just wondering, what you guys think of this: \n\n merged_df[merged_df['geoNetwork.city'] == 'Toronto']['geoNetwork.country'].value_counts()\n\ngives: \n\n Canada 10074\n United States 95\n United Kingdom 6\n China 4\n Indonesia 4\n Denmark 3\n Japan 2\n New Zealand 2\n India 2\n Taiwan 1\n Iran 1\n Italy 1\n Argentina 1\n\nThere are many examples like this in the data. \n\nJust wondering, what you guys think?", "392414": "Geolocation databases have a significant error base, especially at the city level. For instance, one of those databases, Maxmind, is 99.8% accurate on a country level, 90% accurate on a state level in the US, and 86% accurate for cities in the US within a 50 kilometer radius.\nhttps://support.maxmind.com/geoip-faq/geoip2-and-geoip-legacy-databases/how-accurate-are-your-geoip2-and-geoip-legacy-databases/", "392425": "Thank you for your reply. Good to know. \nSo, what do you think, should we ignore it? \nI quickly checked and it seems that just in the 'Toronto' example, there are 4.5% of the total 'Toronto' revenue coming from 'not Canadian Toronto'.", "392463": "6.8% of Toronto+USA rows have sales, while 1.2% of Toronto+Canada have sales. US vs Canada sales % overall are 3% vs 0.7%.\n\nBetween that and the idea that country data is more accurate than city data, I would guess your 95 Toronto+US rows are in the US and not in Toronto.", "392612": "Well, it depends on how the geo information is detected. I've noticed that the for this particular example for all Toronto+USA cases, the `'geoNetwork.networkDomain'` is not Canadian. Is it possible that the city information is extracted from the IP address, but the country info is coming from the provider (Domain)? \n\nAlso I was wondering, how about vpn services?", "392647": "There can be people using VPN or even Tor. But VPN won't make the contradictory country & city pairs, right?", "392798": "Yes I don't think it would make for contradictory city/countries. It might make for different geo info between sessions however.", "392800": "Wouldn't 'geoNetwork.networkDomain' also be extracted from the IP address? I believe so.", "392863": "It makes sense! Yes! \nBut then if all the geo info is extracted from the same source (say IP), then why not all 'Toronto's are in Canada?", "392870": "Here are a few examples: \n\n Toronto\tAmericas\tCanada\tNaN\tbell.ca\t Ontario\n\n Toronto\tAsia\tJapan\tNaN\taccess-internet.ne.jp\tOntario\n\n Toronto\tAsia\tChina\tNaN\tNaN\t Ontario\n\nSo which one is more accurate?", "392891": "For the Japan example, the domain corresponds to the country. Goes in the direction of country being more accurate than city...", "392897": "Yep. That's what I thought as well, but the 'China' example killed the theory! \n\nI guess I'm missing somthing here...", "392905": "They didn't detect the domain for the China example, but that doesn't mean it's not China.", "392911": "But then there is no way to tell if it's 'Toronto' or 'China', right?!", "392916": "Toronto is a village in County Durham, in England.", "392923": "Not with certainty, but chances are it is China since generally country is more accurate than city. Also a missing domain name leads me to believe it is China more than Toronto.", "392935": "Good to know! Thanks!", "392938": "I see. OK. \nThank you for the clarification.", "1267083": "In Italy they could be confused with Taranto", "2181443": "There must be an error in the data-set because in Denmark we do not have tree city's name Toronto." }, "source": "meta" }