{"cells":[{"metadata":{"_uuid":"73d8ffff6547cc858d46ff171d218512fdb26ec3"},"cell_type":"markdown","source":"# Fixing Conflicts in the geoNetwork Attributes"},{"metadata":{"trusted":true,"_uuid":"1e8124061f4fc590e8258df6883bf5f53c4deeb0"},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport random\nimport re\n\nfrom sklearn.preprocessing import OneHotEncoder\nfrom sklearn.ensemble import RandomForestClassifier","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b31eb6a128b399baf9fa54248f2a2d68080c2238"},"cell_type":"markdown","source":"In this notebook, we will investigate the data accuracy and consistency issues in the geoNetwork attributes and attempt to remedy the errors."},{"metadata":{"_uuid":"46fb15171f9bd876e608cb9ff3df1be7953a8767"},"cell_type":"markdown","source":"First we load the processed training data. The processing is done with my other kernel script \"GStore Revenue Data Preprocessing\"."},{"metadata":{"trusted":true,"_uuid":"f5f2473442f8128d5a62eddc56e76938a9dda055"},"cell_type":"code","source":"dat = pd.read_pickle(\"../input/gstore-revenue-data-preprocessing/train.pkl\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6f97a9c6412f763a3bbc35bb4967e230fe1644dc"},"cell_type":"markdown","source":"Now we focus our attention on the location-related data in the dataset, which is stored in the *geoNetwork* section."},{"metadata":{"trusted":false,"_uuid":"1db63a752617efda3365faf012c5ac68e2168268"},"cell_type":"code","source":"geo_colnames = [c for c in dat.columns if re.match(r'geoNetwork', c) is not None]","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"2888a23c7f5c392b39add2a311ac6d2c41246df6"},"cell_type":"code","source":"geo_colnames","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5734e29fecc3eba7042ca32765d2d6171284f5ca"},"cell_type":"markdown","source":"For now, we ignore the *networkDomain* attribute, although it might prove useful later."},{"metadata":{"trusted":false,"_uuid":"40aa41097933d3c99e0d35158460ee93f31a9d3e"},"cell_type":"code","source":"pure_geo_columns = [c for c in geo_colnames if c != 'geoNetwork.networkDomain']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"efcbb29e7ecd31f03f7e33f007531efb72060965"},"cell_type":"markdown","source":"---------------------------------------"},{"metadata":{"trusted":false,"_uuid":"ec22bebb472fcae6264c46a649934e2a503e0624"},"cell_type":"code","source":"for c in pure_geo_columns:\n    dat.loc[:, c] = dat[c].cat.add_categories('N/A').fillna('N/A')","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"6eaee7014b17353f217ddc9db4c8e581aa3405de"},"cell_type":"code","source":"selected = dat.loc[:, dat.columns.isin(pure_geo_columns)].copy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"ee9b7892702de008d4ad6a0deb4ad02b8bf9375f"},"cell_type":"code","source":"selected.groupby(pure_geo_columns).size().reset_index().head(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"39a201cea854580910376f944fc47c129afaffa4"},"cell_type":"markdown","source":"If we look at the first few columns of the *geoNetwork* columns aggregated and counted by attribute values, it is apparent that there are many mistakes and missing values in the data. Most importantly, we see some city names repeated multiple times, often with obviously wrong countries and continents. While there are some cases where cities across the world share names, it makes up only a small portion of all such cases. It might seem there are no other way than to manually investigate each of the cases."},{"metadata":{"_uuid":"8a070e868b5ee9de92e9a123ce40e2995ea08a69"},"cell_type":"markdown","source":"## Method 1. Careful Investigation"},{"metadata":{"_uuid":"3ec54e4704f99b091b4f9301f34a6f88a4dabf6f"},"cell_type":"markdown","source":"If we look closely, we can find that within the six attributes, there are two groups of attributes, three in each group, in which they tend to agree with each other more often than not. The first group is ('geoNetwork.continent', 'geoNetwork.country', 'geoNetwork.subContinent'), and the second group is ('geoNetwork.city', 'geoNetwork.metro', 'geoNetwork.region').\n\nLet us first look at the first group:"},{"metadata":{"trusted":false,"_uuid":"7a4132183f6a2b44fbce57cd64a2d8d985a94a22"},"cell_type":"code","source":"country_part = selected.groupby(\n    ['geoNetwork.continent', 'geoNetwork.country',\n     'geoNetwork.subContinent']).size().reset_index()\n\ncountry_part.head(20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9724671e38ee4b003b8131ba291e5edb901654fd"},"cell_type":"markdown","source":"If we try to find duplicated country names in this subset of data, we will actually find none! Therefore there should not be any factual conflicts within attributes in this group."},{"metadata":{"trusted":false,"_uuid":"e98f7b23b9fa4496fb5f1b10c881d4001cc73e45"},"cell_type":"code","source":"country_part[country_part['geoNetwork.country'].duplicated(keep=False)]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"07c05d5d19afb03c3b1ca1be101654ffff15c15a"},"cell_type":"markdown","source":"Things are more interesting in the other group of attributes:"},{"metadata":{"trusted":false,"_uuid":"97e622bb7d619ac91d3c42f1603bc0614d726f6e"},"cell_type":"code","source":"city_part = selected.groupby(\n    ['geoNetwork.city', 'geoNetwork.metro',\n     'geoNetwork.region']).size().reset_index()\n\ncity_part.head(20)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"84c4473b0ea7b878bea45325c2acb6845ae12414"},"cell_type":"markdown","source":"As we see below, there actually exist some duplicated city names in unique combinations of (city, metro, region). If we look closely, most of the cases are caused by missing values that only exist in part of the rows, but in some cases it is actually two different cities with the same names (Guadalajara Spain vs Guadalajara Mexico, Kansas City\t Kansas vs Kansas City Missouri, etc.). If we can pick out those special cases, we can resolve all conflicts within this subset of attributes, and we will only have to investigate if there are any non-error conflicts between the two attribute groups."},{"metadata":{"trusted":false,"_uuid":"c1bff16c547c8405c011997dc31a7e0fe5adc217"},"cell_type":"code","source":"city_part[city_part['geoNetwork.city'].duplicated(keep=False) & (city_part['geoNetwork.city'] != 'N/A')]","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"ec1646d6ca19decc19babd8450ec2485417a6834"},"cell_type":"code","source":"city_part.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"445e6c0e47cae2d67dfc49083bc8c05cff473cd3"},"cell_type":"code","source":"city_part.groupby(['geoNetwork.city', 'geoNetwork.region']).size().sort_values(ascending=False).head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"28aa5323f4de80d3da3e7954424ca12e98c3a795"},"cell_type":"markdown","source":"If we combine the city and region field, we will be able to uniquely determine the identity of a city (as long as the city attribute is present). Therefore, we only have to fix N/A issues in the region field."},{"metadata":{"trusted":false,"_uuid":"81bca6b2d35baae5d464a2ad6c833dc2e0d8d2c4"},"cell_type":"code","source":"pairs = [('Colombo', 'Western Province'), ('Doha', 'Doha'),\n         ('Guatemala City', 'Guatemala Department'), ('Hanoi', 'Hanoi'),\n         ('Minsk', 'Minsk Region'), ('Nairobi', 'Nairobi County'), ('Tbilisi',\n                                                                    'Tbilisi')]\n\nfor c, r in pairs:\n    dat.loc[(dat['geoNetwork.city'] == c) &\n            (dat['geoNetwork.region'] == 'N/A'), 'geoNetwork.region'] = r","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bb5d5121177e1db0b0619cc6e569e8fab9578743"},"cell_type":"markdown","source":"There are still many cases where the city is not given but the region is available. In these cases, we can still safely drop \"metro\", as in none of the cases, the metro attribute is present."},{"metadata":{"trusted":false,"_uuid":"f7a8e8dc1eca259aa75b4f72834703af0a650e9e"},"cell_type":"code","source":"city_part[city_part['geoNetwork.city'] == 'N/A']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fde7cbc1e4ac8276afb60c18b02543e763ab77d1"},"cell_type":"markdown","source":"We have now determined that the *country* attribute group can be determined from country alone, and the *city* attribute group can be determined by city + region. We have now shrunk the 6 geolocation attributes into 3."},{"metadata":{"trusted":false,"_uuid":"499304b563be3605a1957a27029ed4199bb6733f"},"cell_type":"code","source":"selected = dat.loc[:, dat.columns.isin(pure_geo_columns)].copy()\ncc = selected.groupby(['geoNetwork.city', 'geoNetwork.region','geoNetwork.country']).size().reset_index()\n\ncc.loc[(cc['geoNetwork.city'].duplicated(keep=False) &\n        (cc['geoNetwork.city'] != 'N/A'))\n       | (cc['geoNetwork.region'].duplicated(keep=False) & (\n           (cc['geoNetwork.city'] == 'N/A') &\n           (cc['geoNetwork.region'] != 'N/A')))].head(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"167d87034ddb6bc7a9f4426940f9c15e23bcc4ff"},"cell_type":"markdown","source":"We now see that almost all of the conflicts between (city, region) and country are due to errors. There are still some ambiguities, but since 1) we do not worry too match about per-record accuracy of the data 2) we cannot determine which subgroup of attributes we should trust more when a conflict happens, we might as well accept an okay solution and resolve all these conflicts by setting to the most common triples. We have to exclude cases where both *city* and *region* are NA because it does not make sense to infer anything about *country* from NAs."},{"metadata":{"trusted":false,"_uuid":"c47afb972f32cb09d5655cb6b4a263c505d56063"},"cell_type":"code","source":"most_common = dat.groupby([\n    'geoNetwork.city', 'geoNetwork.region'\n])['geoNetwork.country'].apply(lambda x: x.mode()).reset_index()\nmost_common.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"9bc21db02181d5f750ec7805167246a024ab7958"},"cell_type":"code","source":"for idx, row in most_common.iterrows():\n    dat.loc[(dat['geoNetwork.city'] == row['geoNetwork.city']) &\n            (dat['geoNetwork.region'] == row['geoNetwork.region']) &\n            ((dat['geoNetwork.city'] != 'N/A') |\n             ((dat['geoNetwork.region'] != 'N/A'))\n             ), 'geoNetwork.country'] = row['geoNetwork.country']\n\nselected = dat.loc[:, dat.columns.isin(pure_geo_columns)].copy()\ncc = selected.groupby(\n    ['geoNetwork.city', 'geoNetwork.region',\n     'geoNetwork.country']).size().reset_index()\ncc.head(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cc0b4148e79c97ff7adaad62fe7ac1eb162ac5c3"},"cell_type":"markdown","source":"We have resolved the vast majority of geolocation data conflicts!"},{"metadata":{"_uuid":"640801fd3aa38e10abee37ae2edae404494d9d60"},"cell_type":"markdown","source":"## Method 2. Correction via Attribute Value Association"},{"metadata":{"_uuid":"2c32925b548f3810f5c55ee03258086efe692d63"},"cell_type":"markdown","source":"Suppose the dataset size and anomaly number are even larger than what we have here, and a manual investigation of the data error is not feasible. We need some semi-reliable ways to fix errors in the geolocation data. Luckily, since geographical data are typically attributes describing fixed facts (cities do not randomly switch continental allegiance after all), we can depend on the typical relationships between attribute values and generalise it to determine what a reasonable attribute value combination should look like when suspected anomalies happen. We can learn the distribution of data values and exploit the learned distribution to identify and correct error. One way to do this is to use a classifier to predict one column value based on others, and find values in the target column that do not match up with the data value or have low prediction confidence. As most classifiers tend to learn the most likely value of the target given input variables, we can use the prediction of the classifier to identify unlikely values in the target column given the other columns. Here, we will be using a simple random forest. As we are using the random forest to model the data distribution instead of actually making predictions, accuracy is not really our biggest concern. In fact, we may want to artificially limit the learning capacity of the algorithm so unlikely edge cases that tend to be data anomalies will not be memorised."},{"metadata":{"_uuid":"ba45fd9ab2c4f3b14f377393bde1467d93034df9"},"cell_type":"markdown","source":"We have to convert the original data into one-hot format for the network to ingest. For comparison with method 1, we read a new data frame from the original. We still assign missing data a placeholder value."},{"metadata":{"trusted":false,"_uuid":"36f6890d4b98107da99a3191a79495c1e4f62e3b"},"cell_type":"code","source":"dat2 = pd.read_pickle(\"../input/gstore-revenue-data-preprocessing/train.pkl\")\nfor c in pure_geo_columns:\n    dat2.loc[:, c] = dat2[c].cat.add_categories('N/A').fillna('N/A')\n    \nselected = dat2.loc[:, pure_geo_columns].copy()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e1fc0e8aad6608b0f049a29eb417c636883c8a3c"},"cell_type":"markdown","source":"To simplify the caluclation, we assume we have performed some basic analysis like in method 1 and determined that *continent* and *subcontinent* are strictly determined by *country*, and *metro* can almost always be inferred from *city* and *region*. Therefore, we have only 3 attributes to work with."},{"metadata":{"trusted":false,"_uuid":"63cd314bd2c5a4aed4234f3efa7b00934ce76b5f"},"cell_type":"code","source":"key_attributes = ['geoNetwork.city', 'geoNetwork.region', 'geoNetwork.country']\nselected[key_attributes].describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"efe5dc0de5fdbb85b910de40113f9b7029de45e6"},"cell_type":"markdown","source":"Same as before, we have three columns of categorical data. We now convert them into one-hot encoding."},{"metadata":{"trusted":false,"_uuid":"0eb1a144f422d31e1504d05e403fcc4976e9c655"},"cell_type":"code","source":"enc = OneHotEncoder()\ntransformed = enc.fit_transform(selected[key_attributes].apply(lambda x: x.cat.codes))\nN, D = transformed.shape\nN, D","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"96dc262296f64ddd5055e73b24ec40e1fb2d8239"},"cell_type":"markdown","source":"Here, we will be using the *country* column as the target and (*city*, *region*) as the input for the random forest."},{"metadata":{"trusted":false,"_uuid":"b1bcb6c3ddc381370f6ae70cad80cdad60e2db4f"},"cell_type":"code","source":"clf = RandomForestClassifier()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"8fd420d38362000308640b90fd417c383a4a9f34"},"cell_type":"code","source":"clf.fit(transformed[:, :-222], selected['geoNetwork.country'].cat.codes)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"859d23f92e68595b6f9ad5f6eb32b2b5fb30810e"},"cell_type":"markdown","source":"We set suspect anomaly cases as those whose predicted *country* value does not match up with the value in data:"},{"metadata":{"trusted":false,"_uuid":"3a1ab0faad94d5a0a8874752a06f41797dce5911"},"cell_type":"code","source":"pred = clf.predict(transformed[:, :-222])\nis_anomaly = pred != selected['geoNetwork.country'].cat.codes","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"70258b86533e2cf0d3bb0f68c8e5fd7e66b56f61"},"cell_type":"markdown","source":"We look at the first few columns of the detected anomaly cases. We exclude the cases with double NAs in the input columns because it is unrealistic to expect a reliable prediction in those cases. As we see here, the random forest does a pretty decent job at picking out obvious conflicting cases."},{"metadata":{"trusted":false,"_uuid":"fac581c573a12c43f5d9f9d899bc1d99be148368"},"cell_type":"code","source":"anomaly_cases = selected[is_anomaly][(selected['geoNetwork.city'] != 'N/A') | (\n    selected['geoNetwork.region'] != 'N/A')][key_attributes]\nanomaly_cases.head(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a3dd7deaec320827599c749215b780abb2d8b173"},"cell_type":"markdown","source":"We may also look at prediction confidences to find if there are cases where the random forest is uncertain which value is most probable:"},{"metadata":{"trusted":false,"_uuid":"ee29fe721f42e888d5fecffa0ed77404b9d3e9cb"},"cell_type":"code","source":"certainty = clf.predict_proba(transformed[:, :-222])","execution_count":null,"outputs":[]},{"metadata":{"trusted":false,"_uuid":"f6f45122459c1565836982b333b9187efa2a11f9"},"cell_type":"code","source":"uncertain_idx = np.max(certainty, axis=1) < 0.95\nuncertain_cases = selected[key_attributes][uncertain_idx]\nuncertain_cases.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"63b438087bc862ebf874316e26da13924b382947"},"cell_type":"markdown","source":"Obviously, when we don't know anything about the city, it is impossible for there to be any confidence in the prediction. So we will have to exclude the cases where both *city* and *region* are missing."},{"metadata":{"trusted":false,"_uuid":"2183aec49fc2cde9a718a2fdb1b6ae5e1eaefd3e"},"cell_type":"code","source":"uncertain_cases[(uncertain_cases['geoNetwork.city'] != 'N/A')\n                & (uncertain_cases['geoNetwork.region'] != 'N/A')].groupby(key_attributes).size().reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2df8627db545ab8435f2aa457277d503b00f6f9b"},"cell_type":"markdown","source":"As we see, there are not really any interesting cases where a 0.95 confidence thresholding tells us the algorithm might be making a mistake identifying anomalies. We may alter the confidence level, but the effect on the resulting detected anomalies is minimal.\n\nUsing a classifier for anomaly detection does not fix missing values in *region* as manual investigation did, but it is able to achieve nearly the same level of error fixing and allow us to reduce the chances of strange outliers throwing off further analysis. We now apply the error fixing results using the classifier method."},{"metadata":{"trusted":false,"_uuid":"68a84e54f06ff3bd1d1bc7a00291fc0ce755ceba"},"cell_type":"code","source":"not_na_idx = (selected['geoNetwork.city'] !=\n              'N/A') | (selected['geoNetwork.region'] != 'N/A')\n\ntarget = pd.Categorical.from_codes(\n    pred, categories=selected['geoNetwork.country'].cat.categories)\n\ndat2.loc[not_na_idx, 'geoNetwork.country'] = target[not_na_idx]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2f8ba98c3bf87ff9ca1421d5d5927101be5c2a0f"},"cell_type":"markdown","source":"Finally, let us look at where the two methods yield different columns:"},{"metadata":{"trusted":false,"_uuid":"857087160c2697a7919bddc45d6b5db29ca51688"},"cell_type":"code","source":"diff_idx = np.any(dat[key_attributes] != dat2[key_attributes], axis=1)\npd.concat([dat[key_attributes][diff_idx], dat2[key_attributes][diff_idx]], axis=1).head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6a93d541e9db6452f27c68aa3272d937eecbec71"},"cell_type":"markdown","source":"We can show that the only type of difference between these two methods is the assigning of values to NAs in the *region* column we did in the manual method. If we exclude all cases where the original *region* value is missing, the results of both methods are identical."},{"metadata":{"trusted":false,"_uuid":"af8e275742070c8e925f136e8bb5e5ca59ede5cd"},"cell_type":"code","source":"diff_idx = np.any(dat[key_attributes] != dat2[key_attributes], axis=1) & (dat2['geoNetwork.region'] != 'N/A')\npd.concat([dat[key_attributes][diff_idx], dat2[key_attributes][diff_idx]], axis=1).head(10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"353f1e98cca4fa87ee1c7d52e696c28c9b9f38f3"},"cell_type":"markdown","source":"We now export the fixing results of both methods."},{"metadata":{"trusted":false,"_uuid":"5dc35f9338295606e99e87c9c7e6028bb0e2784e"},"cell_type":"code","source":"for c in pure_geo_columns:\n    dat.loc[:, c] = dat[c].cat.remove_categories('N/A')\n\ndat.to_pickle(\"manual_geo_fix.pkl\")\n\nfor c in pure_geo_columns:\n    dat2.loc[:, c] = dat2[c].cat.remove_categories('N/A')\n\ndat2.to_pickle(\"classifier_geo_fix.pkl\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cd1f34aa9b313c36e425247372cb71b72031d106"},"cell_type":"markdown","source":"## Afterthoughts"},{"metadata":{"_uuid":"51c3cf77d83cebeee5f2ec087430881212f42f24"},"cell_type":"markdown","source":"It seems strange that so many obvious errors exist in the dataset. What if they are not really data errors? What if there is a reason for the inconsistencies? There are a few possibilities where such a situation would arise, for instance if there are two data sources, one from on-device GPS or registration location and one from ISP location. Let us see if we can find any clues from the domain names by looking at anomaly cases where the domain name field is not empty:"},{"metadata":{"trusted":true,"_uuid":"618dd24d2898786fbf515b3d632e1a8774c4ca36"},"cell_type":"code","source":"original = pd.read_pickle(\"../input/gstore-revenue-data-preprocessing/train.pkl\")\noriginal[is_anomaly & (~pd.isna(original['geoNetwork.networkDomain'])) &\n         (~pd.isna(original['geoNetwork.city'])\n          | ~pd.isna(original['geoNetwork.region']))][\n              key_attributes + ['geoNetwork.networkDomain']].head(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1b90ed094cc9cbe953eb190623a139be41b8b469"},"cell_type":"markdown","source":"Clearly there is a close relationship between *country* and *networkDomain*. The domain names here seem to suggest that they are the ISP of the user/session connection. We might make an educated guess and deduce that the ISP domain is determined from the client IP address recorded for the session, and the *country* attribute is entirely dependent on this domain. But what about the *city* attribute? How is it determined then? Is it more precise than *country* or otherwise? We are not quite sure. One more thing we can do, is to see which one is more stable for a given user or session."},{"metadata":{"trusted":true,"_uuid":"a7dc123c4302245e7338ada70ed2cd267809b407"},"cell_type":"code","source":"original[key_attributes + ['fullVisitorId']].dropna().groupby('fullVisitorId')[[\n    'geoNetwork.city', 'geoNetwork.country'\n]].nunique().mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e4bb83f69256d3adb29eb4927ef9a9c18f935a37"},"cell_type":"code","source":"original[key_attributes + ['sessionId']].dropna().groupby('sessionId')[[\n    'geoNetwork.city', 'geoNetwork.country'\n]].nunique().mean()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e0c1973855a213ad4b1f6220aed35249af83ba69"},"cell_type":"markdown","source":"As we see here, the *city* attribute seems to be slightly more stable for a given user or session. *Country* might even change half way through a session in some rare cases. We do not have much more evidence, but so far, there is slightly more reason to believe that the *city* attribute is a slightly more accurate attribute than *country*, and the fixes we did earlier seem reasonable."},{"metadata":{"trusted":true,"_uuid":"ec61fa11c13da78a06feb66cd121ce37b4c0ac36"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"hide_input":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}