{
  "id": 10819,
  "title": "Unique/Missing Train and Test Values",
  "url": "/competitions/avazu-ctr-prediction/discussion/10819",
  "author_name": "",
  "post_date": "2014-11-01T18:25:03.160Z",
  "votes": 13,
  "comment_count": 22,
  "views": 8367,
  "content": "<p>Hi all,</p>\n<p>I've computed the number of unique values for each key for both train and test datasets, suposing that variables are categorical. Results are pasted below, too bad the kaggle post system messed up my formatting.</p>\n<p>Key: name of the key</p>\n<p>#Test: number of unique values in test</p>\n<p>Unique Test: values present in test but not in train</p>\n<p>#Train: number unique values in train</p>\n<p>Unique Train: values present in train but not in test</p>\n<p>-------------------------------</p>\n<p>Key, #Test, Unique Test, #Train, Unique Train <br>id 4769401 4769401 47686351 47686351 <br>hour 24 24 240 240 <br>C1 7 0 7 0 <br>banner_pos 7 0 7 0 <br>site_id 2919 86 4715 1882 <br>site_domain 3681 252 7733 4304 <br>site_category 21 0 26 5 <br>app_id 2904 239 7941 5276 <br>app_domain 139 6 433 300 <br>app_category 22 0 38 16 <br>device_id 435864 358114 2787564 2709814 <br>device_ip 978837 606975 6081079 5709217 <br>device_os 10 0 15 5 <br>device_make 246 1 283 38 <br>device_model 5530 53 8320 2843 <br>device_type 5 1 4 0 <br>device_conn_type 4 0 4 0 <br>device_geo_country 227 1 232 6 <br>C17 1480 408 2592 1520 <br>C18 8 0 8 0 <br>C19 9 0 9 0 <br>C20 294 64 457 227 <br>C21 4 0 4 0 <br>C22 39 4 61 26 <br>C23 163 0 171 8 <br>C24 31 2 53 24</p>\n<p>--------------------------</p>\n<p>There are some interesecting facts from this:</p>\n<p>- The set of hours in test are completely different than in train. Maybe a good cross-validation strategy is to split the dataset by hours, then.</p>\n<p>- There is a high number of device_id, device_ip unique in both datasets</p>\n<p>- Some columns C17, C20 seem not to be categorical variables, but integer ranges instead. It&#180;s hard to know. Any ideas?</p>\n\n<p>I think people should find a way to deal with missing values, they are present in almost every single column.</p>",
  "messages": [
    {
      "id": "57169",
      "postDate": "11/01/2014 18:25:03",
      "content": "<p>Hi all,</p>\n<p>I've computed the number of unique values for each key for both train and test datasets, suposing that variables are categorical. Results are pasted below, too bad the kaggle post system messed up my formatting.</p>\n<p>Key: name of the key</p>\n<p>#Test: number of unique values in test</p>\n<p>Unique Test: values present in test but not in train</p>\n<p>#Train: number unique values in train</p>\n<p>Unique Train: values present in train but not in test</p>\n<p>-------------------------------</p>\n<p>Key, #Test, Unique Test, #Train, Unique Train <br>id 4769401 4769401 47686351 47686351 <br>hour 24 24 240 240 <br>C1 7 0 7 0 <br>banner_pos 7 0 7 0 <br>site_id 2919 86 4715 1882 <br>site_domain 3681 252 7733 4304 <br>site_category 21 0 26 5 <br>app_id 2904 239 7941 5276 <br>app_domain 139 6 433 300 <br>app_category 22 0 38 16 <br>device_id 435864 358114 2787564 2709814 <br>device_ip 978837 606975 6081079 5709217 <br>device_os 10 0 15 5 <br>device_make 246 1 283 38 <br>device_model 5530 53 8320 2843 <br>device_type 5 1 4 0 <br>device_conn_type 4 0 4 0 <br>device_geo_country 227 1 232 6 <br>C17 1480 408 2592 1520 <br>C18 8 0 8 0 <br>C19 9 0 9 0 <br>C20 294 64 457 227 <br>C21 4 0 4 0 <br>C22 39 4 61 26 <br>C23 163 0 171 8 <br>C24 31 2 53 24</p>\n<p>--------------------------</p>\n<p>There are some interesecting facts from this:</p>\n<p>- The set of hours in test are completely different than in train. Maybe a good cross-validation strategy is to split the dataset by hours, then.</p>\n<p>- There is a high number of device_id, device_ip unique in both datasets</p>\n<p>- Some columns C17, C20 seem not to be categorical variables, but integer ranges instead. It&#180;s hard to know. Any ideas?</p>\n\n<p>I think people should find a way to deal with missing values, they are present in almost every single column.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57266",
      "postDate": "11/04/2014 03:57:07",
      "content": "<p>Thank you for doing this. This is useful information.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57267",
      "postDate": "11/04/2014 04:15:49",
      "content": "<p>I looked at my counts of unique values for the test set and compared to your numbers. Your test set counts for device_id and device_ip match what I got for app_domain and app_id. Also, I have 2919 unique values for C3, 3681 for C4, 21 values for banner_pos. The numbers look right but the column names appear to be different from mine.</p>\n<p>Is there any chance you mixed them up? If not, I need to check how I am reading these. I got the same counts in R and Python.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57595",
      "postDate": "11/10/2014 12:43:31",
      "content": "<p>I have not found any click variable in test_rev2.csv.Do you guys also face the same thing</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57596",
      "postDate": "11/10/2014 12:49:22",
      "content": "<p>[quote=Saptarshi Ray;57595]</p>\n<p>I have not found any click variable in test_rev2.csv.Do you guys also face the same thing</p>\n<p>[/quote]</p>\n<p>what will u predict if you have everything?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57600",
      "postDate": "11/10/2014 13:38:02",
      "content": "<p>Saptarshi,</p>\n<p>What Abhishek is saying is that not having the click variable in the test set is correct. You need to predict it.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57604",
      "postDate": "11/10/2014 14:13:25",
      "content": "<p>Ok . Now I have got it;Thanks for the correction.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57929",
      "postDate": "11/12/2014 17:01:49",
      "content": "<p style=\"text-align: left\">What do you think about app* features?</p>\n<p style=\"text-align: left\">What does app_id means? app domain? App category?</p>\n<p>Site_id - site where ad showed.</p>\n<p><span style=\"line-height: 1.4\">Device_id - browser.</span></p>\n<p>Is app_id something like publisher id?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57932",
      "postDate": "11/12/2014 17:31:02",
      "content": "<p>[quote=Shalnovv;57929]</p>\n<p style=\"text-align: left\">What do you think about app* features?</p>\n<p style=\"text-align: left\">What does app_id means? app domain? App category?</p>\n<p>[/quote]</p>\n<p>You can be served a&nbsp;mobile ad either from a webpage you are browsing, or from an application you are running, e.g.,</p>\n<p>http://www.mobyaffiliates.com/wp-content/uploads/2013/10/leadbolt-ad-formats.jpg</p>\n<p>Category is probably something like, e.g., Lifestyle, Games, etc. Domain might be something like sub-category (although, I'm just guessing on that one).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57992",
      "postDate": "11/13/2014 15:00:58",
      "content": "<p>If it is mobile app id. Why site id and app id present in one string?</p>\n<p>What does it means?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58008",
      "postDate": "11/13/2014 16:46:03",
      "content": "<p>[quote=Shalnovv;57992]</p>\n<p>If it is mobile app id. Why site id and app id present in one string?</p>\n<p>What does it means?</p>\n<p>[/quote]</p>\n<p>In those columns, replace&nbsp;d41d8cd9 with NULL. You'll see the ad was either served on a site or an app.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58332",
      "postDate": "11/18/2014 18:45:20",
      "content": "<p>It's harder to see the NULL fields now - I think they're encoded per column?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58338",
      "postDate": "11/18/2014 19:36:40",
      "content": "<p>NULLs for site_id, site_domain, site_category: 85f751fd, c4e18dd6, 50e219e0</p>\n<p>NULLs for app_id, app_domain, app_category: ecad2386, 7801e8d9, 07d7df22<br><br></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58339",
      "postDate": "11/18/2014 19:37:31",
      "content": "<p>Bless you!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58344",
      "postDate": "11/18/2014 20:58:37",
      "content": "<p>[quote=laserwolf;58338]</p>\n<p>NULLs for site_id, site_domain, site_category: 85f751fd, c4e18dd6, 50e219e0</p>\n<p>NULLs for app_id, app_domain, app_category: ecad2386, 7801e8d9, 07d7df22<br><br></p>\n<p>[/quote]</p>\n<p>How did you come up with the hash values for NULL? Was your observation something like, when you have 3 distinct values for site_*, you have the same triplet [ecad2386, 7801e8d9, 07d7df22] occuring in app_*, indicating NULL here and vice versa.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58349",
      "postDate": "11/18/2014 21:35:18",
      "content": "<p>yes - I was just working it out when laserwolf wrote their post</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "58359",
      "postDate": "11/19/2014 00:26:20",
      "content": "<p>For site_category it's not clear what to count as NULL:</p>\n<p><code>50e219e0 16 537 234 <br>f028772b 12 657 073 <br>28905ebd &nbsp;7 377 208 <br>3e814130 &nbsp;3 050 306 <br>f66779e6 &nbsp; &nbsp;252 451 <br>75fa27f6 &nbsp; &nbsp;160 985 <br></code></p>\n<p>At least top2 values could be considered nulls. Same with other columns, nulls distribution seems to be very different from what it used to be. Different encoding? Different sampling?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59081",
      "postDate": "11/27/2014 09:02:17",
      "content": "<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59084",
      "postDate": "11/27/2014 09:09:58",
      "content": "<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59085",
      "postDate": "11/27/2014 09:14:30",
      "content": "<p>[quote=Steve Wang;59084]</p>\n<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>\n<p>[/quote]</p>\n<p>There was no intention of data cracking, merely curious about what can and cannot be cleaned before training.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59118",
      "postDate": "11/27/2014 21:59:38",
      "content": "<p>All null values should be identified, this takes out data cracking.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59119",
      "postDate": "11/27/2014 22:03:47",
      "content": "<p>@Frank, I'm not sure I understand what you mean.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59939",
      "postDate": "12/10/2014 08:25:26",
      "content": "<p>[quote=Steve Wang;59084]</p>\n<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>\n<p>[/quote]</p>\n\n<p>Of course, it's your contest to run&nbsp;however you see fit, but inferring the NULL values and then training separate models for apps and sites gave me a significant boost on the leaderboard (&#8593; 34 places). It seems very odd to obfuscate NULL&nbsp;values in a data set.<br><br>In these contests, I would recommend providing all salient features that an in-house Data Scientist might leverage to model the problem. Training separate models for different user scenarios (mobile apps vs.&nbsp;sites)&nbsp;and using client-local time of day, are definitely two things I would normally use&nbsp;(and have applied&nbsp;on similar problems).</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 57266,
      "author_name": "alkhwarizmi",
      "author_url": "",
      "post_date": "11/04/2014 03:57:07",
      "content": "<p>Thank you for doing this. This is useful information.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57267,
      "author_name": "alkhwarizmi",
      "author_url": "",
      "post_date": "11/04/2014 04:15:49",
      "content": "<p>I looked at my counts of unique values for the test set and compared to your numbers. Your test set counts for device_id and device_ip match what I got for app_domain and app_id. Also, I have 2919 unique values for C3, 3681 for C4, 21 values for banner_pos. The numbers look right but the column names appear to be different from mine.</p>\n<p>Is there any chance you mixed them up? If not, I need to check how I am reading these. I got the same counts in R and Python.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57595,
      "author_name": "saptarshi207",
      "author_url": "",
      "post_date": "11/10/2014 12:43:31",
      "content": "<p>I have not found any click variable in test_rev2.csv.Do you guys also face the same thing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57596,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "11/10/2014 12:49:22",
      "content": "<p>[quote=Saptarshi Ray;57595]</p>\n<p>I have not found any click variable in test_rev2.csv.Do you guys also face the same thing</p>\n<p>[/quote]</p>\n<p>what will u predict if you have everything?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57600,
      "author_name": "guillempalou",
      "author_url": "",
      "post_date": "11/10/2014 13:38:02",
      "content": "<p>Saptarshi,</p>\n<p>What Abhishek is saying is that not having the click variable in the test set is correct. You need to predict it.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57604,
      "author_name": "saptarshi207",
      "author_url": "",
      "post_date": "11/10/2014 14:13:25",
      "content": "<p>Ok . Now I have got it;Thanks for the correction.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57929,
      "author_name": "shalnovv",
      "author_url": "",
      "post_date": "11/12/2014 17:01:49",
      "content": "<p style=\"text-align: left\">What do you think about app* features?</p>\n<p style=\"text-align: left\">What does app_id means? app domain? App category?</p>\n<p>Site_id - site where ad showed.</p>\n<p><span style=\"line-height: 1.4\">Device_id - browser.</span></p>\n<p>Is app_id something like publisher id?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57932,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "11/12/2014 17:31:02",
      "content": "<p>[quote=Shalnovv;57929]</p>\n<p style=\"text-align: left\">What do you think about app* features?</p>\n<p style=\"text-align: left\">What does app_id means? app domain? App category?</p>\n<p>[/quote]</p>\n<p>You can be served a&nbsp;mobile ad either from a webpage you are browsing, or from an application you are running, e.g.,</p>\n<p>http://www.mobyaffiliates.com/wp-content/uploads/2013/10/leadbolt-ad-formats.jpg</p>\n<p>Category is probably something like, e.g., Lifestyle, Games, etc. Domain might be something like sub-category (although, I'm just guessing on that one).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57992,
      "author_name": "shalnovv",
      "author_url": "",
      "post_date": "11/13/2014 15:00:58",
      "content": "<p>If it is mobile app id. Why site id and app id present in one string?</p>\n<p>What does it means?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58008,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "11/13/2014 16:46:03",
      "content": "<p>[quote=Shalnovv;57992]</p>\n<p>If it is mobile app id. Why site id and app id present in one string?</p>\n<p>What does it means?</p>\n<p>[/quote]</p>\n<p>In those columns, replace&nbsp;d41d8cd9 with NULL. You'll see the ad was either served on a site or an app.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58332,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "11/18/2014 18:45:20",
      "content": "<p>It's harder to see the NULL fields now - I think they're encoded per column?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58338,
      "author_name": "vvarkey",
      "author_url": "",
      "post_date": "11/18/2014 19:36:40",
      "content": "<p>NULLs for site_id, site_domain, site_category: 85f751fd, c4e18dd6, 50e219e0</p>\n<p>NULLs for app_id, app_domain, app_category: ecad2386, 7801e8d9, 07d7df22<br><br></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58339,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "11/18/2014 19:37:31",
      "content": "<p>Bless you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58344,
      "author_name": "phanisrikanth",
      "author_url": "",
      "post_date": "11/18/2014 20:58:37",
      "content": "<p>[quote=laserwolf;58338]</p>\n<p>NULLs for site_id, site_domain, site_category: 85f751fd, c4e18dd6, 50e219e0</p>\n<p>NULLs for app_id, app_domain, app_category: ecad2386, 7801e8d9, 07d7df22<br><br></p>\n<p>[/quote]</p>\n<p>How did you come up with the hash values for NULL? Was your observation something like, when you have 3 distinct values for site_*, you have the same triplet [ecad2386, 7801e8d9, 07d7df22] occuring in app_*, indicating NULL here and vice versa.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58349,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "11/18/2014 21:35:18",
      "content": "<p>yes - I was just working it out when laserwolf wrote their post</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 58359,
      "author_name": "dmitrynozhnin",
      "author_url": "",
      "post_date": "11/19/2014 00:26:20",
      "content": "<p>For site_category it's not clear what to count as NULL:</p>\n<p><code>50e219e0 16 537 234 <br>f028772b 12 657 073 <br>28905ebd &nbsp;7 377 208 <br>3e814130 &nbsp;3 050 306 <br>f66779e6 &nbsp; &nbsp;252 451 <br>75fa27f6 &nbsp; &nbsp;160 985 <br></code></p>\n<p>At least top2 values could be considered nulls. Same with other columns, nulls distribution seems to be very different from what it used to be. Different encoding? Different sampling?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59081,
      "author_name": "billderose",
      "author_url": "",
      "post_date": "11/27/2014 09:02:17",
      "content": "<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59084,
      "author_name": "stevewang0",
      "author_url": "",
      "post_date": "11/27/2014 09:09:58",
      "content": "<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59085,
      "author_name": "billderose",
      "author_url": "",
      "post_date": "11/27/2014 09:14:30",
      "content": "<p>[quote=Steve Wang;59084]</p>\n<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>\n<p>[/quote]</p>\n<p>There was no intention of data cracking, merely curious about what can and cannot be cleaned before training.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59118,
      "author_name": "frankhasbani",
      "author_url": "",
      "post_date": "11/27/2014 21:59:38",
      "content": "<p>All null values should be identified, this takes out data cracking.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59119,
      "author_name": "billderose",
      "author_url": "",
      "post_date": "11/27/2014 22:03:47",
      "content": "<p>@Frank, I'm not sure I understand what you mean.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59939,
      "author_name": "brundage",
      "author_url": "",
      "post_date": "12/10/2014 08:25:26",
      "content": "<p>[quote=Steve Wang;59084]</p>\n<p>[quote=Bill DeRose;59081]</p>\n<p>There seem to be multiple reports of how NULL values are encoded in the dataset. So far I've seen that all NULL values are hashed to 'd41d8cd9'. Laserwolf above gives multiple different hashes of NULL depending on the column, and I've also noticed a lot of '-1' in C23.</p>\n<p>Which of these are confirmed to be NULL by Avazu?</p>\n<p>[/quote]</p>\n<p>As to provent from data cracking problems, I will not provide this kind of information, and data cracking is not encouraged&#65292;sorry for inconvenience</p>\n<p>[/quote]</p>\n\n<p>Of course, it's your contest to run&nbsp;however you see fit, but inferring the NULL values and then training separate models for apps and sites gave me a significant boost on the leaderboard (&#8593; 34 places). It seems very odd to obfuscate NULL&nbsp;values in a data set.<br><br>In these contests, I would recommend providing all salient features that an in-house Data Scientist might leverage to model the problem. Training separate models for different user scenarios (mobile apps vs.&nbsp;sites)&nbsp;and using client-local time of day, are definitely two things I would normally use&nbsp;(and have applied&nbsp;on similar problems).</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "57169": "",
    "57266": "",
    "57267": "",
    "57595": "",
    "57596": "",
    "57600": "",
    "57604": "",
    "57929": "",
    "57932": "",
    "57992": "",
    "58008": "",
    "58332": "",
    "58338": "",
    "58339": "",
    "58344": "",
    "58349": "",
    "58359": "",
    "59081": "",
    "59084": "",
    "59085": "",
    "59118": "",
    "59119": "",
    "59939": ""
  },
  "source": "meta"
}