{
  "id": 11075,
  "title": "Feature range from test data",
  "url": "/competitions/avazu-ctr-prediction/discussion/11075",
  "author_name": "",
  "post_date": "2014-11-28T05:10:12.307Z",
  "votes": 1,
  "comment_count": 6,
  "views": 1667,
  "content": "<p>Are we allowed to extract the range of feature values from the test data, as long as we do not use the test data feature vectors directly in training the model? For example, suppose we have device IPs A,B,C in the training data, and device IPs A,B,C,D in the test data, can we use knowledge of the whole set A,B,C,D for the range of device IPs?</p>",
  "messages": [
    {
      "id": "59138",
      "postDate": "11/28/2014 05:10:12",
      "content": "<p>Are we allowed to extract the range of feature values from the test data, as long as we do not use the test data feature vectors directly in training the model? For example, suppose we have device IPs A,B,C in the training data, and device IPs A,B,C,D in the test data, can we use knowledge of the whole set A,B,C,D for the range of device IPs?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59140",
      "postDate": "11/28/2014 06:05:21",
      "content": "<p>Oh yeah, sure you can.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59187",
      "postDate": "11/28/2014 21:30:35",
      "content": "<p>Thanks for the response. I would really also like an official confirmation from the competition admin. ;-)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59228",
      "postDate": "11/29/2014 17:13:26",
      "content": "<p>@Basu</p>\n<p>Are you thinking of building a validation set that mimic the test set based on&nbsp; the total number of values in common?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59243",
      "postDate": "11/29/2014 20:27:08",
      "content": "<p>I haven't really thought that far out. ;-) It just seems to me that for the hashed values of site_id, site_domain, device_id etc. if we don't have knowledge of the values they can take in the test data, it is not meaningful to extrapolate from the values in the training data only.</p>\n<p>Because technically this would be using the test data for training, even if indirectly, I would like to clarify from the admins the extent to which this is permissible, to avoid surprises later on.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59247",
      "postDate": "11/30/2014 00:18:46",
      "content": "<p>I don't recall seeing anything in the rules restricting the use of information from the test data for training.</p>\n<p>Can you point it out for me?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59249",
      "postDate": "11/30/2014 00:48:11",
      "content": "<p>There isn't any such mention in the rules, but this issue was raised in a recently-finished competition, where one of the admins responded as follows:</p>\n<p><a href=\"https://www.kaggle.com/c/seizure-prediction/forums/t/10790/use-of-test-data/57025#post57025\">Use of test data</a></p>\n<p>which is why I wanted to clarify it in this case.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 59140,
      "author_name": "xiaozhouwang",
      "author_url": "",
      "post_date": "11/28/2014 06:05:21",
      "content": "<p>Oh yeah, sure you can.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59187,
      "author_name": "ushnishbasu",
      "author_url": "",
      "post_date": "11/28/2014 21:30:35",
      "content": "<p>Thanks for the response. I would really also like an official confirmation from the competition admin. ;-)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59228,
      "author_name": "herimanitra",
      "author_url": "",
      "post_date": "11/29/2014 17:13:26",
      "content": "<p>@Basu</p>\n<p>Are you thinking of building a validation set that mimic the test set based on&nbsp; the total number of values in common?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59243,
      "author_name": "ushnishbasu",
      "author_url": "",
      "post_date": "11/29/2014 20:27:08",
      "content": "<p>I haven't really thought that far out. ;-) It just seems to me that for the hashed values of site_id, site_domain, device_id etc. if we don't have knowledge of the values they can take in the test data, it is not meaningful to extrapolate from the values in the training data only.</p>\n<p>Because technically this would be using the test data for training, even if indirectly, I would like to clarify from the admins the extent to which this is permissible, to avoid surprises later on.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59247,
      "author_name": "vvarkey",
      "author_url": "",
      "post_date": "11/30/2014 00:18:46",
      "content": "<p>I don't recall seeing anything in the rules restricting the use of information from the test data for training.</p>\n<p>Can you point it out for me?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59249,
      "author_name": "ushnishbasu",
      "author_url": "",
      "post_date": "11/30/2014 00:48:11",
      "content": "<p>There isn't any such mention in the rules, but this issue was raised in a recently-finished competition, where one of the admins responded as follows:</p>\n<p><a href=\"https://www.kaggle.com/c/seizure-prediction/forums/t/10790/use-of-test-data/57025#post57025\">Use of test data</a></p>\n<p>which is why I wanted to clarify it in this case.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "59138": "",
    "59140": "",
    "59187": "",
    "59228": "",
    "59243": "",
    "59247": "",
    "59249": ""
  },
  "source": "meta"
}