{
  "id": 75675,
  "title": "Difference between datasets",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/75675",
  "author_name": "PaoloRampazzo",
  "post_date": "2018-12-24T21:01:29.403000",
  "votes": 0,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi all!\nWell, although we all have a lot of uncertainties about the data, something that caught my attention was the fact that apart from the train.csv, there are other three files of information. If we can get information from these files not only to tune our algorithm but also to get the predictions for the test dataset, what is the point in naming it train.csv ? I mean, is that allowed that I use all the information available to compose a bigger train and use it all to create my model ?</p>\n\n<p>Thanks in advance! </p>",
  "messages": [
    {
      "id": 444807,
      "postDate": "2018-12-24T21:27:08.257Z",
      "content": "<p>Hi! Paolo, the name train.csv doesn't mean anything in my own opinion, I'm currently using a full trainig_dataset including all the information provided with multiple levels of aggregation.\nI think is totally allowed to use all the information.</p>",
      "rawMarkdown": "Hi! Paolo, the name train.csv doesn't mean anything in my own opinion, I'm currently using a full trainig_dataset including all the information provided with multiple levels of aggregation.\nI think is totally allowed to use all the information.",
      "votes": 4,
      "replies": [
        {
          "id": 446925,
          "postDate": "2018-12-28T21:58:55.570Z",
          "content": "<p>Thank you Carlos for your reply! That's what I thought as well</p>",
          "rawMarkdown": "Thank you Carlos for your reply! That's what I thought as well"
        }
      ]
    },
    {
      "id": 446179,
      "postDate": "2018-12-27T16:47:31.927Z",
      "content": "<p>I think thats rt, you got to combine train csv with historical transactions as well as New Merchants transaction and train your model. However for test csv there is no intersection with New Merchants I believe, so you got to create similar features based on historical transactions ,and use those for predictions , thats what I got.</p>",
      "rawMarkdown": "I think thats rt, you got to combine train csv with historical transactions as well as New Merchants transaction and train your model. However for test csv there is no intersection with New Merchants I believe, so you got to create similar features based on historical transactions ,and use those for predictions , thats what I got.",
      "votes": 1,
      "replies": [
        {
          "id": 446926,
          "postDate": "2018-12-28T22:00:40.340Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 446927,
          "postDate": "2018-12-28T22:01:02.640Z",
          "content": "<p>Thanks for your answer! However, you can check that every id from test.csv can be found either on historical or in new merchants. It means that we have available information for all the test data set in order to create a model!</p>",
          "rawMarkdown": "Thanks for your answer! However, you can check that every id from test.csv can be found either on historical or in new merchants. It means that we have available information for all the test data set in order to create a model!"
        }
      ]
    },
    {
      "id": 444799,
      "postDate": "2018-12-24T21:01:29.403Z",
      "content": "<p>Hi all!\nWell, although we all have a lot of uncertainties about the data, something that caught my attention was the fact that apart from the train.csv, there are other three files of information. If we can get information from these files not only to tune our algorithm but also to get the predictions for the test dataset, what is the point in naming it train.csv ? I mean, is that allowed that I use all the information available to compose a bigger train and use it all to create my model ?</p>\n\n<p>Thanks in advance! </p>",
      "rawMarkdown": "Hi all!\nWell, although we all have a lot of uncertainties about the data, something that caught my attention was the fact that apart from the train.csv, there are other three files of information. If we can get information from these files not only to tune our algorithm but also to get the predictions for the test dataset, what is the point in naming it train.csv ? I mean, is that allowed that I use all the information available to compose a bigger train and use it all to create my model ?\n\nThanks in advance! "
    }
  ],
  "comments": [
    {
      "id": 444807,
      "author_name": "C4rl05/V",
      "author_url": "",
      "post_date": "2018-12-24T21:27:08.257000",
      "content": "<p>Hi! Paolo, the name train.csv doesn't mean anything in my own opinion, I'm currently using a full trainig_dataset including all the information provided with multiple levels of aggregation.\nI think is totally allowed to use all the information.</p>",
      "votes": 4,
      "replies": [
        {
          "id": 446925,
          "author_name": "PaoloRampazzo",
          "author_url": "",
          "post_date": "2018-12-28T21:58:55.570000",
          "content": "<p>Thank you Carlos for your reply! That's what I thought as well</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 446179,
      "author_name": "AnirbanK",
      "author_url": "",
      "post_date": "2018-12-27T16:47:31.927000",
      "content": "<p>I think thats rt, you got to combine train csv with historical transactions as well as New Merchants transaction and train your model. However for test csv there is no intersection with New Merchants I believe, so you got to create similar features based on historical transactions ,and use those for predictions , thats what I got.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 446926,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-12-28T22:00:40.340000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 446927,
          "author_name": "PaoloRampazzo",
          "author_url": "",
          "post_date": "2018-12-28T22:01:02.640000",
          "content": "<p>Thanks for your answer! However, you can check that every id from test.csv can be found either on historical or in new merchants. It means that we have available information for all the test data set in order to create a model!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "444807": "Hi! Paolo, the name train.csv doesn't mean anything in my own opinion, I'm currently using a full trainig_dataset including all the information provided with multiple levels of aggregation.\nI think is totally allowed to use all the information.",
    "446179": "I think thats rt, you got to combine train csv with historical transactions as well as New Merchants transaction and train your model. However for test csv there is no intersection with New Merchants I believe, so you got to create similar features based on historical transactions ,and use those for predictions , thats what I got.",
    "444799": "Hi all!\nWell, although we all have a lot of uncertainties about the data, something that caught my attention was the fact that apart from the train.csv, there are other three files of information. If we can get information from these files not only to tune our algorithm but also to get the predictions for the test dataset, what is the point in naming it train.csv ? I mean, is that allowed that I use all the information available to compose a bigger train and use it all to create my model ?\n\nThanks in advance! "
  }
}