{
  "id": 8245,
  "title": "Has anyone managed to join the 3 datasets provided?",
  "url": "/competitions/acquire-valued-shoppers-challenge/discussion/8245",
  "author_name": "",
  "post_date": "2014-05-21T16:36:57.200Z",
  "votes": 1,
  "comment_count": 6,
  "views": 2536,
  "content": "<p>Hello there!</p>\n\n<p>I joined the trainHistory with offers and with transactions on the columns mentioned in Data page but I'm not getting all the rows in trainHistory</p>\n<p>only about ~57000 out of ~150000.</p>\n<p>This is also the case when joining testHistory, offers and transactions.</p>\n<p>I get only ~40000 out of ~160000.</p>\n\n<p>Am I doing something/understading wrong?</p>\n\n<p>Cheers!</p>",
  "messages": [
    {
      "id": "45999",
      "postDate": "05/21/2014 16:36:57",
      "content": "<p>Hello there!</p>\n\n<p>I joined the trainHistory with offers and with transactions on the columns mentioned in Data page but I'm not getting all the rows in trainHistory</p>\n<p>only about ~57000 out of ~150000.</p>\n<p>This is also the case when joining testHistory, offers and transactions.</p>\n<p>I get only ~40000 out of ~160000.</p>\n\n<p>Am I doing something/understading wrong?</p>\n\n<p>Cheers!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "46022",
      "postDate": "05/21/2014 22:31:22",
      "content": "<p>I think if you do an inner join of all 3 tables, then customers with zero transactions with the same category/brand/company as their offer would drop out.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "46023",
      "postDate": "05/21/2014 23:06:13",
      "content": "<p>Ok, but how can a client get on offer with 0 transactions?</p>\n<p>From this column</p>\n<p>quantity - The number of units one must purchase to get the discount</p>\n<p>in offers dataset</p>\n<p>I deduce that a client must have at least one transaction with a certain quantity of the&nbsp;</p>\n<p>category/brand/company tuple to receive on offer.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "46029",
      "postDate": "05/22/2014 00:51:28",
      "content": "<p>[quote=SiberiaV2;46023]</p>\n<p>Ok, but how can a client get on offer with 0 transactions?</p>\n<p>[/quote]</p>\n<p>Siberia, you deduction is wrong. Customer can get a discount coupon for a product that (s)he never bought before in order to entice her/him to switch buying preference to this product manufactured by a competitor.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "46164",
      "postDate": "05/24/2014 07:25:11",
      "content": "<p>Transaction history ends up right before the day when he buy it, so it can be zero purches&nbsp;of this exactly product. In other case baseline &quot;prior category company brand&quot; should&nbsp;be meaningless</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "48690",
      "postDate": "06/05/2014 07:00:38",
      "content": "<p>will I be able to join correctly if I join transactions with history, then join offers to the joined file?</p>\n\n<p>eg. join transactions with history with exact matches of 'id' and 'chain', then join offers with the joined file with exact matches of 'offer', 'category', 'brand', 'company'. at each stages take out rows without exact matches.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "48697",
      "postDate": "06/05/2014 09:26:18",
      "content": "<p>I joined first history and offers by offers. Then transactions to HistOffer by id,chain. There are (if I remember correctly) 3 customers in train that are not &quot;joinable&quot; at every transactions line, since they also went to another chain. You can still read &quot;join these lines&quot; via id only and taking from Histoffer only repattrips and repeater (for which same ID is enough to be a valid join) and set remaining columns to NA.</p>\n\n<p>Atm, I am not using this file though, and depending on your purpose it is good enough or not:</p>\n<p>1) If you want to associate with every customer in the transactions file the offer they received plus the history data independent of whether the offer is in the same category,brand and company than the given transaction--&gt; it should be fine</p>\n\n<p>2) if you are interested in having for every transactions line, the history data and the offer only when it is corresponding to the right company,brand and category--&gt;then it does NOT work, but maybe a join by all 6 keys might work.</p>\n\n<p>Wrt to time it takes, if you split the transaction file to customers in train and in Test, it takes around 10 min in Fortran. How much time it would take in a database I do not know, but I am curious...anyone done it?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 46022,
      "author_name": "breakfastpirate",
      "author_url": "",
      "post_date": "05/21/2014 22:31:22",
      "content": "<p>I think if you do an inner join of all 3 tables, then customers with zero transactions with the same category/brand/company as their offer would drop out.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 46023,
      "author_name": "",
      "author_url": "",
      "post_date": "05/21/2014 23:06:13",
      "content": "<p>Ok, but how can a client get on offer with 0 transactions?</p>\n<p>From this column</p>\n<p>quantity - The number of units one must purchase to get the discount</p>\n<p>in offers dataset</p>\n<p>I deduce that a client must have at least one transaction with a certain quantity of the&nbsp;</p>\n<p>category/brand/company tuple to receive on offer.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 46029,
      "author_name": "odessa",
      "author_url": "",
      "post_date": "05/22/2014 00:51:28",
      "content": "<p>[quote=SiberiaV2;46023]</p>\n<p>Ok, but how can a client get on offer with 0 transactions?</p>\n<p>[/quote]</p>\n<p>Siberia, you deduction is wrong. Customer can get a discount coupon for a product that (s)he never bought before in order to entice her/him to switch buying preference to this product manufactured by a competitor.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 46164,
      "author_name": "dremovd",
      "author_url": "",
      "post_date": "05/24/2014 07:25:11",
      "content": "<p>Transaction history ends up right before the day when he buy it, so it can be zero purches&nbsp;of this exactly product. In other case baseline &quot;prior category company brand&quot; should&nbsp;be meaningless</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 48690,
      "author_name": "jacktracy",
      "author_url": "",
      "post_date": "06/05/2014 07:00:38",
      "content": "<p>will I be able to join correctly if I join transactions with history, then join offers to the joined file?</p>\n\n<p>eg. join transactions with history with exact matches of 'id' and 'chain', then join offers with the joined file with exact matches of 'offer', 'category', 'brand', 'company'. at each stages take out rows without exact matches.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 48697,
      "author_name": "sandros",
      "author_url": "",
      "post_date": "06/05/2014 09:26:18",
      "content": "<p>I joined first history and offers by offers. Then transactions to HistOffer by id,chain. There are (if I remember correctly) 3 customers in train that are not &quot;joinable&quot; at every transactions line, since they also went to another chain. You can still read &quot;join these lines&quot; via id only and taking from Histoffer only repattrips and repeater (for which same ID is enough to be a valid join) and set remaining columns to NA.</p>\n\n<p>Atm, I am not using this file though, and depending on your purpose it is good enough or not:</p>\n<p>1) If you want to associate with every customer in the transactions file the offer they received plus the history data independent of whether the offer is in the same category,brand and company than the given transaction--&gt; it should be fine</p>\n\n<p>2) if you are interested in having for every transactions line, the history data and the offer only when it is corresponding to the right company,brand and category--&gt;then it does NOT work, but maybe a join by all 6 keys might work.</p>\n\n<p>Wrt to time it takes, if you split the transaction file to customers in train and in Test, it takes around 10 min in Fortran. How much time it would take in a database I do not know, but I am curious...anyone done it?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "45999": "",
    "46022": "",
    "46023": "",
    "46029": "",
    "46164": "",
    "48690": "",
    "48697": ""
  },
  "source": "meta"
}