{
  "id": 38188,
  "title": "Solution LB 39",
  "content": "<p>Congratulations to everyone who completed this competition. This is in fact my first kaggle competition and I really enjoy it . Here I am sharing my solution.  Advice would be highly appreciated ! </p>\n\n<h2>Solution</h2>\n\n<p>The problem is formulated as approximating P(u,p | user's prior purchase history) which stands for how much likely user u would repurchase product p given prior purchase history.</p>\n\n<p>The main model should be a binary classifer and features created manually or automatically are fed to the classfier for generating predictions.</p>\n\n<h3>Features</h3>\n\n<p>I construted both manual features and automatic features using unsupervised learning and neural networks.</p>\n\n<p>Manual features include statistics of prior purchase history. As for automatic features, I used the following </p>\n\nLDA\n\n<ul>\n<li>each user is treated as a document and each product is treated as a word. </li>\n<li>generate topic representation of each user and each product</li>\n<li>calculate score by taking inner product for  </li>\n<li>similar operations on aisle and department level</li>\n<li>both score  and compressed topic representations of users/items serves as good features</li>\n</ul>\n\nWORD2VEC\n\n<ul>\n<li>similar as LDA, but only on product level</li>\n</ul>\n\nLSTM\n\n<ul>\n<li>Interval between user u's sequential purchase of product p is modeled as a time sequence.</li>\n<li>Use LSTM to construct regression model for predicting next value of this time sequence.</li>\n<li>The predicted next interval serves as a good feature.</li>\n</ul>\n\nDREAM\n\n<ul>\n<li>RNN and bayesian personalized rank based Model, refer to <a href=\"https://github.com/LaceyChen17/DREAM\">this repo</a> for my implementation</li>\n<li>DREAM provides  scores, dynamic user representaions and item embeddings.</li>\n<li>It captures the sequential information such as periodicity in users' prior purchase history.</li>\n</ul>\n\n<h3>Classifier</h3>\n\n<p>I constructed both lightgbm model and xgboost model. </p>\n\n<h3>Optimization</h3>\n\n<p>I used bayesian optimization to tune my lightgbm model.</p>\n\n<h3>Post-classification</h3>\n\n<p>I used <a href=\"https://www.kaggle.com/tarobxl/parallel-version-of-faron-s-script/\">this script</a> to contruct orders from  pair. Thanks to faron, shing and tarbox !</p>\n\n<h3>Ensemble</h3>\n\n<p>I trained big models (500+ features), median models(260 + features) and small models(80+ features). Final submissions were generated by bagging top models using median.</p>\n\n<p>Here is <a href=\"https://github.com/LaceyChen17/instacart-market-basket-analysis/blob/master/README.md\">link</a> for my code. It's a bit messy (´・ω・`) </p>",
  "comments": [
    {
      "id": 214256,
      "author_name": "",
      "author_url": "",
      "post_date": "2017-08-16T12:07:18.050000",
      "content": "<p>Great solution. I am in Beijing too. Would you mind add my WeChat 472508502 to make a friend? May we do research about 2nd and 3rd code together?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 214260,
          "author_name": "Yihong Chen",
          "author_url": "",
          "post_date": "2017-08-16T12:20:52.163000",
          "content": "<p>deleted</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "comment_count": 2,
  "messages": [
    {
      "id": 214236,
      "postDate": "2017-08-16T10:50:34.220Z",
      "content": "<p>Congratulations to everyone who completed this competition. This is in fact my first kaggle competition and I really enjoy it . Here I am sharing my solution.  Advice would be highly appreciated ! </p>\n\n<h2>Solution</h2>\n\n<p>The problem is formulated as approximating P(u,p | user's prior purchase history) which stands for how much likely user u would repurchase product p given prior purchase history.</p>\n\n<p>The main model should be a binary classifer and features created manually or automatically are fed to the classfier for generating predictions.</p>\n\n<h3>Features</h3>\n\n<p>I construted both manual features and automatic features using unsupervised learning and neural networks.</p>\n\n<p>Manual features include statistics of prior purchase history. As for automatic features, I used the following </p>\n\nLDA\n\n<ul>\n<li>each user is treated as a document and each product is treated as a word. </li>\n<li>generate topic representation of each user and each product</li>\n<li>calculate score by taking inner product for  </li>\n<li>similar operations on aisle and department level</li>\n<li>both score  and compressed topic representations of users/items serves as good features</li>\n</ul>\n\nWORD2VEC\n\n<ul>\n<li>similar as LDA, but only on product level</li>\n</ul>\n\nLSTM\n\n<ul>\n<li>Interval between user u's sequential purchase of product p is modeled as a time sequence.</li>\n<li>Use LSTM to construct regression model for predicting next value of this time sequence.</li>\n<li>The predicted next interval serves as a good feature.</li>\n</ul>\n\nDREAM\n\n<ul>\n<li>RNN and bayesian personalized rank based Model, refer to <a href=\"https://github.com/LaceyChen17/DREAM\">this repo</a> for my implementation</li>\n<li>DREAM provides  scores, dynamic user representaions and item embeddings.</li>\n<li>It captures the sequential information such as periodicity in users' prior purchase history.</li>\n</ul>\n\n<h3>Classifier</h3>\n\n<p>I constructed both lightgbm model and xgboost model. </p>\n\n<h3>Optimization</h3>\n\n<p>I used bayesian optimization to tune my lightgbm model.</p>\n\n<h3>Post-classification</h3>\n\n<p>I used <a href=\"https://www.kaggle.com/tarobxl/parallel-version-of-faron-s-script/\">this script</a> to contruct orders from  pair. Thanks to faron, shing and tarbox !</p>\n\n<h3>Ensemble</h3>\n\n<p>I trained big models (500+ features), median models(260 + features) and small models(80+ features). Final submissions were generated by bagging top models using median.</p>\n\n<p>Here is <a href=\"https://github.com/LaceyChen17/instacart-market-basket-analysis/blob/master/README.md\">link</a> for my code. It's a bit messy (´・ω・`) </p>",
      "rawMarkdown": "Congratulations to everyone who completed this competition. This is in fact my first kaggle competition and I really enjoy it . Here I am sharing my solution.  Advice would be highly appreciated ! \n\n## Solution\n\nThe problem is formulated as approximating P(u,p | user's prior purchase history) which stands for how much likely user u would repurchase product p given prior purchase history.\n\nThe main model should be a binary classifer and features created manually or automatically are fed to the classfier for generating predictions.\n\n### Features\nI construted both manual features and automatic features using unsupervised learning and neural networks.\n\nManual features include statistics of prior purchase history. As for automatic features, I used the following \n\n##### LDA\n* each user is treated as a document and each product is treated as a word. \n* generate topic representation of each user and each product\n* calculate score by taking inner product for ",
      "votes": 18
    },
    {
      "id": 214256,
      "postDate": "2017-08-16T12:07:18.050Z",
      "content": "<p>Great solution. I am in Beijing too. Would you mind add my WeChat 472508502 to make a friend? May we do research about 2nd and 3rd code together?</p>",
      "rawMarkdown": "Great solution. I am in Beijing too. Would you mind add my WeChat 472508502 to make a friend? May we do research about 2nd and 3rd code together?",
      "replies": [
        {
          "id": 214260,
          "postDate": "2017-08-16T12:20:52.163Z",
          "content": "<p>deleted</p>",
          "rawMarkdown": "deleted"
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-20T02:02:26Z",
  "competition_id": 6644,
  "competition_slug": "basket-analysis",
  "team_id": 697280,
  "solution_url": "/competitions/basket-analysis/writeups/octave-solution-lb-39"
}