{
  "id": 10161,
  "title": "Approach sharing",
  "url": "/competitions/avito-prohibited-content/discussion/10161",
  "author_name": "",
  "post_date": "2014-08-30T09:20:24.540Z",
  "votes": 4,
  "comment_count": 17,
  "views": 7340,
  "content": "<p>I had not much time to compete, so I can describe my approach right now.<br>I started a week before the end of competition and had only two days. So I only ran my model from LSHTC-competition (tf*idf + weighted kNN, k=125): 0.97861 on LB (~0.982 in my local tests). I also added category features to tf*idf-matrix. Today I'll try to blend kNN and vw, but as I understand it can result in +0.004% (it is below Top10).</p>",
  "messages": [
    {
      "id": "52678",
      "postDate": "08/30/2014 09:20:24",
      "content": "<p>I had not much time to compete, so I can describe my approach right now.<br>I started a week before the end of competition and had only two days. So I only ran my model from LSHTC-competition (tf*idf + weighted kNN, k=125): 0.97861 on LB (~0.982 in my local tests). I also added category features to tf*idf-matrix. Today I'll try to blend kNN and vw, but as I understand it can result in +0.004% (it is below Top10).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52694",
      "postDate": "08/30/2014 14:17:18",
      "content": "<p>Alexander, I'm sorry but I don't understand why you'd be sharing this with 2 days still to go.</p>\n<p>Can we please wait for the end of the competition before sharing approaches?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52695",
      "postDate": "08/30/2014 14:34:47",
      "content": "<p>Amazing!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52696",
      "postDate": "08/30/2014 14:41:03",
      "content": "<p>I simply wanted to be the first :)</p>\n<p>Naturally, the majority of participants will describe their approaches after the competition (and I will wait for these descriptions).</p>\n<p>I don't think that my &quot;early&quot; description will do any harm to someone (moreover, I don't think that someone used or will use kNN).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52697",
      "postDate": "08/30/2014 14:44:40",
      "content": "<p>I disagree. Someone will gain/lose position because of this. I'm gracefully asking if you could edit your post and re-post at the end of the competition.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52698",
      "postDate": "08/30/2014 14:46:37",
      "content": "<p>[quote=Giulio;52697]</p>\n<p>Someone will gain/lose position because of this</p>\n<p>[/quote]</p>\n<p>How it can be done?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52699",
      "postDate": "08/30/2014 15:04:04",
      "content": "<p>[quote=Alexander D'yakonov;52698]</p>\n<p>[quote=Giulio;52697]</p>\n<p>Someone will gain/lose position because of this</p>\n<p>[/quote]</p>\n<p>How it can be done?</p>\n<p>[/quote]</p>\n<p>I'm holding off from replying to this now, but I promise I'll answer&nbsp;when the competition is over.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52703",
      "postDate": "08/30/2014 15:19:44",
      "content": "<p>[quote=Giulio;52699]</p>\n<p>I'm holding off from replying to this now, but I promise I'll answer&nbsp;when the competition is over.</p>\n<p>[/quote]</p>\n<p>Ok. But</p>\n<p>1) The message &quot;Beating the benchmark&quot; is more dangerous! And it is not against rules.</p>\n<p>2) If I now remove the post, you (and Abhishek:) ) will get an advantage.</p>\n<p>3) It is impossible to repeat my method in two days;)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52793",
      "postDate": "09/01/2014 00:07:11",
      "content": "<p>Wow, you achieved 5th position with&nbsp;less than 2 days work.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52794",
      "postDate": "09/01/2014 00:08:29",
      "content": "<p>Amazing!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52797",
      "postDate": "09/01/2014 00:30:30",
      "content": "<p>I'm really curious about what golden method you have used.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52835",
      "postDate": "09/01/2014 12:03:13",
      "content": "<p>&#1052;&#1091; final solution is the blending of kNN and vw. For vw I prepared tf*idf-matrix + attributes + categories + some hand-made features.</p>\n<p>I didn't use Foxtrot's code, but I am very grateful to him. He stimulated me to learn vw.</p>\n<p>Thank Triskelion for his useful materials on vw.</p>\n<p>I spent only several days on this competition, so I didn't expect a very high result. But now I have all places on Kaggle from first to fifth:)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52855",
      "postDate": "09/01/2014 14:42:34",
      "content": "<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52914",
      "postDate": "09/02/2014 12:04:19",
      "content": "<p>[quote=Foxtrot;52855]</p>\n<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>\n<p>[/quote]</p>\n<p>&gt;300.000</p>\n<p>cosine similarity - This is often used to compare documents in text mining, try &quot;cosine similarity text mining&quot; in google&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52951",
      "postDate": "09/02/2014 21:08:54",
      "content": "<p>For my part, I had ~0.9737 with an average of unweighted 20-NN's on title and description, separately, with the data segmented by subcategory. I only tried that at the end and didn't have much time for tuning. I suspect that tuning k per subcategory could improve performance.</p>\n<p>[quote=Foxtrot;52855]</p>\n<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>\n<p>[/quote]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set. You don't condense the information in the training set into a weight vector.<br>NB: If you L2-row normalize the vectors, then the ordering given by decreasing cosine similarity is the same as that given by increasing Euclidian distance.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53003",
      "postDate": "09/03/2014 04:12:49",
      "content": "<p>[quote=David Thaler;52951]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set.</p>\n<p>[/quote]</p>\n<p>You can try to compare only pairs that have at least one word in common (or one bigram). Just build inverted index and use it for join. It will not reduce big_O complexity (still n^2) but will reduce number of pairs to compare.</p>\n<p>More sophisticated approach - to compare pairs that have at least N words is common, use local sensitive hashes and so on... May be overkill because this set is not that large.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53024",
      "postDate": "09/03/2014 11:43:28",
      "content": "<p>[quote=David Thaler;52951]</p>\n<p>For my part, I had ~0.9737 with an average of unweighted 20-NN's on title and description, separately, with the data segmented by subcategory. I only tried that at the end and didn't have much time for tuning. I suspect that tuning k per subcategory could improve performance.</p>\n<p>[/quote]</p>\n<p>It was interesting that kNN on titles was better than &quot;Avito best predictions benchmark&quot;.</p>\n<p>I am not sure: ~0.96-0.97!!!</p>\n<p>[quote=David Thaler;52951]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set.</p>\n<p>[/quote]</p>\n<p>Yes. &gt; 2 days to build solution vector:)</p>\n<p>I used Matlab, C++ should be faster...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53032",
      "postDate": "09/03/2014 14:50:04",
      "content": "<p>See also the related problem: All-pairs similarity search. With the right tool and format you can compare token cosine similarity of millions of documents, in seconds.</p>\n<p>https://code.google.com/p/google-all-pairs-similarity-search/</p>\n<p><code>; User specified similarity threshold: 0.9<br>; Found 34807 similar pairs.<br>; Candidates considered: 4988117<br>; Vector intersections performed: 1618864<br>; Total running time: 5 seconds</code></p>\n<p>Could also have a look at how Twitter does fast KNN with DIMSUM:</p>\n<p>https://blog.twitter.com/2014/all-pairs-similarity-via-dimsum</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 52694,
      "author_name": "adjgiulio",
      "author_url": "",
      "post_date": "08/30/2014 14:17:18",
      "content": "<p>Alexander, I'm sorry but I don't understand why you'd be sharing this with 2 days still to go.</p>\n<p>Can we please wait for the end of the competition before sharing approaches?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52695,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "08/30/2014 14:34:47",
      "content": "<p>Amazing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52696,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "08/30/2014 14:41:03",
      "content": "<p>I simply wanted to be the first :)</p>\n<p>Naturally, the majority of participants will describe their approaches after the competition (and I will wait for these descriptions).</p>\n<p>I don't think that my &quot;early&quot; description will do any harm to someone (moreover, I don't think that someone used or will use kNN).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52697,
      "author_name": "adjgiulio",
      "author_url": "",
      "post_date": "08/30/2014 14:44:40",
      "content": "<p>I disagree. Someone will gain/lose position because of this. I'm gracefully asking if you could edit your post and re-post at the end of the competition.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52698,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "08/30/2014 14:46:37",
      "content": "<p>[quote=Giulio;52697]</p>\n<p>Someone will gain/lose position because of this</p>\n<p>[/quote]</p>\n<p>How it can be done?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52699,
      "author_name": "adjgiulio",
      "author_url": "",
      "post_date": "08/30/2014 15:04:04",
      "content": "<p>[quote=Alexander D'yakonov;52698]</p>\n<p>[quote=Giulio;52697]</p>\n<p>Someone will gain/lose position because of this</p>\n<p>[/quote]</p>\n<p>How it can be done?</p>\n<p>[/quote]</p>\n<p>I'm holding off from replying to this now, but I promise I'll answer&nbsp;when the competition is over.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52703,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "08/30/2014 15:19:44",
      "content": "<p>[quote=Giulio;52699]</p>\n<p>I'm holding off from replying to this now, but I promise I'll answer&nbsp;when the competition is over.</p>\n<p>[/quote]</p>\n<p>Ok. But</p>\n<p>1) The message &quot;Beating the benchmark&quot; is more dangerous! And it is not against rules.</p>\n<p>2) If I now remove the post, you (and Abhishek:) ) will get an advantage.</p>\n<p>3) It is impossible to repeat my method in two days;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52793,
      "author_name": "apophenia",
      "author_url": "",
      "post_date": "09/01/2014 00:07:11",
      "content": "<p>Wow, you achieved 5th position with&nbsp;less than 2 days work.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52794,
      "author_name": "abhishek",
      "author_url": "",
      "post_date": "09/01/2014 00:08:29",
      "content": "<p>Amazing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52797,
      "author_name": "kevinandhu",
      "author_url": "",
      "post_date": "09/01/2014 00:30:30",
      "content": "<p>I'm really curious about what golden method you have used.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52835,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "09/01/2014 12:03:13",
      "content": "<p>&#1052;&#1091; final solution is the blending of kNN and vw. For vw I prepared tf*idf-matrix + attributes + categories + some hand-made features.</p>\n<p>I didn't use Foxtrot's code, but I am very grateful to him. He stimulated me to learn vw.</p>\n<p>Thank Triskelion for his useful materials on vw.</p>\n<p>I spent only several days on this competition, so I didn't expect a very high result. But now I have all places on Kaggle from first to fifth:)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52855,
      "author_name": "zygmunt",
      "author_url": "",
      "post_date": "09/01/2014 14:42:34",
      "content": "<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52914,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "09/02/2014 12:04:19",
      "content": "<p>[quote=Foxtrot;52855]</p>\n<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>\n<p>[/quote]</p>\n<p>&gt;300.000</p>\n<p>cosine similarity - This is often used to compare documents in text mining, try &quot;cosine similarity text mining&quot; in google&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52951,
      "author_name": "davidthaler",
      "author_url": "",
      "post_date": "09/02/2014 21:08:54",
      "content": "<p>For my part, I had ~0.9737 with an average of unweighted 20-NN's on title and description, separately, with the data segmented by subcategory. I only tried that at the end and didn't have much time for tuning. I suspect that tuning k per subcategory could improve performance.</p>\n<p>[quote=Foxtrot;52855]</p>\n<p>I'm shocked about using kNN in this competition. Thought that the curse of dimensionality would make this infeasible. What's your data dimensionality and distance metric?</p>\n<p>[/quote]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set. You don't condense the information in the training set into a weight vector.<br>NB: If you L2-row normalize the vectors, then the ordering given by decreasing cosine similarity is the same as that given by increasing Euclidian distance.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53003,
      "author_name": "tuzzeg",
      "author_url": "",
      "post_date": "09/03/2014 04:12:49",
      "content": "<p>[quote=David Thaler;52951]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set.</p>\n<p>[/quote]</p>\n<p>You can try to compare only pairs that have at least one word in common (or one bigram). Just build inverted index and use it for join. It will not reduce big_O complexity (still n^2) but will reduce number of pairs to compare.</p>\n<p>More sophisticated approach - to compare pairs that have at least N words is common, use local sensitive hashes and so on... May be overkill because this set is not that large.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53024,
      "author_name": "dyakonov",
      "author_url": "",
      "post_date": "09/03/2014 11:43:28",
      "content": "<p>[quote=David Thaler;52951]</p>\n<p>For my part, I had ~0.9737 with an average of unweighted 20-NN's on title and description, separately, with the data segmented by subcategory. I only tried that at the end and didn't have much time for tuning. I suspect that tuning k per subcategory could improve performance.</p>\n<p>[/quote]</p>\n<p>It was interesting that kNN on titles was better than &quot;Avito best predictions benchmark&quot;.</p>\n<p>I am not sure: ~0.96-0.97!!!</p>\n<p>[quote=David Thaler;52951]</p>\n<p>For me, the real problem was that it was really&nbsp;slow, because every test row has to be compared to the entire training set.</p>\n<p>[/quote]</p>\n<p>Yes. &gt; 2 days to build solution vector:)</p>\n<p>I used Matlab, C++ should be faster...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53032,
      "author_name": "triskelion",
      "author_url": "",
      "post_date": "09/03/2014 14:50:04",
      "content": "<p>See also the related problem: All-pairs similarity search. With the right tool and format you can compare token cosine similarity of millions of documents, in seconds.</p>\n<p>https://code.google.com/p/google-all-pairs-similarity-search/</p>\n<p><code>; User specified similarity threshold: 0.9<br>; Found 34807 similar pairs.<br>; Candidates considered: 4988117<br>; Vector intersections performed: 1618864<br>; Total running time: 5 seconds</code></p>\n<p>Could also have a look at how Twitter does fast KNN with DIMSUM:</p>\n<p>https://blog.twitter.com/2014/all-pairs-similarity-via-dimsum</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "52678": "",
    "52694": "",
    "52695": "",
    "52696": "",
    "52697": "",
    "52698": "",
    "52699": "",
    "52703": "",
    "52793": "",
    "52794": "",
    "52797": "",
    "52835": "",
    "52855": "",
    "52914": "",
    "52951": "",
    "53003": "",
    "53024": "",
    "53032": ""
  },
  "source": "meta"
}