The results on the private part of the test are really unexpected for me.\nFor the final submission I have used 2 solutions\n1) one with data leakage (using the ideas from this kernel by Markus F \nhttps://www.kaggle.com/friedchips/the-missing-link\nFor this submission I got 0.82 on the dataset and was sure that it would be my best solution, but on the private set it got only 0.60. It is really a puzzle for me to understand this result, I still have no idea about it.\n2) one without using data leakage. I selected my kernel with neural network, mostly wondering how efficient NN can be on such a small dataset. And it worked rather good! I got 0.71 on the public set, the same score I got on the public set https://www.kaggle.com/skondrash/nn-lstm-fft-features?scriptVersionId=12728595
", "comments": [ { "id": 515403, "author_name": "Costa.Medvedev", "author_url": "", "post_date": "2019-04-12T15:05:24.303000", "content": "Regarding the puzzle in pp1.\nAssigning y_train surfaces to test data using linked series works well on public data because the linkage is better for public rather than for private. I was doing updates of a decent amount of surfaces and submitted multiple results with no change in LB.\nAlso the linkage (version from Markus applied to test+train) has multiple cases when the same chain has more than one surface. Therefore it can override correct model prediction with incorrect surface for many series in one go.
", "votes": 1, "replies": [ { "id": 515552, "author_name": "Sergey K.", "author_url": "", "post_date": "2019-04-12T18:59:01.120000", "content": "I did not use the linkage between train and test (simply did not have enough time for it), I only took predictions of run+id on the test set and within each run I used most frequent answer (from my 2 best kernels), with correction for run+ids that obviously were made of 2 surfaces. So the point is not in linkage between train and test. I guess that private part has more poorly predicted surface groups, that were unpredicted by my kernels almost within the whole run+id, for example, hard_tiles.
", "votes": 0, "replies": [] } ] }, { "id": 515185, "author_name": "Ievgenii Krevenets", "author_url": "", "post_date": "2019-04-12T09:29:50.120000", "content": "Thank you for sharing your solution! I also thought to use LSTM and CNN there but \"data leakage\" grabbed all my last few days. Never trust in public leaderboard :)
", "votes": 1, "replies": [ { "id": 515556, "author_name": "Sergey K.", "author_url": "", "post_date": "2019-04-12T19:08:02.867000", "content": "For sure, results are a big surprise)
", "votes": 1, "replies": [] } ] }, { "id": 515814, "author_name": "Prashanth Thangavel", "author_url": "", "post_date": "2019-04-13T07:00:52.387000", "content": "Congrats! Thanks for sharing the solution 👍🏻
", "votes": 0, "replies": [ { "id": 516220, "author_name": "Sergey K.", "author_url": "", "post_date": "2019-04-13T19:53:55.483000", "content": "Thank you! You're welcome)
", "votes": 0, "replies": [] } ] } ], "comment_count": 6, "messages": [ { "id": 514984, "postDate": "2019-04-12T04:59:42.443Z", "content": "The results on the private part of the test are really unexpected for me.\nFor the final submission I have used 2 solutions\n1) one with data leakage (using the ideas from this kernel by Markus F \nhttps://www.kaggle.com/friedchips/the-missing-link\nFor this submission I got 0.82 on the dataset and was sure that it would be my best solution, but on the private set it got only 0.60. It is really a puzzle for me to understand this result, I still have no idea about it.\n2) one without using data leakage. I selected my kernel with neural network, mostly wondering how efficient NN can be on such a small dataset. And it worked rather good! I got 0.71 on the public set, the same score I got on the public set https://www.kaggle.com/skondrash/nn-lstm-fft-features?scriptVersionId=12728595
", "rawMarkdown": "The results on the private part of the test are really unexpected for me.\nFor the final submission I have used 2 solutions\n1) one with data leakage (using the ideas from this kernel by Markus F \n[https://www.kaggle.com/friedchips/the-missing-link](https://www.kaggle.com/friedchips/the-missing-link)\nFor this submission I got 0.82 on the dataset and was sure that it would be my best solution, but on the private set it got only 0.60. It is really a puzzle for me to understand this result, I still have no idea about it.\n2) one without using data leakage. I selected my kernel with neural network, mostly wondering how efficient NN can be on such a small dataset. And it worked rather good! I got 0.71 on the public set, the same score I got on the public set [https://www.kaggle.com/skondrash/nn-lstm-fft-features?scriptVersionId=12728595](https://www.kaggle.com/skondrash/nn-lstm-fft-features?scriptVersionId=12728595)\n", "votes": 8 }, { "id": 515403, "postDate": "2019-04-12T15:05:24.303Z", "content": "Regarding the puzzle in pp1.\nAssigning y_train surfaces to test data using linked series works well on public data because the linkage is better for public rather than for private. I was doing updates of a decent amount of surfaces and submitted multiple results with no change in LB.\nAlso the linkage (version from Markus applied to test+train) has multiple cases when the same chain has more than one surface. Therefore it can override correct model prediction with incorrect surface for many series in one go.
", "rawMarkdown": "Regarding the puzzle in pp1.\nAssigning y_train surfaces to test data using linked series works well on public data because the linkage is better for public rather than for private. I was doing updates of a decent amount of surfaces and submitted multiple results with no change in LB.\nAlso the linkage (version from Markus applied to test+train) has multiple cases when the same chain has more than one surface. Therefore it can override correct model prediction with incorrect surface for many series in one go.", "votes": 1, "replies": [ { "id": 515552, "postDate": "2019-04-12T18:59:01.120Z", "content": "I did not use the linkage between train and test (simply did not have enough time for it), I only took predictions of run+id on the test set and within each run I used most frequent answer (from my 2 best kernels), with correction for run+ids that obviously were made of 2 surfaces. So the point is not in linkage between train and test. I guess that private part has more poorly predicted surface groups, that were unpredicted by my kernels almost within the whole run+id, for example, hard_tiles.
", "rawMarkdown": "I did not use the linkage between train and test (simply did not have enough time for it), I only took predictions of run+id on the test set and within each run I used most frequent answer (from my 2 best kernels), with correction for run+ids that obviously were made of 2 surfaces. So the point is not in linkage between train and test. I guess that private part has more poorly predicted surface groups, that were unpredicted by my kernels almost within the whole run+id, for example, hard_tiles." } ] }, { "id": 515185, "postDate": "2019-04-12T09:29:50.120Z", "content": "Thank you for sharing your solution! I also thought to use LSTM and CNN there but \"data leakage\" grabbed all my last few days. Never trust in public leaderboard :)
", "rawMarkdown": "Thank you for sharing your solution! I also thought to use LSTM and CNN there but \"data leakage\" grabbed all my last few days. Never trust in public leaderboard :)", "votes": 1, "replies": [ { "id": 515556, "postDate": "2019-04-12T19:08:02.867Z", "content": "For sure, results are a big surprise)
", "rawMarkdown": "For sure, results are a big surprise)", "votes": 1 } ] }, { "id": 515814, "postDate": "2019-04-13T07:00:52.387Z", "content": "Congrats! Thanks for sharing the solution 👍🏻
", "rawMarkdown": "Congrats! Thanks for sharing the solution 👍🏻", "replies": [ { "id": 516220, "postDate": "2019-04-13T19:53:55.483Z", "content": "Thank you! You're welcome)
", "rawMarkdown": "Thank you! You're welcome)" } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T15:43:43Z", "competition_id": 13242, "competition_slug": "career-con-2019", "team_id": 2908940, "solution_url": "/competitions/career-con-2019/writeups/sergey-k-41th-place-without-using-data-leakage" }