It seems like most people are using gradient boosted trees(like xgb/lightgbm). Is anyone using any other type of algorithm?
", "messages": [ { "id": "565370", "postDate": "06/30/2019 20:36:07", "content": "It seems like most people are using gradient boosted trees(like xgb/lightgbm). Is anyone using any other type of algorithm?
", "rawMarkdown": "It seems like most people are using gradient boosted trees(like xgb/lightgbm). Is anyone using any other type of algorithm?", "votes": null }, { "id": "565684", "postDate": "07/01/2019 09:24:29", "content": "Logistic Regression, and you should create many new features.
", "rawMarkdown": "Logistic Regression, and you should create many new features.", "votes": null }, { "id": "565691", "postDate": "07/01/2019 09:26:51", "content": "Currently using lgbm.
\n\nNeural nets should work quite well: gnn, gcn and others.
", "rawMarkdown": "Currently using lgbm.\n\nNeural nets should work quite well: gnn, gcn and others.", "votes": null }, { "id": "565767", "postDate": "07/01/2019 11:01:35", "content": "I use LightGBM.
", "rawMarkdown": "I use LightGBM.", "votes": null }, { "id": "566841", "postDate": "07/02/2019 16:45:33", "content": "you guys got 7th place using only lgbm?
", "rawMarkdown": "you guys got 7th place using only lgbm?", "votes": null }, { "id": "566856", "postDate": "07/02/2019 16:57:51", "content": "I mean they have a scientist at the French Alternative Energies and Atomic Energy Commission, who probably has amazing domain knowledge and insights. Paired up with 4 good data scientists in real life (and kaggle grandmasters) i'd say its a pretty good team
", "rawMarkdown": "I mean they have a scientist at the French Alternative Energies and Atomic Energy Commission, who probably has amazing domain knowledge and insights. Paired up with 4 good data scientists in real life (and kaggle grandmasters) i'd say its a pretty good team", "votes": null }, { "id": "567669", "postDate": "07/03/2019 20:29:34", "content": "Lightgbm so far, trying to get as far as possible using features engineering. Later, I will try other models and stack (probably the last two weeks).
", "rawMarkdown": "Lightgbm so far, trying to get as far as possible using features engineering. Later, I will try other models and stack (probably the last two weeks).", "votes": null }, { "id": "570125", "postDate": "07/07/2019 20:18:13", "content": "Really nice if so!
", "rawMarkdown": "Really nice if so!", "votes": null }, { "id": "570200", "postDate": "07/08/2019 01:46:28", "content": "Doing a lot of XGB and LGBM these days, but also tend to lean towards Neural Networks. Going to try out CatBoost to further put it in my toolbox.
", "rawMarkdown": "Doing a lot of XGB and LGBM these days, but also tend to lean towards Neural Networks. Going to try out CatBoost to further put it in my toolbox.", "votes": null }, { "id": "570752", "postDate": "07/08/2019 18:28:54", "content": "I've joined the competition this last week so I'm using gradient boosting for now.
\n\nHowever I think a good customized neural network can outperform any boosting algorithm here (particularly without domaine knowledge).
\n\nAnd of course domaine knowledge will help some competitors to build some killer features.
", "rawMarkdown": "I've joined the competition this last week so I'm using gradient boosting for now. \n\nHowever I think a good customized neural network can outperform any boosting algorithm here (particularly without domaine knowledge). \n\nAnd of course domaine knowledge will help some competitors to build some killer features.", "votes": null }, { "id": "574970", "postDate": "07/14/2019 19:10:03", "content": "Looking for a person who will say that thinks than nns work well and he is using them actually =) So many people thinks good about nns in this task)
", "rawMarkdown": "Looking for a person who will say that thinks than nns work well and he is using them actually =) So many people thinks good about nns in this task)", "votes": null }, { "id": "575255", "postDate": "07/15/2019 08:09:38", "content": "No need to use NN to know it work better here, given the nature of the data ( Unless maybe you're able to build tons of great and powerful features for Gradient boosting)
\n\nYou can still take a look at Heng's Starter kit if NN interests you ( And you can be sure he's using NN for his score)
", "rawMarkdown": "No need to use NN to know it work better here, given the nature of the data ( Unless maybe you're able to build tons of great and powerful features for Gradient boosting)\n\nYou can still take a look at Heng's [Starter kit](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93972) if NN interests you ( And you can be sure he's using NN for his score)", "votes": null }, { "id": "575406", "postDate": "07/15/2019 11:37:33", "content": "Using customized NN. Had to put a lot of thought into designing it to make it work well
", "rawMarkdown": "Using customized NN. Had to put a lot of thought into designing it to make it work well", "votes": null }, { "id": "575418", "postDate": "07/15/2019 11:49:11", "content": "@makahana did you build it from scratch, or started from public NN kernel and improved it?\nWorking on NN myself, but it's performing far worse than tree models so far
", "rawMarkdown": "makahana did you build it from scratch, or started from public NN kernel and improved it?\nWorking on NN myself, but it's performing far worse than tree models so far", "votes": null }, { "id": "575440", "postDate": "07/15/2019 12:15:45", "content": "I don't see any public customized NN kernel (I mean something like graph CNN or customized RNN )
", "rawMarkdown": "I don't see any public customized NN kernel (I mean something like graph CNN or customized RNN )", "votes": null }, { "id": "575441", "postDate": "07/15/2019 12:21:22", "content": "You're right but there's this kernel:\nhttps://www.kaggle.com/todnewman/keras-neural-net-for-champs
", "rawMarkdown": "You're right but there's this kernel:\nhttps://www.kaggle.com/todnewman/keras-neural-net-for-champs", "votes": null }, { "id": "575538", "postDate": "07/15/2019 15:07:13", "content": "Built from scratch using keras... If you input precomputed features, Im guessing tree models will work just as well. What I think is interesting is to avoid having to do this and just input raw coordinates (although this is not as easy as it sounds).
", "rawMarkdown": "Built from scratch using keras... If you input precomputed features, Im guessing tree models will work just as well. What I think is interesting is to avoid having to do this and just input raw coordinates (although this is not as easy as it sounds).", "votes": null }, { "id": "575709", "postDate": "07/15/2019 22:26:11", "content": "A plain neural net (FastAI.Tabular). But I'am stuck at LB -1.2 for some time. So I might dig into gnn.
", "rawMarkdown": "A plain neural net (FastAI.Tabular). But I'am stuck at LB -1.2 for some time. So I might dig into gnn.", "votes": null }, { "id": "576784", "postDate": "07/16/2019 02:34:06", "content": "Top of leaderboard uses deep learning, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/99752#574461 and the comments below it.
", "rawMarkdown": "Top of leaderboard uses deep learning, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/99752#574461 and the comments below it.", "votes": null }, { "id": "580702", "postDate": "07/20/2019 15:43:06", "content": "I tried a Cramér-von Mises on the distributions of x, y, z under the projection of index0/1 and all combinations. This as a pattern recognition in 3d. I also tried R/Phi, and only R. And also a second degree polynomial of the CvM measures. Faggedaboudit!
", "rawMarkdown": "I tried a Cramér-von Mises on the distributions of x, y, z under the projection of index0/1 and all combinations. This as a pattern recognition in 3d. I also tried R/Phi, and only R. And also a second degree polynomial of the CvM measures. Faggedaboudit!", "votes": null } ], "comments": [ { "id": 565684, "author_name": "tenffe", "author_url": "", "post_date": "07/01/2019 09:24:29", "content": "Logistic Regression, and you should create many new features.
", "votes": null, "replies": [] }, { "id": 565691, "author_name": "artgor", "author_url": "", "post_date": "07/01/2019 09:26:51", "content": "Currently using lgbm.
\n\nNeural nets should work quite well: gnn, gcn and others.
", "votes": null, "replies": [ { "id": 566841, "author_name": "returnofsputnik", "author_url": "", "post_date": "07/02/2019 16:45:33", "content": "you guys got 7th place using only lgbm?
", "votes": null, "replies": [] }, { "id": 566856, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "07/02/2019 16:57:51", "content": "I mean they have a scientist at the French Alternative Energies and Atomic Energy Commission, who probably has amazing domain knowledge and insights. Paired up with 4 good data scientists in real life (and kaggle grandmasters) i'd say its a pretty good team
", "votes": null, "replies": [] }, { "id": 570125, "author_name": "muhakabartay", "author_url": "", "post_date": "07/07/2019 20:18:13", "content": "Really nice if so!
", "votes": null, "replies": [] } ] }, { "id": 565767, "author_name": "mochidan", "author_url": "", "post_date": "07/01/2019 11:01:35", "content": "I use LightGBM.
", "votes": null, "replies": [] }, { "id": 567669, "author_name": "yassinealouini", "author_url": "", "post_date": "07/03/2019 20:29:34", "content": "Lightgbm so far, trying to get as far as possible using features engineering. Later, I will try other models and stack (probably the last two weeks).
", "votes": null, "replies": [] }, { "id": 570200, "author_name": "shadyman", "author_url": "", "post_date": "07/08/2019 01:46:28", "content": "Doing a lot of XGB and LGBM these days, but also tend to lean towards Neural Networks. Going to try out CatBoost to further put it in my toolbox.
", "votes": null, "replies": [] }, { "id": 570752, "author_name": "serigne", "author_url": "", "post_date": "07/08/2019 18:28:54", "content": "I've joined the competition this last week so I'm using gradient boosting for now.
\n\nHowever I think a good customized neural network can outperform any boosting algorithm here (particularly without domaine knowledge).
\n\nAnd of course domaine knowledge will help some competitors to build some killer features.
", "votes": null, "replies": [ { "id": 574970, "author_name": "kirill702b", "author_url": "", "post_date": "07/14/2019 19:10:03", "content": "Looking for a person who will say that thinks than nns work well and he is using them actually =) So many people thinks good about nns in this task)
", "votes": null, "replies": [] }, { "id": 575255, "author_name": "serigne", "author_url": "", "post_date": "07/15/2019 08:09:38", "content": "No need to use NN to know it work better here, given the nature of the data ( Unless maybe you're able to build tons of great and powerful features for Gradient boosting)
\n\nYou can still take a look at Heng's Starter kit if NN interests you ( And you can be sure he's using NN for his score)
", "votes": null, "replies": [] } ] }, { "id": 575406, "author_name": "makahana", "author_url": "", "post_date": "07/15/2019 11:37:33", "content": "Using customized NN. Had to put a lot of thought into designing it to make it work well
", "votes": null, "replies": [ { "id": 575418, "author_name": "stanislavblinov", "author_url": "", "post_date": "07/15/2019 11:49:11", "content": "@makahana did you build it from scratch, or started from public NN kernel and improved it?\nWorking on NN myself, but it's performing far worse than tree models so far
", "votes": null, "replies": [] }, { "id": 575440, "author_name": "serigne", "author_url": "", "post_date": "07/15/2019 12:15:45", "content": "I don't see any public customized NN kernel (I mean something like graph CNN or customized RNN )
", "votes": null, "replies": [] }, { "id": 575441, "author_name": "stanislavblinov", "author_url": "", "post_date": "07/15/2019 12:21:22", "content": "You're right but there's this kernel:\nhttps://www.kaggle.com/todnewman/keras-neural-net-for-champs
", "votes": null, "replies": [] }, { "id": 575538, "author_name": "makahana", "author_url": "", "post_date": "07/15/2019 15:07:13", "content": "Built from scratch using keras... If you input precomputed features, Im guessing tree models will work just as well. What I think is interesting is to avoid having to do this and just input raw coordinates (although this is not as easy as it sounds).
", "votes": null, "replies": [] } ] }, { "id": 575709, "author_name": "joatom", "author_url": "", "post_date": "07/15/2019 22:26:11", "content": "A plain neural net (FastAI.Tabular). But I'am stuck at LB -1.2 for some time. So I might dig into gnn.
", "votes": null, "replies": [ { "id": 576784, "author_name": "cpmpml", "author_url": "", "post_date": "07/16/2019 02:34:06", "content": "Top of leaderboard uses deep learning, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/99752#574461 and the comments below it.
", "votes": null, "replies": [] } ] }, { "id": 580702, "author_name": "glimmung", "author_url": "", "post_date": "07/20/2019 15:43:06", "content": "I tried a Cramér-von Mises on the distributions of x, y, z under the projection of index0/1 and all combinations. This as a pattern recognition in 3d. I also tried R/Phi, and only R. And also a second degree polynomial of the CvM measures. Faggedaboudit!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "565370": "It seems like most people are using gradient boosted trees(like xgb/lightgbm). Is anyone using any other type of algorithm?", "565684": "Logistic Regression, and you should create many new features.", "565691": "Currently using lgbm.\n\nNeural nets should work quite well: gnn, gcn and others.", "565767": "I use LightGBM.", "566841": "you guys got 7th place using only lgbm?", "566856": "I mean they have a scientist at the French Alternative Energies and Atomic Energy Commission, who probably has amazing domain knowledge and insights. Paired up with 4 good data scientists in real life (and kaggle grandmasters) i'd say its a pretty good team", "567669": "Lightgbm so far, trying to get as far as possible using features engineering. Later, I will try other models and stack (probably the last two weeks).", "570125": "Really nice if so!", "570200": "Doing a lot of XGB and LGBM these days, but also tend to lean towards Neural Networks. Going to try out CatBoost to further put it in my toolbox.", "570752": "I've joined the competition this last week so I'm using gradient boosting for now. \n\nHowever I think a good customized neural network can outperform any boosting algorithm here (particularly without domaine knowledge). \n\nAnd of course domaine knowledge will help some competitors to build some killer features.", "574970": "Looking for a person who will say that thinks than nns work well and he is using them actually =) So many people thinks good about nns in this task)", "575255": "No need to use NN to know it work better here, given the nature of the data ( Unless maybe you're able to build tons of great and powerful features for Gradient boosting)\n\nYou can still take a look at Heng's [Starter kit](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93972) if NN interests you ( And you can be sure he's using NN for his score)", "575406": "Using customized NN. Had to put a lot of thought into designing it to make it work well", "575418": "makahana did you build it from scratch, or started from public NN kernel and improved it?\nWorking on NN myself, but it's performing far worse than tree models so far", "575440": "I don't see any public customized NN kernel (I mean something like graph CNN or customized RNN )", "575441": "You're right but there's this kernel:\nhttps://www.kaggle.com/todnewman/keras-neural-net-for-champs", "575538": "Built from scratch using keras... If you input precomputed features, Im guessing tree models will work just as well. What I think is interesting is to avoid having to do this and just input raw coordinates (although this is not as easy as it sounds).", "575709": "A plain neural net (FastAI.Tabular). But I'am stuck at LB -1.2 for some time. So I might dig into gnn.", "576784": "Top of leaderboard uses deep learning, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/99752#574461 and the comments below it.", "580702": "I tried a Cramér-von Mises on the distributions of x, y, z under the projection of index0/1 and all combinations. This as a pattern recognition in 3d. I also tried R/Phi, and only R. And also a second degree polynomial of the CvM measures. Faggedaboudit!" }, "source": "meta" }