Hi everyone!\nI participate for a first time and I am a little bit confused about scores. I could get -1.4 with only geometrical features trained one model for every type. After this added qm9 features and got something about -1.5. So the question is what can transfer to -2.0 or -2.5 - tuning lgbm parameters for each type or more feature engineering on qm9? Or maybe I am loosing something else? Thanks in advance!
", "messages": [ { "id": 574452, "postDate": "2019-07-13T20:47:10.073Z", "content": "Hi everyone!\nI participate for a first time and I am a little bit confused about scores. I could get -1.4 with only geometrical features trained one model for every type. After this added qm9 features and got something about -1.5. So the question is what can transfer to -2.0 or -2.5 - tuning lgbm parameters for each type or more feature engineering on qm9? Or maybe I am loosing something else? Thanks in advance!
", "rawMarkdown": "Hi everyone!\nI participate for a first time and I am a little bit confused about scores. I could get -1.4 with only geometrical features trained one model for every type. After this added qm9 features and got something about -1.5. So the question is what can transfer to -2.0 or -2.5 - tuning lgbm parameters for each type or more feature engineering on qm9? Or maybe I am loosing something else? Thanks in advance!", "votes": 16 }, { "id": 574485, "postDate": "2019-07-14T00:16:37.213Z", "content": "From -1.5 to -2.0: better features, better models, 150% more efforts. \nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts. \nFrom -1.5 to -1.51: ask people how to obtain -2.0 or -2.5. No one will reply you how to do that.
", "rawMarkdown": "From -1.5 to -2.0: better features, better models, 150% more efforts. \nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts. \nFrom -1.5 to -1.51: ask people how to obtain -2.0 or -2.5. No one will reply you how to do that. ", "votes": 8, "replies": [ { "id": 574561, "postDate": "2019-07-14T05:05:59.697Z", "content": "\n\n\nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts.
\n
You don't know that ;)
", "rawMarkdown": "> From -2.0 to -2.5: extremely good features and models, 200% more efforts.\n\nYou don't know that ;)", "votes": 2 }, { "id": 574573, "postDate": "2019-07-14T05:36:32.973Z", "content": "We know a part of it: -2.0 to maybe -2.2.
", "rawMarkdown": "We know a part of it: -2.0 to maybe -2.2.", "votes": 1 }, { "id": 574756, "postDate": "2019-07-14T12:36:40.657Z", "content": "Do you use nns?
", "rawMarkdown": "Do you use nns?", "votes": 1 }, { "id": 574812, "postDate": "2019-07-14T14:11:09.410Z", "content": "Well is there any way which doesn't include 200% more efforts and all this?\nI'm kinda tired already ;)
", "rawMarkdown": "Well is there any way which doesn't include 200% more efforts and all this?\nI'm kinda tired already ;)" }, { "id": 595010, "postDate": "2019-08-08T18:30:26.050Z", "content": "\n\n\nWe know a part of it: -2.0 to maybe -2.2.
\n
@khahuras actually you didn't know and resorted to my proposal:
\n\n\n\n\n\n \nSo the question is what can transfer to -2.0 or -2.5\nTeaming ;)
\n
;)
", "rawMarkdown": "> We know a part of it: -2.0 to maybe -2.2.\n\n@khahuras actually you didn't know and resorted to my proposal:\n\n> So the question is what can transfer to -2.0 or -2.5\n> \n> Teaming ;)\n\n;)" }, { "id": 595028, "postDate": "2019-08-08T19:02:33.933Z", "content": "We have a GCN model which may score around -2.5 before teaming but not yet submitted, based on CV score. It’s fine if you believe it or not. But publicly in front of my teammates it’s better to me not to lie.
\n\nBy the way, why are you concerning other teams’ business too much? I see your comment like a troll comment, not constructive, not requesting.
", "rawMarkdown": "We have a GCN model which may score around -2.5 before teaming but not yet submitted, based on CV score. It’s fine if you believe it or not. But publicly in front of my teammates it’s better to me not to lie. \n\nBy the way, why are you concerning other teams’ business too much? I see your comment like a troll comment, not constructive, not requesting.", "votes": 2 }, { "id": 595150, "postDate": "2019-08-08T22:43:29.257Z", "content": "\n\n\nWe have a GCN model which may score around -2.5
\n \nWe know a part of it: -2.0 to maybe -2.2.
\n
Lots of maybes when it is very simple to check: submit and you get the score.
\n\nI'll stop here but I can't resist debunking unsupported claims in general. Nothing personal for sure.
\n\nI wish you all the best in the remainder of this competition.
", "rawMarkdown": "> We have a GCN model which may score around -2.5 \n\n> We know a part of it: -2.0 to maybe -2.2.\n\nLots of maybes when it is very simple to check: submit and you get the score.\n\nI'll stop here but I can't resist debunking unsupported claims in general. Nothing personal for sure.\n\nI wish you all the best in the remainder of this competition.\n", "votes": 3 } ] }, { "id": 574559, "postDate": "2019-07-14T05:01:18.927Z", "content": "It is interesting that I couldn’t really improve the score with qm9 dataset. Maybe it is not so useful?
", "rawMarkdown": "It is interesting that I couldn’t really improve the score with qm9 dataset. Maybe it is not so useful?", "votes": 6, "replies": [ { "id": 574566, "postDate": "2019-07-14T05:13:16.230Z", "content": "Good to know, it will save me time to not look at it ;)
", "rawMarkdown": "Good to know, it will save me time to not look at it ;)", "votes": 1 }, { "id": 574757, "postDate": "2019-07-14T12:37:00.047Z", "content": "Good for you:)
", "rawMarkdown": "Good for you:)", "votes": 1 }, { "id": 576732, "postDate": "2019-07-16T00:18:36.803Z", "content": "Same here. Found out after days of attempts and multiple mixed runs :(
", "rawMarkdown": "Same here. Found out after days of attempts and multiple mixed runs :(" }, { "id": 584157, "postDate": "2019-07-25T13:59:53.593Z", "content": "Physicist here and can safely say that QM9 results will not add much because the phenomenon related to the values in QM9 are not very related to scalar coupling
", "rawMarkdown": "Physicist here and can safely say that QM9 results will not add much because the phenomenon related to the values in QM9 are not very related to scalar coupling", "votes": 1 } ] }, { "id": 574974, "postDate": "2019-07-14T19:22:13.763Z", "content": "Working on additional data. It seems to be very helpful. But is there a common approach to deal with the data that is presented only for train dataset?
", "rawMarkdown": "Working on additional data. It seems to be very helpful. But is there a common approach to deal with the data that is presented only for train dataset?", "votes": 1, "replies": [ { "id": 575002, "postDate": "2019-07-14T20:39:01.160Z", "content": "With data provided only for train dataset common approach would be to use that data as target and make a model which predicts same values for test set.
", "rawMarkdown": "With data provided only for train dataset common approach would be to use that data as target and make a model which predicts same values for test set. ", "votes": 1 }, { "id": 575144, "postDate": "2019-07-15T05:34:08.320Z", "content": "And then to use this variables as features, right? But how to use the knowledge that target is a sum of this features?
", "rawMarkdown": "And then to use this variables as features, right? But how to use the knowledge that target is a sum of this features?", "votes": 1 } ] }, { "id": 574758, "postDate": "2019-07-14T12:40:07.797Z", "content": "So much expectations of NNs here ... I going to dig deeper!
", "rawMarkdown": "So much expectations of NNs here ... I going to dig deeper!", "votes": 1 }, { "id": 574464, "postDate": "2019-07-13T22:09:55.627Z", "content": "Do you mean graph nns? Or just regular?
", "rawMarkdown": "Do you mean graph nns? Or just regular? ", "votes": 1, "replies": [ { "id": 574465, "postDate": "2019-07-13T22:19:40.733Z", "content": "I'm not a NN expert but he published a GCN paper predicting molecular properties.
", "rawMarkdown": "I'm not a NN expert but he published a GCN paper predicting molecular properties.", "votes": 1 } ] }, { "id": 574861, "postDate": "2019-07-14T15:38:22.870Z", "content": "\n\n\nSo the question is what can transfer to -2.0 or -2.5
\n
Teaming ;)
", "rawMarkdown": "> So the question is what can transfer to -2.0 or -2.5\n\nTeaming ;)", "votes": 2, "replies": [ { "id": 574975, "postDate": "2019-07-14T19:23:49.110Z", "content": "Hope it is not the only way:)
", "rawMarkdown": "Hope it is not the only way:)", "votes": 1 }, { "id": 596556, "postDate": "2019-08-10T20:53:24.503Z", "content": "@cpmpml You were right. We had to team up with Kirill to break through -2. :)
", "rawMarkdown": "@cpmpml You were right. We had to team up with Kirill to break through -2. :)", "votes": 1 }, { "id": 596570, "postDate": "2019-08-10T21:42:38.703Z", "content": "Teaming is indeed the best way to make progress quickly. I am lucky to have teamed with Ahmet.
", "rawMarkdown": "Teaming is indeed the best way to make progress quickly. I am lucky to have teamed with Ahmet.", "votes": 2 } ] }, { "id": 574711, "postDate": "2019-07-14T11:14:00.207Z", "content": "Based on my experience so far and some observations, I guess that a strategy combining a gradient boosting trees model with a graph neural network would result in a score < -2.0. \nI haven't yet used any GNN model thus my observation is just speculation for now. I will update this comment once I have some data. 😊
", "rawMarkdown": "Based on my experience so far and some observations, I guess that a strategy combining a gradient boosting trees model with a graph neural network would result in a score < -2.0. \nI haven't yet used any GNN model thus my observation is just speculation for now. I will update this comment once I have some data. 😊 ", "votes": 2 }, { "id": 574461, "postDate": "2019-07-13T21:57:19.960Z", "content": "I am totally guessing here but the order of importance is something like this: \n1. Features\n2. NNs\n3. something else\n4. lgbm tuning
\n\nI am fairly certain that LB #1 is using NNs
", "rawMarkdown": "I am totally guessing here but the order of importance is something like this: \n1. Features\n2. NNs\n3. something else\n4. lgbm tuning\n\nI am fairly certain that [LB #1 is using NNs](https://www.kaggle.com/c/champs-scalar-coupling/discussion/99069#573599)", "votes": 2, "replies": [ { "id": 574564, "postDate": "2019-07-14T05:12:12.117Z", "content": "> I am fairly certain that LB #1 is using NNs
\n\ntop 2 @outrunner uses NNs as well. And top 3 @lamdang used NNs when he won Quora competition. And current top 4 @cedben disclosed he uses NNs.
\n\nThe list probably goes on for a while.
", "rawMarkdown": "> I am fairly certain that LB #1 is using NNs\n\ntop 2 @outrunner uses NNs as well. And top 3 @lamdang used NNs when he won Quora competition. And current top 4 @cedben disclosed he uses NNs. \n\nThe list probably goes on for a while.", "votes": 2 }, { "id": 574678, "postDate": "2019-07-14T09:56:54.927Z", "content": "I wonder if there is anyone with Lightgbm in top 10.
", "rawMarkdown": "I wonder if there is anyone with Lightgbm in top 10.", "votes": 2 }, { "id": 576789, "postDate": "2019-07-16T02:47:16.243Z", "content": "\n\n\nI wonder if there is anyone with Lightgbm in top 10.
\n
Maybe you know the answer, now that you're 9th ? ;)
", "rawMarkdown": "> I wonder if there is anyone with Lightgbm in top 10.\n\nMaybe you know the answer, now that you're 9th ? ;)", "votes": 2 }, { "id": 576944, "postDate": "2019-07-16T07:26:22.917Z", "content": "I only know 10% of the answer now:)
", "rawMarkdown": "I only know 10% of the answer now:)", "votes": 3 }, { "id": 576963, "postDate": "2019-07-16T07:52:15.653Z", "content": "Good, you're not using lgb, or not just lgb then ;)
", "rawMarkdown": "Good, you're not using lgb, or not just lgb then ;)" }, { "id": 583491, "postDate": "2019-07-24T14:41:29.183Z", "rawMarkdown": "", "isDeleted": true }, { "id": 583492, "postDate": "2019-07-24T14:42:07.823Z", "rawMarkdown": "", "isDeleted": true }, { "id": 583617, "postDate": "2019-07-24T17:55:00.417Z", "content": "@flemi219 LightGBM is not a NN...
\nNN is a neural network. LightGBM uses decision trees.
i believe it will be neural networks/deep learning
", "rawMarkdown": "i believe it will be neural networks/deep learning", "votes": 2 }, { "id": 593244, "postDate": "2019-08-06T10:46:07.853Z", "content": "Hi, Kirill! BTW, I see that you and your teammate are doing very well in this competition, great!... :) I am currently 13th at Severstahl competition (but its deadline is not very soon, so everything may change). In the next week or two will concentrate on this competition on molecular properties. Actually it is almost my speciality, magnetism.. \n Lets keep in touch, if in 1-2 weeks our models will be complementary (i.e. scores on each bond types complement each other well), we may join in a team. Also, I am looking for a team on 2 medical imaging competitions which are taking place in the moment. They actually pretty similar to Severstahl competition...
", "rawMarkdown": "Hi, Kirill! BTW, I see that you and your teammate are doing very well in this competition, great!... :) I am currently 13th at Severstahl competition (but its deadline is not very soon, so everything may change). In the next week or two will concentrate on this competition on molecular properties. Actually it is almost my speciality, magnetism.. \n Lets keep in touch, if in 1-2 weeks our models will be complementary (i.e. scores on each bond types complement each other well), we may join in a team. Also, I am looking for a team on 2 medical imaging competitions which are taking place in the moment. They actually pretty similar to Severstahl competition...", "replies": [ { "id": 593539, "postDate": "2019-08-06T18:50:14.373Z", "content": "Good luck! BTW, Do you have experience in GNNs?)
", "rawMarkdown": "Good luck! BTW, Do you have experience in GNNs?) ", "votes": 1 } ] }, { "id": 574575, "postDate": "2019-07-14T05:55:25.060Z", "content": "Good question....which I believe no one will answer = =!
\n\nBut I think GBM can at least reach around -1.7
", "rawMarkdown": "Good question....which I believe no one will answer = =!\n\nBut I think GBM can at least reach around -1.7", "replies": [ { "id": 578010, "postDate": "2019-07-17T09:25:20.580Z", "content": "I've got -1.712 with feature ingeneering and LightGBM. I'm almost has no experience with NNs but I'm trying to apply NNs to my dataset and at the moment I have no significant improvements. It looks as I should in the short time seriously improve my knowledge of neural networks. I wonder if it is possible.
", "rawMarkdown": "I've got -1.712 with feature ingeneering and LightGBM. I'm almost has no experience with NNs but I'm trying to apply NNs to my dataset and at the moment I have no significant improvements. It looks as I should in the short time seriously improve my knowledge of neural networks. I wonder if it is possible.", "votes": 4 } ] }, { "id": 574641, "postDate": "2019-07-14T08:28:48.137Z", "rawMarkdown": "", "isDeleted": true }, { "id": 574605, "postDate": "2019-07-14T07:02:37.053Z", "content": "Geometric features, ideas from public kernels, eliminating useless features and lgbm can reach around -2, so it is definitely worth to read the kernels.
", "rawMarkdown": "Geometric features, ideas from public kernels, eliminating useless features and lgbm can reach around -2, so it is definitely worth to read the kernels.", "votes": 15, "isDeleted": true, "replies": [ { "id": 583324, "postDate": "2019-07-24T10:43:20.680Z", "content": "Thanks for insight\nHow many features do you have in the best single lgbm model?
", "rawMarkdown": "Thanks for insight\nHow many features do you have in the best single lgbm model?" }, { "id": 583717, "postDate": "2019-07-24T21:53:22.617Z", "content": "Around 70 features. (it depends on the type)
", "rawMarkdown": "Around 70 features. (it depends on the type)", "votes": 1, "isDeleted": true }, { "id": 583762, "postDate": "2019-07-25T00:14:21.487Z", "content": "May I ask do you use lofo-importance to make feature selection?
", "rawMarkdown": "May I ask do you use lofo-importance to make feature selection?", "votes": 1 }, { "id": 584264, "postDate": "2019-07-25T16:27:51.340Z", "content": "Mostly permutation importance that I learned from a kernel https://www.kaggle.com/speedwagon/permutation-importance\nThis is fast and works reasonably well. LOFO would probably be more scientific, but slower.
", "rawMarkdown": "Mostly permutation importance that I learned from a kernel https://www.kaggle.com/speedwagon/permutation-importance\nThis is fast and works reasonably well. LOFO would probably be more scientific, but slower.", "votes": 4, "isDeleted": true }, { "id": 584598, "postDate": "2019-07-26T07:58:01.357Z", "content": "Thanks, it seems like I should focus more on feature engineering.
", "rawMarkdown": "Thanks, it seems like I should focus more on feature engineering." } ] } ], "comments": [ { "id": 574485, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-07-14T00:16:37.213000", "content": "From -1.5 to -2.0: better features, better models, 150% more efforts. \nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts. \nFrom -1.5 to -1.51: ask people how to obtain -2.0 or -2.5. No one will reply you how to do that.
", "votes": 8, "replies": [ { "id": 574561, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-14T05:05:59.697000", "content": "\n\n\nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts.
\n
You don't know that ;)
", "votes": 2, "replies": [] }, { "id": 574573, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-07-14T05:36:32.973000", "content": "We know a part of it: -2.0 to maybe -2.2.
", "votes": 1, "replies": [] }, { "id": 574756, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T12:36:40.657000", "content": "Do you use nns?
", "votes": 1, "replies": [] }, { "id": 574812, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-07-14T14:11:09.410000", "content": "Well is there any way which doesn't include 200% more efforts and all this?\nI'm kinda tired already ;)
", "votes": 0, "replies": [] }, { "id": 595010, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-08T18:30:26.050000", "content": "\n\n\nWe know a part of it: -2.0 to maybe -2.2.
\n
@khahuras actually you didn't know and resorted to my proposal:
\n\n\n\n\n\n \nSo the question is what can transfer to -2.0 or -2.5\nTeaming ;)
\n
;)
", "votes": 0, "replies": [] }, { "id": 595028, "author_name": "Kha Vo", "author_url": "", "post_date": "2019-08-08T19:02:33.933000", "content": "We have a GCN model which may score around -2.5 before teaming but not yet submitted, based on CV score. It’s fine if you believe it or not. But publicly in front of my teammates it’s better to me not to lie.
\n\nBy the way, why are you concerning other teams’ business too much? I see your comment like a troll comment, not constructive, not requesting.
", "votes": 2, "replies": [] }, { "id": 595150, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-08T22:43:29.257000", "content": "\n\n\nWe have a GCN model which may score around -2.5
\n \nWe know a part of it: -2.0 to maybe -2.2.
\n
Lots of maybes when it is very simple to check: submit and you get the score.
\n\nI'll stop here but I can't resist debunking unsupported claims in general. Nothing personal for sure.
\n\nI wish you all the best in the remainder of this competition.
", "votes": 3, "replies": [] } ] }, { "id": 574559, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T05:01:18.927000", "content": "It is interesting that I couldn’t really improve the score with qm9 dataset. Maybe it is not so useful?
", "votes": 6, "replies": [ { "id": 574566, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-14T05:13:16.230000", "content": "Good to know, it will save me time to not look at it ;)
", "votes": 1, "replies": [] }, { "id": 574757, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T12:37:00.047000", "content": "Good for you:)
", "votes": 1, "replies": [] }, { "id": 576732, "author_name": "Y Paudel", "author_url": "", "post_date": "2019-07-16T00:18:36.803000", "content": "Same here. Found out after days of attempts and multiple mixed runs :(
", "votes": 0, "replies": [] }, { "id": 584157, "author_name": "Joe Kuro", "author_url": "", "post_date": "2019-07-25T13:59:53.593000", "content": "Physicist here and can safely say that QM9 results will not add much because the phenomenon related to the values in QM9 are not very related to scalar coupling
", "votes": 1, "replies": [] } ] }, { "id": 574974, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T19:22:13.763000", "content": "Working on additional data. It seems to be very helpful. But is there a common approach to deal with the data that is presented only for train dataset?
", "votes": 1, "replies": [ { "id": 575002, "author_name": "alijs", "author_url": "", "post_date": "2019-07-14T20:39:01.160000", "content": "With data provided only for train dataset common approach would be to use that data as target and make a model which predicts same values for test set.
", "votes": 1, "replies": [] }, { "id": 575144, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-15T05:34:08.320000", "content": "And then to use this variables as features, right? But how to use the knowledge that target is a sum of this features?
", "votes": 1, "replies": [] } ] }, { "id": 574758, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T12:40:07.797000", "content": "So much expectations of NNs here ... I going to dig deeper!
", "votes": 1, "replies": [] }, { "id": 574464, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-13T22:09:55.627000", "content": "Do you mean graph nns? Or just regular?
", "votes": 1, "replies": [ { "id": 574465, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-07-13T22:19:40.733000", "content": "I'm not a NN expert but he published a GCN paper predicting molecular properties.
", "votes": 1, "replies": [] } ] }, { "id": 574861, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-14T15:38:22.870000", "content": "\n\n\nSo the question is what can transfer to -2.0 or -2.5
\n
Teaming ;)
", "votes": 2, "replies": [ { "id": 574975, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T19:23:49.110000", "content": "Hope it is not the only way:)
", "votes": 1, "replies": [] }, { "id": 596556, "author_name": "Sergey Zlobin", "author_url": "", "post_date": "2019-08-10T20:53:24.503000", "content": "@cpmpml You were right. We had to team up with Kirill to break through -2. :)
", "votes": 1, "replies": [] }, { "id": 596570, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-10T21:42:38.703000", "content": "Teaming is indeed the best way to make progress quickly. I am lucky to have teamed with Ahmet.
", "votes": 2, "replies": [] } ] }, { "id": 574711, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2019-07-14T11:14:00.207000", "content": "Based on my experience so far and some observations, I guess that a strategy combining a gradient boosting trees model with a graph neural network would result in a score < -2.0. \nI haven't yet used any GNN model thus my observation is just speculation for now. I will update this comment once I have some data. 😊
", "votes": 2, "replies": [] }, { "id": 574461, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-07-13T21:57:19.960000", "content": "I am totally guessing here but the order of importance is something like this: \n1. Features\n2. NNs\n3. something else\n4. lgbm tuning
\n\nI am fairly certain that LB #1 is using NNs
", "votes": 2, "replies": [ { "id": 574564, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-14T05:12:12.117000", "content": "> I am fairly certain that LB #1 is using NNs
\n\ntop 2 @outrunner uses NNs as well. And top 3 @lamdang used NNs when he won Quora competition. And current top 4 @cedben disclosed he uses NNs.
\n\nThe list probably goes on for a while.
", "votes": 2, "replies": [] }, { "id": 574678, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-07-14T09:56:54.927000", "content": "I wonder if there is anyone with Lightgbm in top 10.
", "votes": 2, "replies": [] }, { "id": 576789, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-16T02:47:16.243000", "content": "\n\n\nI wonder if there is anyone with Lightgbm in top 10.
\n
Maybe you know the answer, now that you're 9th ? ;)
", "votes": 2, "replies": [] }, { "id": 576944, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-07-16T07:26:22.917000", "content": "I only know 10% of the answer now:)
", "votes": 3, "replies": [] }, { "id": 576963, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-16T07:52:15.653000", "content": "Good, you're not using lgb, or not just lgb then ;)
", "votes": 0, "replies": [] }, { "id": 583491, "author_name": "", "author_url": "", "post_date": "2019-07-24T14:41:29.183000", "content": "", "votes": 0, "replies": [] }, { "id": 583492, "author_name": "", "author_url": "", "post_date": "2019-07-24T14:42:07.823000", "content": "", "votes": 0, "replies": [] }, { "id": 583617, "author_name": "fnands", "author_url": "", "post_date": "2019-07-24T17:55:00.417000", "content": "@flemi219 LightGBM is not a NN...
\nNN is a neural network. LightGBM uses decision trees.
i believe it will be neural networks/deep learning
", "votes": 2, "replies": [] }, { "id": 593244, "author_name": "Alexander", "author_url": "", "post_date": "2019-08-06T10:46:07.853000", "content": "Hi, Kirill! BTW, I see that you and your teammate are doing very well in this competition, great!... :) I am currently 13th at Severstahl competition (but its deadline is not very soon, so everything may change). In the next week or two will concentrate on this competition on molecular properties. Actually it is almost my speciality, magnetism.. \n Lets keep in touch, if in 1-2 weeks our models will be complementary (i.e. scores on each bond types complement each other well), we may join in a team. Also, I am looking for a team on 2 medical imaging competitions which are taking place in the moment. They actually pretty similar to Severstahl competition...
", "votes": 0, "replies": [ { "id": 593539, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-08-06T18:50:14.373000", "content": "Good luck! BTW, Do you have experience in GNNs?)
", "votes": 1, "replies": [] } ] }, { "id": 574575, "author_name": "littlebeandog", "author_url": "", "post_date": "2019-07-14T05:55:25.060000", "content": "Good question....which I believe no one will answer = =!
\n\nBut I think GBM can at least reach around -1.7
", "votes": 0, "replies": [ { "id": 578010, "author_name": "Boris Silantev", "author_url": "", "post_date": "2019-07-17T09:25:20.580000", "content": "I've got -1.712 with feature ingeneering and LightGBM. I'm almost has no experience with NNs but I'm trying to apply NNs to my dataset and at the moment I have no significant improvements. It looks as I should in the short time seriously improve my knowledge of neural networks. I wonder if it is possible.
", "votes": 4, "replies": [] } ] }, { "id": 574641, "author_name": "", "author_url": "", "post_date": "2019-07-14T08:28:48.137000", "content": "", "votes": 0, "replies": [] }, { "id": 574605, "author_name": "", "author_url": "", "post_date": "2019-07-14T07:02:37.053000", "content": "Geometric features, ideas from public kernels, eliminating useless features and lgbm can reach around -2, so it is definitely worth to read the kernels.
", "votes": 15, "replies": [ { "id": 583324, "author_name": "rushakov", "author_url": "", "post_date": "2019-07-24T10:43:20.680000", "content": "Thanks for insight\nHow many features do you have in the best single lgbm model?
", "votes": 0, "replies": [] }, { "id": 583717, "author_name": "", "author_url": "", "post_date": "2019-07-24T21:53:22.617000", "content": "Around 70 features. (it depends on the type)
", "votes": 1, "replies": [] }, { "id": 583762, "author_name": "fwj7", "author_url": "", "post_date": "2019-07-25T00:14:21.487000", "content": "May I ask do you use lofo-importance to make feature selection?
", "votes": 1, "replies": [] }, { "id": 584264, "author_name": "", "author_url": "", "post_date": "2019-07-25T16:27:51.340000", "content": "Mostly permutation importance that I learned from a kernel https://www.kaggle.com/speedwagon/permutation-importance\nThis is fast and works reasonably well. LOFO would probably be more scientific, but slower.
", "votes": 4, "replies": [] }, { "id": 584598, "author_name": "fwj7", "author_url": "", "post_date": "2019-07-26T07:58:01.357000", "content": "Thanks, it seems like I should focus more on feature engineering.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "574452": "Hi everyone!\nI participate for a first time and I am a little bit confused about scores. I could get -1.4 with only geometrical features trained one model for every type. After this added qm9 features and got something about -1.5. So the question is what can transfer to -2.0 or -2.5 - tuning lgbm parameters for each type or more feature engineering on qm9? Or maybe I am loosing something else? Thanks in advance!", "574485": "From -1.5 to -2.0: better features, better models, 150% more efforts. \nFrom -2.0 to -2.5: extremely good features and models, 200% more efforts. \nFrom -1.5 to -1.51: ask people how to obtain -2.0 or -2.5. No one will reply you how to do that. ", "574559": "It is interesting that I couldn’t really improve the score with qm9 dataset. Maybe it is not so useful?", "574974": "Working on additional data. It seems to be very helpful. But is there a common approach to deal with the data that is presented only for train dataset?", "574758": "So much expectations of NNs here ... I going to dig deeper!", "574464": "Do you mean graph nns? Or just regular? ", "574861": "> So the question is what can transfer to -2.0 or -2.5\n\nTeaming ;)", "574711": "Based on my experience so far and some observations, I guess that a strategy combining a gradient boosting trees model with a graph neural network would result in a score < -2.0. \nI haven't yet used any GNN model thus my observation is just speculation for now. I will update this comment once I have some data. 😊 ", "574461": "I am totally guessing here but the order of importance is something like this: \n1. Features\n2. NNs\n3. something else\n4. lgbm tuning\n\nI am fairly certain that [LB #1 is using NNs](https://www.kaggle.com/c/champs-scalar-coupling/discussion/99069#573599)", "574460": "i believe it will be neural networks/deep learning", "593244": "Hi, Kirill! BTW, I see that you and your teammate are doing very well in this competition, great!... :) I am currently 13th at Severstahl competition (but its deadline is not very soon, so everything may change). In the next week or two will concentrate on this competition on molecular properties. Actually it is almost my speciality, magnetism.. \n Lets keep in touch, if in 1-2 weeks our models will be complementary (i.e. scores on each bond types complement each other well), we may join in a team. Also, I am looking for a team on 2 medical imaging competitions which are taking place in the moment. They actually pretty similar to Severstahl competition...", "574575": "Good question....which I believe no one will answer = =!\n\nBut I think GBM can at least reach around -1.7", "574641": "", "574605": "Geometric features, ideas from public kernels, eliminating useless features and lgbm can reach around -2, so it is definitely worth to read the kernels." } }