{ "id": 102918, "title": "Your best NN (not GNN) score?", "url": "/competitions/champs-scalar-coupling/discussion/102918", "author_name": "heng", "post_date": "2019-08-06T01:19:33.929000", "votes": 12, "comment_count": 12, "views": 0, "content": "

We are training a NN base on this great kernel in recent days (using features from another great kernel). So far, our best LB score is -1.533.

\n\n

Validation score:\n1JHC : -0.3433880794815648\n2JHH : -1.988209030784158\n1JHN : -0.6291790261096047\n2JHN : -1.9922648592440666\n2JHC : -1.6449381956111688\n3JHH : -1.8786889566917344\n3JHC : -1.5559262222896761\n3JHN : -2.349979988173275

\n\n

What's your best NN score? 😄

", "messages": [ { "id": 592917, "postDate": "2019-08-06T01:19:33.930Z", "content": "

We are training a NN base on this great kernel in recent days (using features from another great kernel). So far, our best LB score is -1.533.

\n\n

Validation score:\n1JHC : -0.3433880794815648\n2JHH : -1.988209030784158\n1JHN : -0.6291790261096047\n2JHN : -1.9922648592440666\n2JHC : -1.6449381956111688\n3JHH : -1.8786889566917344\n3JHC : -1.5559262222896761\n3JHN : -2.349979988173275

\n\n

What's your best NN score? 😄

", "rawMarkdown": "We are training a NN base on [this great kernel](https://www.kaggle.com/todnewman/keras-neural-net-for-champs) in recent days (using features from [another great kernel](https://www.kaggle.com/marcelotamashiro/lgb-public-kernels-plus-more-features)). So far, our best LB score is -1.533. \n\nValidation score:\n1JHC : -0.3433880794815648\n2JHH : -1.988209030784158\n1JHN : -0.6291790261096047\n2JHN : -1.9922648592440666\n2JHC : -1.6449381956111688\n3JHH : -1.8786889566917344\n3JHC : -1.5559262222896761\n3JHN : -2.349979988173275\n\nWhat's your best NN score? 😄 \n", "votes": 11 }, { "id": 595541, "postDate": "2019-08-09T10:45:44.233Z", "content": "

Plain NN (tabular, Keras)

\n\n

Single fold validation: log(MAE)\n1JHC : -1.0668\n2JHH : -2.63113\n1JHN : -1.63487\n2JHN : -2.41966\n2JHC : -2.00082\n3JHH : -2.38288\n3JHC : -1.96888\n3JHN : -2.75022

\n\n

mean : -2.10691\nLB : -2.096

", "rawMarkdown": "Plain NN (tabular, Keras)\n\nSingle fold validation: log(MAE)\n1JHC : -1.0668\n2JHH : -2.63113\n1JHN : -1.63487\n2JHN : -2.41966\n2JHC : -2.00082\n3JHH : -2.38288\n3JHC : -1.96888\n3JHN : -2.75022\n\nmean : -2.10691\nLB : -2.096", "votes": 5, "replies": [ { "id": 595597, "postDate": "2019-08-09T12:33:51.493Z", "content": "

Awesome!

", "rawMarkdown": "Awesome!" }, { "id": 595882, "postDate": "2019-08-09T19:36:37.747Z", "rawMarkdown": "", "isDeleted": true }, { "id": 595944, "postDate": "2019-08-09T22:16:30.063Z", "content": "

Indeed, I do normalize all features with ScikitLearn StandardScaler

", "rawMarkdown": "Indeed, I do normalize all features with ScikitLearn StandardScaler" }, { "id": 596112, "postDate": "2019-08-10T07:21:19.290Z", "content": "

Do you use LightGBM or similar? From my experience GB trees works better with tabular data vs MLP NN.

", "rawMarkdown": "Do you use LightGBM or similar? From my experience GB trees works better with tabular data vs MLP NN." }, { "id": 596197, "postDate": "2019-08-10T09:44:55.683Z", "content": "

Indeed, you can get way better than -2 with lgb too, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/98444#581857

", "rawMarkdown": "Indeed, you can get way better than -2 with lgb too, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/98444#581857" } ] }, { "id": 593878, "postDate": "2019-08-07T08:10:52.870Z", "content": "

heya, not really what you asked but I use similar features with LGB and get the following scores:

\n\n

type 0: score: -0.37177809364989006\ntype 3: score: -1.8415483569108544\ntype 1: score: -1.0627788403748317\ntype 4: score: -1.6180205433774424\ntype 2: score: -1.1955816071253371\ntype 6: score: -1.7630544731733746\ntype 5: score: -1.1844071044220386\ntype 7: score: -2.016667921831149

\n\n

for an overall LB like -1,48. seems like a NN does worse on some, but better on others, so it might be worth adapting for your stack.

", "rawMarkdown": "heya, not really what you asked but I use similar features with LGB and get the following scores:\n\ntype 0: score: -0.37177809364989006\ntype 3: score: -1.8415483569108544\ntype 1: score: -1.0627788403748317\ntype 4: score: -1.6180205433774424\ntype 2: score: -1.1955816071253371\ntype 6: score: -1.7630544731733746\ntype 5: score: -1.1844071044220386\ntype 7: score: -2.016667921831149\n\nfor an overall LB like -1,48. seems like a NN does worse on some, but better on others, so it might be worth adapting for your stack.", "votes": 1 }, { "id": 593806, "postDate": "2019-08-07T06:28:54.043Z", "content": "

Part of my model is a plain NN (Fastai Tabular) which reaches LB -1.529. \nI'm only using structures.csv, train.csv and test.csv to generate festures.

", "rawMarkdown": "Part of my model is a plain NN (Fastai Tabular) which reaches LB -1.529. \nI'm only using structures.csv, train.csv and test.csv to generate festures.", "replies": [ { "id": 593851, "postDate": "2019-08-07T07:39:18.613Z", "content": "

Good job! \nIf you don't mind, may I ask how many features you using? \ntrain_test_split or kfold?

", "rawMarkdown": "Good job! \nIf you don't mind, may I ask how many features you using? \ntrain_test_split or kfold?" }, { "id": 593946, "postDate": "2019-08-07T10:23:30.277Z", "content": "

Sorry got the LB score wrong for the nn. It's only -1.529. \nI'm not folding on the nn. It is randomly sampled per type with 10% validation.\nUsing about 160 -190 features (need to check at home later).

", "rawMarkdown": "Sorry got the LB score wrong for the nn. It's only -1.529. \nI'm not folding on the nn. It is randomly sampled per type with 10% validation.\nUsing about 160 -190 features (need to check at home later)." } ] }, { "id": 594521, "postDate": "2019-08-08T06:06:02.130Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 595517, "postDate": "2019-08-09T10:14:41.783Z", "content": "

not my kernels ... 👀

", "rawMarkdown": "not my kernels ... 👀 " } ] } ], "comments": [ { "id": 595541, "author_name": "Philippe Lonjoux", "author_url": "", "post_date": "2019-08-09T10:45:44.233000", "content": "

Plain NN (tabular, Keras)

\n\n

Single fold validation: log(MAE)\n1JHC : -1.0668\n2JHH : -2.63113\n1JHN : -1.63487\n2JHN : -2.41966\n2JHC : -2.00082\n3JHH : -2.38288\n3JHC : -1.96888\n3JHN : -2.75022

\n\n

mean : -2.10691\nLB : -2.096

", "votes": 5, "replies": [ { "id": 595597, "author_name": "heng", "author_url": "", "post_date": "2019-08-09T12:33:51.493000", "content": "

Awesome!

", "votes": 0, "replies": [] }, { "id": 595882, "author_name": "", "author_url": "", "post_date": "2019-08-09T19:36:37.747000", "content": "", "votes": 0, "replies": [] }, { "id": 595944, "author_name": "Philippe Lonjoux", "author_url": "", "post_date": "2019-08-09T22:16:30.063000", "content": "

Indeed, I do normalize all features with ScikitLearn StandardScaler

", "votes": 0, "replies": [] }, { "id": 596112, "author_name": "Sergey Zlobin", "author_url": "", "post_date": "2019-08-10T07:21:19.290000", "content": "

Do you use LightGBM or similar? From my experience GB trees works better with tabular data vs MLP NN.

", "votes": 0, "replies": [] }, { "id": 596197, "author_name": "CPMP", "author_url": "", "post_date": "2019-08-10T09:44:55.683000", "content": "

Indeed, you can get way better than -2 with lgb too, see https://www.kaggle.com/c/champs-scalar-coupling/discussion/98444#581857

", "votes": 0, "replies": [] } ] }, { "id": 593878, "author_name": "Proletheus", "author_url": "", "post_date": "2019-08-07T08:10:52.870000", "content": "

heya, not really what you asked but I use similar features with LGB and get the following scores:

\n\n

type 0: score: -0.37177809364989006\ntype 3: score: -1.8415483569108544\ntype 1: score: -1.0627788403748317\ntype 4: score: -1.6180205433774424\ntype 2: score: -1.1955816071253371\ntype 6: score: -1.7630544731733746\ntype 5: score: -1.1844071044220386\ntype 7: score: -2.016667921831149

\n\n

for an overall LB like -1,48. seems like a NN does worse on some, but better on others, so it might be worth adapting for your stack.

", "votes": 1, "replies": [] }, { "id": 593806, "author_name": "Jo Tom", "author_url": "", "post_date": "2019-08-07T06:28:54.043000", "content": "

Part of my model is a plain NN (Fastai Tabular) which reaches LB -1.529. \nI'm only using structures.csv, train.csv and test.csv to generate festures.

", "votes": 0, "replies": [ { "id": 593851, "author_name": "heng", "author_url": "", "post_date": "2019-08-07T07:39:18.613000", "content": "

Good job! \nIf you don't mind, may I ask how many features you using? \ntrain_test_split or kfold?

", "votes": 0, "replies": [] }, { "id": 593946, "author_name": "Jo Tom", "author_url": "", "post_date": "2019-08-07T10:23:30.277000", "content": "

Sorry got the LB score wrong for the nn. It's only -1.529. \nI'm not folding on the nn. It is randomly sampled per type with 10% validation.\nUsing about 160 -190 features (need to check at home later).

", "votes": 0, "replies": [] } ] }, { "id": 594521, "author_name": "", "author_url": "", "post_date": "2019-08-08T06:06:02.130000", "content": "", "votes": 1, "replies": [ { "id": 595517, "author_name": "heng", "author_url": "", "post_date": "2019-08-09T10:14:41.783000", "content": "

not my kernels ... 👀

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "592917": "We are training a NN base on [this great kernel](https://www.kaggle.com/todnewman/keras-neural-net-for-champs) in recent days (using features from [another great kernel](https://www.kaggle.com/marcelotamashiro/lgb-public-kernels-plus-more-features)). So far, our best LB score is -1.533. \n\nValidation score:\n1JHC : -0.3433880794815648\n2JHH : -1.988209030784158\n1JHN : -0.6291790261096047\n2JHN : -1.9922648592440666\n2JHC : -1.6449381956111688\n3JHH : -1.8786889566917344\n3JHC : -1.5559262222896761\n3JHN : -2.349979988173275\n\nWhat's your best NN score? 😄 \n", "595541": "Plain NN (tabular, Keras)\n\nSingle fold validation: log(MAE)\n1JHC : -1.0668\n2JHH : -2.63113\n1JHN : -1.63487\n2JHN : -2.41966\n2JHC : -2.00082\n3JHH : -2.38288\n3JHC : -1.96888\n3JHN : -2.75022\n\nmean : -2.10691\nLB : -2.096", "593878": "heya, not really what you asked but I use similar features with LGB and get the following scores:\n\ntype 0: score: -0.37177809364989006\ntype 3: score: -1.8415483569108544\ntype 1: score: -1.0627788403748317\ntype 4: score: -1.6180205433774424\ntype 2: score: -1.1955816071253371\ntype 6: score: -1.7630544731733746\ntype 5: score: -1.1844071044220386\ntype 7: score: -2.016667921831149\n\nfor an overall LB like -1,48. seems like a NN does worse on some, but better on others, so it might be worth adapting for your stack.", "593806": "Part of my model is a plain NN (Fastai Tabular) which reaches LB -1.529. \nI'm only using structures.csv, train.csv and test.csv to generate festures.", "594521": "" } }