\n type error_mpnn error_lgbm error_stack\n0 1JHC 0.599302 0.437251 0.425867\n1 1JHN 0.406581 0.255192 0.251781\n2 2JHC 0.268458 0.268458 0.268458\n3 2JHH 0.147382 0.084441 0.082816\n4 2JHN 0.190512 0.124846 0.120789\n5 3JHC 0.285617 0.189488 0.187293\n6 3JHH 0.171340 0.110828 0.109682\n7 3JHN 0.192206 0.085231 0.084882\n
We have 2 types of models and 1 stack using FFNN. We have not finished LGBM on 2JHC yet (estimated CV will be ~0.22). Our features are not copies from public kernels, they are self-thought of, or are being pulled from QM9, openbabel, etc.
\n\nFuture ideas we are working on: make MPNN predictions better, run 2JHC on cloud, XGB, Catboost (for diversity)
\n\nThe more teammates we have, then we can add it to the stack and make the predictions better. Looking for diverse solutions that don't show bad CV/good LB (indicates overfitting) or very good CV/bad LB (indicates leak in features)
\n\nThings that make a good teammate:\n1. You have good CV in areas where we need improvement\n2. You have good CV in 2JHH, 3JHH, 2JHN, or 3JHN (improvement in these scores leads to faster improvement of overall Log MAE)\n3. You have access to GPU on your local machine or cloud (useful for training MPNN)\n4. You are motivated\n5. Your score is not obtained by blending with \"Blend of Blends\" kernels. We are looking for strong models that can work well on their own and don't require blend with public LB overfit...
\n\nIf interested, comment here or message me by going to my profile and clicking \"Contact User\"
", "messages": [ { "id": 600308, "postDate": "2019-08-16T00:40:31.590Z", "content": "\n type error_mpnn error_lgbm error_stack\n0 1JHC 0.599302 0.437251 0.425867\n1 1JHN 0.406581 0.255192 0.251781\n2 2JHC 0.268458 0.268458 0.268458\n3 2JHH 0.147382 0.084441 0.082816\n4 2JHN 0.190512 0.124846 0.120789\n5 3JHC 0.285617 0.189488 0.187293\n6 3JHH 0.171340 0.110828 0.109682\n7 3JHN 0.192206 0.085231 0.084882\n
We have 2 types of models and 1 stack using FFNN. We have not finished LGBM on 2JHC yet (estimated CV will be ~0.22). Our features are not copies from public kernels, they are self-thought of, or are being pulled from QM9, openbabel, etc.
\n\nFuture ideas we are working on: make MPNN predictions better, run 2JHC on cloud, XGB, Catboost (for diversity)
\n\nThe more teammates we have, then we can add it to the stack and make the predictions better. Looking for diverse solutions that don't show bad CV/good LB (indicates overfitting) or very good CV/bad LB (indicates leak in features)
\n\nThings that make a good teammate:\n1. You have good CV in areas where we need improvement\n2. You have good CV in 2JHH, 3JHH, 2JHN, or 3JHN (improvement in these scores leads to faster improvement of overall Log MAE)\n3. You have access to GPU on your local machine or cloud (useful for training MPNN)\n4. You are motivated\n5. Your score is not obtained by blending with \"Blend of Blends\" kernels. We are looking for strong models that can work well on their own and don't require blend with public LB overfit...
\n\nIf interested, comment here or message me by going to my profile and clicking \"Contact User\"
", "rawMarkdown": "```\n\ttype\terror_mpnn\terror_lgbm\terror_stack\n0\t1JHC\t0.599302\t0.437251\t0.425867\n1\t1JHN\t0.406581\t0.255192\t0.251781\n2\t2JHC\t0.268458\t0.268458\t0.268458\n3\t2JHH\t0.147382\t0.084441\t0.082816\n4\t2JHN\t0.190512\t0.124846\t0.120789\n5\t3JHC\t0.285617\t0.189488\t0.187293\n6\t3JHH\t0.171340\t0.110828\t0.109682\n7\t3JHN\t0.192206\t0.085231\t0.084882\n```\n\nWe have 2 types of models and 1 stack using FFNN. We have not finished LGBM on 2JHC yet (estimated CV will be ~0.22). Our features are not copies from public kernels, they are self-thought of, or are being pulled from QM9, openbabel, etc.\n\n1. MPNN has -1.37 Log MAE, scores -1.448 on public LB\n2. LightGBM has -1.798 Log MAE, scores -1.752 on public LB\n3. FFNN stack has -1.813 Log MAE, scores -1.781 on public LB\n\nFuture ideas we are working on: make MPNN predictions better, run 2JHC on cloud, XGB, Catboost (for diversity)\n\nThe more teammates we have, then we can add it to the stack and make the predictions better. Looking for diverse solutions that don't show bad CV/good LB (indicates overfitting) or very good CV/bad LB (indicates leak in features)\n\nThings that make a good teammate:\n1. You have good CV in areas where we need improvement\n2. You have good CV in 2JHH, 3JHH, 2JHN, or 3JHN (improvement in these scores leads to faster improvement of overall Log MAE)\n3. You have access to GPU on your local machine or cloud (useful for training MPNN)\n4. You are motivated\n5. Your score is not obtained by blending with \"Blend of Blends\" kernels. We are looking for strong models that can work well on their own and don't require blend with public LB overfit...\n\nIf interested, comment here or message me by going to my profile and clicking \"Contact User\"", "votes": 7 }, { "id": 600377, "postDate": "2019-08-16T03:13:36.110Z", "content": "Thank you so much for sharing. Hope you could find a good teammate.
\n\nMay I ask why you select NN instead of LGB for stacking? I only tried xgb for stacking so far. Thank you!
", "rawMarkdown": "Thank you so much for sharing. Hope you could find a good teammate. \n\nMay I ask why you select NN instead of LGB for stacking? I only tried xgb for stacking so far. Thank you!", "votes": 1, "replies": [ { "id": 600423, "postDate": "2019-08-16T05:20:32.110Z", "content": "In Home Credit Default Risk competition, my teammate used Logistic Regression to stack and I used FFNN to stack. It worked well. Then it also happened to work well in the Santander competition and well in other competitions. I prefer it because it is easy to set up, and the nondeterministic nature of it means you can run it multiple times and after averaging maybe can get higher score than just a stack with XGB. The stack I wrote in the discussion post was not averaged over multiple runs (if it were I think it would marginally improve score)
", "rawMarkdown": "In Home Credit Default Risk competition, my teammate used Logistic Regression to stack and I used FFNN to stack. It worked well. Then it also happened to work well in the Santander competition and well in other competitions. I prefer it because it is easy to set up, and the nondeterministic nature of it means you can run it multiple times and after averaging maybe can get higher score than just a stack with XGB. The stack I wrote in the discussion post was not averaged over multiple runs (if it were I think it would marginally improve score)", "votes": 1 }, { "id": 600548, "postDate": "2019-08-16T09:02:13.430Z", "content": "May I ask do you use NN or just LGB and XGB for stacking ?
", "rawMarkdown": "May I ask do you use NN or just LGB and XGB for stacking ?" }, { "id": 600714, "postDate": "2019-08-16T13:42:47.267Z", "content": "NN
", "rawMarkdown": "NN" } ] }, { "id": 600525, "postDate": "2019-08-16T08:28:22.610Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 600377, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2019-08-16T03:13:36.110000", "content": "Thank you so much for sharing. Hope you could find a good teammate.
\n\nMay I ask why you select NN instead of LGB for stacking? I only tried xgb for stacking so far. Thank you!
", "votes": 1, "replies": [ { "id": 600423, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-08-16T05:20:32.110000", "content": "In Home Credit Default Risk competition, my teammate used Logistic Regression to stack and I used FFNN to stack. It worked well. Then it also happened to work well in the Santander competition and well in other competitions. I prefer it because it is easy to set up, and the nondeterministic nature of it means you can run it multiple times and after averaging maybe can get higher score than just a stack with XGB. The stack I wrote in the discussion post was not averaged over multiple runs (if it were I think it would marginally improve score)
", "votes": 1, "replies": [] }, { "id": 600548, "author_name": "fwj7", "author_url": "", "post_date": "2019-08-16T09:02:13.430000", "content": "May I ask do you use NN or just LGB and XGB for stacking ?
", "votes": 0, "replies": [] }, { "id": 600714, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-08-16T13:42:47.267000", "content": "NN
", "votes": 0, "replies": [] } ] }, { "id": 600525, "author_name": "", "author_url": "", "post_date": "2019-08-16T08:28:22.610000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "600308": "```\n\ttype\terror_mpnn\terror_lgbm\terror_stack\n0\t1JHC\t0.599302\t0.437251\t0.425867\n1\t1JHN\t0.406581\t0.255192\t0.251781\n2\t2JHC\t0.268458\t0.268458\t0.268458\n3\t2JHH\t0.147382\t0.084441\t0.082816\n4\t2JHN\t0.190512\t0.124846\t0.120789\n5\t3JHC\t0.285617\t0.189488\t0.187293\n6\t3JHH\t0.171340\t0.110828\t0.109682\n7\t3JHN\t0.192206\t0.085231\t0.084882\n```\n\nWe have 2 types of models and 1 stack using FFNN. We have not finished LGBM on 2JHC yet (estimated CV will be ~0.22). Our features are not copies from public kernels, they are self-thought of, or are being pulled from QM9, openbabel, etc.\n\n1. MPNN has -1.37 Log MAE, scores -1.448 on public LB\n2. LightGBM has -1.798 Log MAE, scores -1.752 on public LB\n3. FFNN stack has -1.813 Log MAE, scores -1.781 on public LB\n\nFuture ideas we are working on: make MPNN predictions better, run 2JHC on cloud, XGB, Catboost (for diversity)\n\nThe more teammates we have, then we can add it to the stack and make the predictions better. Looking for diverse solutions that don't show bad CV/good LB (indicates overfitting) or very good CV/bad LB (indicates leak in features)\n\nThings that make a good teammate:\n1. You have good CV in areas where we need improvement\n2. You have good CV in 2JHH, 3JHH, 2JHN, or 3JHN (improvement in these scores leads to faster improvement of overall Log MAE)\n3. You have access to GPU on your local machine or cloud (useful for training MPNN)\n4. You are motivated\n5. Your score is not obtained by blending with \"Blend of Blends\" kernels. We are looking for strong models that can work well on their own and don't require blend with public LB overfit...\n\nIf interested, comment here or message me by going to my profile and clicking \"Contact User\"", "600377": "Thank you so much for sharing. Hope you could find a good teammate. \n\nMay I ask why you select NN instead of LGB for stacking? I only tried xgb for stacking so far. Thank you!", "600525": "" } }