Hello everyone, \nmy NN model has been terrible in this competition, but I still believe that the NN model is very helpful for stacking~~~\nAnyone could share best NN model CV&LB if you used it?
\n\nThank you!
", "messages": [ { "id": "479711", "postDate": "02/27/2019 11:05:30", "content": "Hello everyone, \nmy NN model has been terrible in this competition, but I still believe that the NN model is very helpful for stacking~~~\nAnyone could share best NN model CV&LB if you used it?
\n\nThank you!
", "rawMarkdown": "Hello everyone, \nmy NN model has been terrible in this competition, but I still believe that the NN model is very helpful for stacking~~~\nAnyone could share best NN model CV&LB if you used it?\n\nThank you!", "votes": null }, { "id": "479743", "postDate": "02/27/2019 11:42:29", "content": "my teammate have a nn with 3.599 private and 3.680 public. @Lisz
", "rawMarkdown": "my teammate have a nn with 3.599 private and 3.680 public. @Lisz", "votes": null }, { "id": "479754", "postDate": "02/27/2019 11:50:54", "content": "Cool!! can you detail describe your model structure design?
", "rawMarkdown": "Cool!! can you detail describe your model structure design?", "votes": null }, { "id": "479818", "postDate": "02/27/2019 12:59:05", "content": "it's not a single model. This result is used \"law of total probability\" same as 1st Solution.\nprob(outlier) * (-33.21928) + (1 - prob(outlier)) * regression(without outliers)
\n\nthe prob(outlier) is modeled by nn. regression(without outliers) is a lgb model.
\n\nthree sequences in this part
\n a. history
\n b. history & authorized == Y
\n c. new
\neach sequences sorted by timestamp, remain last 200 transactions
\neach transactions used some embedding feature:
\nauthorized(only in history), category_1/2/3, installments, month_lag, pre_transactions_gap ... and so on
use GRU & Attention Layer: like https://www.kaggle.com/suicaokhoailang/lstm-attention-baseline-0-672-lb
\n\nused features from lgb, uses z-score to normalize
\nuse a MissValueLayer to deal NaN Value. like https://paste.ubuntu.com/p/DgJD6yyGS2/\nyou can use it in keras, like:
\ndf.fillna(-10000)
\ninput = MissValueLayer(-10000)(input)\n
then FC, BN, PRelu, large rate Dropout, some shortcut
\n\na. binary task to predict outliers
\nb. regression to predict target
Because of I only submit nn once. So I don't know actully improve each part...\nIntuitively, the dense part is most important. \nmulti-task has some help and more robust.\nSequence is useless in cv, I keep it only for diversity
", "rawMarkdown": "it's not a single model. This result is used \"law of total probability\" same as 1st Solution.\nprob(outlier) * (-33.21928) + (1 - prob(outlier)) * regression(without outliers)\n\nthe prob(outlier) is modeled by nn. regression(without outliers) is a lgb model.\n\n##the nn model is formed of: \n\n### Sequence part \nthree sequences in this part \n a. history \n b. history & authorized == Y \n c. new \neach sequences sorted by timestamp, remain last 200 transactions \neach transactions used some embedding feature: \nauthorized(only in history), category_1/2/3, installments, month_lag, pre_transactions_gap ... and so on \n\nuse GRU & Attention Layer: like https://www.kaggle.com/suicaokhoailang/lstm-attention-baseline-0-672-lb \n\n### dense part\nused features from lgb, uses z-score to normalize \nuse a MissValueLayer to deal NaN Value. like https://paste.ubuntu.com/p/DgJD6yyGS2/\nyou can use it in keras, like: \n\n```\ndf.fillna(-10000) \ninput = MissValueLayer(-10000)(input)\n```\n\nthen FC, BN, PRelu, large rate Dropout, some shortcut \n\n### multi-task\na. binary task to predict outliers \nb. regression to predict target \n\n### PS.\nBecause of I only submit nn once. So I don't know actully improve each part...\nIntuitively, the dense part is most important. \nmulti-task has some help and more robust.\nSequence is useless in cv, I keep it only for diversity", "votes": null }, { "id": "479868", "postDate": "02/27/2019 14:15:56", "content": "Grateful
", "rawMarkdown": "Grateful", "votes": null } ], "comments": [ { "id": 479743, "author_name": "ddongjian0002", "author_url": "", "post_date": "02/27/2019 11:42:29", "content": "my teammate have a nn with 3.599 private and 3.680 public. @Lisz
", "votes": null, "replies": [ { "id": 479754, "author_name": "kangzhang", "author_url": "", "post_date": "02/27/2019 11:50:54", "content": "Cool!! can you detail describe your model structure design?
", "votes": null, "replies": [] }, { "id": 479818, "author_name": "lishaozhe1992", "author_url": "", "post_date": "02/27/2019 12:59:05", "content": "it's not a single model. This result is used \"law of total probability\" same as 1st Solution.\nprob(outlier) * (-33.21928) + (1 - prob(outlier)) * regression(without outliers)
\n\nthe prob(outlier) is modeled by nn. regression(without outliers) is a lgb model.
\n\nthree sequences in this part
\n a. history
\n b. history & authorized == Y
\n c. new
\neach sequences sorted by timestamp, remain last 200 transactions
\neach transactions used some embedding feature:
\nauthorized(only in history), category_1/2/3, installments, month_lag, pre_transactions_gap ... and so on
use GRU & Attention Layer: like https://www.kaggle.com/suicaokhoailang/lstm-attention-baseline-0-672-lb
\n\nused features from lgb, uses z-score to normalize
\nuse a MissValueLayer to deal NaN Value. like https://paste.ubuntu.com/p/DgJD6yyGS2/\nyou can use it in keras, like:
\ndf.fillna(-10000)
\ninput = MissValueLayer(-10000)(input)\n
then FC, BN, PRelu, large rate Dropout, some shortcut
\n\na. binary task to predict outliers
\nb. regression to predict target
Because of I only submit nn once. So I don't know actully improve each part...\nIntuitively, the dense part is most important. \nmulti-task has some help and more robust.\nSequence is useless in cv, I keep it only for diversity
", "votes": null, "replies": [] }, { "id": 479868, "author_name": "kangzhang", "author_url": "", "post_date": "02/27/2019 14:15:56", "content": "Grateful
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "479711": "Hello everyone, \nmy NN model has been terrible in this competition, but I still believe that the NN model is very helpful for stacking~~~\nAnyone could share best NN model CV&LB if you used it?\n\nThank you!", "479743": "my teammate have a nn with 3.599 private and 3.680 public. @Lisz", "479754": "Cool!! can you detail describe your model structure design?", "479818": "it's not a single model. This result is used \"law of total probability\" same as 1st Solution.\nprob(outlier) * (-33.21928) + (1 - prob(outlier)) * regression(without outliers)\n\nthe prob(outlier) is modeled by nn. regression(without outliers) is a lgb model.\n\n##the nn model is formed of: \n\n### Sequence part \nthree sequences in this part \n a. history \n b. history & authorized == Y \n c. new \neach sequences sorted by timestamp, remain last 200 transactions \neach transactions used some embedding feature: \nauthorized(only in history), category_1/2/3, installments, month_lag, pre_transactions_gap ... and so on \n\nuse GRU & Attention Layer: like https://www.kaggle.com/suicaokhoailang/lstm-attention-baseline-0-672-lb \n\n### dense part\nused features from lgb, uses z-score to normalize \nuse a MissValueLayer to deal NaN Value. like https://paste.ubuntu.com/p/DgJD6yyGS2/\nyou can use it in keras, like: \n\n```\ndf.fillna(-10000) \ninput = MissValueLayer(-10000)(input)\n```\n\nthen FC, BN, PRelu, large rate Dropout, some shortcut \n\n### multi-task\na. binary task to predict outliers \nb. regression to predict target \n\n### PS.\nBecause of I only submit nn once. So I don't know actully improve each part...\nIntuitively, the dense part is most important. \nmulti-task has some help and more robust.\nSequence is useless in cv, I keep it only for diversity", "479868": "Grateful" }, "source": "meta" }