![\"enter](\"https://i.imgsafe.org/91d5795.png\")
I was reading raddar's post about their advance data exploration and inference about the anonymous feature sets, I must admit you guys did a fantastic job there. In contrast, we did not invest much time on EDA as we found out likelihood encoding of interactions improving the result exponentially in the early stage of the competition. Since then, I mostly focused on creating different representation of the interactions in different levels and Daniel took care of the Stacking and ensembling part. Jeremy (idle_speculation) also had a fantastic set of features which helped us in the later stage of the competition. the image below illustrates our workflow in a nutshell:
\n\n
We also tried a lot of other things but didn't have much time to finish training on them- especially worth mentioning RGF. As already noted in the forum, performance wise it was almost on par with XGB but very slow.
\n\nI extend my greetings to the other winners and thank Kaggle and BNP Paribas for hosting such a nice competition. I would also like to thank my teammates Daniel and Jeremy for their valuable contributions.
\n\nLast but not the least, Laurea - keep rocking :)
", "comments": [ { "id": 115869, "author_name": "Faron", "author_url": "", "post_date": "2016-04-20T14:54:40.970000", "content": "[quote=JibyBabu;115855]
\n\nCongrats Biswaroop!
\n\nBiswaroop/Kaggle Masters,\nAs rcason and Andrey posted, do you have references to know about this Bayesian coding of features?\nI am a newbie in data science and is excited to explore this hack :)
\n\nThanks,\nJiby
\n\n[/quote]
\n\nSome references:
\n\ncongratulations, and thanks a lot for the sharing.\nI got confuse your guys minus 0.15*ET in the L3? can you teach me why minus but not add? generally we blending by adding ,isnt it?
", "votes": 1, "replies": [] }, { "id": 115646, "author_name": "khyh", "author_url": "", "post_date": "2016-04-19T13:58:25.857000", "content": "Congratulations and thank you for sharing.
\n\nIt seems like the 1st place team and the 2nd place team took a quite different approach and resulted in the similar score. Really interesting.
", "votes": 1, "replies": [] }, { "id": 115642, "author_name": "Shubin", "author_url": "", "post_date": "2016-04-19T13:30:00.733000", "content": "congrats. could you please share your structure for nn? from l1 to l2 and l2 to l3. impressive work.
", "votes": 1, "replies": [] }, { "id": 115627, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2016-04-19T12:32:36.577000", "content": "Very Nice approach. Well-illustrated and well done for your great performance :)
", "votes": 1, "replies": [] }, { "id": 115617, "author_name": "Faron", "author_url": "", "post_date": "2016-04-19T10:47:33.240000", "content": "congratz to you guys! I like your approach. :)
", "votes": 2, "replies": [] }, { "id": 115583, "author_name": "Andrey Vykhodtsev", "author_url": "", "post_date": "2016-04-19T07:48:15.170000", "content": "Congratulations, well done! And thanks a lot for sharing!
\n\nI always look at such pipelines with awe, as I never could build something that complex without becoming a complete mess.\nI always wonder if there is a good writeup somewhere on the technique of build ing the pipeline itself. Or some code examples?
\n\nIf I may also ask the esteemed community to share some good links on Bayesian coding of features?
", "votes": 0, "replies": [] }, { "id": 116293, "author_name": "", "author_url": "", "post_date": "2016-04-22T23:52:56.780000", "content": "Congratulations on landing in 2nd place. Very interesting approach. I am intrigued about lasso being used to identify variables used to create interactions. Does lasso have something like a variable importance table or is this done through an iterative process whilst comparing something like log loss?
", "votes": 0, "replies": [] }, { "id": 115867, "author_name": "Balaji SR", "author_url": "", "post_date": "2016-04-20T14:52:42.087000", "content": "Congrats Biswaroop and Team
", "votes": 0, "replies": [] }, { "id": 115855, "author_name": "Jiby Babu", "author_url": "", "post_date": "2016-04-20T14:00:10.327000", "content": "Congrats Biswaroop!
\n\nBiswaroop/Kaggle Masters,\nAs rcason and Andrey posted, do you have references to know about this Bayesian coding of features?\nI am a newbie in data science and is excited to explore this hack :)
\n\nThanks,\nJiby
", "votes": 0, "replies": [] }, { "id": 115758, "author_name": "Ferris Wu", "author_url": "", "post_date": "2016-04-19T22:55:08.597000", "content": "In pipeline 1 & 5, did you tune parameters for each model on each data partition, e.g. using small eta in XGB? What's the number of folds to create L2 meta features?
\n\nIn pipeline 2, 3 & 4, please correct me if I'm wrong :) by 'interaction' do you mean the out of fold mean target grouped by cate features? How does the binning work? Quantiles?
\n\nSorry to bother you with so many questions, really appreciate it if you can answer some of them. I feel I'm so far away from true masters like you guys.
", "votes": 0, "replies": [] }, { "id": 115731, "author_name": "Jiwei Liu", "author_url": "", "post_date": "2016-04-19T18:43:21.877000", "content": "Congratulations!! Bishwarup B, could you please talk more about likelihood/Bayesian encoding? Thank you very much!
", "votes": 0, "replies": [] }, { "id": 115636, "author_name": "Zichen Wang", "author_url": "", "post_date": "2016-04-19T12:54:30.447000", "content": "Congrats! Impressive levels of ensemble.
", "votes": 0, "replies": [] }, { "id": 115632, "author_name": "Rodolphe Lampe", "author_url": "", "post_date": "2016-04-19T12:47:57.597000", "content": "Congratulations for your 2nd place.\nI wonder how you coded the pipeline #4.
", "votes": 0, "replies": [] }, { "id": 115620, "author_name": "Ferris Wu", "author_url": "", "post_date": "2016-04-19T10:57:26.037000", "content": "Thanks for sharing. I can see there are so many details behind this, and a lot of work.
", "votes": 0, "replies": [] }, { "id": 115609, "author_name": "Andy White", "author_url": "", "post_date": "2016-04-19T09:59:50.990000", "content": "Thank you for sharing this, and congratulations.
\n\nDoes "bagged 20 times" mean running 20 different random configurations of that base model and averaging the output?
\n\nWhat is the reasoning behind subtracting the ET predictions at L3?
", "votes": 0, "replies": [] }, { "id": 115591, "author_name": "Darragh", "author_url": "", "post_date": "2016-04-19T08:38:51.177000", "content": "Great work team.
", "votes": 0, "replies": [] } ], "comment_count": 17, "messages": [ { "id": 115581, "postDate": "2016-04-19T07:38:58.867Z", "content": "I was reading raddar's post about their advance data exploration and inference about the anonymous feature sets, I must admit you guys did a fantastic job there. In contrast, we did not invest much time on EDA as we found out likelihood encoding of interactions improving the result exponentially in the early stage of the competition. Since then, I mostly focused on creating different representation of the interactions in different levels and Daniel took care of the Stacking and ensembling part. Jeremy (idle_speculation) also had a fantastic set of features which helped us in the later stage of the competition. the image below illustrates our workflow in a nutshell:
\n\n
We also tried a lot of other things but didn't have much time to finish training on them- especially worth mentioning RGF. As already noted in the forum, performance wise it was almost on par with XGB but very slow.
\n\nI extend my greetings to the other winners and thank Kaggle and BNP Paribas for hosting such a nice competition. I would also like to thank my teammates Daniel and Jeremy for their valuable contributions.
\n\nLast but not the least, Laurea - keep rocking :)
", "rawMarkdown": "I was reading raddar's post about their advance data exploration and inference about the anonymous feature sets, I must admit you guys did a fantastic job there. In contrast, we did not invest much time on EDA as we found out likelihood encoding of interactions improving the result exponentially in the early stage of the competition. Since then, I mostly focused on creating different representation of the interactions in different levels and Daniel took care of the Stacking and ensembling part. Jeremy (idle_speculation) also had a fantastic set of features which helped us in the later stage of the competition. the image below illustrates our workflow in a nutshell:\r\n\r\n![enter image description here][1]\r\n\r\n\r\n [1]: https://i.imgsafe.org/91d5795.png\r\n\r\nWe also tried a lot of other things but didn't have much time to finish training on them- especially worth mentioning RGF. As already noted in the forum, performance wise it was almost on par with XGB but very slow.\r\n\r\nI extend my greetings to the other winners and thank Kaggle and BNP Paribas for hosting such a nice competition. I would also like to thank my teammates Daniel and Jeremy for their valuable contributions.\r\n\r\nLast but not the least, Laurea - keep rocking :)", "votes": 62 }, { "id": 115869, "postDate": "2016-04-20T14:54:40.970Z", "content": "[quote=JibyBabu;115855]
\n\nCongrats Biswaroop!
\n\nBiswaroop/Kaggle Masters,\nAs rcason and Andrey posted, do you have references to know about this Bayesian coding of features?\nI am a newbie in data science and is excited to explore this hack :)
\n\nThanks,\nJiby
\n\n[/quote]
\n\nSome references:
\n\ncongratulations, and thanks a lot for the sharing.\nI got confuse your guys minus 0.15*ET in the L3? can you teach me why minus but not add? generally we blending by adding ,isnt it?
", "rawMarkdown": "congratulations, and thanks a lot for the sharing.\r\nI got confuse your guys minus 0.15*ET in the L3? can you teach me why minus but not add? generally we blending by adding ,isnt it?", "votes": 1 }, { "id": 115646, "postDate": "2016-04-19T13:58:25.857Z", "content": "Congratulations and thank you for sharing.
\n\nIt seems like the 1st place team and the 2nd place team took a quite different approach and resulted in the similar score. Really interesting.
", "rawMarkdown": "Congratulations and thank you for sharing.\r\n\r\nIt seems like the 1st place team and the 2nd place team took a quite different approach and resulted in the similar score. Really interesting.", "votes": 1 }, { "id": 115642, "postDate": "2016-04-19T13:30:00.733Z", "content": "congrats. could you please share your structure for nn? from l1 to l2 and l2 to l3. impressive work.
", "rawMarkdown": "congrats. could you please share your structure for nn? from l1 to l2 and l2 to l3. impressive work. ", "votes": 1 }, { "id": 115627, "postDate": "2016-04-19T12:32:36.577Z", "content": "Very Nice approach. Well-illustrated and well done for your great performance :)
", "rawMarkdown": "Very Nice approach. Well-illustrated and well done for your great performance :)", "votes": 1 }, { "id": 115617, "postDate": "2016-04-19T10:47:33.240Z", "content": "congratz to you guys! I like your approach. :)
", "rawMarkdown": "congratz to you guys! I like your approach. :)", "votes": 2 }, { "id": 115583, "postDate": "2016-04-19T07:48:15.170Z", "content": "Congratulations, well done! And thanks a lot for sharing!
\n\nI always look at such pipelines with awe, as I never could build something that complex without becoming a complete mess.\nI always wonder if there is a good writeup somewhere on the technique of build ing the pipeline itself. Or some code examples?
\n\nIf I may also ask the esteemed community to share some good links on Bayesian coding of features?
", "rawMarkdown": "Congratulations, well done! And thanks a lot for sharing!\r\n\r\nI always look at such pipelines with awe, as I never could build something that complex without becoming a complete mess.\r\nI always wonder if there is a good writeup somewhere on the technique of build ing the pipeline itself. Or some code examples?\r\n\r\nIf I may also ask the esteemed community to share some good links on Bayesian coding of features?" }, { "id": 116293, "postDate": "2016-04-22T23:52:56.780Z", "content": "Congratulations on landing in 2nd place. Very interesting approach. I am intrigued about lasso being used to identify variables used to create interactions. Does lasso have something like a variable importance table or is this done through an iterative process whilst comparing something like log loss?
", "rawMarkdown": "Congratulations on landing in 2nd place. Very interesting approach. I am intrigued about lasso being used to identify variables used to create interactions. Does lasso have something like a variable importance table or is this done through an iterative process whilst comparing something like log loss?" }, { "id": 115867, "postDate": "2016-04-20T14:52:42.087Z", "content": "Congrats Biswaroop and Team
", "rawMarkdown": "Congrats Biswaroop and Team" }, { "id": 115855, "postDate": "2016-04-20T14:00:10.327Z", "content": "Congrats Biswaroop!
\n\nBiswaroop/Kaggle Masters,\nAs rcason and Andrey posted, do you have references to know about this Bayesian coding of features?\nI am a newbie in data science and is excited to explore this hack :)
\n\nThanks,\nJiby
", "rawMarkdown": "Congrats Biswaroop!\r\n\r\nBiswaroop/Kaggle Masters,\r\nAs rcason and Andrey posted, do you have references to know about this Bayesian coding of features?\r\nI am a newbie in data science and is excited to explore this hack :)\r\n\r\nThanks,\r\nJiby" }, { "id": 115758, "postDate": "2016-04-19T22:55:08.597Z", "content": "In pipeline 1 & 5, did you tune parameters for each model on each data partition, e.g. using small eta in XGB? What's the number of folds to create L2 meta features?
\n\nIn pipeline 2, 3 & 4, please correct me if I'm wrong :) by 'interaction' do you mean the out of fold mean target grouped by cate features? How does the binning work? Quantiles?
\n\nSorry to bother you with so many questions, really appreciate it if you can answer some of them. I feel I'm so far away from true masters like you guys.
", "rawMarkdown": "In pipeline 1 & 5, did you tune parameters for each model on each data partition, e.g. using small eta in XGB? What's the number of folds to create L2 meta features?\r\n\r\nIn pipeline 2, 3 & 4, please correct me if I'm wrong :) by 'interaction' do you mean the out of fold mean target grouped by cate features? How does the binning work? Quantiles?\r\n\r\nSorry to bother you with so many questions, really appreciate it if you can answer some of them. I feel I'm so far away from true masters like you guys." }, { "id": 115731, "postDate": "2016-04-19T18:43:21.877Z", "content": "Congratulations!! Bishwarup B, could you please talk more about likelihood/Bayesian encoding? Thank you very much!
", "rawMarkdown": "Congratulations!! Bishwarup B, could you please talk more about likelihood/Bayesian encoding? Thank you very much! " }, { "id": 115636, "postDate": "2016-04-19T12:54:30.447Z", "content": "Congrats! Impressive levels of ensemble.
", "rawMarkdown": "Congrats! Impressive levels of ensemble." }, { "id": 115632, "postDate": "2016-04-19T12:47:57.597Z", "content": "Congratulations for your 2nd place.\nI wonder how you coded the pipeline #4.
", "rawMarkdown": "Congratulations for your 2nd place.\r\nI wonder how you coded the pipeline #4." }, { "id": 115620, "postDate": "2016-04-19T10:57:26.037Z", "content": "Thanks for sharing. I can see there are so many details behind this, and a lot of work.
", "rawMarkdown": "Thanks for sharing. I can see there are so many details behind this, and a lot of work." }, { "id": 115609, "postDate": "2016-04-19T09:59:50.990Z", "content": "Thank you for sharing this, and congratulations.
\n\nDoes "bagged 20 times" mean running 20 different random configurations of that base model and averaging the output?
\n\nWhat is the reasoning behind subtracting the ET predictions at L3?
", "rawMarkdown": "Thank you for sharing this, and congratulations.\r\n\r\nDoes \"bagged 20 times\" mean running 20 different random configurations of that base model and averaging the output?\r\n\r\nWhat is the reasoning behind subtracting the ET predictions at L3?" }, { "id": 115591, "postDate": "2016-04-19T08:38:51.177Z", "content": "Great work team.
", "rawMarkdown": "Great work team." } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T12:28:28Z", "competition_id": 4852, "competition_slug": "bnp-paribas-cardif-claims-management", "team_id": 277015, "solution_url": "/competitions/bnp-paribas-cardif-claims-management/writeups/escalated-chi-escalated-chi-2nd-place-solution" }