![\"enter](\"http://i.imgur.com/P9LoFhN.png\")
So glad we won this one! After having such good data findings it would have been a shame not to win :)
\n\nIt is clear that this competition was not about ensembling but feature engineering, and we are happy that we found out the major key to winning this competition early on.\nTherefore, we had lots of time to exploit our findings to the max and maintain the top position ever since. \nThe trick was very simple, and I even shared a hint to that early in the competition (without me understanding its importance at that time):\nhttps://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19240/analysis-of-duplicate-variables-correlated-variables-large-post/110095#post110095
\n\nWe figured, that data unanonymizing was important and in the end we were able to figure out some of variables' meaning:
\n\nv40 - date of observation
\n\nv40-v50 - contract start date
\n\nv50 - days since contract startdate (in other words, claimdate)
\n\nv10 - contract term in months
\n\nand few others for remaining days of the contract:
\n\nv12=v10*(365.25)/12-v50
\n\nv34=v10*(365.25)/12-v40
\n\nNow add this knowledge to the assumption that v22 is a customer and v56/v113 is a product type, \nand you will see very obvious patterns, that one contract may have several claims during contract lifespan (R: group_by(v22,v40-v50) %>% arrange(v50)):
\n\n[v22,v40-v50] [v50] [target]
\n\n[ZLS 12840] 197 1
\n\n[ZLS 12840] 962 1
\n\n[ZLS 12840] 1437 1
\n\n[ZLS 12840] 1498 NA
\n\n[ZLS 12840] 1501 1
\n\n[ZLS 12840] 1726 1
\n\n[ZLS 12840] 1788 NA
\n\n[ZLS 12840] 1882 NA
\n\n[ZLS 12840] 2418 1
\n\n[ZLS 12840] 3352 NA
\n\n[ZLS 12840] 3370 NA
\n\n[ZLS 12840] 3909 NA
\n\nSo we could drop the categorical {v22} level i.i.d. assumption, and move to panel data structured categorical levels such as: \n{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc;
\n\nJust looking at the data this way, it was obvious that target value is very persistent for each {v22,v40-v50} level - \ni.e. if a time series starts with target=1, it usually ends with target=1; if it starts with target=0 it often ends with target=0 too; claims which target shifts 0->1 are quite rare (1->0 only few cases);\nSo in the end it all resulted to correctly imputing target sequence for each level - which we done using lag(target) and lead(target) variables;\nTo our surprise, these variables were not overfitting LB, and in the end we made so many lag/lead variables that it was possible to drop v22 column, \nand didn't even use v22 mean target techniques disccussed in the forums, which probably many top teams did anyway.
\n\nOur best single xgboost model achieved 0.42347 public LB (0.42193 private), and the model takes itself only about half an hour to train on 12-thread cpu. \nTo seal the deal, in last 2 weeks we experimented with other techniques and build few stack models with tens of different methodology models, such as nnets, \nlinear SVM's, elastic nets, xgboosts with count:poisson, rank:pairwise, etc.\nI personally enjoyed working with regularized greedy forests, which were almost on par with xgboost.
\n\nThe role of ensembling may not as important as other competition though we have tried several diverse models.
\n\n-models with tsne feature from continous variables
\n\n-models by levels of certain varaible (for example,var5)
\n\n-knn models on likehihood encoding varibles
\n\nTo sum up: \nAs most of you, we were stuck at 0.45x for a long time and it took 3-4 weeks of dedicated time of looking and exploring the data in Excel to end up with panel time-series data, which was the key to success.\nHaving such knowledge about the data could have gotten you to top10 without too much effort.\nIn the end we created a useless model for Bnp, as our lead(target) variables use information from the future:)
\n\nAnd for guys who wants to succeed -
\n\na) when starting a new competition, create simple xgboost model and use feature.importance to get a nice start to discovering important features - then stop making models and work with the data.
\n\nb) do not underestimate the power of knowing what data you are working with
\n\nc) dedicate some time for data exploration and try to understand how people visualize the data in the forums
\n\nd) look for data patterns, especially if it has many categorical variables
\n\ne) spend some time reading forums of past competitions, especially winning materials
\n\nf) keep eyes on overfitting
\n\nP.S. special thanks to this blog post http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/ \n"I found that the out-of-fold CV predictions of categorized lag features were very important. As far as I saw in the forum, many of the participants may have not created these features."\nThis shook our beliefs and assumptions about data being i.i.d in {v22} levels, and it took us only 1 day to utilize this and claim the top1 rank for the rest of the copmetition:)
\n\nP.S.S huge thanks to Laurae for the input to data exploration. I believe it made this competition more interesting for most of us.
\n\nP.S.S. team name came up when I decided to join forces with Davut, and the cartoon was playing in the background- for both of us it was our one of favorite childhood cartoons :)
", "comments": [ { "id": 115643, "author_name": "ToonRoge", "author_url": "", "post_date": "2016-04-19T13:41:26.633000", "content": "I am impressed by the cleverness of the winning solution, but also by the incredible complexity of the 2nd solution, it should take a lot of discipline, organization and coding skills to be able to make such solution reproducible and keep the code clean etc... I would have never be able to find these solutions myself.
\n\nCan I just make some reflections (without trying to diminish the performance of the winners - just a reflection ...):
\n\n--> Is the main goal of this competition getting a useful model, I am not sure that they have got the maximum out of this competition... Is the main goal to get good PR and attract new people, knowing how to code etc... I guess they succeed!
\n\nThanks for the competition and again congratz to the winners!\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)
", "votes": 21, "replies": [] }, { "id": 115563, "author_name": "raddar", "author_url": "", "post_date": "2016-04-19T06:24:47.990000", "content": "Thank you all,
\n\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)
\n\nIt is actually quite simple. lets say you have variable X; you sort that X and calculate rank values for each X value. then take X[rank] - X[rank-1] and plot a distribution plot of that. You will see very interesting distribution of several spikes.
\n\nTo put it simply, lets say you have 0.00049, 0.00049,....0.00049,0.00098,0.00098,....0.00147,0.00147. See the pattern :) THen it just comes to simple optim function to search for median values between "jumping points" (X[rank] - X[rank-1]) distribution.
\n\nv40,v10,v50,v18,v28 were the variables, where this seperation is present. There are some more variables which are a mixture of this and gaussian density (meaning there are many integers, but also fractional numbers in that variable) - However I did not dig into it.
", "votes": 10, "replies": [] }, { "id": 115534, "author_name": "Davut Polat", "author_url": "", "post_date": "2016-04-19T01:48:05.597000", "content": "all credicts goes to raddar for discovering this :)
", "votes": 7, "replies": [] }, { "id": 115666, "author_name": "Laurae", "author_url": "", "post_date": "2016-04-19T14:58:06.660000", "content": "[quote=ToonRoge;115643]
\n\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)
\n\n[/quote]
\n\nWith all I posted you can create many features... many! But not every feature is useful. At the end, I used only 24 features (yes, only 24) in my final model without any stacking/post-ensembling, it got 0.44493 Private LB. But some of these features needed... several hours to compute :(
\n\nI went for a type of "credit scoring" where 10 of my features were scores about the label, all coming from pure EDA and visualizations. Other 14 features were T-SNE and variable interactions. Sometimes I would get strange results if I added a new feature.
\n\nxgboost is pretty absurd in performance when it comes to take the most from engineered variables.
\n\nExample of strange results I could get: sometimes it's either everything or nothing in-between. V is one of my engineered features. It could get a near perfect accuracy on some buckets where target=0, but was still failing very hard at separating 0 and 1 when they were together.
\n\n
[quote=ToonRoge;115643]
\n\nSomeone working at BNP, having the dictionary of the data and insight in the business process, should be able to outperform this 2nd model with a much simpler and IT friendly model in a couple of days, or am I wrong here?
\n\n[/quote]
\n\nNot sure. I think 0.44 is a reasonable target if you have the full insight on the data. Check Jack's (Japan) model here, not only it is fast but it gives a very good performance in this competition. But that's still a "black box" model.
\n\nYou have two important paths in a business when dealing with such machine learning project:
\n\nThe latter is mostly preferred. But there are cases where the first suggestion is better. Here if they wanted the highest transparency (to tell the customer the reason whether the claim is fulfilled to be accelerated or not), they would go the second way. What they will also look at is the false positive rate :)
\n\n[quote=raddar;115661]
\n\nvar114 has strong correlation with var40 (which is observation date). so var114 has to do something with dates.
\n\nOne of my hypothesis is that var114 is a datetime and var40 is a date format. the time value caused the correlation not be linearly perfect.
\n\n[/quote]
\n\nFor v40 (I'm doing the other way, there is something very funny at the end), look at v13 / v19 / v27 / v35 / v45 / v50 / v57 / v114. I suspect there is something simpler than this 8-way relation. I get 0.999876 R^2 with 0.033133 RMSE. Did not cross-validate though.
\n\nedit: RMSE 0.037213 using only v50 and v114 with appropriate cuts on v114.
\n\n![\"enter](\"http://i.imgur.com/0rcU6oL.png\")
I got the following plane of v50/v40/v114, which seems to look curved.
\n\n![\"enter](\"http://i.imgur.com/noTxFBR.png\")
Without v19, I get RMSE=0.1701. I suspect compounding noise makes the residuals explode.
\n\nHere is a funny picture using v50 and v114 :)
\n\n![\"enter](\"http://i.imgur.com/108KwZH.png\")
edit: instructions below to reproduce not working, I have no idea how I got that picture using v50/v114 because I cannot reproduce it :(
\n\nTo get that picture, you need to do the following linear separations in v50 and v114:
\n\nYou should have 8 columns.
\n\nOr use this formula if you are keen at mathematics programming:
\n\n![\"enter](\"http://i.imgur.com/EJ637i3.png\")
It's always inspiring seeing people smash the LB. Nicely done!
", "votes": 8, "replies": [] }, { "id": 115656, "author_name": "Naokazu Mizuta", "author_url": "", "post_date": "2016-04-19T14:29:05.863000", "content": "Congrats to the winners!
\n\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.
\n\n@raddar Do you have any ideas or make assumptions of what v114 means?
", "votes": 5, "replies": [] }, { "id": 115578, "author_name": "raddar", "author_url": "", "post_date": "2016-04-19T07:32:19.240000", "content": "[quote=happydemiologist;115573]
\n\nCongratulations, raddar. You both really got into the data. Could you please further simplify the example for laypersons like myself. I see that variable X is sorted (?), but the median is now calculated for each point in the variable?
\n\n[/quote]
\n\nmaybe this simple script will help:
\n\nv50=sort(train$v50)
\n\nv50_diff=v50[2:length(v50)]- v50[1:(length(v50)-1)]
\n\ntable(round(v50_diff,5))
\n\nnow if you looked at the table, you will see that table values are increasing by a factor of 0.00147*x, which is strong indication of v50 being a integer variable
", "votes": 5, "replies": [] }, { "id": 115572, "author_name": "raddar", "author_url": "", "post_date": "2016-04-19T07:16:38.933000", "content": "My teammate Song works in insurance, and I tackled the data from a banking perspective (I actually thought we were making a model for a bank for long time, lol). So we convinced ourselves that these were dates, because from our experience it seemed logical, that a person which claim was approved at least once would indicate a "good" customer feature. and the person who is constantly rejected (target=0) would indicate persistent fraudulent behaviour.
\n\nBy the way all the data has another interesting property I have not shared :) when you look at i.e. {v22,v40-v50,v56} level and look at all other variables you will notice that they are all constants in time :) So in the end we made a conclusion that all variables are some kind filled in application forms; if you calculated hashes on numericals, there are only 30k unique hashes, which kind of support our hypothesis. That is why people saw some interesting behaviour, when removing many variables would help their score:)
", "votes": 5, "replies": [] }, { "id": 115515, "author_name": "Ivan Gorshkov", "author_url": "", "post_date": "2016-04-19T01:02:19.927000", "content": "Great job! Can you guys please share your strategies for looking for data patterns, visualizing, etc in more details
\n\n:)
", "votes": 5, "replies": [] }, { "id": 115579, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2016-04-19T07:35:08.460000", "content": "[quote=beluga;115576]
\n\nCongratulations Guys. Impressive Feature Engineering and well deserved first place!\nThis competition was a tough nut to crack.
\n\n[/quote]
\n\nNice (late) LB surge by you too!
\n\nTo dexter Lab:
\n\nCongrats . Awesome performance and great insights!
\n\nInterestingly, we never figured out many of these things you mentioned (although we had various speculations that may have come close) and we used kind of brutal force transformations of the data to try and minimize the gap with you guys.
", "votes": 6, "replies": [] }, { "id": 115791, "author_name": "Keiku", "author_url": "", "post_date": "2016-04-20T05:15:17.820000", "content": "@raddar Congratulations. I am glad that my blog post was helpful to you. I am looking forward to your blog post.
", "votes": 3, "replies": [] }, { "id": 115663, "author_name": "Naokazu Mizuta", "author_url": "", "post_date": "2016-04-19T14:50:12.417000", "content": "[quote=dkyang;115657]
\n\nIt's very interesting to see that good results come from different approaches. I'm curious that how do you find var114 is important?\n[quote=Naokazu Mizuta;115656]
\n\nCongrats to the winners!
\n\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.
\n\n@raddar Do you have any ideas or make assumptions of what v114 means?
\n\n[/quote]
\n\n[/quote]
\n\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.
", "votes": 3, "replies": [] }, { "id": 115549, "author_name": "Alexander Pushkin", "author_url": "", "post_date": "2016-04-19T04:07:42.887000", "content": "I had a very haphazard approach consisting mostly of ensembling xgboost and random forests. I was too lazy to dig into the data. The public scripts helped me with fine-tuning parameters. And then I came up with some (ridiculously ambitious) conjectures about how to combine two highly correlated solutions. (These often didn't produce any better scores.)
\n\nFor the benefit of my fellow beginners:
\n\nTypically in ensembling we want a lot of uncorrelated/independent solutions. I think the reason an ensemble (which is typically just an arithmetic mean) performs better than individuals is essentially two-fold:
\n\nI still can't figure out how you guys got the numbers from the data. I tried figuring it out even knowing. I can come close, but not calculate the number to convert to whole numbers for v40 and v50 without "knowing" it. I looked at the deltas of each column and that gets you close, but the noise in the data really screws up the last few decimal points.
\n\nActually I take that back, v50 is fairly easy to figure out. v40 seems to be the hard case. Did you guys use all the data to figure out the average delta (basically taking bins of all the data, number of observations and smooth it out?) I always seem to be ending up at around 0.000724289 which seems very close, buy does not even come close to giving whole numbers.
", "votes": 3, "replies": [] }, { "id": 115584, "author_name": "Inoddy", "author_url": "", "post_date": "2016-04-19T07:51:37.993000", "content": "[quote=raddar;115563]
\n\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)
\n\n[/quote]
\n\nPeople may ask why does, say, scaling features them rounding them improve the logloss? Surely rounding throws away information! I did a PhD thesis on this a few years ago so I'll add my two cents here.
\n\nRegression and classification work by estimating E[y | x] and it is assumed that x is measured without error.
\n\nBut in this case, apparently, the x values had some random error added to them. What difference does this make? Surely, one intuitively thinks, some random error in x just results in random error in y.
\n\nWhen x has an error the estimation of E[y | x] can be biased. Have a look at https://en.wikipedia.org/wiki/Errors-in-variables_models. So any algorithm such as xgboost will produce biased estimates of the target.
\n\nThe winning solution appears to have sorted out the cause of the x error and actually removed it by rounding. Hence the extra classification accuracy. Reducing error in x has reduced the estimation error in y.
", "votes": 4, "replies": [] }, { "id": 115565, "author_name": "raddar", "author_url": "", "post_date": "2016-04-19T06:30:21.243000", "content": "By the way, my feature exploration ended up with using Excel, so cannot provide any good R code. But something like plotting variable ecdf functions of target and non target levels in a same plot is a good thing to have :)
", "votes": 4, "replies": [] }, { "id": 115537, "author_name": "Andy White", "author_url": "", "post_date": "2016-04-19T02:09:58.820000", "content": "I really wasn't expecting to learn any secrets about the data itself. Now I feel like I got to the end of The Usual Suspects and I need to watch it all again! Congratulations
", "votes": 4, "replies": [] }, { "id": 115528, "author_name": "Thiago Brito", "author_url": "", "post_date": "2016-04-19T01:32:06.243000", "content": "Wow! Congratulations for the winners! What a really nice post!!
\n\nI have two questions, did you use any tecnique in order to identify this patterns ? How did you get there?
\n\nAnother thing that I realize is: was this anonimization good for Bnp Paribas? I think that if they put this features information, they could have better models (of course you didn't need to spend so many hours trying to identify what every feature is).
\n\nWhat is your opinion about this?
\n\nCongrats, you did a really impressive job!!!!
", "votes": 4, "replies": [] }, { "id": 117785, "author_name": "happycube", "author_url": "", "post_date": "2016-04-30T22:03:30.543000", "content": "Here's an interesting bit from the thank-you post which puts a different spin on this thread:
\n\nOne interesting fact is that, although these analyses did not reveal the true meaning of the variables, it has discovered some really interesting business patterns which are actually not information from observations in the future. This insight, which is rightfully reflected in the leaderboard, is a demonstration of the efficiency of some feature engineering methodologies.
", "votes": 1, "replies": [] }, { "id": 116157, "author_name": "Davut Polat", "author_url": "", "post_date": "2016-04-22T09:13:08.767000", "content": "[quote=Manideep;116155]
\n\nHi, congratulations on winning the competition and thanks a lot for sharing the solution.
\n\nIn one of the previous posts Davut Polat has mentioned that you guys have created 7K features and used all of them in the model. May I know the logic behind creating so many features and the kind of features you created. And also in general, it would be great if you guys can throw some light on when and howto go for feature expansion and if possible the steps to tackle such a huge data set. Thank you.
\n\n[/quote]
\n\nit was mostly 1-hot encode for categoricals that is why we got many features, we removed half of features then, lag/lead features which raddar mentioned and 1-hot encoded features
", "votes": 1, "replies": [] }, { "id": 115978, "author_name": "", "author_url": "", "post_date": "2016-04-21T06:12:41.467000", "content": "[quote=raddar;115612]
\n\n~350 features.
\n\n[/quote]\nCongratulations and thanks a lot the great post!
\n\nWould you please share more details about these features? Also, would you please also explain how lead(target) and lag(target) are used? Are they used as features?
", "votes": 1, "replies": [] }, { "id": 115634, "author_name": "Giba", "author_url": "", "post_date": "2016-04-19T12:51:21.447000", "content": "Congratulations Dexter's Lab! That is a fantastic finding. Well deserved victory! Our approach was basically brute force attack. I was just wondering the competition scores if only those features names are not anonymized...
", "votes": 1, "replies": [] }, { "id": 115630, "author_name": "Humpty Dumpty", "author_url": "", "post_date": "2016-04-19T12:37:08.910000", "content": "Cryptanalysis
", "votes": 1, "replies": [] }, { "id": 115613, "author_name": "Pietro Marinelli", "author_url": "", "post_date": "2016-04-19T10:06:43.437000", "content": "Grazie mille!! \n... and most of all thanks for sharing your ideas ... it's really great to finish a competition and learn something more ...
", "votes": 1, "replies": [] }, { "id": 115580, "author_name": "raddar", "author_url": "", "post_date": "2016-04-19T07:38:05.607000", "content": "We figured, that all other teams just did brutal approximation on our findings, probably using many models with v22;v40 and v22;v50 interactions:)
", "votes": 1, "replies": [] }, { "id": 115576, "author_name": "beluga", "author_url": "", "post_date": "2016-04-19T07:28:29.607000", "content": "Congratulations Guys. Impressive Feature Engineering and well deserved first place!\nThis competition was a tough nut to crack.
", "votes": 1, "replies": [] }, { "id": 115561, "author_name": "Andrey Gavrilenko", "author_url": "", "post_date": "2016-04-19T06:01:48.243000", "content": "Congratulations, that is awesome! Thanks for sharing this with all of us!
", "votes": 1, "replies": [] }, { "id": 115556, "author_name": "Gert", "author_url": "", "post_date": "2016-04-19T05:06:28.563000", "content": "Very well done Dexter's lab, congratulations, also to other winners and top players. It was a tough competition, but fun!
", "votes": 1, "replies": [] }, { "id": 115535, "author_name": "Humberto Brandão, Ph.D.", "author_url": "", "post_date": "2016-04-19T02:02:31.687000", "content": "Congrats guys!!! Impressive...
", "votes": 1, "replies": [] }, { "id": 115526, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:27:22.723000", "content": "", "votes": 1, "replies": [] }, { "id": 115514, "author_name": "", "author_url": "", "post_date": "2016-04-19T00:55:14.520000", "content": "", "votes": 1, "replies": [] }, { "id": 116470, "author_name": "", "author_url": "", "post_date": "2016-04-24T17:08:58.007000", "content": "", "votes": 2, "replies": [] }, { "id": 115996, "author_name": "", "author_url": "", "post_date": "2016-04-21T09:06:40.390000", "content": "", "votes": 2, "replies": [] }, { "id": 115661, "author_name": "", "author_url": "", "post_date": "2016-04-19T14:47:21.920000", "content": "", "votes": 2, "replies": [] }, { "id": 115612, "author_name": "", "author_url": "", "post_date": "2016-04-19T10:04:29.180000", "content": "", "votes": 2, "replies": [] }, { "id": 115554, "author_name": "", "author_url": "", "post_date": "2016-04-19T04:44:55.040000", "content": "", "votes": 2, "replies": [] }, { "id": 115536, "author_name": "", "author_url": "", "post_date": "2016-04-19T02:08:59.840000", "content": "", "votes": 2, "replies": [] }, { "id": 115533, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:46:44.387000", "content": "", "votes": 2, "replies": [] }, { "id": 115521, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:15:06.663000", "content": "", "votes": 2, "replies": [] }, { "id": 115520, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:11:15.093000", "content": "", "votes": 2, "replies": [] }, { "id": 115522, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:17:29.037000", "content": "", "votes": 0, "replies": [] }, { "id": 356655, "author_name": "", "author_url": "", "post_date": "2018-07-14T06:10:55.143000", "content": "", "votes": 0, "replies": [] }, { "id": 120888, "author_name": "", "author_url": "", "post_date": "2016-05-21T12:13:48.417000", "content": "", "votes": 0, "replies": [] }, { "id": 120887, "author_name": "", "author_url": "", "post_date": "2016-05-21T12:13:04.790000", "content": "", "votes": 0, "replies": [] }, { "id": 116155, "author_name": "", "author_url": "", "post_date": "2016-04-22T09:07:39.710000", "content": "", "votes": 0, "replies": [] }, { "id": 116099, "author_name": "", "author_url": "", "post_date": "2016-04-22T01:35:18.600000", "content": "", "votes": 0, "replies": [] }, { "id": 116029, "author_name": "", "author_url": "", "post_date": "2016-04-21T15:51:49.977000", "content": "", "votes": 0, "replies": [] }, { "id": 116018, "author_name": "", "author_url": "", "post_date": "2016-04-21T13:53:23.837000", "content": "", "votes": 0, "replies": [] }, { "id": 115795, "author_name": "", "author_url": "", "post_date": "2016-04-20T06:40:23.753000", "content": "", "votes": 0, "replies": [] }, { "id": 115788, "author_name": "", "author_url": "", "post_date": "2016-04-20T05:01:51.113000", "content": "", "votes": 0, "replies": [] }, { "id": 115785, "author_name": "", "author_url": "", "post_date": "2016-04-20T04:02:01.823000", "content": "", "votes": 0, "replies": [] }, { "id": 115772, "author_name": "", "author_url": "", "post_date": "2016-04-20T02:25:56.347000", "content": "", "votes": 0, "replies": [] }, { "id": 115685, "author_name": "", "author_url": "", "post_date": "2016-04-19T15:58:04.210000", "content": "", "votes": 0, "replies": [] }, { "id": 115678, "author_name": "", "author_url": "", "post_date": "2016-04-19T15:37:48.957000", "content": "", "votes": 0, "replies": [] }, { "id": 115657, "author_name": "", "author_url": "", "post_date": "2016-04-19T14:37:41.643000", "content": "", "votes": 0, "replies": [] }, { "id": 115651, "author_name": "", "author_url": "", "post_date": "2016-04-19T14:07:38.377000", "content": "", "votes": 0, "replies": [] }, { "id": 115644, "author_name": "", "author_url": "", "post_date": "2016-04-19T13:47:19.230000", "content": "", "votes": 0, "replies": [] }, { "id": 115624, "author_name": "", "author_url": "", "post_date": "2016-04-19T11:52:00.343000", "content": "", "votes": 0, "replies": [] }, { "id": 115619, "author_name": "", "author_url": "", "post_date": "2016-04-19T10:57:12.520000", "content": "", "votes": 0, "replies": [] }, { "id": 115611, "author_name": "", "author_url": "", "post_date": "2016-04-19T10:02:17.453000", "content": "", "votes": 0, "replies": [] }, { "id": 115596, "author_name": "", "author_url": "", "post_date": "2016-04-19T09:14:10.417000", "content": "", "votes": 0, "replies": [] }, { "id": 115582, "author_name": "", "author_url": "", "post_date": "2016-04-19T07:44:00.567000", "content": "", "votes": 0, "replies": [] }, { "id": 115573, "author_name": "", "author_url": "", "post_date": "2016-04-19T07:20:44.463000", "content": "", "votes": 0, "replies": [] }, { "id": 115568, "author_name": "", "author_url": "", "post_date": "2016-04-19T06:58:34.050000", "content": "", "votes": 0, "replies": [] }, { "id": 115564, "author_name": "", "author_url": "", "post_date": "2016-04-19T06:26:16.977000", "content": "", "votes": 0, "replies": [] }, { "id": 115546, "author_name": "", "author_url": "", "post_date": "2016-04-19T03:36:16.310000", "content": "", "votes": 0, "replies": [] }, { "id": 115545, "author_name": "", "author_url": "", "post_date": "2016-04-19T03:33:27.603000", "content": "", "votes": 0, "replies": [] }, { "id": 115543, "author_name": "", "author_url": "", "post_date": "2016-04-19T03:23:10.790000", "content": "", "votes": 0, "replies": [] }, { "id": 115541, "author_name": "", "author_url": "", "post_date": "2016-04-19T03:14:54.317000", "content": "", "votes": 0, "replies": [] }, { "id": 115538, "author_name": "", "author_url": "", "post_date": "2016-04-19T02:35:58.787000", "content": "", "votes": 0, "replies": [] }, { "id": 115527, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:30:42.280000", "content": "", "votes": 0, "replies": [] }, { "id": 115525, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:21:30.583000", "content": "", "votes": 0, "replies": [] }, { "id": 115523, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:18:29.783000", "content": "", "votes": 0, "replies": [] }, { "id": 115519, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:10:08.073000", "content": "", "votes": 0, "replies": [] }, { "id": 115518, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:06:00.107000", "content": "", "votes": 0, "replies": [] }, { "id": 115516, "author_name": "", "author_url": "", "post_date": "2016-04-19T01:03:27.540000", "content": "", "votes": -13, "replies": [] } ], "comment_count": 76, "messages": [ { "id": 115513, "postDate": "2016-04-19T00:49:01.853Z", "content": "So glad we won this one! After having such good data findings it would have been a shame not to win :)
\n\nIt is clear that this competition was not about ensembling but feature engineering, and we are happy that we found out the major key to winning this competition early on.\nTherefore, we had lots of time to exploit our findings to the max and maintain the top position ever since. \nThe trick was very simple, and I even shared a hint to that early in the competition (without me understanding its importance at that time):\nhttps://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19240/analysis-of-duplicate-variables-correlated-variables-large-post/110095#post110095
\n\nWe figured, that data unanonymizing was important and in the end we were able to figure out some of variables' meaning:
\n\nv40 - date of observation
\n\nv40-v50 - contract start date
\n\nv50 - days since contract startdate (in other words, claimdate)
\n\nv10 - contract term in months
\n\nand few others for remaining days of the contract:
\n\nv12=v10*(365.25)/12-v50
\n\nv34=v10*(365.25)/12-v40
\n\nNow add this knowledge to the assumption that v22 is a customer and v56/v113 is a product type, \nand you will see very obvious patterns, that one contract may have several claims during contract lifespan (R: group_by(v22,v40-v50) %>% arrange(v50)):
\n\n[v22,v40-v50] [v50] [target]
\n\n[ZLS 12840] 197 1
\n\n[ZLS 12840] 962 1
\n\n[ZLS 12840] 1437 1
\n\n[ZLS 12840] 1498 NA
\n\n[ZLS 12840] 1501 1
\n\n[ZLS 12840] 1726 1
\n\n[ZLS 12840] 1788 NA
\n\n[ZLS 12840] 1882 NA
\n\n[ZLS 12840] 2418 1
\n\n[ZLS 12840] 3352 NA
\n\n[ZLS 12840] 3370 NA
\n\n[ZLS 12840] 3909 NA
\n\nSo we could drop the categorical {v22} level i.i.d. assumption, and move to panel data structured categorical levels such as: \n{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc;
\n\nJust looking at the data this way, it was obvious that target value is very persistent for each {v22,v40-v50} level - \ni.e. if a time series starts with target=1, it usually ends with target=1; if it starts with target=0 it often ends with target=0 too; claims which target shifts 0->1 are quite rare (1->0 only few cases);\nSo in the end it all resulted to correctly imputing target sequence for each level - which we done using lag(target) and lead(target) variables;\nTo our surprise, these variables were not overfitting LB, and in the end we made so many lag/lead variables that it was possible to drop v22 column, \nand didn't even use v22 mean target techniques disccussed in the forums, which probably many top teams did anyway.
\n\nOur best single xgboost model achieved 0.42347 public LB (0.42193 private), and the model takes itself only about half an hour to train on 12-thread cpu. \nTo seal the deal, in last 2 weeks we experimented with other techniques and build few stack models with tens of different methodology models, such as nnets, \nlinear SVM's, elastic nets, xgboosts with count:poisson, rank:pairwise, etc.\nI personally enjoyed working with regularized greedy forests, which were almost on par with xgboost.
\n\nThe role of ensembling may not as important as other competition though we have tried several diverse models.
\n\n-models with tsne feature from continous variables
\n\n-models by levels of certain varaible (for example,var5)
\n\n-knn models on likehihood encoding varibles
\n\nTo sum up: \nAs most of you, we were stuck at 0.45x for a long time and it took 3-4 weeks of dedicated time of looking and exploring the data in Excel to end up with panel time-series data, which was the key to success.\nHaving such knowledge about the data could have gotten you to top10 without too much effort.\nIn the end we created a useless model for Bnp, as our lead(target) variables use information from the future:)
\n\nAnd for guys who wants to succeed -
\n\na) when starting a new competition, create simple xgboost model and use feature.importance to get a nice start to discovering important features - then stop making models and work with the data.
\n\nb) do not underestimate the power of knowing what data you are working with
\n\nc) dedicate some time for data exploration and try to understand how people visualize the data in the forums
\n\nd) look for data patterns, especially if it has many categorical variables
\n\ne) spend some time reading forums of past competitions, especially winning materials
\n\nf) keep eyes on overfitting
\n\nP.S. special thanks to this blog post http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/ \n"I found that the out-of-fold CV predictions of categorized lag features were very important. As far as I saw in the forum, many of the participants may have not created these features."\nThis shook our beliefs and assumptions about data being i.i.d in {v22} levels, and it took us only 1 day to utilize this and claim the top1 rank for the rest of the copmetition:)
\n\nP.S.S huge thanks to Laurae for the input to data exploration. I believe it made this competition more interesting for most of us.
\n\nP.S.S. team name came up when I decided to join forces with Davut, and the cartoon was playing in the background- for both of us it was our one of favorite childhood cartoons :)
", "rawMarkdown": "So glad we won this one! After having such good data findings it would have been a shame not to win :)\r\n\r\nIt is clear that this competition was not about ensembling but feature engineering, and we are happy that we found out the major key to winning this competition early on.\r\nTherefore, we had lots of time to exploit our findings to the max and maintain the top position ever since. \r\nThe trick was very simple, and I even shared a hint to that early in the competition (without me understanding its importance at that time):\r\nhttps://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19240/analysis-of-duplicate-variables-correlated-variables-large-post/110095#post110095\r\n\r\nWe figured, that data unanonymizing was important and in the end we were able to figure out some of variables' meaning:\r\n\r\nv40 - date of observation\r\n\r\nv40-v50 - contract start date\r\n\r\nv50 - days since contract startdate (in other words, claimdate)\r\n\r\nv10 - contract term in months\r\n\r\nand few others for remaining days of the contract:\r\n\r\nv12=v10*(365.25)/12-v50 \r\n\r\nv34=v10*(365.25)/12-v40\r\n\r\nNow add this knowledge to the assumption that v22 is a customer and v56/v113 is a product type, \r\nand you will see very obvious patterns, that one contract may have several claims during contract lifespan (R: group_by(v22,v40-v50) %>% arrange(v50)):\r\n\r\n[v22,v40-v50] [v50] [target]\r\n\r\n[ZLS 12840] 197 1\r\n\r\n[ZLS 12840] 962 1\r\n\r\n[ZLS 12840] 1437 1\r\n\r\n[ZLS 12840] 1498 NA\r\n\r\n[ZLS 12840] 1501 1\r\n\r\n[ZLS 12840] 1726 1\r\n\r\n[ZLS 12840] 1788 NA\r\n\r\n[ZLS 12840] 1882 NA\r\n\r\n[ZLS 12840] 2418 1\r\n\r\n[ZLS 12840] 3352 NA\r\n\r\n[ZLS 12840] 3370 NA\r\n\r\n[ZLS 12840] 3909 NA\r\n\r\nSo we could drop the categorical {v22} level i.i.d. assumption, and move to panel data structured categorical levels such as: \r\n{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc; \r\n\r\nJust looking at the data this way, it was obvious that target value is very persistent for each {v22,v40-v50} level - \r\ni.e. if a time series starts with target=1, it usually ends with target=1; if it starts with target=0 it often ends with target=0 too; claims which target shifts 0->1 are quite rare (1->0 only few cases);\r\nSo in the end it all resulted to correctly imputing target sequence for each level - which we done using lag(target) and lead(target) variables;\r\nTo our surprise, these variables were not overfitting LB, and in the end we made so many lag/lead variables that it was possible to drop v22 column, \r\nand didn't even use v22 mean target techniques disccussed in the forums, which probably many top teams did anyway.\r\n\r\nOur best single xgboost model achieved 0.42347 public LB (0.42193 private), and the model takes itself only about half an hour to train on 12-thread cpu. \r\nTo seal the deal, in last 2 weeks we experimented with other techniques and build few stack models with tens of different methodology models, such as nnets, \r\nlinear SVM's, elastic nets, xgboosts with count:poisson, rank:pairwise, etc.\r\nI personally enjoyed working with regularized greedy forests, which were almost on par with xgboost.\r\n\r\nThe role of ensembling may not as important as other competition though we have tried several diverse models. \r\n\r\n-models with tsne feature from continous variables\r\n\r\n-models by levels of certain varaible (for example,var5) \r\n\r\n-knn models on likehihood encoding varibles\r\n\r\n\r\n\r\nTo sum up: \r\nAs most of you, we were stuck at 0.45x for a long time and it took 3-4 weeks of dedicated time of looking and exploring the data in Excel to end up with panel time-series data, which was the key to success.\r\nHaving such knowledge about the data could have gotten you to top10 without too much effort.\r\nIn the end we created a useless model for Bnp, as our lead(target) variables use information from the future:)\r\n\r\nAnd for guys who wants to succeed - \r\n\r\na) when starting a new competition, create simple xgboost model and use feature.importance to get a nice start to discovering important features - then stop making models and work with the data.\r\n\r\nb) do not underestimate the power of knowing what data you are working with\r\n\r\nc) dedicate some time for data exploration and try to understand how people visualize the data in the forums\r\n\r\nd) look for data patterns, especially if it has many categorical variables\r\n\r\ne) spend some time reading forums of past competitions, especially winning materials\r\n\r\nf) keep eyes on overfitting\r\n\r\n\r\nP.S. special thanks to this blog post http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/ \r\n\"I found that the out-of-fold CV predictions of categorized lag features were very important. As far as I saw in the forum, many of the participants may have not created these features.\"\r\nThis shook our beliefs and assumptions about data being i.i.d in {v22} levels, and it took us only 1 day to utilize this and claim the top1 rank for the rest of the copmetition:)\r\n\r\nP.S.S huge thanks to Laurae for the input to data exploration. I believe it made this competition more interesting for most of us.\r\n\r\nP.S.S. team name came up when I decided to join forces with Davut, and the cartoon was playing in the background- for both of us it was our one of favorite childhood cartoons :)\r\n", "votes": 180 }, { "id": 115643, "postDate": "2016-04-19T13:41:26.633Z", "content": "I am impressed by the cleverness of the winning solution, but also by the incredible complexity of the 2nd solution, it should take a lot of discipline, organization and coding skills to be able to make such solution reproducible and keep the code clean etc... I would have never be able to find these solutions myself.
\n\nCan I just make some reflections (without trying to diminish the performance of the winners - just a reflection ...):
\n\n--> Is the main goal of this competition getting a useful model, I am not sure that they have got the maximum out of this competition... Is the main goal to get good PR and attract new people, knowing how to code etc... I guess they succeed!
\n\nThanks for the competition and again congratz to the winners!\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)
", "rawMarkdown": "I am impressed by the cleverness of the winning solution, but also by the incredible complexity of the 2nd solution, it should take a lot of discipline, organization and coding skills to be able to make such solution reproducible and keep the code clean etc... I would have never be able to find these solutions myself.\r\n\r\nCan I just make some reflections (without trying to diminish the performance of the winners - just a reflection ...):\r\n\r\n - How useful is it if for BNP if you anonymyze the data and the winning solution spend three weeks to reverse this work in order to get the crucial insight they need in the data... now, after the competition the data is no longer anonymous anyway...\r\n - Also the 2nd best solution (and probably many others) are very complex, but I like to think that they somehow *substitute the lack of insight in the data (because of anonymization), by complexity*. Someone working at BNP, having the dictionary of the data and insight in the business process, should be able to outperform this 2nd model with a much simpler and IT friendly model in a couple of days, or am I wrong here?\r\n - On top of this all ... the winning solution turns out to use 'lead' variables to predict the response...\r\n\r\n--> Is the main goal of this competition getting a useful model, I am not sure that they have got the maximum out of this competition... Is the main goal to get good PR and attract new people, knowing how to code etc... I guess they succeed!\r\n\r\nThanks for the competition and again congratz to the winners!\r\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)", "votes": 21 }, { "id": 115563, "postDate": "2016-04-19T06:24:47.990Z", "content": "Thank you all,
\n\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)
\n\nIt is actually quite simple. lets say you have variable X; you sort that X and calculate rank values for each X value. then take X[rank] - X[rank-1] and plot a distribution plot of that. You will see very interesting distribution of several spikes.
\n\nTo put it simply, lets say you have 0.00049, 0.00049,....0.00049,0.00098,0.00098,....0.00147,0.00147. See the pattern :) THen it just comes to simple optim function to search for median values between "jumping points" (X[rank] - X[rank-1]) distribution.
\n\nv40,v10,v50,v18,v28 were the variables, where this seperation is present. There are some more variables which are a mixture of this and gaussian density (meaning there are many integers, but also fractional numbers in that variable) - However I did not dig into it.
", "rawMarkdown": "Thank you all,\r\n\r\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)\r\n\r\nIt is actually quite simple. lets say you have variable X; you sort that X and calculate rank values for each X value. then take X[rank] - X[rank-1] and plot a distribution plot of that. You will see very interesting distribution of several spikes.\r\n\r\nTo put it simply, lets say you have 0.00049, 0.00049,....0.00049,0.00098,0.00098,....0.00147,0.00147. See the pattern :) THen it just comes to simple optim function to search for median values between \"jumping points\" (X[rank] - X[rank-1]) distribution.\r\n\r\nv40,v10,v50,v18,v28 were the variables, where this seperation is present. There are some more variables which are a mixture of this and gaussian density (meaning there are many integers, but also fractional numbers in that variable) - However I did not dig into it.\r\n\r\n", "votes": 10 }, { "id": 115534, "postDate": "2016-04-19T01:48:05.597Z", "content": "all credicts goes to raddar for discovering this :)
", "rawMarkdown": "all credicts goes to raddar for discovering this :)", "votes": 7 }, { "id": 115666, "postDate": "2016-04-19T14:58:06.660Z", "content": "[quote=ToonRoge;115643]
\n\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)
\n\n[/quote]
\n\nWith all I posted you can create many features... many! But not every feature is useful. At the end, I used only 24 features (yes, only 24) in my final model without any stacking/post-ensembling, it got 0.44493 Private LB. But some of these features needed... several hours to compute :(
\n\nI went for a type of "credit scoring" where 10 of my features were scores about the label, all coming from pure EDA and visualizations. Other 14 features were T-SNE and variable interactions. Sometimes I would get strange results if I added a new feature.
\n\nxgboost is pretty absurd in performance when it comes to take the most from engineered variables.
\n\nExample of strange results I could get: sometimes it's either everything or nothing in-between. V is one of my engineered features. It could get a near perfect accuracy on some buckets where target=0, but was still failing very hard at separating 0 and 1 when they were together.
\n\n
[quote=ToonRoge;115643]
\n\nSomeone working at BNP, having the dictionary of the data and insight in the business process, should be able to outperform this 2nd model with a much simpler and IT friendly model in a couple of days, or am I wrong here?
\n\n[/quote]
\n\nNot sure. I think 0.44 is a reasonable target if you have the full insight on the data. Check Jack's (Japan) model here, not only it is fast but it gives a very good performance in this competition. But that's still a "black box" model.
\n\nYou have two important paths in a business when dealing with such machine learning project:
\n\nThe latter is mostly preferred. But there are cases where the first suggestion is better. Here if they wanted the highest transparency (to tell the customer the reason whether the claim is fulfilled to be accelerated or not), they would go the second way. What they will also look at is the false positive rate :)
\n\n[quote=raddar;115661]
\n\nvar114 has strong correlation with var40 (which is observation date). so var114 has to do something with dates.
\n\nOne of my hypothesis is that var114 is a datetime and var40 is a date format. the time value caused the correlation not be linearly perfect.
\n\n[/quote]
\n\nFor v40 (I'm doing the other way, there is something very funny at the end), look at v13 / v19 / v27 / v35 / v45 / v50 / v57 / v114. I suspect there is something simpler than this 8-way relation. I get 0.999876 R^2 with 0.033133 RMSE. Did not cross-validate though.
\n\nedit: RMSE 0.037213 using only v50 and v114 with appropriate cuts on v114.
\n\n
I got the following plane of v50/v40/v114, which seems to look curved.
\n\n
Without v19, I get RMSE=0.1701. I suspect compounding noise makes the residuals explode.
\n\nHere is a funny picture using v50 and v114 :)
\n\n
edit: instructions below to reproduce not working, I have no idea how I got that picture using v50/v114 because I cannot reproduce it :(
\n\nTo get that picture, you need to do the following linear separations in v50 and v114:
\n\nYou should have 8 columns.
\n\nOr use this formula if you are keen at mathematics programming:
\n\n
It's always inspiring seeing people smash the LB. Nicely done!
", "rawMarkdown": "It's always inspiring seeing people smash the LB. Nicely done!", "votes": 8 }, { "id": 115656, "postDate": "2016-04-19T14:29:05.863Z", "content": "Congrats to the winners!
\n\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.
\n\n@raddar Do you have any ideas or make assumptions of what v114 means?
", "rawMarkdown": "Congrats to the winners!\r\n\r\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.\r\n\r\n@raddar Do you have any ideas or make assumptions of what v114 means?\r\n", "votes": 5 }, { "id": 115578, "postDate": "2016-04-19T07:32:19.240Z", "content": "[quote=happydemiologist;115573]
\n\nCongratulations, raddar. You both really got into the data. Could you please further simplify the example for laypersons like myself. I see that variable X is sorted (?), but the median is now calculated for each point in the variable?
\n\n[/quote]
\n\nmaybe this simple script will help:
\n\nv50=sort(train$v50)
\n\nv50_diff=v50[2:length(v50)]- v50[1:(length(v50)-1)]
\n\ntable(round(v50_diff,5))
\n\nnow if you looked at the table, you will see that table values are increasing by a factor of 0.00147*x, which is strong indication of v50 being a integer variable
", "rawMarkdown": "[quote=happydemiologist;115573]\r\n\r\nCongratulations, raddar. You both really got into the data. Could you please further simplify the example for laypersons like myself. I see that variable X is sorted (?), but the median is now calculated for each point in the variable?\r\n\r\n[/quote]\r\n\r\nmaybe this simple script will help:\r\n\r\nv50=sort(train$v50)\r\n\r\nv50_diff=v50[2:length(v50)]- v50[1:(length(v50)-1)]\r\n\r\ntable(round(v50_diff,5))\r\n\r\n\r\nnow if you looked at the table, you will see that table values are increasing by a factor of 0.00147*x, which is strong indication of v50 being a integer variable\r\n", "votes": 5 }, { "id": 115572, "postDate": "2016-04-19T07:16:38.933Z", "content": "My teammate Song works in insurance, and I tackled the data from a banking perspective (I actually thought we were making a model for a bank for long time, lol). So we convinced ourselves that these were dates, because from our experience it seemed logical, that a person which claim was approved at least once would indicate a "good" customer feature. and the person who is constantly rejected (target=0) would indicate persistent fraudulent behaviour.
\n\nBy the way all the data has another interesting property I have not shared :) when you look at i.e. {v22,v40-v50,v56} level and look at all other variables you will notice that they are all constants in time :) So in the end we made a conclusion that all variables are some kind filled in application forms; if you calculated hashes on numericals, there are only 30k unique hashes, which kind of support our hypothesis. That is why people saw some interesting behaviour, when removing many variables would help their score:)
", "rawMarkdown": "My teammate Song works in insurance, and I tackled the data from a banking perspective (I actually thought we were making a model for a bank for long time, lol). So we convinced ourselves that these were dates, because from our experience it seemed logical, that a person which claim was approved at least once would indicate a \"good\" customer feature. and the person who is constantly rejected (target=0) would indicate persistent fraudulent behaviour.\r\n\r\nBy the way all the data has another interesting property I have not shared :) when you look at i.e. {v22,v40-v50,v56} level and look at all other variables you will notice that they are all constants in time :) So in the end we made a conclusion that all variables are some kind filled in application forms; if you calculated hashes on numericals, there are only 30k unique hashes, which kind of support our hypothesis. That is why people saw some interesting behaviour, when removing many variables would help their score:)\r\n\r\n\r\n\r\n\r\n\r\n", "votes": 5 }, { "id": 115515, "postDate": "2016-04-19T01:02:19.927Z", "content": "Great job! Can you guys please share your strategies for looking for data patterns, visualizing, etc in more details
\n\n:)
", "rawMarkdown": "Great job! Can you guys please share your strategies for looking for data patterns, visualizing, etc in more details\r\n\r\n:)", "votes": 5 }, { "id": 115579, "postDate": "2016-04-19T07:35:08.460Z", "content": "[quote=beluga;115576]
\n\nCongratulations Guys. Impressive Feature Engineering and well deserved first place!\nThis competition was a tough nut to crack.
\n\n[/quote]
\n\nNice (late) LB surge by you too!
\n\nTo dexter Lab:
\n\nCongrats . Awesome performance and great insights!
\n\nInterestingly, we never figured out many of these things you mentioned (although we had various speculations that may have come close) and we used kind of brutal force transformations of the data to try and minimize the gap with you guys.
", "rawMarkdown": "[quote=beluga;115576]\r\n\r\nCongratulations Guys. Impressive Feature Engineering and well deserved first place!\r\nThis competition was a tough nut to crack. \r\n\r\n[/quote]\r\n\r\nNice (late) LB surge by you too!\r\n\r\nTo dexter Lab:\r\n\r\nCongrats . Awesome performance and great insights!\r\n\r\nInterestingly, we never figured out many of these things you mentioned (although we had various speculations that may have come close) and we used kind of brutal force transformations of the data to try and minimize the gap with you guys. \r\n\r\n", "votes": 6 }, { "id": 115791, "postDate": "2016-04-20T05:15:17.820Z", "content": "@raddar Congratulations. I am glad that my blog post was helpful to you. I am looking forward to your blog post.
", "rawMarkdown": "@raddar Congratulations. I am glad that my blog post was helpful to you. I am looking forward to your blog post.", "votes": 3 }, { "id": 115663, "postDate": "2016-04-19T14:50:12.417Z", "content": "[quote=dkyang;115657]
\n\nIt's very interesting to see that good results come from different approaches. I'm curious that how do you find var114 is important?\n[quote=Naokazu Mizuta;115656]
\n\nCongrats to the winners!
\n\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.
\n\n@raddar Do you have any ideas or make assumptions of what v114 means?
\n\n[/quote]
\n\n[/quote]
\n\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.
", "rawMarkdown": "[quote=dkyang;115657]\r\n\r\nIt's very interesting to see that good results come from different approaches. I'm curious that how do you find var114 is important?\r\n[quote=Naokazu Mizuta;115656]\r\n\r\nCongrats to the winners!\r\n\r\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.\r\n\r\n@raddar Do you have any ideas or make assumptions of what v114 means?\r\n\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.", "votes": 3 }, { "id": 115549, "postDate": "2016-04-19T04:07:42.887Z", "content": "I had a very haphazard approach consisting mostly of ensembling xgboost and random forests. I was too lazy to dig into the data. The public scripts helped me with fine-tuning parameters. And then I came up with some (ridiculously ambitious) conjectures about how to combine two highly correlated solutions. (These often didn't produce any better scores.)
\n\nFor the benefit of my fellow beginners:
\n\nTypically in ensembling we want a lot of uncorrelated/independent solutions. I think the reason an ensemble (which is typically just an arithmetic mean) performs better than individuals is essentially two-fold:
\n\nI still can't figure out how you guys got the numbers from the data. I tried figuring it out even knowing. I can come close, but not calculate the number to convert to whole numbers for v40 and v50 without "knowing" it. I looked at the deltas of each column and that gets you close, but the noise in the data really screws up the last few decimal points.
\n\nActually I take that back, v50 is fairly easy to figure out. v40 seems to be the hard case. Did you guys use all the data to figure out the average delta (basically taking bins of all the data, number of observations and smooth it out?) I always seem to be ending up at around 0.000724289 which seems very close, buy does not even come close to giving whole numbers.
", "rawMarkdown": "I still can't figure out how you guys got the numbers from the data. I tried figuring it out even knowing. I can come close, but not calculate the number to convert to whole numbers for v40 and v50 without \"knowing\" it. I looked at the deltas of each column and that gets you close, but the noise in the data really screws up the last few decimal points. \r\n\r\nActually I take that back, v50 is fairly easy to figure out. v40 seems to be the hard case. Did you guys use all the data to figure out the average delta (basically taking bins of all the data, number of observations and smooth it out?) I always seem to be ending up at around 0.000724289 which seems very close, buy does not even come close to giving whole numbers.", "votes": 3 }, { "id": 115584, "postDate": "2016-04-19T07:51:37.993Z", "content": "[quote=raddar;115563]
\n\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)
\n\n[/quote]
\n\nPeople may ask why does, say, scaling features them rounding them improve the logloss? Surely rounding throws away information! I did a PhD thesis on this a few years ago so I'll add my two cents here.
\n\nRegression and classification work by estimating E[y | x] and it is assumed that x is measured without error.
\n\nBut in this case, apparently, the x values had some random error added to them. What difference does this make? Surely, one intuitively thinks, some random error in x just results in random error in y.
\n\nWhen x has an error the estimation of E[y | x] can be biased. Have a look at https://en.wikipedia.org/wiki/Errors-in-variables_models. So any algorithm such as xgboost will produce biased estimates of the target.
\n\nThe winning solution appears to have sorted out the cause of the x error and actually removed it by rounding. Hence the extra classification accuracy. Reducing error in x has reduced the estimation error in y.
", "rawMarkdown": "[quote=raddar;115563]\r\n\r\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)\r\n\r\n[/quote]\r\n\r\nPeople may ask why does, say, scaling features them rounding them improve the logloss? Surely rounding throws away information! I did a PhD thesis on this a few years ago so I'll add my two cents here.\r\n\r\nRegression and classification work by estimating E[y | x] and it is assumed that x is measured without error.\r\n\r\nBut in this case, apparently, the x values had some random error added to them. What difference does this make? Surely, one intuitively thinks, some random error in x just results in random error in y.\r\n\r\nWhen x has an error the estimation of E[y | x] can be biased. Have a look at https://en.wikipedia.org/wiki/Errors-in-variables_models. So any algorithm such as xgboost will produce biased estimates of the target.\r\n\r\nThe winning solution appears to have sorted out the cause of the x error and actually removed it by rounding. Hence the extra classification accuracy. Reducing error in x has reduced the estimation error in y.\r\n\r\n", "votes": 4 }, { "id": 115565, "postDate": "2016-04-19T06:30:21.243Z", "content": "By the way, my feature exploration ended up with using Excel, so cannot provide any good R code. But something like plotting variable ecdf functions of target and non target levels in a same plot is a good thing to have :)
", "rawMarkdown": "By the way, my feature exploration ended up with using Excel, so cannot provide any good R code. But something like plotting variable ecdf functions of target and non target levels in a same plot is a good thing to have :)", "votes": 4 }, { "id": 115537, "postDate": "2016-04-19T02:09:58.820Z", "content": "I really wasn't expecting to learn any secrets about the data itself. Now I feel like I got to the end of The Usual Suspects and I need to watch it all again! Congratulations
", "rawMarkdown": "I really wasn't expecting to learn any secrets about the data itself. Now I feel like I got to the end of The Usual Suspects and I need to watch it all again! Congratulations", "votes": 4 }, { "id": 115528, "postDate": "2016-04-19T01:32:06.243Z", "content": "Wow! Congratulations for the winners! What a really nice post!!
\n\nI have two questions, did you use any tecnique in order to identify this patterns ? How did you get there?
\n\nAnother thing that I realize is: was this anonimization good for Bnp Paribas? I think that if they put this features information, they could have better models (of course you didn't need to spend so many hours trying to identify what every feature is).
\n\nWhat is your opinion about this?
\n\nCongrats, you did a really impressive job!!!!
", "rawMarkdown": "Wow! Congratulations for the winners! What a really nice post!!\r\n\r\nI have two questions, did you use any tecnique in order to identify this patterns ? How did you get there?\r\n\r\nAnother thing that I realize is: was this anonimization good for Bnp Paribas? I think that if they put this features information, they could have better models (of course you didn't need to spend so many hours trying to identify what every feature is).\r\n\r\nWhat is your opinion about this?\r\n\r\nCongrats, you did a really impressive job!!!! ", "votes": 4 }, { "id": 117785, "postDate": "2016-04-30T22:03:30.543Z", "content": "Here's an interesting bit from the thank-you post which puts a different spin on this thread:
\n\nOne interesting fact is that, although these analyses did not reveal the true meaning of the variables, it has discovered some really interesting business patterns which are actually not information from observations in the future. This insight, which is rightfully reflected in the leaderboard, is a demonstration of the efficiency of some feature engineering methodologies.
", "rawMarkdown": "Here's an interesting bit from the thank-you post which puts a different spin on this thread:\r\n\r\n*One interesting fact is that, although these analyses did not reveal the true meaning of the variables, it has discovered some really interesting business patterns which are actually not information from observations in the future. This insight, which is rightfully reflected in the leaderboard, is a demonstration of the efficiency of some feature engineering methodologies.*", "votes": 1 }, { "id": 116157, "postDate": "2016-04-22T09:13:08.767Z", "content": "[quote=Manideep;116155]
\n\nHi, congratulations on winning the competition and thanks a lot for sharing the solution.
\n\nIn one of the previous posts Davut Polat has mentioned that you guys have created 7K features and used all of them in the model. May I know the logic behind creating so many features and the kind of features you created. And also in general, it would be great if you guys can throw some light on when and howto go for feature expansion and if possible the steps to tackle such a huge data set. Thank you.
\n\n[/quote]
\n\nit was mostly 1-hot encode for categoricals that is why we got many features, we removed half of features then, lag/lead features which raddar mentioned and 1-hot encoded features
", "rawMarkdown": "[quote=Manideep;116155]\r\n\r\nHi, congratulations on winning the competition and thanks a lot for sharing the solution.\r\n\r\nIn one of the previous posts Davut Polat has mentioned that you guys have created 7K features and used all of them in the model. May I know the logic behind creating so many features and the kind of features you created. And also in general, it would be great if you guys can throw some light on when and howto go for feature expansion and if possible the steps to tackle such a huge data set. Thank you.\r\n\r\n\r\n\r\n[/quote]\r\n\r\nit was mostly 1-hot encode for categoricals that is why we got many features, we removed half of features then, lag/lead features which raddar mentioned and 1-hot encoded features", "votes": 1 }, { "id": 115978, "postDate": "2016-04-21T06:12:41.467Z", "content": "[quote=raddar;115612]
\n\n~350 features.
\n\n[/quote]\nCongratulations and thanks a lot the great post!
\n\nWould you please share more details about these features? Also, would you please also explain how lead(target) and lag(target) are used? Are they used as features?
", "rawMarkdown": "[quote=raddar;115612]\r\n\r\n~350 features.\r\n\r\n[/quote]\r\nCongratulations and thanks a lot the great post!\r\n\r\nWould you please share more details about these features? Also, would you please also explain how lead(target) and lag(target) are used? Are they used as features?", "votes": 1 }, { "id": 115634, "postDate": "2016-04-19T12:51:21.447Z", "content": "Congratulations Dexter's Lab! That is a fantastic finding. Well deserved victory! Our approach was basically brute force attack. I was just wondering the competition scores if only those features names are not anonymized...
", "rawMarkdown": "Congratulations Dexter's Lab! That is a fantastic finding. Well deserved victory! Our approach was basically brute force attack. I was just wondering the competition scores if only those features names are not anonymized...", "votes": 1 }, { "id": 115630, "postDate": "2016-04-19T12:37:08.910Z", "content": "Cryptanalysis
", "rawMarkdown": "Cryptanalysis", "votes": 1 }, { "id": 115613, "postDate": "2016-04-19T10:06:43.437Z", "content": "Grazie mille!! \n... and most of all thanks for sharing your ideas ... it's really great to finish a competition and learn something more ...
", "rawMarkdown": "Grazie mille!! \r\n... and most of all thanks for sharing your ideas ... it's really great to finish a competition and learn something more ...", "votes": 1 }, { "id": 115580, "postDate": "2016-04-19T07:38:05.607Z", "content": "We figured, that all other teams just did brutal approximation on our findings, probably using many models with v22;v40 and v22;v50 interactions:)
", "rawMarkdown": "We figured, that all other teams just did brutal approximation on our findings, probably using many models with v22;v40 and v22;v50 interactions:)", "votes": 1 }, { "id": 115576, "postDate": "2016-04-19T07:28:29.607Z", "content": "Congratulations Guys. Impressive Feature Engineering and well deserved first place!\nThis competition was a tough nut to crack.
", "rawMarkdown": "Congratulations Guys. Impressive Feature Engineering and well deserved first place!\r\nThis competition was a tough nut to crack. \r\n", "votes": 1 }, { "id": 115561, "postDate": "2016-04-19T06:01:48.243Z", "content": "Congratulations, that is awesome! Thanks for sharing this with all of us!
", "rawMarkdown": "Congratulations, that is awesome! Thanks for sharing this with all of us!", "votes": 1 }, { "id": 115556, "postDate": "2016-04-19T05:06:28.563Z", "content": "Very well done Dexter's lab, congratulations, also to other winners and top players. It was a tough competition, but fun!
", "rawMarkdown": "Very well done Dexter's lab, congratulations, also to other winners and top players. It was a tough competition, but fun!", "votes": 1 }, { "id": 115535, "postDate": "2016-04-19T02:02:31.687Z", "content": "Congrats guys!!! Impressive...
", "rawMarkdown": "Congrats guys!!! Impressive...\r\n\r\n", "votes": 1 }, { "id": 115526, "postDate": "2016-04-19T01:27:22.723Z", "content": "[quote=idle_speculation;115525]
\n\nHats off to you guys. I'm still not sure what any of this data means.
\n\n[/quote]\nBut your team almost got the same LB value~!! Congratulation~How did you achieve that ?
", "rawMarkdown": "[quote=idle_speculation;115525]\r\n\r\nHats off to you guys. I'm still not sure what any of this data means.\r\n\r\n[/quote]\r\nBut your team almost got the same LB value~!! Congratulation~How did you achieve that ?", "votes": 1 }, { "id": 115514, "postDate": "2016-04-19T00:55:14.520Z", "content": "chapeau, congratz & well deserved. I'm impressed.
", "rawMarkdown": "chapeau, congratz & well deserved. I'm impressed.", "votes": 1 }, { "id": 116470, "postDate": "2016-04-24T17:08:58.007Z", "content": "Congratulations! Very insightful discovery!\nCould you please explain the meaning of "lag(target) and lead(target)" ?
\n\nMany thanks :)
", "rawMarkdown": "Congratulations! Very insightful discovery!\r\nCould you please explain the meaning of \"lag(target) and lead(target)\" ?\r\n\r\nMany thanks :)", "votes": 2 }, { "id": 115996, "postDate": "2016-04-21T09:06:40.390Z", "content": "Congratulations to winners...Hope all top10 master will share their ideas....Thanks.
", "rawMarkdown": "Congratulations to winners...Hope all top10 master will share their ideas....Thanks.", "votes": 2 }, { "id": 115661, "postDate": "2016-04-19T14:47:21.920Z", "content": "var114 has strong correlation with var40 (which is observation date). so var114 has to do something with dates.
\n\nOne of my hypothesis is that var114 is a datetime and var40 is a date format. the time value caused the correlation not be linearly perfect.
", "rawMarkdown": "var114 has strong correlation with var40 (which is observation date). so var114 has to do something with dates.\r\n\r\nOne of my hypothesis is that var114 is a datetime and var40 is a date format. the time value caused the correlation not be linearly perfect. ", "votes": 2 }, { "id": 115612, "postDate": "2016-04-19T10:04:29.180Z", "content": "~350 features.
", "rawMarkdown": "~350 features.", "votes": 2 }, { "id": 115554, "postDate": "2016-04-19T04:44:55.040Z", "content": "Congratulations!. By any chance is it possible for you guys to share your approach/thought process for data exploration in a form of tutorial as learning something given a practical use case is very effective. Like MadScientist I also am not able to figure out how you guys got the numbers from the data. I hope I am not asking too much. Thanks again for sharing you solution.
", "rawMarkdown": "Congratulations!. By any chance is it possible for you guys to share your approach/thought process for data exploration in a form of tutorial as learning something given a practical use case is very effective. Like **MadScientist** I also am not able to figure out how you guys got the numbers from the data. I hope I am not asking too much. Thanks again for sharing you solution.", "votes": 2 }, { "id": 115536, "postDate": "2016-04-19T02:08:59.840Z", "content": "Great job - there's lots to learn from the master kagglers :-)
", "rawMarkdown": "Great job - there's lots to learn from the master kagglers :-) ", "votes": 2 }, { "id": 115533, "postDate": "2016-04-19T01:46:44.387Z", "content": "Congratulations! I realized how important data exploration is.
\n\nI couldn't find out the meaning of features, and maybe that's the reason of limitation of my LB score...
", "rawMarkdown": "Congratulations! I realized how important data exploration is.\r\n\r\nI couldn't find out the meaning of features, and maybe that's the reason of limitation of my LB score...", "votes": 2 }, { "id": 115521, "postDate": "2016-04-19T01:15:06.663Z", "content": "Congratulations to Dexter's Lab team members!
\n\n@raddar, it's really a wonderful data exploration. We tried to find the secret of the "golden features" for months with no luck!
\n\nAny chance for a more detail R code for the exploration?
\n\nThank you.
", "rawMarkdown": "Congratulations to Dexter's Lab team members! \r\n\r\n@raddar, it's really a wonderful data exploration. We tried to find the secret of the \"golden features\" for months with no luck! \r\n\r\nAny chance for a more detail R code for the exploration? \r\n\r\nThank you. ", "votes": 2 }, { "id": 115520, "postDate": "2016-04-19T01:11:15.093Z", "content": "Congrats to the winners and thanks for disseminating tips along the competition.
\n\nSpecial thanks to Professor Laurae for all his comments ! Please continue with your long posts: they were amazing.
", "rawMarkdown": "Congrats to the winners and thanks for disseminating tips along the competition.\r\n\r\nSpecial thanks to Professor Laurae for all his comments ! Please continue with your long posts: they were amazing.", "votes": 2 }, { "id": 115522, "postDate": "2016-04-19T01:17:29.037Z", "content": "[quote=KarlLiu;115516]
\n\nI forgot to select my final two submissions. Does anyone know if Kaggle will use the most recent two submissions to calculate my final result? Thank you.
\n\n[/quote]
\n\nNot the most recent, it will select the two highest private LB scoring by default
", "rawMarkdown": "[quote=KarlLiu;115516]\r\n\r\nI forgot to select my final two submissions. Does anyone know if Kaggle will use the most recent two submissions to calculate my final result? Thank you. \r\n\r\n[/quote]\r\n\r\nNot the most recent, it will select the two highest private LB scoring by default" }, { "id": 356655, "postDate": "2018-07-14T06:10:55.143Z", "content": "What is the accuracy of your model?
", "rawMarkdown": "What is the accuracy of your model?" }, { "id": 120888, "postDate": "2016-05-21T12:13:48.417Z", "content": "Can someone please explain me the meaning of lag and lead features in this competition's context?
", "rawMarkdown": "Can someone please explain me the meaning of lag and lead features in this competition's context?" }, { "id": 120887, "postDate": "2016-05-21T12:13:04.790Z", "content": "[quote=raddar;115513]
\n\nSo glad we won this one! After having such good data findings it would have been a shame not to win :)
\n\nIt is clear that this competition was not about ensembling but feature engineering, and we are happy that we found out the major key to winning this competition early on.\nTherefore, we had lots of time to exploit our findings to the max and maintain the top position ever since. \nThe trick was very simple, and I even shared a hint to that early in the competition (without me understanding its importance at that time):\nhttps://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19240/analysis-of-duplicate-variables-correlated-variables-large-post/110095#post110095
\n\nWe figured, that data unanonymizing was important and in the end we were able to figure out some of variables' meaning:
\n\nv40 - date of observation
\n\nv40-v50 - contract start date
\n\nv50 - days since contract startdate (in other words, claimdate)
\n\nv10 - contract term in months
\n\nand few others for remaining days of the contract:
\n\nv12=v10*(365.25)/12-v50
\n\nv34=v10*(365.25)/12-v40
\n\nNow add this knowledge to the assumption that v22 is a customer and v56/v113 is a product type, \nand you will see very obvious patterns, that one contract may have several claims during contract lifespan (R: group_by(v22,v40-v50) %>% arrange(v50)):
\n\n[v22,v40-v50] [v50] [target]
\n\n[ZLS 12840] 197 1
\n\n[ZLS 12840] 962 1
\n\n[ZLS 12840] 1437 1
\n\n[ZLS 12840] 1498 NA
\n\n[ZLS 12840] 1501 1
\n\n[ZLS 12840] 1726 1
\n\n[ZLS 12840] 1788 NA
\n\n[ZLS 12840] 1882 NA
\n\n[ZLS 12840] 2418 1
\n\n[ZLS 12840] 3352 NA
\n\n[ZLS 12840] 3370 NA
\n\n[ZLS 12840] 3909 NA
\n\nSo we could drop the categorical {v22} level i.i.d. assumption, and move to panel data structured categorical levels such as: \n{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc;
\n\nJust looking at the data this way, it was obvious that target value is very persistent for each {v22,v40-v50} level - \ni.e. if a time series starts with target=1, it usually ends with target=1; if it starts with target=0 it often ends with target=0 too; claims which target shifts 0->1 are quite rare (1->0 only few cases);\nSo in the end it all resulted to correctly imputing target sequence for each level - which we done using lag(target) and lead(target) variables;\nTo our surprise, these variables were not overfitting LB, and in the end we made so many lag/lead variables that it was possible to drop v22 column, \nand didn't even use v22 mean target techniques disccussed in the forums, which probably many top teams did anyway.
\n\nOur best single xgboost model achieved 0.42347 public LB (0.42193 private), and the model takes itself only about half an hour to train on 12-thread cpu. \nTo seal the deal, in last 2 weeks we experimented with other techniques and build few stack models with tens of different methodology models, such as nnets, \nlinear SVM's, elastic nets, xgboosts with count:poisson, rank:pairwise, etc.\nI personally enjoyed working with regularized greedy forests, which were almost on par with xgboost.
\n\nThe role of ensembling may not as important as other competition though we have tried several diverse models.
\n\n-models with tsne feature from continous variables
\n\n-models by levels of certain varaible (for example,var5)
\n\n-knn models on likehihood encoding varibles
\n\nTo sum up: \nAs most of you, we were stuck at 0.45x for a long time and it took 3-4 weeks of dedicated time of looking and exploring the data in Excel to end up with panel time-series data, which was the key to success.\nHaving such knowledge about the data could have gotten you to top10 without too much effort.\nIn the end we created a useless model for Bnp, as our lead(target) variables use information from the future:)
\n\nAnd for guys who wants to succeed -
\n\na) when starting a new competition, create simple xgboost model and use feature.importance to get a nice start to discovering important features - then stop making models and work with the data.
\n\nb) do not underestimate the power of knowing what data you are working with
\n\nc) dedicate some time for data exploration and try to understand how people visualize the data in the forums
\n\nd) look for data patterns, especially if it has many categorical variables
\n\ne) spend some time reading forums of past competitions, especially winning materials
\n\nf) keep eyes on overfitting
\n\nP.S. special thanks to this blog post http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/ \n"I found that the out-of-fold CV predictions of categorized lag features were very important. As far as I saw in the forum, many of the participants may have not created these features."\nThis shook our beliefs and assumptions about data being i.i.d in {v22} levels, and it took us only 1 day to utilize this and claim the top1 rank for the rest of the copmetition:)
\n\nP.S.S huge thanks to Laurae for the input to data exploration. I believe it made this competition more interesting for most of us.
\n\nP.S.S. team name came up when I decided to join forces with Davut, and the cartoon was playing in the background- for both of us it was our one of favorite childhood cartoons :)
\n\n[/quote]
\n\nHi Raddar,
\n\nCan you please explain me what do you mean by lag(target) and lead(target) variables in this competition's context? I also did not understand the following notations, {v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc;. Can you please briefly describe what exactly are you doing here? Appreciate your help. Thanks!
", "rawMarkdown": "[quote=raddar;115513]\r\n\r\nSo glad we won this one! After having such good data findings it would have been a shame not to win :)\r\n\r\nIt is clear that this competition was not about ensembling but feature engineering, and we are happy that we found out the major key to winning this competition early on.\r\nTherefore, we had lots of time to exploit our findings to the max and maintain the top position ever since. \r\nThe trick was very simple, and I even shared a hint to that early in the competition (without me understanding its importance at that time):\r\nhttps://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19240/analysis-of-duplicate-variables-correlated-variables-large-post/110095#post110095\r\n\r\nWe figured, that data unanonymizing was important and in the end we were able to figure out some of variables' meaning:\r\n\r\nv40 - date of observation\r\n\r\nv40-v50 - contract start date\r\n\r\nv50 - days since contract startdate (in other words, claimdate)\r\n\r\nv10 - contract term in months\r\n\r\nand few others for remaining days of the contract:\r\n\r\nv12=v10*(365.25)/12-v50 \r\n\r\nv34=v10*(365.25)/12-v40\r\n\r\nNow add this knowledge to the assumption that v22 is a customer and v56/v113 is a product type, \r\nand you will see very obvious patterns, that one contract may have several claims during contract lifespan (R: group_by(v22,v40-v50) %>% arrange(v50)):\r\n\r\n[v22,v40-v50] [v50] [target]\r\n\r\n[ZLS 12840] 197 1\r\n\r\n[ZLS 12840] 962 1\r\n\r\n[ZLS 12840] 1437 1\r\n\r\n[ZLS 12840] 1498 NA\r\n\r\n[ZLS 12840] 1501 1\r\n\r\n[ZLS 12840] 1726 1\r\n\r\n[ZLS 12840] 1788 NA\r\n\r\n[ZLS 12840] 1882 NA\r\n\r\n[ZLS 12840] 2418 1\r\n\r\n[ZLS 12840] 3352 NA\r\n\r\n[ZLS 12840] 3370 NA\r\n\r\n[ZLS 12840] 3909 NA\r\n\r\nSo we could drop the categorical {v22} level i.i.d. assumption, and move to panel data structured categorical levels such as: \r\n{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc; \r\n\r\nJust looking at the data this way, it was obvious that target value is very persistent for each {v22,v40-v50} level - \r\ni.e. if a time series starts with target=1, it usually ends with target=1; if it starts with target=0 it often ends with target=0 too; claims which target shifts 0->1 are quite rare (1->0 only few cases);\r\nSo in the end it all resulted to correctly imputing target sequence for each level - which we done using lag(target) and lead(target) variables;\r\nTo our surprise, these variables were not overfitting LB, and in the end we made so many lag/lead variables that it was possible to drop v22 column, \r\nand didn't even use v22 mean target techniques disccussed in the forums, which probably many top teams did anyway.\r\n\r\nOur best single xgboost model achieved 0.42347 public LB (0.42193 private), and the model takes itself only about half an hour to train on 12-thread cpu. \r\nTo seal the deal, in last 2 weeks we experimented with other techniques and build few stack models with tens of different methodology models, such as nnets, \r\nlinear SVM's, elastic nets, xgboosts with count:poisson, rank:pairwise, etc.\r\nI personally enjoyed working with regularized greedy forests, which were almost on par with xgboost.\r\n\r\nThe role of ensembling may not as important as other competition though we have tried several diverse models. \r\n\r\n-models with tsne feature from continous variables\r\n\r\n-models by levels of certain varaible (for example,var5) \r\n\r\n-knn models on likehihood encoding varibles\r\n\r\n\r\n\r\nTo sum up: \r\nAs most of you, we were stuck at 0.45x for a long time and it took 3-4 weeks of dedicated time of looking and exploring the data in Excel to end up with panel time-series data, which was the key to success.\r\nHaving such knowledge about the data could have gotten you to top10 without too much effort.\r\nIn the end we created a useless model for Bnp, as our lead(target) variables use information from the future:)\r\n\r\nAnd for guys who wants to succeed - \r\n\r\na) when starting a new competition, create simple xgboost model and use feature.importance to get a nice start to discovering important features - then stop making models and work with the data.\r\n\r\nb) do not underestimate the power of knowing what data you are working with\r\n\r\nc) dedicate some time for data exploration and try to understand how people visualize the data in the forums\r\n\r\nd) look for data patterns, especially if it has many categorical variables\r\n\r\ne) spend some time reading forums of past competitions, especially winning materials\r\n\r\nf) keep eyes on overfitting\r\n\r\n\r\nP.S. special thanks to this blog post http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/ \r\n\"I found that the out-of-fold CV predictions of categorized lag features were very important. As far as I saw in the forum, many of the participants may have not created these features.\"\r\nThis shook our beliefs and assumptions about data being i.i.d in {v22} levels, and it took us only 1 day to utilize this and claim the top1 rank for the rest of the copmetition:)\r\n\r\nP.S.S huge thanks to Laurae for the input to data exploration. I believe it made this competition more interesting for most of us.\r\n\r\nP.S.S. team name came up when I decided to join forces with Davut, and the cartoon was playing in the background- for both of us it was our one of favorite childhood cartoons :)\r\n\r\n\r\n[/quote]\r\n\r\nHi Raddar,\r\n\r\nCan you please explain me what do you mean by lag(target) and lead(target) variables in this competition's context? I also did not understand the following notations, ***{v22,v40-v50} => sort(v50), {v22,v40-v50,v56} => sort(v50), {v22} => sort(v40), etc;.*** Can you please briefly describe what exactly are you doing here? Appreciate your help. Thanks!" }, { "id": 116155, "postDate": "2016-04-22T09:07:39.710Z", "content": "Hi, congratulations on winning the competition and thanks a lot for sharing the solution.
\n\nIn one of the previous posts Davut Polat has mentioned that you guys have created 7K features and used all of them in the model. May I know the logic behind creating so many features and the kind of features you created. And also in general, it would be great if you guys can throw some light on when and howto go for feature expansion and if possible the steps to tackle such a huge data set. Thank you.
", "rawMarkdown": "Hi, congratulations on winning the competition and thanks a lot for sharing the solution.\r\n\r\nIn one of the previous posts Davut Polat has mentioned that you guys have created 7K features and used all of them in the model. May I know the logic behind creating so many features and the kind of features you created. And also in general, it would be great if you guys can throw some light on when and howto go for feature expansion and if possible the steps to tackle such a huge data set. Thank you.\r\n\r\n" }, { "id": 116099, "postDate": "2016-04-22T01:35:18.600Z", "content": "Congratulations Dexter Team, great feature engineering. thanks for sharing winning solution.
", "rawMarkdown": "Congratulations Dexter Team, great feature engineering. thanks for sharing winning solution." }, { "id": 116029, "postDate": "2016-04-21T15:51:49.977Z", "content": "Congratulations on winning this one. You have very generously explained your approach. Good learning from this. Thanks guys.
", "rawMarkdown": "Congratulations on winning this one. You have very generously explained your approach. Good learning from this. Thanks guys. " }, { "id": 116018, "postDate": "2016-04-21T13:53:23.837Z", "content": "Ingenious exploration. Congratulations to both of you guys.
", "rawMarkdown": "Ingenious exploration. Congratulations to both of you guys." }, { "id": 115795, "postDate": "2016-04-20T06:40:23.753Z", "content": "Thanks for your reply, I learned a lot. Next time I will try to use variables with fewer missing values :)
\n\n[quote=Naokazu Mizuta;115785]
\n\nYes, mainly because a variable that has a half of missing values has less impact on the whole data. While there are lots of numeric variables other than I used, when grouped by v114, they almost move in conjunction with v1, which I used as a representative.
\n\n[/quote]
", "rawMarkdown": "Thanks for your reply, I learned a lot. Next time I will try to use variables with fewer missing values :)\r\n\r\n[quote=Naokazu Mizuta;115785]\r\n\r\nYes, mainly because a variable that has a half of missing values has less impact on the whole data. While there are lots of numeric variables other than I used, when grouped by v114, they almost move in conjunction with v1, which I used as a representative.\r\n\r\n[/quote]\r\n" }, { "id": 115788, "postDate": "2016-04-20T05:01:51.113Z", "content": "Congrats to all winners and new Masters! I learnt a lot in this competition since the focus was more on feature engineering. The kind of insights Raddar came up with were unimaginable (at least for me when the features were anonymized). I mainly end up using Extra trees and XGB and created different models and then used them in an ensemble. My best model used stacked features from some simple classifiers like k-NN and entropy based decision trees. I combined these features with original features and then created ensemble model. Ensemble weights were optimized on out-of-sample validation split.
\n\nBest,\nabhijit
", "rawMarkdown": "Congrats to all winners and new Masters! I learnt a lot in this competition since the focus was more on feature engineering. The kind of insights Raddar came up with were unimaginable (at least for me when the features were anonymized). I mainly end up using Extra trees and XGB and created different models and then used them in an ensemble. My best model used stacked features from some simple classifiers like k-NN and entropy based decision trees. I combined these features with original features and then created ensemble model. Ensemble weights were optimized on out-of-sample validation split.\r\n\r\nBest,\r\nabhijit" }, { "id": 115785, "postDate": "2016-04-20T04:02:01.823Z", "content": "[quote=dkyang;115772]
\n\nI guess why you only consider variables that have almost no missing values is that , if original variables have many missing values, the the combinations will have more missing values because of mathematical caculation. Am I right?
\n\nI also tried combinations between numeric variables, I only chose variables that have high feature importance, but it didn't work well. I explored the combinations then, and I found some features have many NaN and Inf.
\n\n[quote=Naokazu Mizuta;115663]
\n\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.
\n\n[/quote]
\n\n[/quote]
\n\nYes, mainly because a variable that has a half of missing values has less impact on the whole data. While there are lots of numeric variables other than I used, when grouped by v114, they almost move in conjunction with v1, which I used as a representative.
", "rawMarkdown": "[quote=dkyang;115772]\r\n\r\nI guess why you only consider variables that have almost no missing values is that , if original variables have many missing values, the the combinations will have more missing values because of mathematical caculation. Am I right?\r\n\r\nI also tried combinations between numeric variables, I only chose variables that have high feature importance, but it didn't work well. I explored the combinations then, and I found some features have many NaN and Inf.\r\n\r\n[quote=Naokazu Mizuta;115663]\r\n\r\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.\r\n\r\n[/quote]\r\n\r\n\r\n[/quote]\r\n\r\nYes, mainly because a variable that has a half of missing values has less impact on the whole data. While there are lots of numeric variables other than I used, when grouped by v114, they almost move in conjunction with v1, which I used as a representative." }, { "id": 115772, "postDate": "2016-04-20T02:25:56.347Z", "content": "I guess why you only consider variables that have almost no missing values is that , if original variables have many missing values, the the combinations will have more missing values because of mathematical caculation. Am I right?
\n\nI also tried combinations between numeric variables, I only chose variables that have high feature importance, but it didn't work well. I explored the combinations then, and I found some features have many NaN and Inf.
\n\n[quote=Naokazu Mizuta;115663]
\n\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.
\n\n[/quote]
", "rawMarkdown": "I guess why you only consider variables that have almost no missing values is that , if original variables have many missing values, the the combinations will have more missing values because of mathematical caculation. Am I right?\r\n\r\nI also tried combinations between numeric variables, I only chose variables that have high feature importance, but it didn't work well. I explored the combinations then, and I found some features have many NaN and Inf.\r\n\r\n[quote=Naokazu Mizuta;115663]\r\n\r\nI searched for all of combinations between numeric variables and categorical ones though I only considered those variables that have almost no missing values. So numeric variables I considered are v10, v12, v14, v21, v34, v40, v50, v114.\r\n\r\n[/quote]\r\n" }, { "id": 115685, "postDate": "2016-04-19T15:58:04.210Z", "content": "[quote=Laurae;115666]
\n\n[quote=ToonRoge;115643]
\n\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)
\n\n[/quote]
\n\nWith all I posted you can create many features... many! But not every feature is useful. At the end, I used only 24 features (yes, only 24) in my final model without any stacking/post-ensembling, it got 0.44493 Private LB. But some of these features needed... several hours to compute :(
\n\nI went for a type of "credit scoring" where 10 of my features were scores about the label, all coming from pure EDA and visualizations. Other 14 features were T-SNE and variable interactions. Sometimes I would get strange results if I added a new feature.
\n\nxgboost is pretty absurd in performance when it comes to take the most from engineered variables.
\n\nExample of strange results I could get: sometimes it's either everything or nothing in-between. V is one of my engineered features. It could get a near perfect accuracy on some buckets where target=0, but was still failing very hard at separating 0 and 1 when they were together.
\n\n
[quote=ToonRoge;115643]
\n\nSomeone working at BNP, having the dictionary of the data and insight in the business process, should be able to outperform this 2nd model with a much simpler and IT friendly model in a couple of days, or am I wrong here?
\n\n[/quote]
\n\nNot sure. I think 0.44 is a reasonable target if you have the full insight on the data. Check Jack's (Japan) model here, not only it is fast but it gives a very good performance in this competition. But that's still a "black box" model.
\n\nYou have two important paths in a business when dealing with such machine learning project:
\n\nThe latter is mostly preferred. But there are cases where the first suggestion is better.
\n\n[/quote]I saw the script of Jack. It is nice indeed, but I don't think it exploits the maximum of feature engineering you could do when you actually know the meaning of each variable. He just got an idea of some combinations of variables that work well, but these would probably be trivial for someone who has the data dictionary.
\n\nCongratz with your result, it is a very nice result as well. Especially the fact it comes from one single model.
\n\nI agree with your 2 elements, but a blackbox of 1 XGB is still a lot simpler as the 100 model-ensemble approach...
", "rawMarkdown": "[quote=Laurae;115666]\r\n\r\n[quote=ToonRoge;115643]\r\n\r\nAnd I am still trying to understand all posts of Laurae. (But the things I understand were nice, however I did not find a way to leverage it in improving my solution)\r\n\r\n[/quote]\r\n\r\nWith all I posted you can create many features... many! But not every feature is useful. At the end, I used only 24 features (yes, only 24) in my final model without any stacking/post-ensembling, it got 0.44493 Private LB. But some of these features needed... several hours to compute :(\r\n\r\nI went for a type of \"credit scoring\" where 10 of my features were scores about the label, all coming from pure EDA and visualizations. Other 14 features were T-SNE and variable interactions. Sometimes I would get strange results if I added a new feature.\r\n\r\nxgboost is pretty absurd in performance when it comes to take the most from engineered variables.\r\n\r\nExample of strange results I could get: sometimes it's either everything or nothing in-between. V is one of my engineered features. It could get a near perfect accuracy on some buckets where target=0, but was still failing very hard at separating 0 and 1 when they were together.\r\n\r\n![enter image description here][1]\r\n\r\n[quote=ToonRoge;115643]\r\n\r\nSomeone working at BNP, having the dictionary of the data and insight in the business process, should be able to outperform this 2nd model with a much simpler and IT friendly model in a couple of days, or am I wrong here?\r\n\r\n[/quote]\r\n\r\nNot sure. I think 0.44 is a reasonable target if you have the full insight on the data. Check Jack's (Japan) model [here][2], not only it is fast but it gives a very good performance in this competition. But that's still a \"black box\" model.\r\n\r\nYou have two important paths in a business when dealing with such machine learning project:\r\n\r\n* Either you aim for the best performance and you allow yourself to use a \"black box\" model, or an ensemble of them\r\n* Either you aim for the highest transparency possible and unallow yourself to use a \"black box\" model, because you need to understand exactly the separation in the data to make the final prediction\r\n\r\nThe latter is mostly preferred. But there are cases where the first suggestion is better.\r\n\r\n [1]: http://i.imgur.com/P9LoFhN.png\r\n [2]: https://www.kaggle.com/rsakata/bnp-paribas-cardif-claims-management/xgboost-with-combination-of-factors/discussion\r\n\r\n[/quote]I saw the script of Jack. It is nice indeed, but I don't think it exploits the maximum of feature engineering you could do when you actually know the meaning of each variable. He just got an idea of some combinations of variables that work well, but these would probably be trivial for someone who has the data dictionary.\r\n\r\nCongratz with your result, it is a very nice result as well. Especially the fact it comes from one single model. \r\n\r\nI agree with your 2 elements, but a blackbox of 1 XGB is still a lot simpler as the 100 model-ensemble approach..." }, { "id": 115678, "postDate": "2016-04-19T15:37:48.957Z", "content": "(double post bug)
", "rawMarkdown": "(double post bug)" }, { "id": 115657, "postDate": "2016-04-19T14:37:41.643Z", "content": "It's very interesting to see that good results come from different approaches. I'm curious that how do you find var114 is important?\n[quote=Naokazu Mizuta;115656]
\n\nCongrats to the winners!
\n\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.
\n\n@raddar Do you have any ideas or make assumptions of what v114 means?
\n\n[/quote]
", "rawMarkdown": "It's very interesting to see that good results come from different approaches. I'm curious that how do you find var114 is important?\r\n[quote=Naokazu Mizuta;115656]\r\n\r\nCongrats to the winners!\r\n\r\nI'm a bit surprised to know the methodologies of winning teams are different from mine. I thought most of top teams were exploiting similar ones. I used rounded var114 intensively to detect interaction of variables, especially with v52 and v112. I didn't get so much of gain using v22. What has been bugging me for a while was the relation between v40 and v50. When grouped by v114, the order of v40 and v50 almost coincides and I have been wondering what it means. raddar's comment shed some light on this.\r\n\r\n@raddar Do you have any ideas or make assumptions of what v114 means?\r\n\r\n\r\n[/quote]\r\n" }, { "id": 115651, "postDate": "2016-04-19T14:07:38.377Z", "content": "@raddar and @Dexter's Lab,
\n\nThanks for sharing your approach and tips! Well deserved win!
", "rawMarkdown": "@raddar and @Dexter's Lab,\r\n\r\nThanks for sharing your approach and tips! Well deserved win!" }, { "id": 115644, "postDate": "2016-04-19T13:47:19.230Z", "content": "[quote=KarlLiu;115516]
\n\nI forgot to select my final two submissions. Does anyone know if Kaggle will use the most recent two submissions to calculate my final result? Thank you.
\n\n[/quote]They take the best 2 public LB if you don't select anything, which is probably reasonably good for a lot of competitions. Especially if you don't overfit the public leaderboard and the dataset is big.
", "rawMarkdown": "[quote=KarlLiu;115516]\r\n\r\nI forgot to select my final two submissions. Does anyone know if Kaggle will use the most recent two submissions to calculate my final result? Thank you. \r\n\r\n[/quote]They take the best 2 public LB if you don't select anything, which is probably reasonably good for a lot of competitions. Especially if you don't overfit the public leaderboard and the dataset is big." }, { "id": 115624, "postDate": "2016-04-19T11:52:00.343Z", "content": "Great knowledge to gain here, thanks guys !
", "rawMarkdown": "Great knowledge to gain here, thanks guys !" }, { "id": 115619, "postDate": "2016-04-19T10:57:12.520Z", "content": "Congratulations to the winers!!! and the rest of people for his amazing post in the forums...I learnt a lot!!!!
\n\n@ Raddar, How many new features did you use?
\n\nThanks a lot!!!
", "rawMarkdown": "Congratulations to the winers!!! and the rest of people for his amazing post in the forums...I learnt a lot!!!!\r\n\r\n@ Raddar, How many new features did you use?\r\n\r\nThanks a lot!!!" }, { "id": 115611, "postDate": "2016-04-19T10:02:17.453Z", "content": "Congratulazioni!! How many features did you use in the best single model?
", "rawMarkdown": "Congratulazioni!! How many features did you use in the best single model?" }, { "id": 115596, "postDate": "2016-04-19T09:14:10.417Z", "content": "[quote=raddar;115563]
\n\nThank you all,
\n\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)
\n\nIt is actually quite simple. lets say you have variable X; you sort that X and calculate rank values for each X value. then take X[rank] - X[rank-1] and plot a distribution plot of that. You will see very interesting distribution of several spikes.
\n\nTo put it simply, lets say you have 0.00049, 0.00049,....0.00049,0.00098,0.00098,....0.00147,0.00147. See the pattern :) THen it just comes to simple optim function to search for median values between "jumping points" (X[rank] - X[rank-1]) distribution.
\n\nv40,v10,v50,v18,v28 were the variables, where this seperation is present. There are some more variables which are a mixture of this and gaussian density (meaning there are many integers, but also fractional numbers in that variable) - However I did not dig into it.
\n\n[/quote]\nI posted a script with this technique right after i read your post :)\nhttps://www.kaggle.com/rushter/bnp-paribas-cardif-claims-management/et-classifier/code
\n\nIt seems like no one gained from it. Even i.
", "rawMarkdown": "[quote=raddar;115563]\r\n\r\nThank you all,\r\n\r\nI see everyone is interested about the denominators. Actually the solution how I came up to that was posted in forums weeks ago, and some1 even did visualization for that:)\r\n\r\nIt is actually quite simple. lets say you have variable X; you sort that X and calculate rank values for each X value. then take X[rank] - X[rank-1] and plot a distribution plot of that. You will see very interesting distribution of several spikes.\r\n\r\nTo put it simply, lets say you have 0.00049, 0.00049,....0.00049,0.00098,0.00098,....0.00147,0.00147. See the pattern :) THen it just comes to simple optim function to search for median values between \"jumping points\" (X[rank] - X[rank-1]) distribution.\r\n\r\nv40,v10,v50,v18,v28 were the variables, where this seperation is present. There are some more variables which are a mixture of this and gaussian density (meaning there are many integers, but also fractional numbers in that variable) - However I did not dig into it.\r\n\r\n[/quote]\r\nI posted a script with this technique right after i read your post :)\r\nhttps://www.kaggle.com/rushter/bnp-paribas-cardif-claims-management/et-classifier/code\r\n\r\nIt seems like no one gained from it. Even i. " }, { "id": 115582, "postDate": "2016-04-19T07:44:00.567Z", "content": "Congrats @Dexters lab ,good to see the hidden informations ....\n@Laurae.... Thank u very much for your analysis....You analysis is indepth and very unique....if possible please share some code describing your way of seeing the problem.... I'm interested in seeing the linear models( like lm in R) best performance ..\nThank u again to all who shared their views .. All views do help.. :-) Cheers
", "rawMarkdown": "Congrats @Dexters lab ,good to see the hidden informations ....\r\n@Laurae.... Thank u very much for your analysis....You analysis is indepth and very unique....if possible please share some code describing your way of seeing the problem.... I'm interested in seeing the linear models( like lm in R) best performance ..\r\nThank u again to all who shared their views .. All views do help.. :-) Cheers" }, { "id": 115573, "postDate": "2016-04-19T07:20:44.463Z", "content": "Congratulations, raddar. You both really got into the data. Could you please further simplify the example for laypersons like myself. I see that variable X is sorted (?), but the median is now calculated for each point in the variable?
", "rawMarkdown": "Congratulations, raddar. You both really got into the data. Could you please further simplify the example for laypersons like myself. I see that variable X is sorted (?), but the median is now calculated for each point in the variable?" }, { "id": 115568, "postDate": "2016-04-19T06:58:34.050Z", "content": "Well done, I thought I had an advantage on feature interpretation being member of BNP Paribas Cardif but I couldn't manage to guess what you did.
\n\nThe main question for me, because I tried to find it is : how did you convince yourself v40 and v50 would be dates ?
\n\nEven with noise removal I didn't think to remove at the time, it does not appear as an evidence for me.
", "rawMarkdown": "Well done, I thought I had an advantage on feature interpretation being member of BNP Paribas Cardif but I couldn't manage to guess what you did. \r\n\r\nThe main question for me, because I tried to find it is : how did you convince yourself v40 and v50 would be dates ?\r\n\r\nEven with noise removal I didn't think to remove at the time, it does not appear as an evidence for me." }, { "id": 115564, "postDate": "2016-04-19T06:26:16.977Z", "content": "Thanks for sharing! I had a sense of the importance of feature engineering. But as a rookie of Kaggle competition, I didn't have enough time and was lack of experience of it.
\n\nCongrats and thanks again!
", "rawMarkdown": "Thanks for sharing! I had a sense of the importance of feature engineering. But as a rookie of Kaggle competition, I didn't have enough time and was lack of experience of it.\r\n\r\nCongrats and thanks again!" }, { "id": 115546, "postDate": "2016-04-19T03:36:16.310Z", "content": "Congrats Team!\nCan you share the code used for data exploration?
", "rawMarkdown": "Congrats Team!\r\nCan you share the code used for data exploration?" }, { "id": 115545, "postDate": "2016-04-19T03:33:27.603Z", "content": "Very well deserved win, based on the insights that you discovered in the data.
", "rawMarkdown": "Very well deserved win, based on the insights that you discovered in the data." }, { "id": 115543, "postDate": "2016-04-19T03:23:10.790Z", "content": "Very impressive!!!
\n\nBy the way, I used regularized greedy forest too (part of ensemble). The algorithm is very powerful but is very hard to train properly because it's VERY slow.
", "rawMarkdown": "Very impressive!!!\r\n\r\nBy the way, I used regularized greedy forest too (part of ensemble). The algorithm is very powerful but is very hard to train properly because it's VERY slow. " }, { "id": 115541, "postDate": "2016-04-19T03:14:54.317Z", "content": "Congratulations! It' amazing how you crack the problem.
", "rawMarkdown": "Congratulations! It' amazing how you crack the problem." }, { "id": 115538, "postDate": "2016-04-19T02:35:58.787Z", "content": "Congratulations! How did you discover that v40 was the date?
", "rawMarkdown": "Congratulations! How did you discover that v40 was the date?" }, { "id": 115527, "postDate": "2016-04-19T01:30:42.280Z", "content": "Congratulations! Very insightful explanation.
", "rawMarkdown": "Congratulations! Very insightful explanation." }, { "id": 115525, "postDate": "2016-04-19T01:21:30.583Z", "content": "Hats off to you guys. I'm still not sure what any of this data means.
", "rawMarkdown": "Hats off to you guys. I'm still not sure what any of this data means." }, { "id": 115523, "postDate": "2016-04-19T01:18:29.783Z", "content": "Thanks for sharing. Really impressive data exploration! Well deserved top position.
\n\n@xaviercapdepon thanks for your ET script, helped me at lot when I got stuck.
", "rawMarkdown": "Thanks for sharing. Really impressive data exploration! Well deserved top position.\r\n\r\n@xaviercapdepon thanks for your ET script, helped me at lot when I got stuck." }, { "id": 115519, "postDate": "2016-04-19T01:10:08.073Z", "content": "Congratulations! Thank you so much for sharing!
", "rawMarkdown": "Congratulations! Thank you so much for sharing!" }, { "id": 115518, "postDate": "2016-04-19T01:06:00.107Z", "content": "Very impressive winning solution! Congratulation to you all!
", "rawMarkdown": "Very impressive winning solution! Congratulation to you all!" }, { "id": 115516, "postDate": "2016-04-19T01:03:27.540Z", "content": "Thank you.
", "rawMarkdown": "Thank you. ", "votes": -13 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T12:28:23Z", "competition_id": 4852, "competition_slug": "bnp-paribas-cardif-claims-management", "team_id": 278057, "solution_url": "/competitions/bnp-paribas-cardif-claims-management/writeups/dexter-s-lab-1-dexter-s-lab-winning-solution" }