We have to wait 2 months until the official competition results but it would be interesting to know the public scores that you chose for your submissions. So many people had Public 0.0 but what was actually selected?
\n\nI think whatever approach you took there is an element of luck to this competition. But anyway, mine were 2.1042 and 2.1127.
", "messages": [ { "id": 431108, "postDate": "2018-12-01T16:16:55.080Z", "content": "2.0751 and 2.1015
\n\nP.S. I hope noone selected the 0-score submission intentionally :D
", "rawMarkdown": "2.0751 and 2.1015\n\nP.S. I hope noone selected the 0-score submission intentionally :D", "votes": 3, "replies": [ { "id": 431163, "postDate": "2018-12-01T17:34:47.773Z", "content": "Thanks. I am interested in whether the \"wisdom of the crowds\" can give an indication on the macro-levels for good predictions.
", "rawMarkdown": "Thanks. I am interested in whether the \"wisdom of the crowds\" can give an indication on the macro-levels for good predictions. " } ] }, { "id": 431047, "postDate": "2018-12-01T14:24:02.697Z", "content": "We have to wait 2 months until the official competition results but it would be interesting to know the public scores that you chose for your submissions. So many people had Public 0.0 but what was actually selected?
\n\nI think whatever approach you took there is an element of luck to this competition. But anyway, mine were 2.1042 and 2.1127.
", "rawMarkdown": "We have to wait 2 months until the official competition results but it would be interesting to know the public scores that you chose for your submissions. So many people had Public 0.0 but what was actually selected?\n\nI think whatever approach you took there is an element of luck to this competition. But anyway, mine were 2.1042 and 2.1127.", "votes": 3 }, { "id": 434716, "postDate": "2018-12-06T20:53:43.790Z", "content": "2.1119 and 2.1138 (yes all zeros ;-). Another interesting question - what was the difference in RMSE score (in your CV schema) between all zeros and other submissions? I had about 0.01 better RMSE in my second submission than all zeros (of cause it could be an overfitting).
", "rawMarkdown": "2.1119 and 2.1138 (yes all zeros ;-). Another interesting question - what was the difference in RMSE score (in your CV schema) between all zeros and other submissions? I had about 0.01 better RMSE in my second submission than all zeros (of cause it could be an overfitting). ", "votes": 1 }, { "id": 431749, "postDate": "2018-12-02T21:18:58.490Z", "content": "So far most predictions people have listed are >2. We know that the score for predicting all zeros is 2.1138. To avoid overfitting completely someone could have predicted all zeros. The trouble with that approach is that it would underfit. The only alternative is that all zeros is the best solution which I really hope is not the case. I believe that there is some scope to improve on all zeros however small. Overfitting and underfitting are not mutually exclusive but trying to avoid one generally leads to the other. We won't know for sure until the competition closes which is the best approach. One other observation is that many people chose submissions quite far apart suggesting some uncertainty about the best approach. I don't think this reflects a lack of skill but rather that there is a lot of noise in this competition relative to the signal. It can also be a tactically sensible approach to have one submission more aggressive and one more conservative as a backup if the aggressive entry overfits horribly. It seems of the responses so far that mine were closest to the all zero approach aiming to avoid overfitting at the risk of underfitting. Thanks for the responses so far and I'm still interested in other scores selected. Also do you guys have comments on the nature of the challenge in this competition based on these responses?
", "rawMarkdown": "So far most predictions people have listed are >2. We know that the score for predicting all zeros is 2.1138. To avoid overfitting completely someone could have predicted all zeros. The trouble with that approach is that it would underfit. The only alternative is that all zeros is the best solution which I really hope is not the case. I believe that there is some scope to improve on all zeros however small. Overfitting and underfitting are not mutually exclusive but trying to avoid one generally leads to the other. We won't know for sure until the competition closes which is the best approach. One other observation is that many people chose submissions quite far apart suggesting some uncertainty about the best approach. I don't think this reflects a lack of skill but rather that there is a lot of noise in this competition relative to the signal. It can also be a tactically sensible approach to have one submission more aggressive and one more conservative as a backup if the aggressive entry overfits horribly. It seems of the responses so far that mine were closest to the all zero approach aiming to avoid overfitting at the risk of underfitting. Thanks for the responses so far and I'm still interested in other scores selected. Also do you guys have comments on the nature of the challenge in this competition based on these responses?", "votes": 1, "replies": [ { "id": 431794, "postDate": "2018-12-02T22:59:13.733Z", "content": "For RMSE metric the constant equal to the mean for the test period would give a better result than all-zeros.Of course, we do not know the mean, but we know that it is not strictly 0 (and we could get some estimates from previous periods), so 1e-6 would beat the all-zero prediction
", "rawMarkdown": "For RMSE metric the constant equal to the mean for the test period would give a better result than all-zeros.Of course, we do not know the mean, but we know that it is not strictly 0 (and we could get some estimates from previous periods), so 1e-6 would beat the all-zero prediction", "votes": 2 }, { "id": 431807, "postDate": "2018-12-02T23:31:32.353Z", "content": "Agreed, but estimating the mean at least takes some degree of forecasting skill not required in submitting the sample submission of all zeros. Also, doing it that way means there is some overfitting on most values but it is offset by the decreased underfitting on the others. To a degree it would be interesting to know the mean and standard deviation of submissions as well.
", "rawMarkdown": "Agreed, but estimating the mean at least takes some degree of forecasting skill not required in submitting the sample submission of all zeros. Also, doing it that way means there is some overfitting on most values but it is offset by the decreased underfitting on the others. To a degree it would be interesting to know the mean and standard deviation of submissions as well.", "votes": 1 } ] }, { "id": 431562, "postDate": "2018-12-02T13:50:42.030Z", "content": "2.0398 and 2.0847 ...
", "rawMarkdown": "2.0398 and 2.0847 ...", "votes": 1 }, { "id": 431535, "postDate": "2018-12-02T12:55:18.537Z", "content": "2.0505 and 2.1104 here, (using only oficial v2 datasets, i.e, no web scraping data for the gap interval)
", "rawMarkdown": "2.0505 and 2.1104 here, (using only oficial v2 datasets, i.e, no web scraping data for the gap interval)", "votes": 1 }, { "id": 437592, "postDate": "2018-12-12T06:49:04.967Z", "content": "Hello.Could you give me datasets about this compete.Because the compete entry is shutdown.I can't download from website.I'll appreciate it if I can get it.Have a nice day
", "rawMarkdown": "Hello.Could you give me datasets about this compete.Because the compete entry is shutdown.I can't download from website.I'll appreciate it if I can get it.Have a nice day" }, { "id": 432266, "postDate": "2018-12-03T16:28:05.950Z", "content": "Mine was 2.1000.
\n\nI made a non-zero prediction only for buyers in test_v2.csv, i.e., I put all my effort on predicting repeat-buyers.
", "rawMarkdown": "Mine was 2.1000.\n\nI made a non-zero prediction only for buyers in test_v2.csv, i.e., I put all my effort on predicting repeat-buyers." }, { "id": 431640, "postDate": "2018-12-02T16:56:20.340Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 431781, "postDate": "2018-12-02T22:12:04.463Z", "content": "It depends on how you you train your model (how you set up validation). I had blic a 3-step procedure and only the first step (data pre-processing with json parsing) is in a public kernel. The second (feature extraction and CV slicing) and third (model building) are not public. But I summarised my approach in a dedicated forum post, and I do not have high expectations that my solution will lend high on the private LB. So at the moment it might be useless to publish it (maybe it is plain wrong)
", "rawMarkdown": "It depends on how you you train your model (how you set up validation). I had blic a 3-step procedure and only the first step (data pre-processing with json parsing) is in a public kernel. The second (feature extraction and CV slicing) and third (model building) are not public. But I summarised my approach in a dedicated forum post, and I do not have high expectations that my solution will lend high on the private LB. So at the moment it might be useless to publish it (maybe it is plain wrong)" } ] } ], "comments": [ { "id": 431108, "author_name": "Misha Lisovyi", "author_url": "", "post_date": "2018-12-01T16:16:55.080000", "content": "2.0751 and 2.1015
\n\nP.S. I hope noone selected the 0-score submission intentionally :D
", "votes": 3, "replies": [ { "id": 431163, "author_name": "MichaelP", "author_url": "", "post_date": "2018-12-01T17:34:47.773000", "content": "Thanks. I am interested in whether the \"wisdom of the crowds\" can give an indication on the macro-levels for good predictions.
", "votes": 0, "replies": [] } ] }, { "id": 434716, "author_name": "Mikhail Chesnokov", "author_url": "", "post_date": "2018-12-06T20:53:43.790000", "content": "2.1119 and 2.1138 (yes all zeros ;-). Another interesting question - what was the difference in RMSE score (in your CV schema) between all zeros and other submissions? I had about 0.01 better RMSE in my second submission than all zeros (of cause it could be an overfitting).
", "votes": 1, "replies": [] }, { "id": 431749, "author_name": "MichaelP", "author_url": "", "post_date": "2018-12-02T21:18:58.490000", "content": "So far most predictions people have listed are >2. We know that the score for predicting all zeros is 2.1138. To avoid overfitting completely someone could have predicted all zeros. The trouble with that approach is that it would underfit. The only alternative is that all zeros is the best solution which I really hope is not the case. I believe that there is some scope to improve on all zeros however small. Overfitting and underfitting are not mutually exclusive but trying to avoid one generally leads to the other. We won't know for sure until the competition closes which is the best approach. One other observation is that many people chose submissions quite far apart suggesting some uncertainty about the best approach. I don't think this reflects a lack of skill but rather that there is a lot of noise in this competition relative to the signal. It can also be a tactically sensible approach to have one submission more aggressive and one more conservative as a backup if the aggressive entry overfits horribly. It seems of the responses so far that mine were closest to the all zero approach aiming to avoid overfitting at the risk of underfitting. Thanks for the responses so far and I'm still interested in other scores selected. Also do you guys have comments on the nature of the challenge in this competition based on these responses?
", "votes": 1, "replies": [ { "id": 431794, "author_name": "Misha Lisovyi", "author_url": "", "post_date": "2018-12-02T22:59:13.733000", "content": "For RMSE metric the constant equal to the mean for the test period would give a better result than all-zeros.Of course, we do not know the mean, but we know that it is not strictly 0 (and we could get some estimates from previous periods), so 1e-6 would beat the all-zero prediction
", "votes": 2, "replies": [] }, { "id": 431807, "author_name": "MichaelP", "author_url": "", "post_date": "2018-12-02T23:31:32.353000", "content": "Agreed, but estimating the mean at least takes some degree of forecasting skill not required in submitting the sample submission of all zeros. Also, doing it that way means there is some overfitting on most values but it is offset by the decreased underfitting on the others. To a degree it would be interesting to know the mean and standard deviation of submissions as well.
", "votes": 1, "replies": [] } ] }, { "id": 431562, "author_name": "Chau Huynh", "author_url": "", "post_date": "2018-12-02T13:50:42.030000", "content": "2.0398 and 2.0847 ...
", "votes": 1, "replies": [] }, { "id": 431535, "author_name": "miguel perez", "author_url": "", "post_date": "2018-12-02T12:55:18.537000", "content": "2.0505 and 2.1104 here, (using only oficial v2 datasets, i.e, no web scraping data for the gap interval)
", "votes": 1, "replies": [] }, { "id": 437592, "author_name": "FuengWul", "author_url": "", "post_date": "2018-12-12T06:49:04.967000", "content": "Hello.Could you give me datasets about this compete.Because the compete entry is shutdown.I can't download from website.I'll appreciate it if I can get it.Have a nice day
", "votes": 0, "replies": [] }, { "id": 432266, "author_name": "Daniel Quelali", "author_url": "", "post_date": "2018-12-03T16:28:05.950000", "content": "Mine was 2.1000.
\n\nI made a non-zero prediction only for buyers in test_v2.csv, i.e., I put all my effort on predicting repeat-buyers.
", "votes": 0, "replies": [] }, { "id": 431640, "author_name": "", "author_url": "", "post_date": "2018-12-02T16:56:20.340000", "content": "", "votes": 1, "replies": [ { "id": 431781, "author_name": "Misha Lisovyi", "author_url": "", "post_date": "2018-12-02T22:12:04.463000", "content": "It depends on how you you train your model (how you set up validation). I had blic a 3-step procedure and only the first step (data pre-processing with json parsing) is in a public kernel. The second (feature extraction and CV slicing) and third (model building) are not public. But I summarised my approach in a dedicated forum post, and I do not have high expectations that my solution will lend high on the private LB. So at the moment it might be useless to publish it (maybe it is plain wrong)
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "431108": "2.0751 and 2.1015\n\nP.S. I hope noone selected the 0-score submission intentionally :D", "431047": "We have to wait 2 months until the official competition results but it would be interesting to know the public scores that you chose for your submissions. So many people had Public 0.0 but what was actually selected?\n\nI think whatever approach you took there is an element of luck to this competition. But anyway, mine were 2.1042 and 2.1127.", "434716": "2.1119 and 2.1138 (yes all zeros ;-). Another interesting question - what was the difference in RMSE score (in your CV schema) between all zeros and other submissions? I had about 0.01 better RMSE in my second submission than all zeros (of cause it could be an overfitting). ", "431749": "So far most predictions people have listed are >2. We know that the score for predicting all zeros is 2.1138. To avoid overfitting completely someone could have predicted all zeros. The trouble with that approach is that it would underfit. The only alternative is that all zeros is the best solution which I really hope is not the case. I believe that there is some scope to improve on all zeros however small. Overfitting and underfitting are not mutually exclusive but trying to avoid one generally leads to the other. We won't know for sure until the competition closes which is the best approach. One other observation is that many people chose submissions quite far apart suggesting some uncertainty about the best approach. I don't think this reflects a lack of skill but rather that there is a lot of noise in this competition relative to the signal. It can also be a tactically sensible approach to have one submission more aggressive and one more conservative as a backup if the aggressive entry overfits horribly. It seems of the responses so far that mine were closest to the all zero approach aiming to avoid overfitting at the risk of underfitting. Thanks for the responses so far and I'm still interested in other scores selected. Also do you guys have comments on the nature of the challenge in this competition based on these responses?", "431562": "2.0398 and 2.0847 ...", "431535": "2.0505 and 2.1104 here, (using only oficial v2 datasets, i.e, no web scraping data for the gap interval)", "437592": "Hello.Could you give me datasets about this compete.Because the compete entry is shutdown.I can't download from website.I'll appreciate it if I can get it.Have a nice day", "432266": "Mine was 2.1000.\n\nI made a non-zero prediction only for buyers in test_v2.csv, i.e., I put all my effort on predicting repeat-buyers.", "431640": "" } }