It's the final week, so you won't (hopefully) post or read any new kernels with ground-breaking techniques. You've already bragged about your great LB scores and envied the folks who've done better. Does that mean there's nothing to talk about? Heck, no!!
\n\nTell us about how obsessive you are. What are the myriad things you tried -- not the techniques themselves but the sheer number of experiments.
\n\nHere are some specific numbers to think about:\n- How many different notebooks/kernel-versions did you create?\n- How many kernels did you clone?\n- How many different model-types/algorithms did you try?\n- How many different models have you trained within a single kernel? Alternatively, how many ways have you split up the training data?\n- What's the highest/lowest number of features that you tried?
\n\nThere's no need to reveal secrets. I don't care whether the things you tried actually worked -- the fun is in the attempt.
\n\nHere are my answers:\n- Notebook/kernel count: I've created 104 different notebooks. Typically I have one notebook per submission, FE experiment, or EDA approach. I'm obsessive about trying new things and keeping each experiment small and focussed.\n- Cloned kernels: I have zero cloned kernels. Even when I want to try someone else's approaches, I'd still rather write the code from scratch. I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n- Algorithm count: I've only tried 4 off-the-shelf regressors. Alas, nothing but the usual suspects.\n- Data splits: I've split up my data (through partitioning and CV) to create 28,560 simultaneous models in a single training run. They still used less memory than the training data itself. (This doesn't count the individual trees in the gradient-boosting models -- just high-level instances of the sklearn model.)\n- Feature counts: I don't actually have an exact number for max-feature-count, but it's under 100. The minimum count was 20.
", "messages": [ { "id": 696353, "postDate": "2019-12-16T14:08:23.400Z", "content": "It's the final week, so you won't (hopefully) post or read any new kernels with ground-breaking techniques. You've already bragged about your great LB scores and envied the folks who've done better. Does that mean there's nothing to talk about? Heck, no!!
\n\nTell us about how obsessive you are. What are the myriad things you tried -- not the techniques themselves but the sheer number of experiments.
\n\nHere are some specific numbers to think about:\n- How many different notebooks/kernel-versions did you create?\n- How many kernels did you clone?\n- How many different model-types/algorithms did you try?\n- How many different models have you trained within a single kernel? Alternatively, how many ways have you split up the training data?\n- What's the highest/lowest number of features that you tried?
\n\nThere's no need to reveal secrets. I don't care whether the things you tried actually worked -- the fun is in the attempt.
\n\nHere are my answers:\n- Notebook/kernel count: I've created 104 different notebooks. Typically I have one notebook per submission, FE experiment, or EDA approach. I'm obsessive about trying new things and keeping each experiment small and focussed.\n- Cloned kernels: I have zero cloned kernels. Even when I want to try someone else's approaches, I'd still rather write the code from scratch. I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n- Algorithm count: I've only tried 4 off-the-shelf regressors. Alas, nothing but the usual suspects.\n- Data splits: I've split up my data (through partitioning and CV) to create 28,560 simultaneous models in a single training run. They still used less memory than the training data itself. (This doesn't count the individual trees in the gradient-boosting models -- just high-level instances of the sklearn model.)\n- Feature counts: I don't actually have an exact number for max-feature-count, but it's under 100. The minimum count was 20.
", "rawMarkdown": "It's the final week, so you won't (hopefully) post or read any new kernels with ground-breaking techniques. You've already bragged about your great LB scores and envied the folks who've done better. Does that mean there's nothing to talk about? Heck, no!!\n\nTell us about how obsessive you are. What are the myriad things you tried -- not the techniques themselves but the sheer number of experiments.\n\nHere are some specific numbers to think about:\n- How many different notebooks/kernel-versions did you create?\n- How many kernels did you clone?\n- How many different model-types/algorithms did you try?\n- How many different models have you trained within a single kernel? Alternatively, how many ways have you split up the training data?\n- What's the highest/lowest number of features that you tried?\n\nThere's no need to reveal secrets. I don't care whether the things you tried actually *worked* -- the fun is in the attempt.\n\nHere are my answers:\n- **Notebook/kernel count**: I've created 104 different notebooks. Typically I have one notebook per submission, FE experiment, or EDA approach. I'm obsessive about trying new things and keeping each experiment small and focussed.\n- **Cloned kernels**: I have zero cloned kernels. Even when I want to try someone else's approaches, I'd still rather write the code from scratch. I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n- **Algorithm count**: I've only tried 4 off-the-shelf regressors. Alas, nothing but the usual suspects.\n- **Data splits**: I've split up my data (through partitioning and CV) to create 28,560 simultaneous models in a single training run. They still used less memory than the training data itself. (This doesn't count the individual trees in the gradient-boosting models -- just high-level instances of the sklearn model.)\n- **Feature counts**: I don't actually have an exact number for max-feature-count, but it's under 100. The minimum count was 20.\n", "votes": 12 }, { "id": 697450, "postDate": "2019-12-17T23:50:00.417Z", "content": "I feel you on the obsession. This is my first competition and I haven't shaved for weeks. I have been disregarding basic life matters due to this. I'm not going to share my numbers because I lost track although I share most of what you said/tried.\nSome learnings I can think of right now: \n- Next time I'm going to take advantage of every single submission slot... Failing to do so (and keeping good track of them) is extremely bad. \n- Made a big improvement after using Vopani's version of his Half & Half\n- Then big improvements after running algorithms that would \"judge\" whether some training data is representative of the test set or not. That is why probing hypothesis against the site's test set is so important.
\n\nAfter hitting my best score it was downhill, then I forgot what I did to get my best score only to find afterwards that it was a result of a mistake in my code.... SO PURE LUCK.
\n\nThanks for reminding everyone that this is supposed to be fun.
", "rawMarkdown": "I feel you on the obsession. This is my first competition and I haven't shaved for weeks. I have been disregarding basic life matters due to this. I'm not going to share my numbers because I lost track although I share most of what you said/tried.\nSome learnings I can think of right now: \n- Next time I'm going to take advantage of every single submission slot... Failing to do so (and keeping good track of them) is extremely bad. \n- Made a big improvement after using Vopani's version of his Half & Half\n- Then big improvements after running algorithms that would \"judge\" whether some training data is representative of the test set or not. That is why probing hypothesis against the site's test set is so important.\n\nAfter hitting my best score it was downhill, then I forgot what I did to get my best score only to find afterwards that it was a result of a mistake in my code.... SO PURE LUCK.\n\nThanks for reminding everyone that this is supposed to be fun.\n\n", "votes": 3, "replies": [ { "id": 697522, "postDate": "2019-12-18T03:12:41.440Z", "content": "I relate a lot to this. Also my first competition, one learning that I took from it: Start the competition by finding a solid validation method. I struggled with validation and also had the problem of not keeping good track of my submissions.
", "rawMarkdown": "I relate a lot to this. Also my first competition, one learning that I took from it: Start the competition by finding a solid validation method. I struggled with validation and also had the problem of not keeping good track of my submissions.", "votes": 2 }, { "id": 697534, "postDate": "2019-12-18T03:48:09.627Z", "content": "Yes, I agree with you. This is the first official competition to me also. I am having same problem like your case.
", "rawMarkdown": "Yes, I agree with you. This is the first official competition to me also. I am having same problem like your case.", "votes": 2 }, { "id": 697626, "postDate": "2019-12-18T07:35:01.067Z", "content": "Agree with on using submissions well.
", "rawMarkdown": "Agree with on using submissions well. " } ] }, { "id": 696688, "postDate": "2019-12-17T00:23:10.567Z", "content": "Wow, so many notebooks! How do you keep yourself organized? I had about 15 in total and I kept overwriting the same ones with new ideas (sometimes this was kinda bad because I wanted to go back XD, not a very efficient version control).
\n\nOne idea that I tried a lot in validation (and maybe about 6 submissions) but didn't perform well in LB (mb i messed up, who knows =p ) was to do classification for the 0s. So basically do regression and \"prune\" the predictions afterwards by classifying readings that should've been zeroes.
", "rawMarkdown": "Wow, so many notebooks! How do you keep yourself organized? I had about 15 in total and I kept overwriting the same ones with new ideas (sometimes this was kinda bad because I wanted to go back XD, not a very efficient version control).\n\nOne idea that I tried a lot in validation (and maybe about 6 submissions) but didn't perform well in LB (mb i messed up, who knows =p ) was to do classification for the 0s. So basically do regression and \"prune\" the predictions afterwards by classifying readings that should've been zeroes.", "votes": 3, "replies": [ { "id": 696903, "postDate": "2019-12-17T08:04:01.983Z", "content": "I tried that same zero-classification strategy you did, with a similar lack of success. I suspect that zero-readings for the trickiest meters are effectively random, or at least impossible to predict given high-level data.
\n\nAs far as organization, I give each notebook a version number and a description as part of the filename, and a \"Summary\" cell at the start. Each submission filename includes the version number of the appropriate notebook, as does each feature extractor I introduce.
\n\nThus, when I see that my best LB submission is V042-submission.csv, I can go straight to the notebook named V042 -- Life the Universe and Llamas, and recall the secret to its success from the first cell:
\n\n\nSummary:
\n \nContinue using the Aztec calendar features from V041, but use llama encoding instead of one-hot.
\n \nThis produced a leaderboard improvement from 1.09 to 0.72, and should thus be maintained for future submissions.
\n
Once you get into the habit, it doesn't actually require much effort. I'd probably do something much more boring with git if Jupyter worked better with VC, but the manual versioning works out okay for me.
", "rawMarkdown": "I tried that same zero-classification strategy you did, with a similar lack of success. I suspect that zero-readings for the trickiest meters are effectively random, or at least impossible to predict given high-level data.\n\nAs far as organization, I give each notebook a version number and a description as part of the filename, and a \"Summary\" cell at the start. Each submission filename includes the version number of the appropriate notebook, as does each feature extractor I introduce.\n\nThus, when I see that my best LB submission is `V042-submission.csv`, I can go straight to the notebook named `V042 -- Life the Universe and Llamas`, and recall the secret to its success from the first cell:\n\n> # Summary:\n> Continue using the Aztec calendar features from V041, but use llama encoding instead of one-hot.\n>\n> This produced a leaderboard improvement from 1.09 to 0.72, and should thus be maintained for future submissions.\n\nOnce you get into the habit, it doesn't actually require much effort. I'd probably do something much more boring with git if Jupyter worked better with VC, but the manual versioning works out okay for me.", "votes": 4 }, { "id": 697187, "postDate": "2019-12-17T15:35:27.717Z", "content": "Wow how you manage to keep that habit of organization is really impressive, thanks a lot for sharing, honestly you were an MVP in this competition, specially with that Outlier analysis kernel, kudos!
", "rawMarkdown": "Wow how you manage to keep that habit of organization is really impressive, thanks a lot for sharing, honestly you were an MVP in this competition, specially with that Outlier analysis kernel, kudos!", "votes": 1 } ] }, { "id": 696729, "postDate": "2019-12-17T01:29:59.663Z", "content": "\n\n\nI did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.
\n
Any recommended starting hyperparams for meter-specific lightgbms? (possibly already shared in other kernels/discussions)
", "rawMarkdown": "> I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n\nAny recommended starting hyperparams for meter-specific lightgbms? (possibly already shared in other kernels/discussions)", "votes": 1, "replies": [ { "id": 696891, "postDate": "2019-12-17T07:38:31.540Z", "content": "I borrowed the parameters from ASHRAE: Divide and Conquer by Vopani (@rohanrao). I've modified them slightly since then, but it made a great starting point.
\n\n\n params = {\"objective\": \"regression\",\n \"num_leaves\": 41,\n \"learning_rate\": 0.049,\n \"bagging_freq\": 5,\n \"bagging_fraction\": 0.51,\n \"feature_fraction\": 0.81,\n \"metric\": \"rmse\"\n }\n
You did a lot. I didn't do that much since I am lazy. getting top score should be patient as I saw you. XD\nTry different things but not made into submission. : Fourier Transform + noise, Genetic Algorithm, Multivarate CNN
", "rawMarkdown": "You did a lot. I didn't do that much since I am lazy. getting top score should be patient as I saw you. XD\nTry different things but not made into submission. : Fourier Transform + noise, Genetic Algorithm, Multivarate CNN", "votes": 1 }, { "id": 696391, "postDate": "2019-12-16T15:01:38.310Z", "content": "I've wanted to write \"...all of them\", but after this
\n\n\n\n\nI've created 104 different notebooks
\n
I reconsidered.
", "rawMarkdown": "I've wanted to write \"...all of them\", but after this\n> I've created 104 different notebooks\n\nI reconsidered.", "votes": 1 }, { "id": 697052, "postDate": "2019-12-17T12:20:52.453Z", "content": "Pretty nice and educating discussion. Thank you @purist1024 for sharing your workflow with us. Mine not so established as yours but anyway, here you go:
\n\nA short list of my insights and things that helped me:\n- Moving the loading/preprocessing/etc on the test set after the training free-uped enough space for train and made the develop-debug cycles much faster\n- Merging data frames loaded from feather ate up lotta memory and couldn't find any discussion about why it is so.. b but just switching back to original csvs, merging and then storing as feather worked.\n- I tried to reuse codes from others as much as possible. Very soon the code could get very messy and lose its integrity.. so I guess it is wise to implement the idea from scratch (but that I guess could take much longer).
\n\nIssues I have struggled with and still struggling:\n- Overfitting\n- Parameter tunning (I feel I haven't found the best)\n- Too long prediction time (~1.5 hours) - I guess due to depth of the tree
", "rawMarkdown": "Pretty nice and educating discussion. Thank you @purist1024 for sharing your workflow with us. Mine not so established as yours but anyway, here you go:\n\n- **How many different notebooks/kernel-versions did you create?** I created around a dozen notebooks which each of them specialized in achieved something, for example, feature engineering only on weather data. These served as a mini-experiment before merging into the one notebook that I used for submission. But still I had the same issue as @prschdt as I was overwriting.\n- **How many kernels did you clone?** I cloned 2 and then stuck with one that I felt is the best for me as a starting point.\nHow many different model-types/algorithms did you try? I first began with a baseline (mean prediction), then RF then just lgbm. I want to try a simple linear regression but probably it is more useful to spend the remaining time on param tuning.\n- **How many different models have you trained within a single kernel?** Alternatively, how many ways have you split up the training data? I used the idea of the half and half model, meaning half of the data one model the other half another model and the half that is not used by the models to be used as validation.\n- **What's the highest/lowest number of features that you tried?** I guess I never went less than 30 and max was 50 but staying around 40 seemed to be efficient enough result wise as well as memory-wise\n\nA short list of my insights and things that helped me:\n- Moving the loading/preprocessing/etc on the test set after the training free-uped enough space for train and made the develop-debug cycles much faster\n- Merging data frames loaded from feather ate up lotta memory and couldn't find any discussion about why it is so.. b but just switching back to original csvs, merging and then storing as feather worked.\n- I tried to reuse codes from others as much as possible. Very soon the code could get very messy and lose its integrity.. so I guess it is wise to implement the idea from scratch (but that I guess could take much longer).\n\nIssues I have struggled with and still struggling:\n- Overfitting\n- Parameter tunning (I feel I haven't found the best)\n- Too long prediction time (~1.5 hours) - I guess due to depth of the tree", "votes": 2 }, { "id": 696490, "postDate": "2019-12-16T17:05:40.513Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 697450, "author_name": "DietHard", "author_url": "", "post_date": "2019-12-17T23:50:00.417000", "content": "I feel you on the obsession. This is my first competition and I haven't shaved for weeks. I have been disregarding basic life matters due to this. I'm not going to share my numbers because I lost track although I share most of what you said/tried.\nSome learnings I can think of right now: \n- Next time I'm going to take advantage of every single submission slot... Failing to do so (and keeping good track of them) is extremely bad. \n- Made a big improvement after using Vopani's version of his Half & Half\n- Then big improvements after running algorithms that would \"judge\" whether some training data is representative of the test set or not. That is why probing hypothesis against the site's test set is so important.
\n\nAfter hitting my best score it was downhill, then I forgot what I did to get my best score only to find afterwards that it was a result of a mistake in my code.... SO PURE LUCK.
\n\nThanks for reminding everyone that this is supposed to be fun.
", "votes": 3, "replies": [ { "id": 697522, "author_name": "Thiago Preischadt", "author_url": "", "post_date": "2019-12-18T03:12:41.440000", "content": "I relate a lot to this. Also my first competition, one learning that I took from it: Start the competition by finding a solid validation method. I struggled with validation and also had the problem of not keeping good track of my submissions.
", "votes": 2, "replies": [] }, { "id": 697534, "author_name": "Hanjoon Choe", "author_url": "", "post_date": "2019-12-18T03:48:09.627000", "content": "Yes, I agree with you. This is the first official competition to me also. I am having same problem like your case.
", "votes": 2, "replies": [] }, { "id": 697626, "author_name": "Anant Joshi", "author_url": "", "post_date": "2019-12-18T07:35:01.067000", "content": "Agree with on using submissions well.
", "votes": 0, "replies": [] } ] }, { "id": 696688, "author_name": "Thiago Preischadt", "author_url": "", "post_date": "2019-12-17T00:23:10.567000", "content": "Wow, so many notebooks! How do you keep yourself organized? I had about 15 in total and I kept overwriting the same ones with new ideas (sometimes this was kinda bad because I wanted to go back XD, not a very efficient version control).
\n\nOne idea that I tried a lot in validation (and maybe about 6 submissions) but didn't perform well in LB (mb i messed up, who knows =p ) was to do classification for the 0s. So basically do regression and \"prune\" the predictions afterwards by classifying readings that should've been zeroes.
", "votes": 3, "replies": [ { "id": 696903, "author_name": "Robert Stockton", "author_url": "", "post_date": "2019-12-17T08:04:01.983000", "content": "I tried that same zero-classification strategy you did, with a similar lack of success. I suspect that zero-readings for the trickiest meters are effectively random, or at least impossible to predict given high-level data.
\n\nAs far as organization, I give each notebook a version number and a description as part of the filename, and a \"Summary\" cell at the start. Each submission filename includes the version number of the appropriate notebook, as does each feature extractor I introduce.
\n\nThus, when I see that my best LB submission is V042-submission.csv, I can go straight to the notebook named V042 -- Life the Universe and Llamas, and recall the secret to its success from the first cell:
\n\n\nSummary:
\n \nContinue using the Aztec calendar features from V041, but use llama encoding instead of one-hot.
\n \nThis produced a leaderboard improvement from 1.09 to 0.72, and should thus be maintained for future submissions.
\n
Once you get into the habit, it doesn't actually require much effort. I'd probably do something much more boring with git if Jupyter worked better with VC, but the manual versioning works out okay for me.
", "votes": 4, "replies": [] }, { "id": 697187, "author_name": "Thiago Preischadt", "author_url": "", "post_date": "2019-12-17T15:35:27.717000", "content": "Wow how you manage to keep that habit of organization is really impressive, thanks a lot for sharing, honestly you were an MVP in this competition, specially with that Outlier analysis kernel, kudos!
", "votes": 1, "replies": [] } ] }, { "id": 696729, "author_name": "HyperSeedOptimized", "author_url": "", "post_date": "2019-12-17T01:29:59.663000", "content": "\n\n\nI did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.
\n
Any recommended starting hyperparams for meter-specific lightgbms? (possibly already shared in other kernels/discussions)
", "votes": 1, "replies": [ { "id": 696891, "author_name": "Robert Stockton", "author_url": "", "post_date": "2019-12-17T07:38:31.540000", "content": "I borrowed the parameters from ASHRAE: Divide and Conquer by Vopani (@rohanrao). I've modified them slightly since then, but it made a great starting point.
\n\n\n params = {\"objective\": \"regression\",\n \"num_leaves\": 41,\n \"learning_rate\": 0.049,\n \"bagging_freq\": 5,\n \"bagging_fraction\": 0.51,\n \"feature_fraction\": 0.81,\n \"metric\": \"rmse\"\n }\n
You did a lot. I didn't do that much since I am lazy. getting top score should be patient as I saw you. XD\nTry different things but not made into submission. : Fourier Transform + noise, Genetic Algorithm, Multivarate CNN
", "votes": 1, "replies": [] }, { "id": 696391, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-16T15:01:38.310000", "content": "I've wanted to write \"...all of them\", but after this
\n\n\n\n\nI've created 104 different notebooks
\n
I reconsidered.
", "votes": 1, "replies": [] }, { "id": 697052, "author_name": "Novin Shahroudi", "author_url": "", "post_date": "2019-12-17T12:20:52.453000", "content": "Pretty nice and educating discussion. Thank you @purist1024 for sharing your workflow with us. Mine not so established as yours but anyway, here you go:
\n\nA short list of my insights and things that helped me:\n- Moving the loading/preprocessing/etc on the test set after the training free-uped enough space for train and made the develop-debug cycles much faster\n- Merging data frames loaded from feather ate up lotta memory and couldn't find any discussion about why it is so.. b but just switching back to original csvs, merging and then storing as feather worked.\n- I tried to reuse codes from others as much as possible. Very soon the code could get very messy and lose its integrity.. so I guess it is wise to implement the idea from scratch (but that I guess could take much longer).
\n\nIssues I have struggled with and still struggling:\n- Overfitting\n- Parameter tunning (I feel I haven't found the best)\n- Too long prediction time (~1.5 hours) - I guess due to depth of the tree
", "votes": 2, "replies": [] }, { "id": 696490, "author_name": "", "author_url": "", "post_date": "2019-12-16T17:05:40.513000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "696353": "It's the final week, so you won't (hopefully) post or read any new kernels with ground-breaking techniques. You've already bragged about your great LB scores and envied the folks who've done better. Does that mean there's nothing to talk about? Heck, no!!\n\nTell us about how obsessive you are. What are the myriad things you tried -- not the techniques themselves but the sheer number of experiments.\n\nHere are some specific numbers to think about:\n- How many different notebooks/kernel-versions did you create?\n- How many kernels did you clone?\n- How many different model-types/algorithms did you try?\n- How many different models have you trained within a single kernel? Alternatively, how many ways have you split up the training data?\n- What's the highest/lowest number of features that you tried?\n\nThere's no need to reveal secrets. I don't care whether the things you tried actually *worked* -- the fun is in the attempt.\n\nHere are my answers:\n- **Notebook/kernel count**: I've created 104 different notebooks. Typically I have one notebook per submission, FE experiment, or EDA approach. I'm obsessive about trying new things and keeping each experiment small and focussed.\n- **Cloned kernels**: I have zero cloned kernels. Even when I want to try someone else's approaches, I'd still rather write the code from scratch. I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n- **Algorithm count**: I've only tried 4 off-the-shelf regressors. Alas, nothing but the usual suspects.\n- **Data splits**: I've split up my data (through partitioning and CV) to create 28,560 simultaneous models in a single training run. They still used less memory than the training data itself. (This doesn't count the individual trees in the gradient-boosting models -- just high-level instances of the sklearn model.)\n- **Feature counts**: I don't actually have an exact number for max-feature-count, but it's under 100. The minimum count was 20.\n", "697450": "I feel you on the obsession. This is my first competition and I haven't shaved for weeks. I have been disregarding basic life matters due to this. I'm not going to share my numbers because I lost track although I share most of what you said/tried.\nSome learnings I can think of right now: \n- Next time I'm going to take advantage of every single submission slot... Failing to do so (and keeping good track of them) is extremely bad. \n- Made a big improvement after using Vopani's version of his Half & Half\n- Then big improvements after running algorithms that would \"judge\" whether some training data is representative of the test set or not. That is why probing hypothesis against the site's test set is so important.\n\nAfter hitting my best score it was downhill, then I forgot what I did to get my best score only to find afterwards that it was a result of a mistake in my code.... SO PURE LUCK.\n\nThanks for reminding everyone that this is supposed to be fun.\n\n", "696688": "Wow, so many notebooks! How do you keep yourself organized? I had about 15 in total and I kept overwriting the same ones with new ideas (sometimes this was kinda bad because I wanted to go back XD, not a very efficient version control).\n\nOne idea that I tried a lot in validation (and maybe about 6 submissions) but didn't perform well in LB (mb i messed up, who knows =p ) was to do classification for the 0s. So basically do regression and \"prune\" the predictions afterwards by classifying readings that should've been zeroes.", "696729": "> I did copy a few sets of model hyper-parameters pretty much verbatim, however, since tuning from scratch is pretty expensive.\n\nAny recommended starting hyperparams for meter-specific lightgbms? (possibly already shared in other kernels/discussions)", "696562": "You did a lot. I didn't do that much since I am lazy. getting top score should be patient as I saw you. XD\nTry different things but not made into submission. : Fourier Transform + noise, Genetic Algorithm, Multivarate CNN", "696391": "I've wanted to write \"...all of them\", but after this\n> I've created 104 different notebooks\n\nI reconsidered.", "697052": "Pretty nice and educating discussion. Thank you @purist1024 for sharing your workflow with us. Mine not so established as yours but anyway, here you go:\n\n- **How many different notebooks/kernel-versions did you create?** I created around a dozen notebooks which each of them specialized in achieved something, for example, feature engineering only on weather data. These served as a mini-experiment before merging into the one notebook that I used for submission. But still I had the same issue as @prschdt as I was overwriting.\n- **How many kernels did you clone?** I cloned 2 and then stuck with one that I felt is the best for me as a starting point.\nHow many different model-types/algorithms did you try? I first began with a baseline (mean prediction), then RF then just lgbm. I want to try a simple linear regression but probably it is more useful to spend the remaining time on param tuning.\n- **How many different models have you trained within a single kernel?** Alternatively, how many ways have you split up the training data? I used the idea of the half and half model, meaning half of the data one model the other half another model and the half that is not used by the models to be used as validation.\n- **What's the highest/lowest number of features that you tried?** I guess I never went less than 30 and max was 50 but staying around 40 seemed to be efficient enough result wise as well as memory-wise\n\nA short list of my insights and things that helped me:\n- Moving the loading/preprocessing/etc on the test set after the training free-uped enough space for train and made the develop-debug cycles much faster\n- Merging data frames loaded from feather ate up lotta memory and couldn't find any discussion about why it is so.. b but just switching back to original csvs, merging and then storing as feather worked.\n- I tried to reuse codes from others as much as possible. Very soon the code could get very messy and lose its integrity.. so I guess it is wise to implement the idea from scratch (but that I guess could take much longer).\n\nIssues I have struggled with and still struggling:\n- Overfitting\n- Parameter tunning (I feel I haven't found the best)\n- Too long prediction time (~1.5 hours) - I guess due to depth of the tree", "696490": "" } }