{
  "id": 301316,
  "title": "Relatively new to Kaggle - Always run Notebooks from the start",
  "url": "/competitions/g-research-crypto-forecasting/discussion/301316",
  "author_name": "",
  "post_date": "2022-01-17T03:36:18.417810200Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi all! </p>\n<p>I'm relatively new to Kaggle doing my first real competition for Crypto Forecasting. My main issue/annoyance I'm experiencing is that I constantly have to rerun my notebook from scratch (i.e., all the way from doing the imports and importing the raw datasets and go from there). And this takes a lot of my time. </p>\n<p>Even when I save versions of my notebook, I still need to run from the beginning since it didn't save anything in the process. </p>\n<p>My question(s) is/are:</p>\n<ul>\n<li>How do you deal with this and what measures to take to avoid this is possible?</li>\n<li>Is there a way so that I can save Dataframes/data created (my features dataset) and don't have to run all the previous cells before it?</li>\n</ul>\n<p>Thanks in advance!!<br>\nVincent </p>",
  "messages": [
    {
      "id": "1652841",
      "postDate": "01/17/2022 03:36:18",
      "content": "<p>Hi all! </p>\n<p>I'm relatively new to Kaggle doing my first real competition for Crypto Forecasting. My main issue/annoyance I'm experiencing is that I constantly have to rerun my notebook from scratch (i.e., all the way from doing the imports and importing the raw datasets and go from there). And this takes a lot of my time. </p>\n<p>Even when I save versions of my notebook, I still need to run from the beginning since it didn't save anything in the process. </p>\n<p>My question(s) is/are:</p>\n<ul>\n<li>How do you deal with this and what measures to take to avoid this is possible?</li>\n<li>Is there a way so that I can save Dataframes/data created (my features dataset) and don't have to run all the previous cells before it?</li>\n</ul>\n<p>Thanks in advance!!<br>\nVincent </p>",
      "rawMarkdown": "Hi all! \n\nI'm relatively new to Kaggle doing my first real competition for Crypto Forecasting. My main issue/annoyance I'm experiencing is that I constantly have to rerun my notebook from scratch (i.e., all the way from doing the imports and importing the raw datasets and go from there). And this takes a lot of my time. \n\nEven when I save versions of my notebook, I still need to run from the beginning since it didn't save anything in the process. \n\nMy question(s) is/are:\n* How do you deal with this and what measures to take to avoid this is possible?\n* Is there a way so that I can save Dataframes/data created (my features dataset) and don't have to run all the previous cells before it?\n\nThanks in advance!!\nVincent",
      "votes": null
    },
    {
      "id": "1653510",
      "postDate": "01/17/2022 15:44:43",
      "content": "<p>Instead of dealing with Kaggle's interface, you could try working with your offline environment using Jupyter notebook and save/download your progress by pickling or CSV, and then uploading your completed file on here once it's done. </p>\n<p>This isn't really possible to do on here if you're constantly restarting the kernel; it's just going to dump any saved files and bring you back to square one. </p>",
      "rawMarkdown": "Instead of dealing with Kaggle's interface, you could try working with your offline environment using Jupyter notebook and save/download your progress by pickling or CSV, and then uploading your completed file on here once it's done. \n\nThis isn't really possible to do on here if you're constantly restarting the kernel; it's just going to dump any saved files and bring you back to square one.",
      "votes": null
    },
    {
      "id": "1653543",
      "postDate": "01/17/2022 16:20:36",
      "content": "<p>I have faced similar issues when i work with kaggle notebooks, i followed these measures so as to save time when running a script:</p>\n<ul>\n<li>Pickling model and reusing it rather than training the model with the same dataset over again</li>\n<li>Constant version saves ( the datasets were however added readily from the previous version itself)</li>\n<li>Following a code structure which requires lesser memory computation so as to avoid the notebook from running out of memory.</li>\n</ul>",
      "rawMarkdown": "I have faced similar issues when i work with kaggle notebooks, i followed these measures so as to save time when running a script:\n- Pickling model and reusing it rather than training the model with the same dataset over again\n- Constant version saves ( the datasets were however added readily from the previous version itself)\n- Following a code structure which requires lesser memory computation so as to avoid the notebook from running out of memory.",
      "votes": null
    },
    {
      "id": "1654952",
      "postDate": "01/19/2022 01:39:40",
      "content": "<p>Thx guys!!</p>",
      "rawMarkdown": "Thx guys!!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1653510,
      "author_name": "vincentwainman",
      "author_url": "",
      "post_date": "01/17/2022 15:44:43",
      "content": "<p>Instead of dealing with Kaggle's interface, you could try working with your offline environment using Jupyter notebook and save/download your progress by pickling or CSV, and then uploading your completed file on here once it's done. </p>\n<p>This isn't really possible to do on here if you're constantly restarting the kernel; it's just going to dump any saved files and bring you back to square one. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1653543,
      "author_name": "gova26",
      "author_url": "",
      "post_date": "01/17/2022 16:20:36",
      "content": "<p>I have faced similar issues when i work with kaggle notebooks, i followed these measures so as to save time when running a script:</p>\n<ul>\n<li>Pickling model and reusing it rather than training the model with the same dataset over again</li>\n<li>Constant version saves ( the datasets were however added readily from the previous version itself)</li>\n<li>Following a code structure which requires lesser memory computation so as to avoid the notebook from running out of memory.</li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1654952,
      "author_name": "vweimer",
      "author_url": "",
      "post_date": "01/19/2022 01:39:40",
      "content": "<p>Thx guys!!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1652841": "Hi all! \n\nI'm relatively new to Kaggle doing my first real competition for Crypto Forecasting. My main issue/annoyance I'm experiencing is that I constantly have to rerun my notebook from scratch (i.e., all the way from doing the imports and importing the raw datasets and go from there). And this takes a lot of my time. \n\nEven when I save versions of my notebook, I still need to run from the beginning since it didn't save anything in the process. \n\nMy question(s) is/are:\n* How do you deal with this and what measures to take to avoid this is possible?\n* Is there a way so that I can save Dataframes/data created (my features dataset) and don't have to run all the previous cells before it?\n\nThanks in advance!!\nVincent",
    "1653510": "Instead of dealing with Kaggle's interface, you could try working with your offline environment using Jupyter notebook and save/download your progress by pickling or CSV, and then uploading your completed file on here once it's done. \n\nThis isn't really possible to do on here if you're constantly restarting the kernel; it's just going to dump any saved files and bring you back to square one.",
    "1653543": "I have faced similar issues when i work with kaggle notebooks, i followed these measures so as to save time when running a script:\n- Pickling model and reusing it rather than training the model with the same dataset over again\n- Constant version saves ( the datasets were however added readily from the previous version itself)\n- Following a code structure which requires lesser memory computation so as to avoid the notebook from running out of memory.",
    "1654952": "Thx guys!!"
  },
  "source": "meta"
}