{
  "id": 164107,
  "title": "Incomplete data download when using Kaggle api in Colab",
  "url": "/competitions/global-wheat-detection/discussion/164107",
  "author_name": "",
  "post_date": "2020-07-04T18:19:05.397245200Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I am trying to download data using Kaggle api into Colab environment for training. I used the following commands.\n<code>\n!mkdir competition_data\n!kaggle competitions download -p competition_data global-wheat-detection\n</code>\n<a href=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F00b449ffc7bf143dc4677a86e46d8b79%2FScreenshot%202020-07-04%20at%2011.41.15%20PM.png?generation=1593886580742927&amp;alt=media\">Screenshot!</a>\nWhen the command ran, it downloaded only 30 image files, sample_submission file and train.csv.zip file.\n<a href=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F0b8a49e0f6049109266dfd3d6906cc64%2FScreenshot%202020-07-04%20at%2011.41.47%20PM.png?generation=1593886679682455&amp;alt=media\">Downloaded File structure!</a></p>\n\n<p>I checked for Kaggle API version and it is Kaggle API 1.5.4.!</p>\n\n<p>Any help in fixing this issue is appreciated. Thanks.</p>",
  "messages": [
    {
      "id": "915434",
      "postDate": "07/04/2020 18:19:05",
      "content": "<p>I am trying to download data using Kaggle api into Colab environment for training. I used the following commands.\n<code>\n!mkdir competition_data\n!kaggle competitions download -p competition_data global-wheat-detection\n</code>\n<a href=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F00b449ffc7bf143dc4677a86e46d8b79%2FScreenshot%202020-07-04%20at%2011.41.15%20PM.png?generation=1593886580742927&amp;alt=media\">Screenshot!</a>\nWhen the command ran, it downloaded only 30 image files, sample_submission file and train.csv.zip file.\n<a href=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F0b8a49e0f6049109266dfd3d6906cc64%2FScreenshot%202020-07-04%20at%2011.41.47%20PM.png?generation=1593886679682455&amp;alt=media\">Downloaded File structure!</a></p>\n\n<p>I checked for Kaggle API version and it is Kaggle API 1.5.4.!</p>\n\n<p>Any help in fixing this issue is appreciated. Thanks.</p>",
      "rawMarkdown": "I am trying to download data using Kaggle api into Colab environment for training. I used the following commands.\n```\n!mkdir competition_data\n!kaggle competitions download -p competition_data global-wheat-detection\n```\n[Screenshot!](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F00b449ffc7bf143dc4677a86e46d8b79%2FScreenshot%202020-07-04%20at%2011.41.15%20PM.png?generation=1593886580742927&amp;alt=media)\nWhen the command ran, it downloaded only 30 image files, sample_submission file and train.csv.zip file.\n[Downloaded File structure!](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F0b8a49e0f6049109266dfd3d6906cc64%2FScreenshot%202020-07-04%20at%2011.41.47%20PM.png?generation=1593886679682455&amp;alt=media)\n\nI checked for Kaggle API version and it is Kaggle API 1.5.4.!\n\nAny help in fixing this issue is appreciated. Thanks.",
      "votes": null
    },
    {
      "id": "915459",
      "postDate": "07/04/2020 18:47:56",
      "content": "<p>You must update to Kaggle API 1.5.6.</p>",
      "rawMarkdown": "You must update to Kaggle API 1.5.6.",
      "votes": null
    },
    {
      "id": "915507",
      "postDate": "07/04/2020 19:34:14",
      "content": "<p>The problem I think is that there are two versions of Kaggle API installed in google colab for python2.7 and python3 separately. Another way to do it would be to import kaggle and do it using python so that it uses the python3 version automatically and downloads the zip file instead. This way you won't have to update kaggle.</p>\n\n<p><code>!kaggle</code>\n<code>!touch /root/.kaggle/kaggle.json</code>\n<code>!chmod 600 /root/.kaggle/kaggle.json</code>\n<code>!echo '{\"username\": \"YOUR_USERNAME\", \"key\": \"YOUR_API_KEY\"}' &gt; /root/.kaggle/kaggle.json</code>\n<code>import kaggle</code>\n<code>kaggle.api.authenticate()</code>\n<code>kaggle.api.competition_download_files(\"global-wheat-detection\", path=YOUR_PATH, quiet=False)</code></p>",
      "rawMarkdown": "The problem I think is that there are two versions of Kaggle API installed in google colab for python2.7 and python3 separately. Another way to do it would be to import kaggle and do it using python so that it uses the python3 version automatically and downloads the zip file instead. This way you won't have to update kaggle.\n\n`!kaggle`\n`!touch /root/.kaggle/kaggle.json`\n`!chmod 600 /root/.kaggle/kaggle.json`\n`!echo '{\"username\": \"YOUR_USERNAME\", \"key\": \"YOUR_API_KEY\"}' &gt; /root/.kaggle/kaggle.json`\n`import kaggle`\n`kaggle.api.authenticate()`\n`kaggle.api.competition_download_files(\"global-wheat-detection\", path=YOUR_PATH, quiet=False)`",
      "votes": null
    },
    {
      "id": "915517",
      "postDate": "07/04/2020 19:40:25",
      "content": "<p>To upgrade to 1.5.6, <code>pip install kaggle --upgrade</code> is not working.\nWe need to use the following command</p>\n\n<blockquote>\n  <p>!pip install --upgrade --force-reinstall --no-deps kaggle </p>\n</blockquote>",
      "rawMarkdown": "To upgrade to 1.5.6, `pip install kaggle --upgrade` is not working.\nWe need to use the following command\n \n&gt; !pip install --upgrade --force-reinstall --no-deps kaggle",
      "votes": null
    },
    {
      "id": "915518",
      "postDate": "07/04/2020 19:41:09",
      "content": "<p>Yeah. Issue is with the version. Thanks</p>",
      "rawMarkdown": "Yeah. Issue is with the version. Thanks",
      "votes": null
    },
    {
      "id": "915519",
      "postDate": "07/04/2020 19:44:04",
      "content": "<p>This worked. Thanks</p>",
      "rawMarkdown": "This worked. Thanks",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 915459,
      "author_name": "matthewmasters",
      "author_url": "",
      "post_date": "07/04/2020 18:47:56",
      "content": "<p>You must update to Kaggle API 1.5.6.</p>",
      "votes": null,
      "replies": [
        {
          "id": 915518,
          "author_name": "devipradeep",
          "author_url": "",
          "post_date": "07/04/2020 19:41:09",
          "content": "<p>Yeah. Issue is with the version. Thanks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 915507,
      "author_name": "jonykarki",
      "author_url": "",
      "post_date": "07/04/2020 19:34:14",
      "content": "<p>The problem I think is that there are two versions of Kaggle API installed in google colab for python2.7 and python3 separately. Another way to do it would be to import kaggle and do it using python so that it uses the python3 version automatically and downloads the zip file instead. This way you won't have to update kaggle.</p>\n\n<p><code>!kaggle</code>\n<code>!touch /root/.kaggle/kaggle.json</code>\n<code>!chmod 600 /root/.kaggle/kaggle.json</code>\n<code>!echo '{\"username\": \"YOUR_USERNAME\", \"key\": \"YOUR_API_KEY\"}' &gt; /root/.kaggle/kaggle.json</code>\n<code>import kaggle</code>\n<code>kaggle.api.authenticate()</code>\n<code>kaggle.api.competition_download_files(\"global-wheat-detection\", path=YOUR_PATH, quiet=False)</code></p>",
      "votes": null,
      "replies": [
        {
          "id": 915519,
          "author_name": "devipradeep",
          "author_url": "",
          "post_date": "07/04/2020 19:44:04",
          "content": "<p>This worked. Thanks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 915517,
      "author_name": "devipradeep",
      "author_url": "",
      "post_date": "07/04/2020 19:40:25",
      "content": "<p>To upgrade to 1.5.6, <code>pip install kaggle --upgrade</code> is not working.\nWe need to use the following command</p>\n\n<blockquote>\n  <p>!pip install --upgrade --force-reinstall --no-deps kaggle </p>\n</blockquote>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "915434": "I am trying to download data using Kaggle api into Colab environment for training. I used the following commands.\n```\n!mkdir competition_data\n!kaggle competitions download -p competition_data global-wheat-detection\n```\n[Screenshot!](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F00b449ffc7bf143dc4677a86e46d8b79%2FScreenshot%202020-07-04%20at%2011.41.15%20PM.png?generation=1593886580742927&amp;alt=media)\nWhen the command ran, it downloaded only 30 image files, sample_submission file and train.csv.zip file.\n[Downloaded File structure!](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1201938%2F0b8a49e0f6049109266dfd3d6906cc64%2FScreenshot%202020-07-04%20at%2011.41.47%20PM.png?generation=1593886679682455&amp;alt=media)\n\nI checked for Kaggle API version and it is Kaggle API 1.5.4.!\n\nAny help in fixing this issue is appreciated. Thanks.",
    "915459": "You must update to Kaggle API 1.5.6.",
    "915507": "The problem I think is that there are two versions of Kaggle API installed in google colab for python2.7 and python3 separately. Another way to do it would be to import kaggle and do it using python so that it uses the python3 version automatically and downloads the zip file instead. This way you won't have to update kaggle.\n\n`!kaggle`\n`!touch /root/.kaggle/kaggle.json`\n`!chmod 600 /root/.kaggle/kaggle.json`\n`!echo '{\"username\": \"YOUR_USERNAME\", \"key\": \"YOUR_API_KEY\"}' &gt; /root/.kaggle/kaggle.json`\n`import kaggle`\n`kaggle.api.authenticate()`\n`kaggle.api.competition_download_files(\"global-wheat-detection\", path=YOUR_PATH, quiet=False)`",
    "915517": "To upgrade to 1.5.6, `pip install kaggle --upgrade` is not working.\nWe need to use the following command\n \n&gt; !pip install --upgrade --force-reinstall --no-deps kaggle",
    "915518": "Yeah. Issue is with the version. Thanks",
    "915519": "This worked. Thanks"
  },
  "source": "meta"
}