{
  "id": 23046,
  "title": "Submission file",
  "url": "/competitions/bosch-production-line-performance/discussion/23046",
  "author_name": "",
  "post_date": "2016-08-23T08:02:33.433Z",
  "votes": null,
  "comment_count": 8,
  "views": 786,
  "content": "<p>Hello,</p>\n\n<p>The sample csv submission file shows very very very many rows with with a response value.</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>Or do we have to submit a csv file with 1.18 million rows showing both zeros and ones?</p>\n\n<p>If so I am at a bit of a loss as to how to do that since my results are also in chunks of 50,000 rows at most, and my old clunky language and laptop can't handle anything longer than that at any given point in time. O.o</p>\n\n<p>Any tips?</p>\n\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "132167",
      "postDate": "08/23/2016 08:02:33",
      "content": "<p>Hello,</p>\n\n<p>The sample csv submission file shows very very very many rows with with a response value.</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>Or do we have to submit a csv file with 1.18 million rows showing both zeros and ones?</p>\n\n<p>If so I am at a bit of a loss as to how to do that since my results are also in chunks of 50,000 rows at most, and my old clunky language and laptop can't handle anything longer than that at any given point in time. O.o</p>\n\n<p>Any tips?</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "Hello,\r\n\r\nThe sample csv submission file shows very very very many rows with with a response value.\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\nOr do we have to submit a csv file with 1.18 million rows showing both zeros and ones?\r\n\r\nIf so I am at a bit of a loss as to how to do that since my results are also in chunks of 50,000 rows at most, and my old clunky language and laptop can't handle anything longer than that at any given point in time. O.o\r\n\r\nAny tips?\r\n\r\nThanks!",
      "votes": null
    },
    {
      "id": "132172",
      "postDate": "08/23/2016 08:41:23",
      "content": "<p>If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's</p>\n\n<p>See here: <a href=\"https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\">https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line</a></p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>",
      "rawMarkdown": "If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's\r\n\r\n\r\nSee here: https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\r\n\r\n    cat file1.csv file2.csv > outputfile.csv",
      "votes": null
    },
    {
      "id": "132175",
      "postDate": "08/23/2016 08:57:47",
      "content": "<p>[quote=Jed Santo Domingo;132167]</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>[/quote]</p>\n\n<p>This makes so much sense.</p>",
      "rawMarkdown": "[quote=Jed Santo Domingo;132167]\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\n\r\n[/quote]\r\n\r\n\r\nThis makes so much sense.",
      "votes": null
    },
    {
      "id": "132194",
      "postDate": "08/23/2016 12:30:31",
      "content": "<p>[quote=Simon Preis;132172]</p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>\n\n<p>[/quote]</p>\n\n<p>Or even</p>\n\n<pre><code>cat file1.csv.gz file2.csv.gz &gt; outputfile.csv.gz\n</code></pre>\n\n<p>if you need to work directly with gzipped files for some reason.</p>",
      "rawMarkdown": "[quote=Simon Preis;132172]\r\n\r\n    cat file1.csv file2.csv > outputfile.csv\r\n\r\n[/quote]\r\n\r\n\r\nOr even\r\n\r\n    cat file1.csv.gz file2.csv.gz > outputfile.csv.gz\r\n\r\nif you need to work directly with gzipped files for some reason.",
      "votes": null
    },
    {
      "id": "132195",
      "postDate": "08/23/2016 12:31:36",
      "content": "<p>[quote=Simon Preis;132172]</p>\n\n<p>If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's</p>\n\n<p>See here: <a href=\"https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\">https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line</a></p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>\n\n<p>[/quote]</p>\n\n<p>Awesome, thanks Simon!</p>",
      "rawMarkdown": "[quote=Simon Preis;132172]\r\n\r\nIf you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's\r\n\r\n\r\nSee here: https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\r\n\r\n    cat file1.csv file2.csv > outputfile.csv\r\n\r\n\r\n[/quote]\r\n\r\nAwesome, thanks Simon!",
      "votes": null
    },
    {
      "id": "132196",
      "postDate": "08/23/2016 12:32:14",
      "content": "<p>[quote=Toby Cheese;132194]</p>\n\n<p>Or even</p>\n\n<pre><code>cat file1.csv.gz file2.csv.gz &gt; outputfile.csv.gz\n</code></pre>\n\n<p>if you need to work directly with gzipped files for some reason.</p>\n\n<p>[/quote]</p>\n\n<p>thanks toby!</p>",
      "rawMarkdown": "[quote=Toby Cheese;132194]\r\n\r\nOr even\r\n\r\n    cat file1.csv.gz file2.csv.gz > outputfile.csv.gz\r\n\r\nif you need to work directly with gzipped files for some reason.\r\n\r\n[/quote]\r\n\r\nthanks toby!",
      "votes": null
    },
    {
      "id": "132197",
      "postDate": "08/23/2016 12:35:59",
      "content": "<p>[quote=inversion;132175]</p>\n\n<p>[quote=Jed Santo Domingo;132167]</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>[/quote]</p>\n\n<p>This makes so much sense.</p>\n\n<p>[/quote]</p>\n\n<p>Yup. We're submitting a million rows of zeros, and just a couple thousand ones. :P But we'll do as instructed.</p>\n\n<p>For a recently concluded Kaggle competition (Brachial Plexus nerve identification) they allowed for something similar, which essentially was to denote only the positives, and anything not mentioned is assumed as null.</p>",
      "rawMarkdown": "[quote=inversion;132175]\r\n\r\n[quote=Jed Santo Domingo;132167]\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\n\r\n[/quote]\r\n\r\n\r\nThis makes so much sense.\r\n\r\n[/quote]\r\n\r\nYup. We're submitting a million rows of zeros, and just a couple thousand ones. :P But we'll do as instructed.\r\n\r\nFor a recently concluded Kaggle competition (Brachial Plexus nerve identification) they allowed for something similar, which essentially was to denote only the positives, and anything not mentioned is assumed as null.",
      "votes": null
    },
    {
      "id": "132205",
      "postDate": "08/23/2016 14:46:16",
      "content": "<p>Most of our metrics don't accept a sparse submission format (with a few exceptions, like the Ultrasound competition where it would be impossible to use a full format) . It makes sense here because of the binary metric format combined with the imbalance, but it is not something we'd use often, since we're usually after dense probabilities.</p>\n\n<p>An unspoken benefit of requiring every value is that you can be sure we received your full and proper submission. There's no guesswork about whether you neglected to cat a file, or some esoteric parsing error where we only score part of what you uploaded but don't surface an error.</p>\n\n<p>Lastly, at the risk of stating the obvious, we do accept compressed files :)  We receive a surprising number of very large uncompressed submission files here at Kaggle...</p>",
      "rawMarkdown": "Most of our metrics don't accept a sparse submission format (with a few exceptions, like the Ultrasound competition where it would be impossible to use a full format) . It makes sense here because of the binary metric format combined with the imbalance, but it is not something we'd use often, since we're usually after dense probabilities.\r\n\r\nAn unspoken benefit of requiring every value is that you can be sure we received your full and proper submission. There's no guesswork about whether you neglected to cat a file, or some esoteric parsing error where we only score part of what you uploaded but don't surface an error.\r\n\r\nLastly, at the risk of stating the obvious, we do accept compressed files :)  We receive a surprising number of very large uncompressed submission files here at Kaggle...",
      "votes": null
    },
    {
      "id": "132206",
      "postDate": "08/23/2016 14:49:12",
      "content": "<p>Thanks for explaining William! :) yup will just do my best then</p>",
      "rawMarkdown": "Thanks for explaining William! :) yup will just do my best then",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 132172,
      "author_name": "simonp",
      "author_url": "",
      "post_date": "08/23/2016 08:41:23",
      "content": "<p>If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's</p>\n\n<p>See here: <a href=\"https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\">https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line</a></p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132175,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "08/23/2016 08:57:47",
      "content": "<p>[quote=Jed Santo Domingo;132167]</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>[/quote]</p>\n\n<p>This makes so much sense.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132194,
      "author_name": "tobycheese",
      "author_url": "",
      "post_date": "08/23/2016 12:30:31",
      "content": "<p>[quote=Simon Preis;132172]</p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>\n\n<p>[/quote]</p>\n\n<p>Or even</p>\n\n<pre><code>cat file1.csv.gz file2.csv.gz &gt; outputfile.csv.gz\n</code></pre>\n\n<p>if you need to work directly with gzipped files for some reason.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132195,
      "author_name": "jedsd3285",
      "author_url": "",
      "post_date": "08/23/2016 12:31:36",
      "content": "<p>[quote=Simon Preis;132172]</p>\n\n<p>If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's</p>\n\n<p>See here: <a href=\"https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\">https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line</a></p>\n\n<pre><code>cat file1.csv file2.csv &gt; outputfile.csv\n</code></pre>\n\n<p>[/quote]</p>\n\n<p>Awesome, thanks Simon!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132196,
      "author_name": "jedsd3285",
      "author_url": "",
      "post_date": "08/23/2016 12:32:14",
      "content": "<p>[quote=Toby Cheese;132194]</p>\n\n<p>Or even</p>\n\n<pre><code>cat file1.csv.gz file2.csv.gz &gt; outputfile.csv.gz\n</code></pre>\n\n<p>if you need to work directly with gzipped files for some reason.</p>\n\n<p>[/quote]</p>\n\n<p>thanks toby!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132197,
      "author_name": "jedsd3285",
      "author_url": "",
      "post_date": "08/23/2016 12:35:59",
      "content": "<p>[quote=inversion;132175]</p>\n\n<p>[quote=Jed Santo Domingo;132167]</p>\n\n<p>Is it possible to just submit a csv file for the IDs of the rows we consider as 1?</p>\n\n<p>[/quote]</p>\n\n<p>This makes so much sense.</p>\n\n<p>[/quote]</p>\n\n<p>Yup. We're submitting a million rows of zeros, and just a couple thousand ones. :P But we'll do as instructed.</p>\n\n<p>For a recently concluded Kaggle competition (Brachial Plexus nerve identification) they allowed for something similar, which essentially was to denote only the positives, and anything not mentioned is assumed as null.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132205,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "08/23/2016 14:46:16",
      "content": "<p>Most of our metrics don't accept a sparse submission format (with a few exceptions, like the Ultrasound competition where it would be impossible to use a full format) . It makes sense here because of the binary metric format combined with the imbalance, but it is not something we'd use often, since we're usually after dense probabilities.</p>\n\n<p>An unspoken benefit of requiring every value is that you can be sure we received your full and proper submission. There's no guesswork about whether you neglected to cat a file, or some esoteric parsing error where we only score part of what you uploaded but don't surface an error.</p>\n\n<p>Lastly, at the risk of stating the obvious, we do accept compressed files :)  We receive a surprising number of very large uncompressed submission files here at Kaggle...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 132206,
      "author_name": "jedsd3285",
      "author_url": "",
      "post_date": "08/23/2016 14:49:12",
      "content": "<p>Thanks for explaining William! :) yup will just do my best then</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "132167": "Hello,\r\n\r\nThe sample csv submission file shows very very very many rows with with a response value.\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\nOr do we have to submit a csv file with 1.18 million rows showing both zeros and ones?\r\n\r\nIf so I am at a bit of a loss as to how to do that since my results are also in chunks of 50,000 rows at most, and my old clunky language and laptop can't handle anything longer than that at any given point in time. O.o\r\n\r\nAny tips?\r\n\r\nThanks!",
    "132172": "If you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's\r\n\r\n\r\nSee here: https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\r\n\r\n    cat file1.csv file2.csv > outputfile.csv",
    "132175": "[quote=Jed Santo Domingo;132167]\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\n\r\n[/quote]\r\n\r\n\r\nThis makes so much sense.",
    "132194": "[quote=Simon Preis;132172]\r\n\r\n    cat file1.csv file2.csv > outputfile.csv\r\n\r\n[/quote]\r\n\r\n\r\nOr even\r\n\r\n    cat file1.csv.gz file2.csv.gz > outputfile.csv.gz\r\n\r\nif you need to work directly with gzipped files for some reason.",
    "132195": "[quote=Simon Preis;132172]\r\n\r\nIf you are only able to create results 50k observations at a time, a quick way would be to concatenate your resulting csv's\r\n\r\n\r\nSee here: https://www.quora.com/How-do-you-combine-multiple-csv-files-into-a-single-file-using-the-command-line\r\n\r\n    cat file1.csv file2.csv > outputfile.csv\r\n\r\n\r\n[/quote]\r\n\r\nAwesome, thanks Simon!",
    "132196": "[quote=Toby Cheese;132194]\r\n\r\nOr even\r\n\r\n    cat file1.csv.gz file2.csv.gz > outputfile.csv.gz\r\n\r\nif you need to work directly with gzipped files for some reason.\r\n\r\n[/quote]\r\n\r\nthanks toby!",
    "132197": "[quote=inversion;132175]\r\n\r\n[quote=Jed Santo Domingo;132167]\r\n\r\nIs it possible to just submit a csv file for the IDs of the rows we consider as 1?\r\n\r\n\r\n[/quote]\r\n\r\n\r\nThis makes so much sense.\r\n\r\n[/quote]\r\n\r\nYup. We're submitting a million rows of zeros, and just a couple thousand ones. :P But we'll do as instructed.\r\n\r\nFor a recently concluded Kaggle competition (Brachial Plexus nerve identification) they allowed for something similar, which essentially was to denote only the positives, and anything not mentioned is assumed as null.",
    "132205": "Most of our metrics don't accept a sparse submission format (with a few exceptions, like the Ultrasound competition where it would be impossible to use a full format) . It makes sense here because of the binary metric format combined with the imbalance, but it is not something we'd use often, since we're usually after dense probabilities.\r\n\r\nAn unspoken benefit of requiring every value is that you can be sure we received your full and proper submission. There's no guesswork about whether you neglected to cat a file, or some esoteric parsing error where we only score part of what you uploaded but don't surface an error.\r\n\r\nLastly, at the risk of stating the obvious, we do accept compressed files :)  We receive a surprising number of very large uncompressed submission files here at Kaggle...",
    "132206": "Thanks for explaining William! :) yup will just do my best then"
  },
  "source": "meta"
}