{
  "id": 407459,
  "title": "How to handle comma in data series?",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/407459",
  "author_name": "",
  "post_date": "2023-05-06T14:27:08.923682200Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>An example image is b9507ab4e9fc.jpg from the training images:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1708071%2F0c052f1ff86c1fc43f113cf8b3842dda%2Fb9507ab4e9fc.jpg?generation=1683382497730755&amp;alt=media\" alt=\"\"></p>\n<p>The ground truth annotations have a data series of:<br>\n<code>\"data-series\": [{\"x\": \"April 1, 2012\", \"y\": 19}, {\"x\": \"Aug. 14, 2013\", \"y\": 111}, {\"x\": \"Dec. 27, 2014\", \"y\": 314}, {\"x\": \"May 10, 2016\", \"y\": 481}]</code></p>\n<p>The proposed format for a row in <code>submission.csv</code> is:  <code>abc123_y,a;b;c;d,horizontal_bar</code>.  But, how to incorporate individual labels with commas?</p>\n<p>One possibility is to just include the commas while doing nothing else:<br>\n<code>b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar</code><br>\nIt seems possible this will not be parsed correctly by the scorer.  And what happens if a label also has a semicolon?</p>\n<p>Another possibility would be to do something like python's <code>csv.writer</code> and have a row like:<br>\n<code>b9507ab4e9fc_y,\"April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016\",horizontal_bar</code><br>\nBut is this read as hoped for by the scorer?</p>\n<p>(Sorry if this has already been discussed, but I'm getting \"Submission Scoring Error\" and guessing at possible reasons why.)</p>",
  "messages": [
    {
      "id": "2248148",
      "postDate": "05/06/2023 14:27:08",
      "content": "<p>An example image is b9507ab4e9fc.jpg from the training images:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1708071%2F0c052f1ff86c1fc43f113cf8b3842dda%2Fb9507ab4e9fc.jpg?generation=1683382497730755&amp;alt=media\" alt=\"\"></p>\n<p>The ground truth annotations have a data series of:<br>\n<code>\"data-series\": [{\"x\": \"April 1, 2012\", \"y\": 19}, {\"x\": \"Aug. 14, 2013\", \"y\": 111}, {\"x\": \"Dec. 27, 2014\", \"y\": 314}, {\"x\": \"May 10, 2016\", \"y\": 481}]</code></p>\n<p>The proposed format for a row in <code>submission.csv</code> is:  <code>abc123_y,a;b;c;d,horizontal_bar</code>.  But, how to incorporate individual labels with commas?</p>\n<p>One possibility is to just include the commas while doing nothing else:<br>\n<code>b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar</code><br>\nIt seems possible this will not be parsed correctly by the scorer.  And what happens if a label also has a semicolon?</p>\n<p>Another possibility would be to do something like python's <code>csv.writer</code> and have a row like:<br>\n<code>b9507ab4e9fc_y,\"April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016\",horizontal_bar</code><br>\nBut is this read as hoped for by the scorer?</p>\n<p>(Sorry if this has already been discussed, but I'm getting \"Submission Scoring Error\" and guessing at possible reasons why.)</p>",
      "rawMarkdown": "An example image is b9507ab4e9fc.jpg from the training images:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1708071%2F0c052f1ff86c1fc43f113cf8b3842dda%2Fb9507ab4e9fc.jpg?generation=1683382497730755&alt=media)\n\nThe ground truth annotations have a data series of:\n`\"data-series\": [{\"x\": \"April 1, 2012\", \"y\": 19}, {\"x\": \"Aug. 14, 2013\", \"y\": 111}, {\"x\": \"Dec. 27, 2014\", \"y\": 314}, {\"x\": \"May 10, 2016\", \"y\": 481}]`\n\nThe proposed format for a row in `submission.csv` is:  `abc123_y,a;b;c;d,horizontal_bar`.  But, how to incorporate individual labels with commas?\n\nOne possibility is to just include the commas while doing nothing else:\n`b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar`\nIt seems possible this will not be parsed correctly by the scorer.  And what happens if a label also has a semicolon?\n\nAnother possibility would be to do something like python's `csv.writer` and have a row like:\n`b9507ab4e9fc_y,\"April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016\",horizontal_bar`\nBut is this read as hoped for by the scorer?\n\n(Sorry if this has already been discussed, but I'm getting \"Submission Scoring Error\" and guessing at possible reasons why.)",
      "votes": null
    },
    {
      "id": "2271468",
      "postDate": "05/24/2023 00:10:11",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/pabyrnes\" target=\"_blank\">@pabyrnes</a>,</p>\n<p>What was the source of your problem?</p>",
      "rawMarkdown": "Hey @pabyrnes,\n\nWhat was the source of your problem?",
      "votes": null
    },
    {
      "id": "2272420",
      "postDate": "05/24/2023 13:23:45",
      "content": "<p>Not really a clear source, but I do not know if the above example would be scored as correct or not.  It's possible the scoring implementation would accept:</p>\n<pre><code>b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar\n</code></pre>\n<p>But, I don't believe this can be confirmed without the scorer source code.  The problem is that there is ambiguity about which comma separates the id and the data.  For instance, using <code>pandas.read_csv()</code> results in a row with <code>id =  2014;May 10</code>, <code>data_series = 2016</code>, and <code>chart_type =  horizontal_bar</code>.  (pandas splits into 4 sections and gives the row an id of <code>b9507ab4e9fc_y April 1  2012;Aug. 14  2013;Dec. 27</code>).  This would result in a score of 0.</p>\n<p>However, if the scorer is custom written to do something like consider the <code>id</code> to be everything before the first <code>,</code>, then the above line would get a score of 1.</p>\n<p>The question here is what the behavior of the (hidden) scoring algorithm is.</p>",
      "rawMarkdown": "Not really a clear source, but I do not know if the above example would be scored as correct or not.  It's possible the scoring implementation would accept:\n```\nb9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar\n```\nBut, I don't believe this can be confirmed without the scorer source code.  The problem is that there is ambiguity about which comma separates the id and the data.  For instance, using `pandas.read_csv()` results in a row with `id =  2014;May 10`, `data_series = 2016`, and `chart_type =  horizontal_bar`.  (pandas splits into 4 sections and gives the row an id of `b9507ab4e9fc_y April 1  2012;Aug. 14  2013;Dec. 27`).  This would result in a score of 0.\n\nHowever, if the scorer is custom written to do something like consider the `id` to be everything before the first `,`, then the above line would get a score of 1.\n\nThe question here is what the behavior of the (hidden) scoring algorithm is.",
      "votes": null
    },
    {
      "id": "2273029",
      "postDate": "05/25/2023 01:07:56",
      "content": "<p>Thanks. I assume they use standard csv reader that accepts double quotes.</p>",
      "rawMarkdown": "Thanks. I assume they use standard csv reader that accepts double quotes.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2271468,
      "author_name": "imoore",
      "author_url": "",
      "post_date": "05/24/2023 00:10:11",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/pabyrnes\" target=\"_blank\">@pabyrnes</a>,</p>\n<p>What was the source of your problem?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2272420,
          "author_name": "pabyrnes",
          "author_url": "",
          "post_date": "05/24/2023 13:23:45",
          "content": "<p>Not really a clear source, but I do not know if the above example would be scored as correct or not.  It's possible the scoring implementation would accept:</p>\n<pre><code>b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar\n</code></pre>\n<p>But, I don't believe this can be confirmed without the scorer source code.  The problem is that there is ambiguity about which comma separates the id and the data.  For instance, using <code>pandas.read_csv()</code> results in a row with <code>id =  2014;May 10</code>, <code>data_series = 2016</code>, and <code>chart_type =  horizontal_bar</code>.  (pandas splits into 4 sections and gives the row an id of <code>b9507ab4e9fc_y April 1  2012;Aug. 14  2013;Dec. 27</code>).  This would result in a score of 0.</p>\n<p>However, if the scorer is custom written to do something like consider the <code>id</code> to be everything before the first <code>,</code>, then the above line would get a score of 1.</p>\n<p>The question here is what the behavior of the (hidden) scoring algorithm is.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2273029,
              "author_name": "imoore",
              "author_url": "",
              "post_date": "05/25/2023 01:07:56",
              "content": "<p>Thanks. I assume they use standard csv reader that accepts double quotes.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2248148": "An example image is b9507ab4e9fc.jpg from the training images:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1708071%2F0c052f1ff86c1fc43f113cf8b3842dda%2Fb9507ab4e9fc.jpg?generation=1683382497730755&alt=media)\n\nThe ground truth annotations have a data series of:\n`\"data-series\": [{\"x\": \"April 1, 2012\", \"y\": 19}, {\"x\": \"Aug. 14, 2013\", \"y\": 111}, {\"x\": \"Dec. 27, 2014\", \"y\": 314}, {\"x\": \"May 10, 2016\", \"y\": 481}]`\n\nThe proposed format for a row in `submission.csv` is:  `abc123_y,a;b;c;d,horizontal_bar`.  But, how to incorporate individual labels with commas?\n\nOne possibility is to just include the commas while doing nothing else:\n`b9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar`\nIt seems possible this will not be parsed correctly by the scorer.  And what happens if a label also has a semicolon?\n\nAnother possibility would be to do something like python's `csv.writer` and have a row like:\n`b9507ab4e9fc_y,\"April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016\",horizontal_bar`\nBut is this read as hoped for by the scorer?\n\n(Sorry if this has already been discussed, but I'm getting \"Submission Scoring Error\" and guessing at possible reasons why.)",
    "2271468": "Hey @pabyrnes,\n\nWhat was the source of your problem?",
    "2272420": "Not really a clear source, but I do not know if the above example would be scored as correct or not.  It's possible the scoring implementation would accept:\n```\nb9507ab4e9fc_y,April 1, 2012;Aug. 14, 2013;Dec. 27, 2014;May 10, 2016,horizontal_bar\n```\nBut, I don't believe this can be confirmed without the scorer source code.  The problem is that there is ambiguity about which comma separates the id and the data.  For instance, using `pandas.read_csv()` results in a row with `id =  2014;May 10`, `data_series = 2016`, and `chart_type =  horizontal_bar`.  (pandas splits into 4 sections and gives the row an id of `b9507ab4e9fc_y April 1  2012;Aug. 14  2013;Dec. 27`).  This would result in a score of 0.\n\nHowever, if the scorer is custom written to do something like consider the `id` to be everything before the first `,`, then the above line would get a score of 1.\n\nThe question here is what the behavior of the (hidden) scoring algorithm is.",
    "2273029": "Thanks. I assume they use standard csv reader that accepts double quotes."
  },
  "source": "meta"
}