{
  "id": 406916,
  "title": "Zero score on \"dot\" graph?",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/406916",
  "author_name": "",
  "post_date": "2023-05-04T11:16:52.255444100Z",
  "votes": 3,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Have you tried to submit only one type of chart? I tried to find out which type of chart get the lowest score, so I tried to submit only four types of charts and keep the other one blank. Surprisingly, if I don't submit the \"dot\" type, I got the same score as all five types. I have checked my code thousands of times. <strong>So, is there really a \"dot\" graph in the test set? Or, when scoring, is our codes running in the same environment as the Kaggle notebook?</strong> That's so odd. Is there anyone facing the same problem?</p>",
  "messages": [
    {
      "id": "2245434",
      "postDate": "05/04/2023 11:16:52",
      "content": "<p>Have you tried to submit only one type of chart? I tried to find out which type of chart get the lowest score, so I tried to submit only four types of charts and keep the other one blank. Surprisingly, if I don't submit the \"dot\" type, I got the same score as all five types. I have checked my code thousands of times. <strong>So, is there really a \"dot\" graph in the test set? Or, when scoring, is our codes running in the same environment as the Kaggle notebook?</strong> That's so odd. Is there anyone facing the same problem?</p>",
      "rawMarkdown": "Have you tried to submit only one type of chart? I tried to find out which type of chart get the lowest score, so I tried to submit only four types of charts and keep the other one blank. Surprisingly, if I don't submit the \"dot\" type, I got the same score as all five types. I have checked my code thousands of times. **So, is there really a \"dot\" graph in the test set? Or, when scoring, is our codes running in the same environment as the Kaggle notebook?** That's so odd. Is there anyone facing the same problem?",
      "votes": null
    },
    {
      "id": "2245467",
      "postDate": "05/04/2023 11:49:32",
      "content": "<p>Yes the dot scores zero (with my exps too)  but maybe this is only in the public dataset ! so the \"zero\" might not be zero in private leaderboard </p>",
      "rawMarkdown": "Yes the dot scores zero (with my exps too)  but maybe this is only in the public dataset ! so the \"zero\" might not be zero in private leaderboard",
      "votes": null
    },
    {
      "id": "2245546",
      "postDate": "05/04/2023 12:42:34",
      "content": "<p>I got it. Thank you very much.  </p>",
      "rawMarkdown": "I got it. Thank you very much.",
      "votes": null
    },
    {
      "id": "2247182",
      "postDate": "05/05/2023 19:28:34",
      "content": "<p>did the same happen to you with horizontal bars?</p>",
      "rawMarkdown": "did the same happen to you with horizontal bars?",
      "votes": null
    },
    {
      "id": "2247191",
      "postDate": "05/05/2023 19:37:21",
      "content": "<p>For me <strong>yes</strong>, my horizontal bar and dot give zero, however, I think there are few \"very few\" horizontal bars on the public board since in general, I have an experiment that showed a slightly better score after doing some post-processing over horizontal bars. ( but it might not be enough to score (0.01 alone !)</p>\n<p>This makes me think that the private data is a challenge, I think there will be a good number of dot bars and a good number of horizontal with scatter also, which may cause a shakeup in the end! </p>",
      "rawMarkdown": "For me **yes**, my horizontal bar and dot give zero, however, I think there are few \"very few\" horizontal bars on the public board since in general, I have an experiment that showed a slightly better score after doing some post-processing over horizontal bars. ( but it might not be enough to score (0.01 alone !)\n\nThis makes me think that the private data is a challenge, I think there will be a good number of dot bars and a good number of horizontal with scatter also, which may cause a shakeup in the end!",
      "votes": null
    },
    {
      "id": "2247200",
      "postDate": "05/05/2023 19:49:43",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/asalhi\" target=\"_blank\">@asalhi</a>, I was wondering about this as my model combines Donut and an Object Detector. Since Donut was terrible on horizontal bars I only submit predictions of horizontal bars with the object detector, which I see it provides much better results. However, when submitting I saw a very little increase (just as you) and made me wonder if there are very little hbars or they are mostly in the private LB</p>",
      "rawMarkdown": "Thanks @asalhi, I was wondering about this as my model combines Donut and an Object Detector. Since Donut was terrible on horizontal bars I only submit predictions of horizontal bars with the object detector, which I see it provides much better results. However, when submitting I saw a very little increase (just as you) and made me wonder if there are very little hbars or they are mostly in the private LB",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2245467,
      "author_name": "asalhi",
      "author_url": "",
      "post_date": "05/04/2023 11:49:32",
      "content": "<p>Yes the dot scores zero (with my exps too)  but maybe this is only in the public dataset ! so the \"zero\" might not be zero in private leaderboard </p>",
      "votes": null,
      "replies": [
        {
          "id": 2245546,
          "author_name": "doventang",
          "author_url": "",
          "post_date": "05/04/2023 12:42:34",
          "content": "<p>I got it. Thank you very much.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2247182,
      "author_name": "alejopaullier",
      "author_url": "",
      "post_date": "05/05/2023 19:28:34",
      "content": "<p>did the same happen to you with horizontal bars?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2247191,
          "author_name": "asalhi",
          "author_url": "",
          "post_date": "05/05/2023 19:37:21",
          "content": "<p>For me <strong>yes</strong>, my horizontal bar and dot give zero, however, I think there are few \"very few\" horizontal bars on the public board since in general, I have an experiment that showed a slightly better score after doing some post-processing over horizontal bars. ( but it might not be enough to score (0.01 alone !)</p>\n<p>This makes me think that the private data is a challenge, I think there will be a good number of dot bars and a good number of horizontal with scatter also, which may cause a shakeup in the end! </p>",
          "votes": null,
          "replies": [
            {
              "id": 2247200,
              "author_name": "alejopaullier",
              "author_url": "",
              "post_date": "05/05/2023 19:49:43",
              "content": "<p>Thanks <a href=\"https://www.kaggle.com/asalhi\" target=\"_blank\">@asalhi</a>, I was wondering about this as my model combines Donut and an Object Detector. Since Donut was terrible on horizontal bars I only submit predictions of horizontal bars with the object detector, which I see it provides much better results. However, when submitting I saw a very little increase (just as you) and made me wonder if there are very little hbars or they are mostly in the private LB</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2245434": "Have you tried to submit only one type of chart? I tried to find out which type of chart get the lowest score, so I tried to submit only four types of charts and keep the other one blank. Surprisingly, if I don't submit the \"dot\" type, I got the same score as all five types. I have checked my code thousands of times. **So, is there really a \"dot\" graph in the test set? Or, when scoring, is our codes running in the same environment as the Kaggle notebook?** That's so odd. Is there anyone facing the same problem?",
    "2245467": "Yes the dot scores zero (with my exps too)  but maybe this is only in the public dataset ! so the \"zero\" might not be zero in private leaderboard",
    "2245546": "I got it. Thank you very much.",
    "2247182": "did the same happen to you with horizontal bars?",
    "2247191": "For me **yes**, my horizontal bar and dot give zero, however, I think there are few \"very few\" horizontal bars on the public board since in general, I have an experiment that showed a slightly better score after doing some post-processing over horizontal bars. ( but it might not be enough to score (0.01 alone !)\n\nThis makes me think that the private data is a challenge, I think there will be a good number of dot bars and a good number of horizontal with scatter also, which may cause a shakeup in the end!",
    "2247200": "Thanks @asalhi, I was wondering about this as my model combines Donut and an Object Detector. Since Donut was terrible on horizontal bars I only submit predictions of horizontal bars with the object detector, which I see it provides much better results. However, when submitting I saw a very little increase (just as you) and made me wonder if there are very little hbars or they are mostly in the private LB"
  },
  "source": "meta"
}