{
  "id": 458274,
  "title": "[Solved]Keeping Submission Score Error",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/458274",
  "author_name": "hyd",
  "post_date": "2023-11-29T03:47:48",
  "votes": 6,
  "comment_count": 14,
  "views": 0,
  "content": "<h1>update:</h1>\n<p>Finally succeeded after add this line.<br>\n<code>\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n</code></p>\n<h1>original post</h1>\n<p>I am pertty sure it's not memroy error,  What are the hidden limitations of the submission file that are not said in data tab?</p>\n<p>tried:</p>\n<ol>\n<li>head(100)  to pass row limitation</li>\n<li>drop_duplicates(subset=['series_id', 'step'])</li>\n<li>remove all other files but keep submission.csv</li>\n<li>sub['score'] = sub['score'].fillna(0.0)</li>\n<li>sort_values</li>\n<li>infer one by one to avoid memory error. test_df = all_test_df.filter(pl.col('series_id').is_in(this_series_ids))</li>\n</ol>\n<pre><code>sub_df = my_inference()\nsub_df = sub_df.drop_duplicates(subset=[, ])\nsub_df = sub_df.sort_values(=[, ]).reset_index(=)\nsub_df[] = np.arange(len(sub_df))\nsub_df[] = sub_df[].astype()\nsub_df[] = sub_df[].astype(str)\nsub_df[] = sub_df[].fillna().astype()\n\n#  sub_df[].().sum()==\nsub_df = sub_df.head().reset_index(=)\nprint(sub_df.describe())\nprint(sub_df.().sum())\nsub_df[[, , , , ]].to_csv(, =)\n</code></pre>\n<p>😅used 9 submission chance still can't figure out why.<br>\nWho can help me😭</p>",
  "messages": [
    {
      "id": 2542069,
      "postDate": "2023-11-29T03:47:48Z",
      "content": "<h1>update:</h1>\n<p>Finally succeeded after add this line.<br>\n<code>\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n</code></p>\n<h1>original post</h1>\n<p>I am pertty sure it's not memroy error,  What are the hidden limitations of the submission file that are not said in data tab?</p>\n<p>tried:</p>\n<ol>\n<li>head(100)  to pass row limitation</li>\n<li>drop_duplicates(subset=['series_id', 'step'])</li>\n<li>remove all other files but keep submission.csv</li>\n<li>sub['score'] = sub['score'].fillna(0.0)</li>\n<li>sort_values</li>\n<li>infer one by one to avoid memory error. test_df = all_test_df.filter(pl.col('series_id').is_in(this_series_ids))</li>\n</ol>\n<pre><code>sub_df = my_inference()\nsub_df = sub_df.drop_duplicates(subset=[, ])\nsub_df = sub_df.sort_values(=[, ]).reset_index(=)\nsub_df[] = np.arange(len(sub_df))\nsub_df[] = sub_df[].astype()\nsub_df[] = sub_df[].astype(str)\nsub_df[] = sub_df[].fillna().astype()\n\n#  sub_df[].().sum()==\nsub_df = sub_df.head().reset_index(=)\nprint(sub_df.describe())\nprint(sub_df.().sum())\nsub_df[[, , , , ]].to_csv(, =)\n</code></pre>\n<p>😅used 9 submission chance still can't figure out why.<br>\nWho can help me😭</p>",
      "rawMarkdown": "# update:\nFinally succeeded after add this line.\n`\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n`\n\n\n\n\n# original post\nI am pertty sure it's not memroy error,  What are the hidden limitations of the submission file that are not said in data tab?\n\ntried:\n1.  head(100)  to pass row limitation\n2. drop_duplicates(subset=['series_id', 'step'])\n3. remove all other files but keep submission.csv\n4. sub['score'] = sub['score'].fillna(0.0)\n5. sort_values\n6. infer one by one to avoid memory error. test_df = all_test_df.filter(pl.col('series_id').is_in(this_series_ids))\n```\nsub_df = my_inference()\nsub_df = sub_df.drop_duplicates(subset=['series_id', 'step'])\nsub_df = sub_df.sort_values(by=['series_id', 'step']).reset_index(drop=True)\nsub_df['row_id'] = np.arange(len(sub_df))\nsub_df['step'] = sub_df['step'].astype(int)\nsub_df['event'] = sub_df['event'].astype(str)\nsub_df['score'] = sub_df['score'].fillna(0.0).astype('float32')\n\n# assert sub_df['score'].isnull().sum()==0.\nsub_df = sub_df.head(100).reset_index(drop=True)\nprint(sub_df.describe())\nprint(sub_df.isnull().sum())\nsub_df[['row_id', 'series_id', 'step', 'event', 'score']].to_csv('submission.csv', index=None)\n```\n\n😅used 9 submission chance still can't figure out why.\nWho can help me😭\n\n\n\n",
      "votes": 6
    },
    {
      "id": 2542102,
      "postDate": "2023-11-29T04:34:17.660Z",
      "content": "<p>May it be due to the large test data? I think you can choose 200 series train data to run inference to see where is the problem. Good luck!</p>",
      "rawMarkdown": "May it be due to the large test data? I think you can choose 200 series train data to run inference to see where is the problem. Good luck!",
      "votes": 1,
      "replies": [
        {
          "id": 2542105,
          "postDate": "2023-11-29T04:42:29.030Z",
          "content": "<p>Not really. My code infer one by one and pass the 200 series training data. It shows submission score error  but not memory error.<br>\nThanks for your help!</p>",
          "rawMarkdown": "Not really. My code infer one by one and pass the 200 series training data. It shows submission score error  but not memory error.\nThanks for your help!"
        }
      ]
    },
    {
      "id": 2544323,
      "postDate": "2023-11-30T18:42:10.400Z",
      "content": "<p>The same thing was happening to me also.  </p>\n<ol>\n<li>Successful submission requires that you filter out the starting step and the ending step of an event for a series.</li>\n<li>Events are a pair happening consecutively. Generally not more than 1 onset and 1 wakeup event per day but not limited. onset can be on the previous day but its pair wakeup can be on the next day.</li>\n<li>Make sure for any day if the model is not predicting onset and wakeup you don't have to force a value those could be days when the device has not been worn and you can remove those days from submission.</li>\n<li>Similarly if a series has no event according to the model you can drop that series.</li>\n<li>NaN are not allowed in submission df.</li>\n<li>The score column should represent the confidence of the predicted event between 0 and 1.</li>\n<li>row_id will be unique for each row irrespective of series.</li>\n<li>sub_df['event'] will have either \"onset\" or \"wakeup\".</li>\n</ol>\n<p>hope this helps and correct me otherwise.</p>",
      "rawMarkdown": "The same thing was happening to me also.  \n\n1. Successful submission requires that you filter out the starting step and the ending step of an event for a series.\n2. Events are a pair happening consecutively. Generally not more than 1 onset and 1 wakeup event per day but not limited. onset can be on the previous day but its pair wakeup can be on the next day.\n3. Make sure for any day if the model is not predicting onset and wakeup you don't have to force a value those could be days when the device has not been worn and you can remove those days from submission.\n4. Similarly if a series has no event according to the model you can drop that series.\n4. NaN are not allowed in submission df.\n5. The score column should represent the confidence of the predicted event between 0 and 1.\n6. row_id will be unique for each row irrespective of series.\n7. sub_df['event'] will have either \"onset\" or \"wakeup\".\n\nhope this helps and correct me otherwise.",
      "votes": -2
    },
    {
      "id": 2542323,
      "postDate": "2023-11-29T07:32:20.227Z",
      "content": "<p>I also need help regarding this submission error, Tried to reduce the number of prediction also not working, copied public notebook code for my submission format still submission scoring error. Something is not right but using scoring function works which really bothers me. </p>",
      "rawMarkdown": "I also need help regarding this submission error, Tried to reduce the number of prediction also not working, copied public notebook code for my submission format still submission scoring error. Something is not right but using scoring function works which really bothers me. ",
      "replies": [
        {
          "id": 2542337,
          "postDate": "2023-11-29T07:44:16.780Z",
          "content": "<p>You can try like this:<br>\n<code>sub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)</code><br>\nand make sure:<br>\nsubmission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) &lt; certain value e.g. 500,000</p>",
          "rawMarkdown": "You can try like this:\n`sub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)`\nand make sure:\nsubmission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) < certain value e.g. 500,000\n",
          "votes": 1,
          "replies": [
            {
              "id": 2542785,
              "postDate": "2023-11-29T14:06:51.233Z",
              "content": "<p>Still hit error</p>",
              "rawMarkdown": "Still hit error"
            },
            {
              "id": 2543332,
              "postDate": "2023-11-30T01:55:58.427Z",
              "content": "<p>😭That's so sad!</p>",
              "rawMarkdown": "😭That's so sad!",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 2542113,
      "postDate": "2023-11-29T05:02:27.763Z",
      "content": "<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> hi, please take a look.<br>\nIs submission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) &lt; certain value e.g. 500,000?<br>\notherwise it will raise submission score error.</p>",
      "rawMarkdown": "@ryanholbrook hi, please take a look.\nIs submission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) < certain value e.g. 500,000?\notherwise it will raise submission score error.\n\n",
      "replies": [
        {
          "id": 2542178,
          "postDate": "2023-11-29T06:03:55.680Z",
          "content": "<p>Finally succeeded after add this line. It seems that's right!<br>\n<code>\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n</code></p>",
          "rawMarkdown": "Finally succeeded after add this line. It seems that's right!\n`\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n`",
          "votes": 3,
          "replies": [
            {
              "id": 2544021,
              "postDate": "2023-11-30T14:42:12.113Z",
              "content": "<p>wow, I presume you got single model at that high score. You're amazing </p>",
              "rawMarkdown": "wow, I presume you got single model at that high score. You're amazing "
            }
          ]
        }
      ]
    },
    {
      "id": 2546531,
      "postDate": "2023-12-02T14:11:44.783Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 2547926,
          "postDate": "2023-12-04T02:48:53.130Z",
          "content": "<p>Yes, I think total count less than 500,000 is ok. Maybe 1000,000 is also fine, but 2,000,000 will fail.</p>",
          "rawMarkdown": "Yes, I think total count less than 500,000 is ok. Maybe 1000,000 is also fine, but 2,000,000 will fail.",
          "replies": [
            {
              "id": 2547961,
              "postDate": "2023-12-04T03:47:13.300Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 2546491,
      "postDate": "2023-12-02T13:39:07.360Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2542102,
      "author_name": "Joseph Zhou",
      "author_url": "",
      "post_date": "2023-11-29T04:34:17.660000",
      "content": "<p>May it be due to the large test data? I think you can choose 200 series train data to run inference to see where is the problem. Good luck!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2542105,
          "author_name": "hyd",
          "author_url": "",
          "post_date": "2023-11-29T04:42:29.030000",
          "content": "<p>Not really. My code infer one by one and pass the 200 series training data. It shows submission score error  but not memory error.<br>\nThanks for your help!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2544323,
      "author_name": "Anand Tirkey",
      "author_url": "",
      "post_date": "2023-11-30T18:42:10.400000",
      "content": "<p>The same thing was happening to me also.  </p>\n<ol>\n<li>Successful submission requires that you filter out the starting step and the ending step of an event for a series.</li>\n<li>Events are a pair happening consecutively. Generally not more than 1 onset and 1 wakeup event per day but not limited. onset can be on the previous day but its pair wakeup can be on the next day.</li>\n<li>Make sure for any day if the model is not predicting onset and wakeup you don't have to force a value those could be days when the device has not been worn and you can remove those days from submission.</li>\n<li>Similarly if a series has no event according to the model you can drop that series.</li>\n<li>NaN are not allowed in submission df.</li>\n<li>The score column should represent the confidence of the predicted event between 0 and 1.</li>\n<li>row_id will be unique for each row irrespective of series.</li>\n<li>sub_df['event'] will have either \"onset\" or \"wakeup\".</li>\n</ol>\n<p>hope this helps and correct me otherwise.</p>",
      "votes": -2,
      "replies": []
    },
    {
      "id": 2542323,
      "author_name": "Ruby Hartono",
      "author_url": "",
      "post_date": "2023-11-29T07:32:20.227000",
      "content": "<p>I also need help regarding this submission error, Tried to reduce the number of prediction also not working, copied public notebook code for my submission format still submission scoring error. Something is not right but using scoring function works which really bothers me. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 2542337,
          "author_name": "hyd",
          "author_url": "",
          "post_date": "2023-11-29T07:44:16.780000",
          "content": "<p>You can try like this:<br>\n<code>sub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)</code><br>\nand make sure:<br>\nsubmission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) &lt; certain value e.g. 500,000</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2542785,
              "author_name": "Ruby Hartono",
              "author_url": "",
              "post_date": "2023-11-29T14:06:51.233000",
              "content": "<p>Still hit error</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2543332,
              "author_name": "hyd",
              "author_url": "",
              "post_date": "2023-11-30T01:55:58.427000",
              "content": "<p>😭That's so sad!</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2542113,
      "author_name": "hyd",
      "author_url": "",
      "post_date": "2023-11-29T05:02:27.763000",
      "content": "<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> hi, please take a look.<br>\nIs submission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) &lt; certain value e.g. 500,000?<br>\notherwise it will raise submission score error.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2542178,
          "author_name": "hyd",
          "author_url": "",
          "post_date": "2023-11-29T06:03:55.680000",
          "content": "<p>Finally succeeded after add this line. It seems that's right!<br>\n<code>\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n</code></p>",
          "votes": 3,
          "replies": [
            {
              "id": 2544021,
              "author_name": "yukiya",
              "author_url": "",
              "post_date": "2023-11-30T14:42:12.113000",
              "content": "<p>wow, I presume you got single model at that high score. You're amazing </p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2546531,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-12-02T14:11:44.783000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 2547926,
          "author_name": "hyd",
          "author_url": "",
          "post_date": "2023-12-04T02:48:53.130000",
          "content": "<p>Yes, I think total count less than 500,000 is ok. Maybe 1000,000 is also fine, but 2,000,000 will fail.</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2547961,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-12-04T03:47:13.300000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2546491,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-12-02T13:39:07.360000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2542069": "# update:\nFinally succeeded after add this line.\n`\nsub_df = sub_df.groupby(by=['series_id', 'event']).head(50).reset_index(drop=True)\n`\n\n\n\n\n# original post\nI am pertty sure it's not memroy error,  What are the hidden limitations of the submission file that are not said in data tab?\n\ntried:\n1.  head(100)  to pass row limitation\n2. drop_duplicates(subset=['series_id', 'step'])\n3. remove all other files but keep submission.csv\n4. sub['score'] = sub['score'].fillna(0.0)\n5. sort_values\n6. infer one by one to avoid memory error. test_df = all_test_df.filter(pl.col('series_id').is_in(this_series_ids))\n```\nsub_df = my_inference()\nsub_df = sub_df.drop_duplicates(subset=['series_id', 'step'])\nsub_df = sub_df.sort_values(by=['series_id', 'step']).reset_index(drop=True)\nsub_df['row_id'] = np.arange(len(sub_df))\nsub_df['step'] = sub_df['step'].astype(int)\nsub_df['event'] = sub_df['event'].astype(str)\nsub_df['score'] = sub_df['score'].fillna(0.0).astype('float32')\n\n# assert sub_df['score'].isnull().sum()==0.\nsub_df = sub_df.head(100).reset_index(drop=True)\nprint(sub_df.describe())\nprint(sub_df.isnull().sum())\nsub_df[['row_id', 'series_id', 'step', 'event', 'score']].to_csv('submission.csv', index=None)\n```\n\n😅used 9 submission chance still can't figure out why.\nWho can help me😭\n\n\n\n",
    "2542102": "May it be due to the large test data? I think you can choose 200 series train data to run inference to see where is the problem. Good luck!",
    "2544323": "The same thing was happening to me also.  \n\n1. Successful submission requires that you filter out the starting step and the ending step of an event for a series.\n2. Events are a pair happening consecutively. Generally not more than 1 onset and 1 wakeup event per day but not limited. onset can be on the previous day but its pair wakeup can be on the next day.\n3. Make sure for any day if the model is not predicting onset and wakeup you don't have to force a value those could be days when the device has not been worn and you can remove those days from submission.\n4. Similarly if a series has no event according to the model you can drop that series.\n4. NaN are not allowed in submission df.\n5. The score column should represent the confidence of the predicted event between 0 and 1.\n6. row_id will be unique for each row irrespective of series.\n7. sub_df['event'] will have either \"onset\" or \"wakeup\".\n\nhope this helps and correct me otherwise.",
    "2542323": "I also need help regarding this submission error, Tried to reduce the number of prediction also not working, copied public notebook code for my submission format still submission scoring error. Something is not right but using scoring function works which really bothers me. ",
    "2542113": "@ryanholbrook hi, please take a look.\nIs submission['series_id'].nunique() == test_series['series_id'].nunique() and len(submission) < certain value e.g. 500,000?\notherwise it will raise submission score error.\n\n",
    "2546531": "",
    "2546491": ""
  }
}