![\"me\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14322371%2F0d88f5e8f95fcdb64346515e69c4538a%2Finbox_2683702_6cd08ef904c00a43127b524078c54ee8_moe-tossing-barney-from-moes.png?generation=1695913445882180&alt=media\")
Are there any requirements for the format of the submission file? So far, I have not been able to create a Notebook with custom events which does not run into a \"Submission Scoring Error\".
\nThe only valid submission that worked was copying the sample_submission.csv:
\n pandas pd\n\ndf = pd.read_csv()\ndf.to_csv(, index=)\nArtifically creating a single event for the first series did not work:
\n pandas pd\n\ntest_series = pd.read_parquet()\n\nsubmission = pd.DataFrame({: [], : [test_series[].loc[]], : [], : , : []})\nsubmission.to_csv(, index=)\nRandomly creating events and creating the submission similar to public scoring notebooks also didn't work:
\n pandas pd\n numpy np\n\ntest_series = pd.read_parquet()\n\nevents = test_series\nevents[] = np.random.rand((test_series))\nevents[] = \nevents.loc[np.random.rand((test_series)) < , ] = \nevents.loc[np.random.rand((test_series)) > , ] = \n\nsubmission = events.loc[events[] != ][[,,,]].copy().reset_index(drop=).reset_index(names=)\nsubmission.to_csv(, index=)\nHave you encountered scoring issues before? If yes, what where the problems/how did you solve them?
\nTo all participants with valid submissions: How do you create your submission file? Do you enforce an equal number of onset and wakeup events? Do you check that every series has at least one event? Do you order the submissions by id and/or step?
Thanks in advance to everyone how can help me with this problem.
", "messages": [ { "id": "2446413", "postDate": "09/19/2023 12:29:22", "content": "Are there any requirements for the format of the submission file? So far, I have not been able to create a Notebook with custom events which does not run into a \"Submission Scoring Error\".
\nThe only valid submission that worked was copying the sample_submission.csv:
\n pandas pd\n\ndf = pd.read_csv()\ndf.to_csv(, index=)\nArtifically creating a single event for the first series did not work:
\n pandas pd\n\ntest_series = pd.read_parquet()\n\nsubmission = pd.DataFrame({: [], : [test_series[].loc[]], : [], : , : []})\nsubmission.to_csv(, index=)\nRandomly creating events and creating the submission similar to public scoring notebooks also didn't work:
\n pandas pd\n numpy np\n\ntest_series = pd.read_parquet()\n\nevents = test_series\nevents[] = np.random.rand((test_series))\nevents[] = \nevents.loc[np.random.rand((test_series)) < , ] = \nevents.loc[np.random.rand((test_series)) > , ] = \n\nsubmission = events.loc[events[] != ][[,,,]].copy().reset_index(drop=).reset_index(names=)\nsubmission.to_csv(, index=)\nHave you encountered scoring issues before? If yes, what where the problems/how did you solve them?
\nTo all participants with valid submissions: How do you create your submission file? Do you enforce an equal number of onset and wakeup events? Do you check that every series has at least one event? Do you order the submissions by id and/or step?
Thanks in advance to everyone how can help me with this problem.
", "rawMarkdown": "Are there any requirements for the format of the submission file? So far, I have not been able to create a Notebook with custom events which does not run into a \"Submission Scoring Error\".\n\nThe only valid submission that worked was copying the sample_submission.csv:\n```py\nimport pandas as pd\n\ndf = pd.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/sample_submission.csv')\ndf.to_csv('submission.csv', index=False)\n```\n\nArtifically creating a single event for the first series did not work:\n```py\nimport pandas as pd\n\ntest_series = pd.read_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet')\n\nsubmission = pd.DataFrame({'row_id': [0], 'series_id': [test_series['series_id'].loc[0]], 'step': [100], 'event': 'onset', 'score': [0.5]})\nsubmission.to_csv('submission.csv', index=False)\n```\n\nRandomly creating events and creating the submission similar to public scoring notebooks also didn't work:\n```py\nimport pandas as pd\nimport numpy as np\n\ntest_series = pd.read_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet')\n\nevents = test_series\nevents['score'] = np.random.rand(len(test_series))\nevents['event'] = 'no'\nevents.loc[np.random.rand(len(test_series)) < 0.01, 'event'] = 'onset'\nevents.loc[np.random.rand(len(test_series)) > 0.99, 'event'] = 'wakeup'\n\nsubmission = events.loc[events[\"event\"] != 'no'][[\"series_id\",\"step\",\"event\",\"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\nsubmission.to_csv('submission.csv', index=False)\n```\n\nHave you encountered scoring issues before? If yes, what where the problems/how did you solve them?\nTo all participants with valid submissions: How do you create your submission file? Do you enforce an equal number of onset and wakeup events? Do you check that every series has at least one event? Do you order the submissions by id and/or step?\n\nThanks in advance to everyone how can help me with this problem.", "votes": null }, { "id": "2446457", "postDate": "09/19/2023 12:54:25", "content": "Here is the code that works for me without any issue
\n ():\n lstCV = (df.series_id, df.smooth)\n lstPOI = []\n (c, v), g groupby(lstCV, cv: (cv[], cv[] != pd.isnull(cv[]))):\n llg = ( item g)\n v :\n lstPOI.extend([] * llg)\n :\n lstPOI.extend([] + (llg - ) * [] + [] llg > [])\n lstPOI\n\ntest = pd.read_parquet(TEST_FILE)\ntest = create_features(test)\ntest = test.drop(columns=[])\n\ntest[] = classifier.predict_proba(test[features].fillna())[:,]\ntest[] = classifier.predict_proba(test[features].fillna())[:,]\n\nsmoothing_length = * \ntest[] = test[].rolling(smoothing_length,center=).mean().fillna(method=).fillna(method=)\ntest[] = test[].rolling(smoothing_length,center=).mean().fillna(method=).fillna(method=)\ntest[] = test[].()\ntest[] = get_event(test)\n\ndf = test.loc[test[] != ][[,,,]].copy().reset_index(drop=).reset_index(names=)\ndf.to_csv(, index=)\nThat is my own submissions cell from my successfully submitted notebook .take a look at this maybe it could help.
\n\nsample_submission = test.loc[test[] != ][[, , , ]].copy().reset_index(drop=).reset_index(names=)\n\n\nsample_submission.to_csv(, index=)\nhttps://www.kaggle.com/code/zulqarnainali/predicator/notebook?scriptVersionId=143196242
", "rawMarkdown": "That is my own submissions cell from my successfully submitted notebook .take a look at this maybe it could help. \n```python\n\n# 📄 Create a sample_submission DataFrame by selecting relevant columns and resetting the index\nsample_submission = test.loc[test[\"event\"] != 0][[\"series_id\", \"step\", \"event\", \"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\n\n# 💾 Save the sample_submission DataFrame to a CSV file\nsample_submission.to_csv('submission.csv', index=False)\n```\n\n\n[https://www.kaggle.com/code/zulqarnainali/predicator/notebook?scriptVersionId=143196242](url)", "votes": null }, { "id": "2446674", "postDate": "09/19/2023 14:40:04", "content": "@zulqarnainali @anilreddyvv Those are the code snippets from several public notebooks I have already investigated. I am aware, that these work as they are. However, these do require a relatively clean prediction of awake/not awake classes. But my current model/system (deep CNN with PyTorch) does predict the probability of the events directly. Thus, I do not use the get_event function to transform my predictions but generate a score value for each step and event directly. Transforming this information into a submission file directly fails and I do not know why.
Thank you mate!!!!!! been struggling with submission scoring error for 2 days, since I was batch processing the test data and concat them, the row_id is inconsistent, and
\n\n\nreset_index(names=\"row_id\")
\n
saved me!!!!🫡
\n
Hi @anilreddyvv
\nMay ask what does represent \"df.smooth\" in get_event function????, because i am trying to create event column in the submission file and i don't know how to get it.
I have the same problem!
", "rawMarkdown": "I have the same problem!", "votes": null } ], "comments": [ { "id": 2446457, "author_name": "anilreddyvv", "author_url": "", "post_date": "09/19/2023 12:54:25", "content": "Here is the code that works for me without any issue
\n ():\n lstCV = (df.series_id, df.smooth)\n lstPOI = []\n (c, v), g groupby(lstCV, cv: (cv[], cv[] != pd.isnull(cv[]))):\n llg = ( item g)\n v :\n lstPOI.extend([] * llg)\n :\n lstPOI.extend([] + (llg - ) * [] + [] llg > [])\n lstPOI\n\ntest = pd.read_parquet(TEST_FILE)\ntest = create_features(test)\ntest = test.drop(columns=[])\n\ntest[] = classifier.predict_proba(test[features].fillna())[:,]\ntest[] = classifier.predict_proba(test[features].fillna())[:,]\n\nsmoothing_length = * \ntest[] = test[].rolling(smoothing_length,center=).mean().fillna(method=).fillna(method=)\ntest[] = test[].rolling(smoothing_length,center=).mean().fillna(method=).fillna(method=)\ntest[] = test[].()\ntest[] = get_event(test)\n\ndf = test.loc[test[] != ][[,,,]].copy().reset_index(drop=).reset_index(names=)\ndf.to_csv(, index=)\nHi @anilreddyvv
\nMay ask what does represent \"df.smooth\" in get_event function????, because i am trying to create event column in the submission file and i don't know how to get it.
That is my own submissions cell from my successfully submitted notebook .take a look at this maybe it could help.
\n\nsample_submission = test.loc[test[] != ][[, , , ]].copy().reset_index(drop=).reset_index(names=)\n\n\nsample_submission.to_csv(, index=)\nhttps://www.kaggle.com/code/zulqarnainali/predicator/notebook?scriptVersionId=143196242
", "votes": null, "replies": [ { "id": 2460182, "author_name": "yuchen2066", "author_url": "", "post_date": "09/28/2023 16:12:47", "content": "Thank you mate!!!!!! been struggling with submission scoring error for 2 days, since I was batch processing the test data and concat them, the row_id is inconsistent, and
\n\n\nreset_index(names=\"row_id\")
\n
saved me!!!!🫡
\n
@zulqarnainali @anilreddyvv Those are the code snippets from several public notebooks I have already investigated. I am aware, that these work as they are. However, these do require a relatively clean prediction of awake/not awake classes. But my current model/system (deep CNN with PyTorch) does predict the probability of the events directly. Thus, I do not use the get_event function to transform my predictions but generate a score value for each step and event directly. Transforming this information into a submission file directly fails and I do not know why.
I have the same problem!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2446413": "Are there any requirements for the format of the submission file? So far, I have not been able to create a Notebook with custom events which does not run into a \"Submission Scoring Error\".\n\nThe only valid submission that worked was copying the sample_submission.csv:\n```py\nimport pandas as pd\n\ndf = pd.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/sample_submission.csv')\ndf.to_csv('submission.csv', index=False)\n```\n\nArtifically creating a single event for the first series did not work:\n```py\nimport pandas as pd\n\ntest_series = pd.read_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet')\n\nsubmission = pd.DataFrame({'row_id': [0], 'series_id': [test_series['series_id'].loc[0]], 'step': [100], 'event': 'onset', 'score': [0.5]})\nsubmission.to_csv('submission.csv', index=False)\n```\n\nRandomly creating events and creating the submission similar to public scoring notebooks also didn't work:\n```py\nimport pandas as pd\nimport numpy as np\n\ntest_series = pd.read_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet')\n\nevents = test_series\nevents['score'] = np.random.rand(len(test_series))\nevents['event'] = 'no'\nevents.loc[np.random.rand(len(test_series)) < 0.01, 'event'] = 'onset'\nevents.loc[np.random.rand(len(test_series)) > 0.99, 'event'] = 'wakeup'\n\nsubmission = events.loc[events[\"event\"] != 'no'][[\"series_id\",\"step\",\"event\",\"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\nsubmission.to_csv('submission.csv', index=False)\n```\n\nHave you encountered scoring issues before? If yes, what where the problems/how did you solve them?\nTo all participants with valid submissions: How do you create your submission file? Do you enforce an equal number of onset and wakeup events? Do you check that every series has at least one event? Do you order the submissions by id and/or step?\n\nThanks in advance to everyone how can help me with this problem.", "2446457": "Here is the code that works for me without any issue\n\n```python\ndef get_event(df):\n lstCV = zip(df.series_id, df.smooth)\n lstPOI = []\n for (c, v), g in groupby(lstCV, lambda cv: (cv[0], cv[1] != 0 and not pd.isnull(cv[1]))):\n llg = sum(1 for item in g)\n if v is False:\n lstPOI.extend([0] * llg)\n else:\n lstPOI.extend(['onset'] + (llg - 2) * [0] + ['wakeup'] if llg > 1 else [0])\n return lstPOI\n\ntest = pd.read_parquet(TEST_FILE)\ntest = create_features(test)\ntest = test.drop(columns=['timestamp'])\n\ntest[\"not_awake\"] = classifier.predict_proba(test[features].fillna(0))[:,0]\ntest[\"awake\"] = classifier.predict_proba(test[features].fillna(0))[:,1]\n\nsmoothing_length = 2 * 231\ntest[\"score\"] = test[\"awake\"].rolling(smoothing_length,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\")\ntest[\"smooth\"] = test[\"not_awake\"].rolling(smoothing_length,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\")\ntest[\"smooth\"] = test[\"smooth\"].round()\ntest[\"event\"] = get_event(test)\n\ndf = test.loc[test[\"event\"] != 0][[\"series_id\",\"step\",\"event\",\"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\ndf.to_csv('submission.csv', index=False)\n```", "2446481": "That is my own submissions cell from my successfully submitted notebook .take a look at this maybe it could help. \n```python\n\n# 📄 Create a sample_submission DataFrame by selecting relevant columns and resetting the index\nsample_submission = test.loc[test[\"event\"] != 0][[\"series_id\", \"step\", \"event\", \"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\n\n# 💾 Save the sample_submission DataFrame to a CSV file\nsample_submission.to_csv('submission.csv', index=False)\n```\n\n\n[https://www.kaggle.com/code/zulqarnainali/predicator/notebook?scriptVersionId=143196242](url)", "2446674": "zulqarnainali @anilreddyvv Those are the code snippets from several public notebooks I have already investigated. I am aware, that these work as they are. However, these do require a relatively clean prediction of awake/not awake classes. But my current model/system (deep CNN with PyTorch) does predict the probability of the events directly. Thus, I do not use the `get_event` function to transform my predictions but generate a score value for each step and event directly. Transforming this information into a submission file directly fails and I do not know why.", "2460182": "Thank you mate!!!!!! been struggling with submission scoring error for 2 days, since I was batch processing the test data and concat them, the row_id is inconsistent, and\n\n >reset_index(names=\"row_id\")\n\nsaved me!!!!🫡\n\n", "2508428": "Hi @anilreddyvv \nMay ask what does represent \"df.smooth\" in get_event function????, because i am trying to create event column in the submission file and i don't know how to get it.", "3111680": "I have the same problem!" }, "source": "meta" }