\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3197853%2F675e7fa54abd45c66324e8545fb7e7ec%2FScreenshot%202025-06-11%20at%2015.14.31.png?generation=1749665705000245&alt=media\")
It seems there wasn't a clear sample submission file provided, or perhaps there was some uncertainty on what format or content was truly expected for the submission. This made it challenging to determine the exact requirements.
", "messages": [ { "id": "3214475", "postDate": "05/31/2025 16:08:15", "content": "It seems there wasn't a clear sample submission file provided, or perhaps there was some uncertainty on what format or content was truly expected for the submission. This made it challenging to determine the exact requirements.
", "rawMarkdown": "It seems there wasn't a clear sample submission file provided, or perhaps there was some uncertainty on what format or content was truly expected for the submission. This made it challenging to determine the exact requirements.", "votes": null }, { "id": "3214491", "postDate": "05/31/2025 16:45:45", "content": "You will get the submission sample from the API @alireza4465
", "rawMarkdown": "You will get the submission sample from the API @alireza4465", "votes": null }, { "id": "3214916", "postDate": "06/01/2025 09:11:29", "content": "I'm having trouble finding sample_submission.csv, too.
\n(I couldn't find it in 'cmi-detect-behavior-with-sensor-data.zip'.)
\nI'd really appreciate it if Mr. @ravi20076 or someone could help me.
We don't need sample_subsmission.csv, you just need to fill in the predict function.
\nIts input is a data from the test set, and its output is a string where you can add feature engineering and model inference code.
", "rawMarkdown": "demo code\n\nWe don't need sample_subsmission.csv, you just need to fill in the predict function.\n\nIts input is a data from the test set, and its output is a string where you can add feature engineering and model inference code.", "votes": null }, { "id": "3215010", "postDate": "06/01/2025 12:35:29", "content": "@yunsuxiaozi
\nI understand now! Thank you very much!!
I'm confused.
\nIn test.csv I see only 2 samples. I mean 2 sequences (which is 2 samples).
I have to predict gesture for only 2 samples? that's it? Create submission.csv with 2 rows and 2 columns?
from instructions:
\n\"
\nBinary F1 on whether the gesture is one of the target or non-target types.
\nMacro F1 on gesture, where all non-target sequences are collapsed into a single non_target class
\n\"
\nthis will be calculated by Kaggle, right? And if I understand Binary F1, I don't understand how Macro F1 works?
@dmitrikoltsov
\nI think there are only a part of the actual test data in test.csv.
\nI think Kaggle will calculate the value as you say. But Im sorry I cannot give a good answer to your question since I haven't understood the metric well, either..
No the test.csv is just a sample. During inference the test.csv will be replaced with the hidden test set, which has more sequences. @sumirekoga is right, there are only a part of the actual test data in test.csv.
Yes the metric will be calculated by Kaggle. You can follow the demo code for submission.
Great, thank you all.
\n`def my_model(sequence, demographics):
\n …..
\n return submission # polars dataframe
def predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\n submission = my_model(sequence, demographics)
\n return submission
inference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )`
For your predict function, it should return the predicted gesture in a string format. You should expect there are missing values in the hidden test set.
", "rawMarkdown": "For your predict function, it should return the predicted gesture in a string format. You should expect there are missing values in the hidden test set.", "votes": null }, { "id": "3218687", "postDate": "06/06/2025 15:10:28", "content": "you mean:
\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\ngesture = my_model(sequence, demographics)
\nreturn gesture # string
so, input to 'predict' function:
\nsequence: pl.DataFrame - is just one sample, dataframe slice with same sequence_id, and with all columns: row_id, sequence_type, sequence_id, sequence_counter, subject, orientation, behavior, phase, gesture, acc_x, acc_y … etc.
\nand demographics: pl.DataFrame - is dataframe with one row
Am I correct?
", "rawMarkdown": "you mean:\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\ngesture = my_model(sequence, demographics)\nreturn gesture # string\n\nso, input to 'predict' function:\nsequence: pl.DataFrame - is just one sample, dataframe slice with same sequence_id, and with all columns: row_id, sequence_type, sequence_id, sequence_counter, subject, orientation, behavior, phase, gesture, acc_x, acc_y ... etc.\nand demographics: pl.DataFrame - is dataframe with one row\n\nAm I correct?", "votes": null }, { "id": "3218724", "postDate": "06/06/2025 16:09:23", "content": "The function part is correct.
\nFor the input sequence, it is just one sample (one sequence with same sequence_id). But it should only include the columns in test.csv, it does not include sequence_type, behavior, phase, gesture.
got it. Thank you. Now it's clear 😀
", "rawMarkdown": "got it. Thank you. Now it's clear 😀", "votes": null }, { "id": "3219194", "postDate": "06/07/2025 09:26:33", "content": "Good luck with the competition. Happy kaggling!
", "rawMarkdown": "Good luck with the competition. Happy kaggling!", "votes": null }, { "id": "3222025", "postDate": "06/11/2025 18:08:24", "content": "I'm still confused.
\nI created model. Trained it.
\nI created predict function:
\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\n x = data_scalers_fillers.create_sensors_test_sample(sequence)
\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)
\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0) # shape: [1, T, D]
\n meta = torch.tensor(meta, dtype=torch.float32) # shape: [1, M]
\n model.eval()
\n with torch.no_grad():
\n outputs = model(x, meta, None)
\n pred = torch.argmax(outputs, dim=1).item()
\n return gesture_mapping_back[pred]
I ran this function on test.csv and it works.
\nand I have this submission code:
\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
if os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )
so, how should I submit? just run this portion of code? Or still have to click submit button? I clicked and got:
\n'''
\nCannot submit
\nYour Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version.
\n'''
\nWhat does it mean?
@dmitrikoltsov In the side bar on the right you must disable internet. Your notebook cannot access the internet when submitting:
\n
OK. Thank you.
\nI submitted and in my 'Submissions' tab I see:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4600911%2Ff2bf945c14ae85174018a4624f4dc396%2F01.png?generation=1749730232146332&alt=media\")
I try to check logs:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4600911%2F8b3a3c2dd2a7361e8c298b2024dea70d%2F02.png?generation=1749730254861786&alt=media\")
So, how can I find what is wrong?
\nThank you.
If it's not an internet or timeout issue, there may be problems with the function itself. Did you properly handle the missing data in the test set? While there are no missing entries in test.csv, there are null values present in the hidden test set.
", "rawMarkdown": "If it's not an internet or timeout issue, there may be problems with the function itself. Did you properly handle the missing data in the test set? While there are no missing entries in test.csv, there are null values present in the hidden test set.", "votes": null }, { "id": "3222801", "postDate": "06/12/2025 14:27:40", "content": "Yes, I handled missing values and during training there were no issues.
\nIn my notebook I have:
inference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )
I didn't create any submission file.
", "rawMarkdown": "Yes, I handled missing values and during training there were no issues.\nIn my notebook I have: \n- loading data\n- model, classes, functions \n- training loop (according to log it ran correctly)\n- and at the end I have:\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0).to(device) # shape: [1, T, D]\n meta = torch.tensor(meta, dtype=torch.float32).to(device) # shape: [1, M]\n model.eval()\n with torch.no_grad():\n outputs = model(x, meta, None)\n pred = torch.argmax(outputs, dim=1).item()\n return gesture_mapping_back[pred]\n\n\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n inference_server.serve()\nelse:\n inference_server.run_local_gateway(\n data_paths=(\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n )\n )\n\nI didn't create any submission file.", "votes": null }, { "id": "3222934", "postDate": "06/12/2025 17:06:35", "content": "In this competion, Kaggle evaluates your submission dynamically by sending data to your model one sequence at a time (online inference).so you're not submitting predictions , you're submitting a live inference server
\nthere's no static test set to predict all at once. Kaggle handles testing your code against hidden test data, calling predict() one sequence at a time
\nYour task here is to implement predict() and ensure it returns a string label quickly enough (less than 30 min)
", "rawMarkdown": "In this competion, Kaggle evaluates your submission dynamically by sending data to your model one sequence at a time (online inference).so you're not submitting predictions , you're submitting a live inference server \n\nthere's no static test set to predict all at once. Kaggle handles testing your code against hidden test data, calling predict() one sequence at a time\n\nYour task here is to implement predict() and ensure it returns a string label quickly enough (less than 30 min)", "votes": null }, { "id": "3225609", "postDate": "06/16/2025 16:27:55", "content": "@dmitrikoltsov
\nI'm a beginner, so I'm not sure I can give good advice.
\nBut I went through the same experience (my notebook ran fine on the public test data, but failed upon submission with that 'Notebook threw exception' message, even though I thought I handled null values carefully).
\nSo I'd like to share how I managed to get through it. There may be better solutions, but maybe it can help in some way.
First, I commented out most of my code and just made a constant prediction (like the template notebook).
\nThen I gradually added parts of the code back in, submitting each time to identify which part was causing the issue.
\nI also tried rewriting the same function in different ways.
It took me a long time (about three days), but now I can submit my predictions! Good luck!
", "rawMarkdown": "dmitrikoltsov \nI'm a beginner, so I'm not sure I can give good advice. \nBut I went through the same experience (my notebook ran fine on the public test data, but failed upon submission with that 'Notebook threw exception' message, even though I thought I handled null values carefully). \nSo I'd like to share how I managed to get through it. There may be better solutions, but maybe it can help in some way.\n\nFirst, I commented out most of my code and just made a constant prediction (like the template notebook). \nThen I gradually added parts of the code back in, submitting each time to identify which part was causing the issue. \nI also tried rewriting the same function in different ways. \n\nIt took me a long time (about three days), but now I can submit my predictions! Good luck!", "votes": null }, { "id": "3225623", "postDate": "06/16/2025 16:44:58", "content": "Thank you for sharing your experience! You took a very methodical approach to debugging, which is great advice for anyone facing similar issues. Gradually reintroducing code is a smart way to pinpoint the problem. Good luck with your future submissions!!
", "rawMarkdown": "Thank you for sharing your experience! You took a very methodical approach to debugging, which is great advice for anyone facing similar issues. Gradually reintroducing code is a smart way to pinpoint the problem. Good luck with your future submissions!!", "votes": null }, { "id": "3227045", "postDate": "06/18/2025 12:10:46", "content": "I thought there is a limit for amount of submissions, no?
", "rawMarkdown": "I thought there is a limit for amount of submissions, no?", "votes": null }, { "id": "3227062", "postDate": "06/18/2025 12:42:06", "content": "You can submit a maximum of five entries per day
", "rawMarkdown": "You can submit a maximum of five entries per day", "votes": null }, { "id": "3228027", "postDate": "06/19/2025 15:24:48", "content": "this is my function:
\n () -> :\n :\n ()\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n ()\n x = torch.tensor(x, dtype=torch.float32).unsqueeze().to(device)\n meta = torch.tensor(meta, dtype=torch.float32).to(device)\n ()\n model.()\n torch.no_grad():\n outputs = model(x, meta, )\n pred = torch.argmax(outputs, dim=).item()\n pred gesture_mapping_back:\n ValueError()\n gesture = gesture_mapping_back[pred]\n ()\n (gesture)\n Exception e:\n ()\n \nthis is from the log:
\n65.9s 24 from predict 0; (56, 336), (1, 8)
\n65.9s 25 from predict 1; (56, 332), (1, 7)
\n65.9s 26 from predict 2; torch.Size([1, 56, 332]), torch.Size([1, 7])
\n65.9s 27 pred: 16, gesture: Forehead - scratch
\n66.3s 28 from predict 0; (51, 336), (1, 8)
\n66.3s 29 from predict 1; (51, 332), (1, 7)
\n66.3s 30 from predict 2; torch.Size([1, 51, 332]), torch.Size([1, 7])
\n66.3s 31 pred: 9, gesture: Neck - pinch skin
so it returns 'guesture' as a string
\nand i still have:
\n\"Submission Format Error\"
I should return a string, right? I return a string. What format do they need?
", "rawMarkdown": "this is my function:\n\n```python\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n try:\n print(f'from predict 0; {sequence.shape}, {demographics.shape}')\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n print(f'from predict 1; {x.shape}, {meta.shape}')\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0).to(device)\n meta = torch.tensor(meta, dtype=torch.float32).to(device)\n print(f'from predict 2; {x.shape}, {meta.shape}')\n model.eval()\n with torch.no_grad():\n outputs = model(x, meta, None)\n pred = torch.argmax(outputs, dim=1).item()\n if pred not in gesture_mapping_back:\n raise ValueError(f\"Invalid prediction index: {pred}\")\n gesture = gesture_mapping_back[pred]\n print(f'pred: {pred}, gesture: {gesture}')\n return str(gesture)\n except Exception as e:\n print(f'Error during predict: {e}')\n return \"UNKNOWN\"\n```\n\n\n\nthis is from the log:\n65.9s\t24\tfrom predict 0; (56, 336), (1, 8)\n65.9s\t25\tfrom predict 1; (56, 332), (1, 7)\n65.9s\t26\tfrom predict 2; torch.Size([1, 56, 332]), torch.Size([1, 7])\n65.9s\t27\tpred: 16, gesture: Forehead - scratch\n66.3s\t28\tfrom predict 0; (51, 336), (1, 8)\n66.3s\t29\tfrom predict 1; (51, 332), (1, 7)\n66.3s\t30\tfrom predict 2; torch.Size([1, 51, 332]), torch.Size([1, 7])\n66.3s\t31\tpred: 9, gesture: Neck - pinch skin\n\nso it returns 'guesture' as a string\n\nand i still have:\n\"Submission Format Error\"\n\nI should return a string, right? I return a string. What format do they need?", "votes": null }, { "id": "3228316", "postDate": "06/20/2025 03:28:28", "content": "@dmitrikoltsov
\nHow about changing your code as below and submitting again?
\nIf it works, you can confirm that some error occurred inside the try block!
\nSubmission would fail if the predict function returns 'UNKNOWN' because it is not included in the gesture list.
return \"UNKNOWN\"
\n->
\nreturn \"Above ear - Pull hair\"
You will get the submission sample from the API @alireza4465
", "votes": null, "replies": [] }, { "id": 3214916, "author_name": "sumirekoga", "author_url": "", "post_date": "06/01/2025 09:11:29", "content": "I'm having trouble finding sample_submission.csv, too.
\n(I couldn't find it in 'cmi-detect-behavior-with-sensor-data.zip'.)
\nI'd really appreciate it if Mr. @ravi20076 or someone could help me.
We don't need sample_subsmission.csv, you just need to fill in the predict function.
\nIts input is a data from the test set, and its output is a string where you can add feature engineering and model inference code.
", "votes": null, "replies": [ { "id": 3215010, "author_name": "sumirekoga", "author_url": "", "post_date": "06/01/2025 12:35:29", "content": "@yunsuxiaozi
\nI understand now! Thank you very much!!
I'm confused.
\nIn test.csv I see only 2 samples. I mean 2 sequences (which is 2 samples).
I have to predict gesture for only 2 samples? that's it? Create submission.csv with 2 rows and 2 columns?
from instructions:
\n\"
\nBinary F1 on whether the gesture is one of the target or non-target types.
\nMacro F1 on gesture, where all non-target sequences are collapsed into a single non_target class
\n\"
\nthis will be calculated by Kaggle, right? And if I understand Binary F1, I don't understand how Macro F1 works?
@dmitrikoltsov
\nI think there are only a part of the actual test data in test.csv.
\nI think Kaggle will calculate the value as you say. But Im sorry I cannot give a good answer to your question since I haven't understood the metric well, either..
No the test.csv is just a sample. During inference the test.csv will be replaced with the hidden test set, which has more sequences. @sumirekoga is right, there are only a part of the actual test data in test.csv.
Yes the metric will be calculated by Kaggle. You can follow the demo code for submission.
Great, thank you all.
\n`def my_model(sequence, demographics):
\n …..
\n return submission # polars dataframe
def predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\n submission = my_model(sequence, demographics)
\n return submission
inference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )`
For your predict function, it should return the predicted gesture in a string format. You should expect there are missing values in the hidden test set.
", "votes": null, "replies": [ { "id": 3218687, "author_name": "dmitrikoltsov", "author_url": "", "post_date": "06/06/2025 15:10:28", "content": "you mean:
\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\ngesture = my_model(sequence, demographics)
\nreturn gesture # string
so, input to 'predict' function:
\nsequence: pl.DataFrame - is just one sample, dataframe slice with same sequence_id, and with all columns: row_id, sequence_type, sequence_id, sequence_counter, subject, orientation, behavior, phase, gesture, acc_x, acc_y … etc.
\nand demographics: pl.DataFrame - is dataframe with one row
Am I correct?
", "votes": null, "replies": [ { "id": 3218724, "author_name": "hearttt", "author_url": "", "post_date": "06/06/2025 16:09:23", "content": "The function part is correct.
\nFor the input sequence, it is just one sample (one sequence with same sequence_id). But it should only include the columns in test.csv, it does not include sequence_type, behavior, phase, gesture.
got it. Thank you. Now it's clear 😀
", "votes": null, "replies": [ { "id": 3219194, "author_name": "hearttt", "author_url": "", "post_date": "06/07/2025 09:26:33", "content": "Good luck with the competition. Happy kaggling!
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ] } ] } ] }, { "id": 3222025, "author_name": "dmitrikoltsov", "author_url": "", "post_date": "06/11/2025 18:08:24", "content": "I'm still confused.
\nI created model. Trained it.
\nI created predict function:
\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:
\n x = data_scalers_fillers.create_sensors_test_sample(sequence)
\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)
\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0) # shape: [1, T, D]
\n meta = torch.tensor(meta, dtype=torch.float32) # shape: [1, M]
\n model.eval()
\n with torch.no_grad():
\n outputs = model(x, meta, None)
\n pred = torch.argmax(outputs, dim=1).item()
\n return gesture_mapping_back[pred]
I ran this function on test.csv and it works.
\nand I have this submission code:
\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
if os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )
so, how should I submit? just run this portion of code? Or still have to click submit button? I clicked and got:
\n'''
\nCannot submit
\nYour Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version.
\n'''
\nWhat does it mean?
@dmitrikoltsov In the side bar on the right you must disable internet. Your notebook cannot access the internet when submitting:
\n
OK. Thank you.
\nI submitted and in my 'Submissions' tab I see:
\n
I try to check logs:
\n
So, how can I find what is wrong?
\nThank you.
If it's not an internet or timeout issue, there may be problems with the function itself. Did you properly handle the missing data in the test set? While there are no missing entries in test.csv, there are null values present in the hidden test set.
", "votes": null, "replies": [ { "id": 3222801, "author_name": "dmitrikoltsov", "author_url": "", "post_date": "06/12/2025 14:27:40", "content": "Yes, I handled missing values and during training there were no issues.
\nIn my notebook I have:
inference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)
\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):
\n inference_server.serve()
\nelse:
\n inference_server.run_local_gateway(
\n data_paths=(
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',
\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',
\n )
\n )
I didn't create any submission file.
", "votes": null, "replies": [ { "id": 3225609, "author_name": "sumirekoga", "author_url": "", "post_date": "06/16/2025 16:27:55", "content": "@dmitrikoltsov
\nI'm a beginner, so I'm not sure I can give good advice.
\nBut I went through the same experience (my notebook ran fine on the public test data, but failed upon submission with that 'Notebook threw exception' message, even though I thought I handled null values carefully).
\nSo I'd like to share how I managed to get through it. There may be better solutions, but maybe it can help in some way.
First, I commented out most of my code and just made a constant prediction (like the template notebook).
\nThen I gradually added parts of the code back in, submitting each time to identify which part was causing the issue.
\nI also tried rewriting the same function in different ways.
It took me a long time (about three days), but now I can submit my predictions! Good luck!
", "votes": null, "replies": [ { "id": 3225623, "author_name": "hearttt", "author_url": "", "post_date": "06/16/2025 16:44:58", "content": "Thank you for sharing your experience! You took a very methodical approach to debugging, which is great advice for anyone facing similar issues. Gradually reintroducing code is a smart way to pinpoint the problem. Good luck with your future submissions!!
", "votes": null, "replies": [] }, { "id": 3227045, "author_name": "dmitrikoltsov", "author_url": "", "post_date": "06/18/2025 12:10:46", "content": "I thought there is a limit for amount of submissions, no?
", "votes": null, "replies": [ { "id": 3227062, "author_name": "hearttt", "author_url": "", "post_date": "06/18/2025 12:42:06", "content": "You can submit a maximum of five entries per day
", "votes": null, "replies": [ { "id": 3228027, "author_name": "dmitrikoltsov", "author_url": "", "post_date": "06/19/2025 15:24:48", "content": "this is my function:
\n () -> :\n :\n ()\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n ()\n x = torch.tensor(x, dtype=torch.float32).unsqueeze().to(device)\n meta = torch.tensor(meta, dtype=torch.float32).to(device)\n ()\n model.()\n torch.no_grad():\n outputs = model(x, meta, )\n pred = torch.argmax(outputs, dim=).item()\n pred gesture_mapping_back:\n ValueError()\n gesture = gesture_mapping_back[pred]\n ()\n (gesture)\n Exception e:\n ()\n \nthis is from the log:
\n65.9s 24 from predict 0; (56, 336), (1, 8)
\n65.9s 25 from predict 1; (56, 332), (1, 7)
\n65.9s 26 from predict 2; torch.Size([1, 56, 332]), torch.Size([1, 7])
\n65.9s 27 pred: 16, gesture: Forehead - scratch
\n66.3s 28 from predict 0; (51, 336), (1, 8)
\n66.3s 29 from predict 1; (51, 332), (1, 7)
\n66.3s 30 from predict 2; torch.Size([1, 51, 332]), torch.Size([1, 7])
\n66.3s 31 pred: 9, gesture: Neck - pinch skin
so it returns 'guesture' as a string
\nand i still have:
\n\"Submission Format Error\"
I should return a string, right? I return a string. What format do they need?
", "votes": null, "replies": [ { "id": 3228316, "author_name": "sumirekoga", "author_url": "", "post_date": "06/20/2025 03:28:28", "content": "@dmitrikoltsov
\nHow about changing your code as below and submitting again?
\nIf it works, you can confirm that some error occurred inside the try block!
\nSubmission would fail if the predict function returns 'UNKNOWN' because it is not included in the gesture list.
return \"UNKNOWN\"
\n->
\nreturn \"Above ear - Pull hair\"
In this competion, Kaggle evaluates your submission dynamically by sending data to your model one sequence at a time (online inference).so you're not submitting predictions , you're submitting a live inference server
\nthere's no static test set to predict all at once. Kaggle handles testing your code against hidden test data, calling predict() one sequence at a time
\nYour task here is to implement predict() and ensure it returns a string label quickly enough (less than 30 min)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3214475": "It seems there wasn't a clear sample submission file provided, or perhaps there was some uncertainty on what format or content was truly expected for the submission. This made it challenging to determine the exact requirements.", "3214491": "You will get the submission sample from the API @alireza4465", "3214916": "I'm having trouble finding sample_submission.csv, too. \n(I couldn't find it in 'cmi-detect-behavior-with-sensor-data.zip'.)\nI'd really appreciate it if Mr. @ravi20076 or someone could help me.", "3214980": "demo code\n\nWe don't need sample_subsmission.csv, you just need to fill in the predict function.\n\nIts input is a data from the test set, and its output is a string where you can add feature engineering and model inference code.", "3215010": "yunsuxiaozi \nI understand now! Thank you very much!!", "3218145": "I'm confused.\nIn test.csv I see only 2 samples. I mean 2 sequences (which is 2 samples).\n\n1. I have to predict gesture for only 2 samples? that's it? Create submission.csv with 2 rows and 2 columns?\n\n2. from instructions:\n\"\nBinary F1 on whether the gesture is one of the target or non-target types.\nMacro F1 on gesture, where all non-target sequences are collapsed into a single non_target class\n\"\nthis will be calculated by Kaggle, right? And if I understand Binary F1, I don't understand how Macro F1 works?", "3218533": "dmitrikoltsov \nI think there are only a part of the actual test data in test.csv.\nI think Kaggle will calculate the value as you say. But Im sorry I cannot give a good answer to your question since I haven't understood the metric well, either..", "3218555": "dmitrikoltsov \n\n1. No the test.csv is just a sample. During inference the test.csv will be replaced with the hidden test set, which has more sequences. @sumirekoga is right, there are only a part of the actual test data in test.csv.\n\n2. Yes the metric will be calculated by Kaggle. You can follow the [demo code](https://www.kaggle.com/code/sohier/cmi-2025-demo-submission) for submission.", "3218599": "Great, thank you all.\n\n1. Am I correct that my code should look like:\n\n`def my_model(sequence, demographics):\n .....\n return submission # polars dataframe\n\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n submission = my_model(sequence, demographics)\n return submission\n\n\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n inference_server.serve()\nelse:\n inference_server.run_local_gateway(\n data_paths=(\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n )\n )`\n\n2. In test.csv (example) there are no missing values. Should we expect the same in real test.csv or should we expect missing values?", "3218613": "For your predict function, it should return the predicted gesture in a string format. You should expect there are missing values in the hidden test set.", "3218687": "you mean:\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\ngesture = my_model(sequence, demographics)\nreturn gesture # string\n\nso, input to 'predict' function:\nsequence: pl.DataFrame - is just one sample, dataframe slice with same sequence_id, and with all columns: row_id, sequence_type, sequence_id, sequence_counter, subject, orientation, behavior, phase, gesture, acc_x, acc_y ... etc.\nand demographics: pl.DataFrame - is dataframe with one row\n\nAm I correct?", "3218724": "The function part is correct.\nFor the input sequence, it is just one sample (one sequence with same sequence_id). But it should only include the columns in test.csv, it does not include sequence_type, behavior, phase, gesture.", "3218738": "got it. Thank you. Now it's clear 😀", "3219194": "Good luck with the competition. Happy kaggling!", "3222025": "I'm still confused.\nI created model. Trained it.\nI created predict function:\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0) # shape: [1, T, D]\n meta = torch.tensor(meta, dtype=torch.float32) # shape: [1, M]\n model.eval()\n with torch.no_grad():\n outputs = model(x, meta, None)\n pred = torch.argmax(outputs, dim=1).item()\n return gesture_mapping_back[pred]\n\nI ran this function on test.csv and it works.\n\nand I have this submission code:\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n inference_server.serve()\nelse:\n inference_server.run_local_gateway(\n data_paths=(\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n )\n )\n\nso, how should I submit? just run this portion of code? Or still have to click submit button? I clicked and got:\n'''\nCannot submit\nYour Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version.\n'''\nWhat does it mean?", "3222028": "dmitrikoltsov In the side bar on the right you must disable internet. Your notebook cannot access the internet when submitting:\n", "3222697": "OK. Thank you.\nI submitted and in my 'Submissions' tab I see:\n\n\nI try to check logs:\n\n\nSo, how can I find what is wrong? \nThank you.", "3222762": "If it's not an internet or timeout issue, there may be problems with the function itself. Did you properly handle the missing data in the test set? While there are no missing entries in test.csv, there are null values present in the hidden test set.", "3222801": "Yes, I handled missing values and during training there were no issues.\nIn my notebook I have: \n- loading data\n- model, classes, functions \n- training loop (according to log it ran correctly)\n- and at the end I have:\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0).to(device) # shape: [1, T, D]\n meta = torch.tensor(meta, dtype=torch.float32).to(device) # shape: [1, M]\n model.eval()\n with torch.no_grad():\n outputs = model(x, meta, None)\n pred = torch.argmax(outputs, dim=1).item()\n return gesture_mapping_back[pred]\n\n\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n inference_server.serve()\nelse:\n inference_server.run_local_gateway(\n data_paths=(\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n )\n )\n\nI didn't create any submission file.", "3222934": "In this competion, Kaggle evaluates your submission dynamically by sending data to your model one sequence at a time (online inference).so you're not submitting predictions , you're submitting a live inference server \n\nthere's no static test set to predict all at once. Kaggle handles testing your code against hidden test data, calling predict() one sequence at a time\n\nYour task here is to implement predict() and ensure it returns a string label quickly enough (less than 30 min)", "3225609": "dmitrikoltsov \nI'm a beginner, so I'm not sure I can give good advice. \nBut I went through the same experience (my notebook ran fine on the public test data, but failed upon submission with that 'Notebook threw exception' message, even though I thought I handled null values carefully). \nSo I'd like to share how I managed to get through it. There may be better solutions, but maybe it can help in some way.\n\nFirst, I commented out most of my code and just made a constant prediction (like the template notebook). \nThen I gradually added parts of the code back in, submitting each time to identify which part was causing the issue. \nI also tried rewriting the same function in different ways. \n\nIt took me a long time (about three days), but now I can submit my predictions! Good luck!", "3225623": "Thank you for sharing your experience! You took a very methodical approach to debugging, which is great advice for anyone facing similar issues. Gradually reintroducing code is a smart way to pinpoint the problem. Good luck with your future submissions!!", "3227045": "I thought there is a limit for amount of submissions, no?", "3227062": "You can submit a maximum of five entries per day", "3228027": "this is my function:\n\n```python\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n try:\n print(f'from predict 0; {sequence.shape}, {demographics.shape}')\n x = data_scalers_fillers.create_sensors_test_sample(sequence)\n meta = data_scalers_fillers.create_demographics_test_sample(demographics)\n print(f'from predict 1; {x.shape}, {meta.shape}')\n x = torch.tensor(x, dtype=torch.float32).unsqueeze(0).to(device)\n meta = torch.tensor(meta, dtype=torch.float32).to(device)\n print(f'from predict 2; {x.shape}, {meta.shape}')\n model.eval()\n with torch.no_grad():\n outputs = model(x, meta, None)\n pred = torch.argmax(outputs, dim=1).item()\n if pred not in gesture_mapping_back:\n raise ValueError(f\"Invalid prediction index: {pred}\")\n gesture = gesture_mapping_back[pred]\n print(f'pred: {pred}, gesture: {gesture}')\n return str(gesture)\n except Exception as e:\n print(f'Error during predict: {e}')\n return \"UNKNOWN\"\n```\n\n\n\nthis is from the log:\n65.9s\t24\tfrom predict 0; (56, 336), (1, 8)\n65.9s\t25\tfrom predict 1; (56, 332), (1, 7)\n65.9s\t26\tfrom predict 2; torch.Size([1, 56, 332]), torch.Size([1, 7])\n65.9s\t27\tpred: 16, gesture: Forehead - scratch\n66.3s\t28\tfrom predict 0; (51, 336), (1, 8)\n66.3s\t29\tfrom predict 1; (51, 332), (1, 7)\n66.3s\t30\tfrom predict 2; torch.Size([1, 51, 332]), torch.Size([1, 7])\n66.3s\t31\tpred: 9, gesture: Neck - pinch skin\n\nso it returns 'guesture' as a string\n\nand i still have:\n\"Submission Format Error\"\n\nI should return a string, right? I return a string. What format do they need?", "3228316": "dmitrikoltsov \nHow about changing your code as below and submitting again?\nIf it works, you can confirm that some error occurred inside the try block!\nSubmission would fail if the predict function returns 'UNKNOWN' because it is not included in the gesture list.\n - - - - - -\nreturn \"UNKNOWN\"\n-> \nreturn \"Above ear - Pull hair\"" }, "source": "meta" }