{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":102335,"databundleVersionId":12518947,"isSourceIdPinned":false,"sourceType":"competition"}],"dockerImageVersionId":31040,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Inference test set is tricky to simulate","metadata":{}},{"cell_type":"markdown","source":"- Dear New Competitors,\nI spent over a week trying to make my first submission and finally succeeded after making tens of attempts.\nWhat was frustrating is that the submission notebook threw exceptions, and one has to rely on guessing to figure out what goes wrong at inference time.\n\nI wrote this notebook to help you simulate the test set as closely as possible during inference time, so that you can debug and test your code effectively.\n\n","metadata":{}},{"cell_type":"markdown","source":"### Partitioning the Columns\n\n**Target Column:**  \n`gesture`\n\n**Common Extra Columns:**  \n`row_id`, `sequence_counter`\n\n**Rotation Columns:**  \n`rot_w`, `rot_x`, `rot_y`, `rot_z`\n\n**Accelerometer Columns:**  \n`acc_x`, `acc_y`, `acc_z`\n\n**Thermophile Columns:**  \n`thm_1`, `thm_2`, `thm_3`, `thm_4`, `thm_5`\n\n**Time-of-Flight (ToF) Columns:**  \n`tof_1_v1`, `tof_1_v2`, ..., `tof_5_v64`  \n<sub>(64 columns for each of the 5 sensors)</sub>\n\n**Grouping Column:**  \n`sequence_id`\n\n**Subject Column:**  \n`subject`\n\n\n\n**Categorical Columns to Encode:**  \n`sex`, `handedness`, `adult_child`","metadata":{}},{"cell_type":"code","source":"from pathlib import Path\nimport polars as pl\nimport numpy as np\nimport os\ntarget_col = \"gesture\"\ncommon_extra_cols = [\"row_id\",\"sequence_counter\"]\nrotation_cols = [ 'rot_w', 'rot_x',\n       'rot_y', 'rot_z']\naccelerometer_cols = ['acc_x', 'acc_y', 'acc_z']\n\nthermophile_columns = [f\"thm_{i}\" for i in range(1, 6)]\ntof_cols = [f\"tof_{i}_v{j}\" for i in range(1, 6) for j in range(64)]\ngrouping_col = \"sequence_id\"\nsubject_col = \"subject\"\nheat_col = \"total_heat\"\n# doing the fe\ncats_to_be = [\"sex\",\"handedness\",\"adult_child\"]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:03:07.383926Z","iopub.execute_input":"2025-06-20T13:03:07.384240Z","iopub.status.idle":"2025-06-20T13:03:07.390762Z","shell.execute_reply.started":"2025-06-20T13:03:07.384206Z","shell.execute_reply":"2025-06-20T13:03:07.389863Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#retrain the model from scratch, but first testing the model that I have\ntrain_df = pl.read_csv('/kaggle/input/cmi-detect-behavior-with-sensor-data/train.csv')\ntrain_demo_df = pl.read_csv('/kaggle/input/cmi-detect-behavior-with-sensor-data/train_demographics.csv')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:01:20.822261Z","iopub.execute_input":"2025-06-20T13:01:20.822591Z","iopub.status.idle":"2025-06-20T13:01:44.387577Z","shell.execute_reply.started":"2025-06-20T13:01:20.822568Z","shell.execute_reply":"2025-06-20T13:01:44.386536Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Start randomly nullifying the dfs' columns\n\nTo simulate missing data, we randomly nullify parts of the DataFrame's columns. This helps identify potential issues during inference.","metadata":{}},{"cell_type":"code","source":"def randomly_nullify_df(train_df,parts_count=2):\n    row_no = train_df.height //parts_count\n    random_indices = np.random.choice(np.arange(row_no))\n    all_cols = train_df.columns\n    for col in all_cols:\n        b_df = train_df.with_columns(pl.when(pl.arange(0,train_df.height).is_in(random_indices)).\n                                     then(None).otherwise(train_df[col]).alias(col))\n    return b_df\n        ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:01.783610Z","iopub.execute_input":"2025-06-20T13:02:01.783921Z","iopub.status.idle":"2025-06-20T13:02:01.789395Z","shell.execute_reply.started":"2025-06-20T13:02:01.783898Z","shell.execute_reply":"2025-06-20T13:02:01.788469Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"preprocessed_demog_df = randomly_nullify_df(train_demo_df)\npreprocessed_quantitative_df = randomly_nullify_df(train_df)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:04.103723Z","iopub.execute_input":"2025-06-20T13:02:04.104122Z","iopub.status.idle":"2025-06-20T13:02:05.527525Z","shell.execute_reply.started":"2025-06-20T13:02:04.104092Z","shell.execute_reply":"2025-06-20T13:02:05.526717Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def randomly_sample(train_df,demog_df,idx=None):\n    rows_no = train_df.height//2\n    subj_range = len(train_df[subject_col].unique())\n    random_idx = np.random.choice(np.arange(subj_range)) if idx is None else idx\n    subjc_id = demog_df[subject_col].to_list()[random_idx]\n    quant_seq_df = train_df.filter(pl.col(subject_col)==subjc_id)\n    demog_seq_df = demog_df.filter(pl.col(subject_col)==subjc_id)\n    return quant_seq_df,demog_seq_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:09.143130Z","iopub.execute_input":"2025-06-20T13:02:09.143497Z","iopub.status.idle":"2025-06-20T13:02:09.149274Z","shell.execute_reply.started":"2025-06-20T13:02:09.143473Z","shell.execute_reply":"2025-06-20T13:02:09.148105Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"##########\n","metadata":{}},{"cell_type":"markdown","source":"#### Randomly sampling from training data with null values \nWe randomly sample from the training data (with nullified values) to simulate inference conditions and test the robustness of the pipeline.","metadata":{}},{"cell_type":"code","source":"import kaggle_evaluation.cmi_inference_server","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:11.943607Z","iopub.execute_input":"2025-06-20T13:02:11.943904Z","iopub.status.idle":"2025-06-20T13:02:11.948094Z","shell.execute_reply.started":"2025-06-20T13:02:11.943881Z","shell.execute_reply":"2025-06-20T13:02:11.947063Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def predict(sequence,demographics) -> str:\n    return 'Text on phone'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:12.744764Z","iopub.execute_input":"2025-06-20T13:02:12.745048Z","iopub.status.idle":"2025-06-20T13:02:12.749381Z","shell.execute_reply.started":"2025-06-20T13:02:12.745026Z","shell.execute_reply":"2025-06-20T13:02:12.748610Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"normal_seq,demog_seq = randomly_sample(preprocessed_quantitative_df,preprocessed_demog_df)\nresult = predict(normal_seq,demog_seq)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:02:51.383882Z","iopub.execute_input":"2025-06-20T13:02:51.384149Z","iopub.status.idle":"2025-06-20T13:02:51.473750Z","shell.execute_reply.started":"2025-06-20T13:02:51.384130Z","shell.execute_reply":"2025-06-20T13:02:51.472922Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Next Steps\n- **Implement the `predict` function** to handle the simulated test data.\n- Use the randomly nullified and sampled data to ensure your pipeline is robust and can handle edge cases effectively.","metadata":{}},{"cell_type":"code","source":"inference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n    inference_server.serve()\nelse:\n    inference_server.run_local_gateway(\n        data_paths=(\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n        )\n    )","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T13:03:10.504842Z","iopub.execute_input":"2025-06-20T13:03:10.505529Z"}},"outputs":[],"execution_count":null}]}