{ "id": 598873, "title": "Assuming 200Hz sampling rate?", "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/598873", "author_name": "", "post_date": "2025-08-13T07:41:52.215690600Z", "votes": 9, "comment_count": 2, "views": 0, "content": "

Hi all,

\n

Many notebooks assume a 200 Hz sampling rate in the following calculation.
\nHowever, judging by the length of each sequence, the provided data appears to be sampled at ~10 Hz.
\nWhen I set time_delta=1/10, my score drops compared to 1/200, and I’m curious why this happens.

\n
 (): \n     (rot_data, pd.DataFrame):\n        quat_values = rot_data[[, , , ]].values\n    :\n        quat_values = rot_data\n    num_samples = quat_values.shape[]\n    angular_vel = np.zeros((num_samples, ))\n     i  (num_samples - ):\n        q_t = quat_values[i]\n        q_t_plus_dt = quat_values[i+]\n         np.(np.isnan(q_t))  np.(np.isclose(q_t, ))  \\\n           np.(np.isnan(q_t_plus_dt))  np.(np.isclose(q_t_plus_dt, )):\n            \n        :\n            rot_t = R.from_quat(q_t)\n            rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n            delta_rot = rot_t.inv() * rot_t_plus_dt\n            angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n         ValueError:\n            \n     angular_vel\n
", "messages": [ { "id": "3268601", "postDate": "08/13/2025 07:41:52", "content": "

Hi all,

\n

Many notebooks assume a 200 Hz sampling rate in the following calculation.
\nHowever, judging by the length of each sequence, the provided data appears to be sampled at ~10 Hz.
\nWhen I set time_delta=1/10, my score drops compared to 1/200, and I’m curious why this happens.

\n
 (): \n     (rot_data, pd.DataFrame):\n        quat_values = rot_data[[, , , ]].values\n    :\n        quat_values = rot_data\n    num_samples = quat_values.shape[]\n    angular_vel = np.zeros((num_samples, ))\n     i  (num_samples - ):\n        q_t = quat_values[i]\n        q_t_plus_dt = quat_values[i+]\n         np.(np.isnan(q_t))  np.(np.isclose(q_t, ))  \\\n           np.(np.isnan(q_t_plus_dt))  np.(np.isclose(q_t_plus_dt, )):\n            \n        :\n            rot_t = R.from_quat(q_t)\n            rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n            delta_rot = rot_t.inv() * rot_t_plus_dt\n            angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n         ValueError:\n            \n     angular_vel\n
", "rawMarkdown": "Hi all,\n\nMany notebooks assume a 200 Hz sampling rate in the following calculation.\nHowever, judging by the length of each sequence, the provided data appears to be sampled at ~10 Hz.\nWhen I set time_delta=1/10, my score drops compared to 1/200, and I’m curious why this happens.\n\n\n```python\ndef calculate_angular_velocity_from_quat(rot_data, time_delta=1/200): # Assuming 200Hz sampling rate\n if isinstance(rot_data, pd.DataFrame):\n quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n else:\n quat_values = rot_data\n num_samples = quat_values.shape[0]\n angular_vel = np.zeros((num_samples, 3))\n for i in range(num_samples - 1):\n q_t = quat_values[i]\n q_t_plus_dt = quat_values[i+1]\n if np.all(np.isnan(q_t)) or np.all(np.isclose(q_t, 0)) or \\\n np.all(np.isnan(q_t_plus_dt)) or np.all(np.isclose(q_t_plus_dt, 0)):\n continue\n try:\n rot_t = R.from_quat(q_t)\n rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n delta_rot = rot_t.inv() * rot_t_plus_dt\n angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n except ValueError:\n pass\n return angular_vel\n```", "votes": null }, { "id": "3268742", "postDate": "08/13/2025 10:38:12", "content": "

The data was downsampled for sure, 10Hz is a reasonable guess for the value after downsampled (I really hope host of competitions in Kaggle would be more clear and straightforward regarding their preprocessing steps! I had the same frustration with GWI competition 2 months ago).
\nYour 'score drop' does not imply any meaningful physical true value- it's just a factor, NN respond different to different normalization factors. There is no real meaning to insert in this function time_delta=1e-8 or time_delta=1e8, you just need to make sure that the result is numbers that are easy for NN to digest- in this case probably 1/200 is better than 1/10 but it has nothing to do with real sampling values.

", "rawMarkdown": "The data was downsampled for sure, 10Hz is a reasonable guess for the value after downsampled (I really hope host of competitions in Kaggle would be more clear and straightforward regarding their preprocessing steps! I had the same frustration with GWI competition 2 months ago). \nYour 'score drop' does not imply any meaningful physical true value- it's just a factor, NN respond different to different normalization factors. There is no real meaning to insert in this function time_delta=1e-8 or time_delta=1e8, you just need to make sure that the result is numbers that are easy for NN to digest- in this case probably 1/200 is better than 1/10 but it has nothing to do with real sampling values.", "votes": null }, { "id": "3269084", "postDate": "08/13/2025 23:33:16", "content": "

Got it, thanks for the clear explanation!

", "rawMarkdown": "Got it, thanks for the clear explanation!", "votes": null } ], "comments": [ { "id": 3268742, "author_name": "shlomoron", "author_url": "", "post_date": "08/13/2025 10:38:12", "content": "

The data was downsampled for sure, 10Hz is a reasonable guess for the value after downsampled (I really hope host of competitions in Kaggle would be more clear and straightforward regarding their preprocessing steps! I had the same frustration with GWI competition 2 months ago).
\nYour 'score drop' does not imply any meaningful physical true value- it's just a factor, NN respond different to different normalization factors. There is no real meaning to insert in this function time_delta=1e-8 or time_delta=1e8, you just need to make sure that the result is numbers that are easy for NN to digest- in this case probably 1/200 is better than 1/10 but it has nothing to do with real sampling values.

", "votes": null, "replies": [ { "id": 3269084, "author_name": "tamadesu217", "author_url": "", "post_date": "08/13/2025 23:33:16", "content": "

Got it, thanks for the clear explanation!

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3268601": "Hi all,\n\nMany notebooks assume a 200 Hz sampling rate in the following calculation.\nHowever, judging by the length of each sequence, the provided data appears to be sampled at ~10 Hz.\nWhen I set time_delta=1/10, my score drops compared to 1/200, and I’m curious why this happens.\n\n\n```python\ndef calculate_angular_velocity_from_quat(rot_data, time_delta=1/200): # Assuming 200Hz sampling rate\n if isinstance(rot_data, pd.DataFrame):\n quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n else:\n quat_values = rot_data\n num_samples = quat_values.shape[0]\n angular_vel = np.zeros((num_samples, 3))\n for i in range(num_samples - 1):\n q_t = quat_values[i]\n q_t_plus_dt = quat_values[i+1]\n if np.all(np.isnan(q_t)) or np.all(np.isclose(q_t, 0)) or \\\n np.all(np.isnan(q_t_plus_dt)) or np.all(np.isclose(q_t_plus_dt, 0)):\n continue\n try:\n rot_t = R.from_quat(q_t)\n rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n delta_rot = rot_t.inv() * rot_t_plus_dt\n angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n except ValueError:\n pass\n return angular_vel\n```", "3268742": "The data was downsampled for sure, 10Hz is a reasonable guess for the value after downsampled (I really hope host of competitions in Kaggle would be more clear and straightforward regarding their preprocessing steps! I had the same frustration with GWI competition 2 months ago). \nYour 'score drop' does not imply any meaningful physical true value- it's just a factor, NN respond different to different normalization factors. There is no real meaning to insert in this function time_delta=1e-8 or time_delta=1e8, you just need to make sure that the result is numbers that are easy for NN to digest- in this case probably 1/200 is better than 1/10 but it has nothing to do with real sampling values.", "3269084": "Got it, thanks for the clear explanation!" }, "source": "meta" }