The competition text says
\n\n\nNote also that there is known sensor communication failure in this dataset, resulting in missing data from some sensors in some sequences.
\n
Are these some sensors fully or partially null through the sequence for these sequences?
", "messages": [ { "id": "3234275", "postDate": "06/27/2025 17:28:28", "content": "The competition text says
\n\n\nNote also that there is known sensor communication failure in this dataset, resulting in missing data from some sensors in some sequences.
\n
Are these some sensors fully or partially null through the sequence for these sequences?
", "rawMarkdown": "The competition text says\n>Note also that there is known sensor communication failure in this dataset, resulting in missing data from some sensors in some sequences.\n\nAre these some sensors **fully** or **partially** null through the sequence for these sequences?", "votes": null }, { "id": "3234317", "postDate": "06/27/2025 17:54:17", "content": "I would say a mixture of both. This code snippet gives this output for one of the inference examples!
\n i (, ):\n pixel_cols_tof = [ p ()]\n existing_tof_cols = [col col pixel_cols_tof col seq_df.columns]\n\n (existing_tof_cols) >= : \n tof_sensor_data = seq_df[existing_tof_cols].replace(-, np.nan)\n\n \n valid_data_ratio = (~tof_sensor_data.isna()).().() / ((tof_sensor_data) * (existing_tof_cols))\n\n valid_data_ratio > : \n seq_df[] = tof_sensor_data.mean(axis=, skipna=)\n seq_df[] = tof_sensor_data.std(axis=, skipna=)\n seq_df[] = tof_sensor_data.(axis=, skipna=)\n seq_df[] = tof_sensor_data.(axis=, skipna=)\n ()\n :\n seq_df[] = \n seq_df[] = \n seq_df[] = \n seq_df[] = \n ()\noutput log:
\n→ Processing TOF aggregations…
\n ⚠️ TOF sensor 1: Sparse data, using zeros
\n ⚠️ TOF sensor 2: Sparse data, using zeros
\n ✅ TOF sensor 3: Valid data (24.2%)
\n ⚠️ TOF sensor 4: Sparse data, using zeros
\n ⚠️ TOF sensor 5: Sparse data, using zeros
There are two sequences in the train data with partial nulls. SEQ_044963 has thm_2 partially missing, and SEQ_062104 has thm_5 and all of tof_5 partially missing. Otherwise sequences either have all nulls or no nulls for a given sensor. Though it is common for only one thermal or one of the tof's to have nulls and the others still ok.
I ran some checks, there's no NaNs on the ToF columns (except for the full NaN on both the 5 ToF columns and the Thm ones, which is intended). I also replaced all -1s with NaNs and checked that there's sequences where we have exactly 1, 2, 3 or 4 ToF sensors full of -1 (same as in train, for example look at sequences where 'Wave hello' is performed), so that doesn't really tell much.
\nI tried playing around with augmentations which fully disable ToF sensors or random occlusions, but could not see any performance increase. I am trying to figure out what the communication failure represents, and whether test set behaves the same as train. I experience a break of CV-LB correlation when using ToF sensors which I do not yet understand.
\nI am wondering if anyone else is also experiencing this or if I am doing something wrong.
", "rawMarkdown": "I ran some checks, there's no NaNs on the ToF columns (except for the full NaN on both the 5 ToF columns and the Thm ones, which is intended). I also replaced all -1s with NaNs and checked that there's sequences where we have exactly 1, 2, 3 or 4 ToF sensors full of -1 (same as in train, for example look at sequences where 'Wave hello' is performed), so that doesn't really tell much.\n\nI tried playing around with augmentations which fully disable ToF sensors or random occlusions, but could not see any performance increase. I am trying to figure out what the communication failure represents, and whether test set behaves the same as train. I experience a break of CV-LB correlation when using ToF sensors which I do not yet understand.\n\nI am wondering if anyone else is also experiencing this or if I am doing something wrong.", "votes": null }, { "id": "3241233", "postDate": "07/04/2025 16:42:00", "content": "I found the same thing. It's probably happening at some point during the lifetime of a subject and persists like that.
", "rawMarkdown": "I found the same thing. It's probably happening at some point during the lifetime of a subject and persists like that.", "votes": null } ], "comments": [ { "id": 3234317, "author_name": "fafa92", "author_url": "", "post_date": "06/27/2025 17:54:17", "content": "I would say a mixture of both. This code snippet gives this output for one of the inference examples!
\n i (, ):\n pixel_cols_tof = [ p ()]\n existing_tof_cols = [col col pixel_cols_tof col seq_df.columns]\n\n (existing_tof_cols) >= : \n tof_sensor_data = seq_df[existing_tof_cols].replace(-, np.nan)\n\n \n valid_data_ratio = (~tof_sensor_data.isna()).().() / ((tof_sensor_data) * (existing_tof_cols))\n\n valid_data_ratio > : \n seq_df[] = tof_sensor_data.mean(axis=, skipna=)\n seq_df[] = tof_sensor_data.std(axis=, skipna=)\n seq_df[] = tof_sensor_data.(axis=, skipna=)\n seq_df[] = tof_sensor_data.(axis=, skipna=)\n ()\n :\n seq_df[] = \n seq_df[] = \n seq_df[] = \n seq_df[] = \n ()\noutput log:
\n→ Processing TOF aggregations…
\n ⚠️ TOF sensor 1: Sparse data, using zeros
\n ⚠️ TOF sensor 2: Sparse data, using zeros
\n ✅ TOF sensor 3: Valid data (24.2%)
\n ⚠️ TOF sensor 4: Sparse data, using zeros
\n ⚠️ TOF sensor 5: Sparse data, using zeros
There are two sequences in the train data with partial nulls. SEQ_044963 has thm_2 partially missing, and SEQ_062104 has thm_5 and all of tof_5 partially missing. Otherwise sequences either have all nulls or no nulls for a given sensor. Though it is common for only one thermal or one of the tof's to have nulls and the others still ok.
I found the same thing. It's probably happening at some point during the lifetime of a subject and persists like that.
", "votes": null, "replies": [] } ] }, { "id": 3241229, "author_name": "andreizamfir", "author_url": "", "post_date": "07/04/2025 16:37:25", "content": "I ran some checks, there's no NaNs on the ToF columns (except for the full NaN on both the 5 ToF columns and the Thm ones, which is intended). I also replaced all -1s with NaNs and checked that there's sequences where we have exactly 1, 2, 3 or 4 ToF sensors full of -1 (same as in train, for example look at sequences where 'Wave hello' is performed), so that doesn't really tell much.
\nI tried playing around with augmentations which fully disable ToF sensors or random occlusions, but could not see any performance increase. I am trying to figure out what the communication failure represents, and whether test set behaves the same as train. I experience a break of CV-LB correlation when using ToF sensors which I do not yet understand.
\nI am wondering if anyone else is also experiencing this or if I am doing something wrong.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3234275": "The competition text says\n>Note also that there is known sensor communication failure in this dataset, resulting in missing data from some sensors in some sequences.\n\nAre these some sensors **fully** or **partially** null through the sequence for these sequences?", "3234317": "I would say a mixture of both. This code snippet gives this output for one of the inference examples!\n\n```python\nfor i in range(1, 6):\n pixel_cols_tof = [f\"tof_{i}_v{p}\" for p in range(64)]\n existing_tof_cols = [col for col in pixel_cols_tof if col in seq_df.columns]\n \n if len(existing_tof_cols) >= 32: # At least half the pixels available\n tof_sensor_data = seq_df[existing_tof_cols].replace(-1, np.nan)\n \n # Check for valid data\n valid_data_ratio = (~tof_sensor_data.isna()).sum().sum() / (len(tof_sensor_data) * len(existing_tof_cols))\n \n if valid_data_ratio > 0.1: # At least 10% valid data\n seq_df[f'tof_{i}_mean'] = tof_sensor_data.mean(axis=1, skipna=True)\n seq_df[f'tof_{i}_std'] = tof_sensor_data.std(axis=1, skipna=True)\n seq_df[f'tof_{i}_min'] = tof_sensor_data.min(axis=1, skipna=True)\n seq_df[f'tof_{i}_max'] = tof_sensor_data.max(axis=1, skipna=True)\n print(f\" ✅ TOF sensor {i}: Valid data ({valid_data_ratio:.1%})\")\n else:\n seq_df[f'tof_{i}_mean'] = 0.0\n seq_df[f'tof_{i}_std'] = 0.0\n seq_df[f'tof_{i}_min'] = 0.0\n seq_df[f'tof_{i}_max'] = 0.0\n print(f\" ⚠️ TOF sensor {i}: Sparse data, using zeros\")\n```\n\n\n\noutput log:\n\n → Processing TOF aggregations...\n ⚠️ TOF sensor 1: Sparse data, using zeros\n ⚠️ TOF sensor 2: Sparse data, using zeros\n ✅ TOF sensor 3: Valid data (24.2%)\n ⚠️ TOF sensor 4: Sparse data, using zeros\n ⚠️ TOF sensor 5: Sparse data, using zeros", "3234488": "There are two sequences in the train data with partial nulls. `SEQ_044963` has thm_2 partially missing, and `SEQ_062104` has thm_5 and all of tof_5 partially missing. Otherwise sequences either have all nulls or no nulls for a given sensor. Though it is common for only one thermal or one of the tof's to have nulls and the others still ok.", "3241229": "I ran some checks, there's no NaNs on the ToF columns (except for the full NaN on both the 5 ToF columns and the Thm ones, which is intended). I also replaced all -1s with NaNs and checked that there's sequences where we have exactly 1, 2, 3 or 4 ToF sensors full of -1 (same as in train, for example look at sequences where 'Wave hello' is performed), so that doesn't really tell much.\n\nI tried playing around with augmentations which fully disable ToF sensors or random occlusions, but could not see any performance increase. I am trying to figure out what the communication failure represents, and whether test set behaves the same as train. I experience a break of CV-LB correlation when using ToF sensors which I do not yet understand.\n\nI am wondering if anyone else is also experiencing this or if I am doing something wrong.", "3241233": "I found the same thing. It's probably happening at some point during the lifetime of a subject and persists like that." }, "source": "meta" }