{
  "id": 598172,
  "title": "Handling the sparsity of the TOF data",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/598172",
  "author_name": "",
  "post_date": "2025-08-09T07:43:56.084856700Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello Peeps,</p>\n<p>I was kind of bothered by the high-dimensionality and the sparisty of the tof sensor data. And I didn't encounter a solution that addressed this concern. So I built a convolutional autoencoder to make the data more NN friendly, and also to be able to train models that are much smaller in size. I am currently experimenting with different ways of post processing the output of the encoder (which is a 5x5x16 tensor). Nonetheless, I have made the model's weights public along with a baseline LSTM that uses the compressed tof data. <br>\nThis is the model <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2Fe6216f66e774fa919fb6521f53b2d07b%2FScreenshot%202025-08-09%20104729.png?generation=1754725671165057&amp;alt=media\" alt=\"Model's architecutre.png\"></p>",
  "messages": [
    {
      "id": "3266480",
      "postDate": "08/09/2025 07:43:56",
      "content": "<p>Hello Peeps,</p>\n<p>I was kind of bothered by the high-dimensionality and the sparisty of the tof sensor data. And I didn't encounter a solution that addressed this concern. So I built a convolutional autoencoder to make the data more NN friendly, and also to be able to train models that are much smaller in size. I am currently experimenting with different ways of post processing the output of the encoder (which is a 5x5x16 tensor). Nonetheless, I have made the model's weights public along with a baseline LSTM that uses the compressed tof data. <br>\nThis is the model <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2Fe6216f66e774fa919fb6521f53b2d07b%2FScreenshot%202025-08-09%20104729.png?generation=1754725671165057&amp;alt=media\" alt=\"Model's architecutre.png\"></p>",
      "rawMarkdown": "Hello Peeps,\n\nI was kind of bothered by the high-dimensionality and the sparisty of the tof sensor data. And I didn't encounter a solution that addressed this concern. So I built a convolutional autoencoder to make the data more NN friendly, and also to be able to train models that are much smaller in size. I am currently experimenting with different ways of post processing the output of the encoder (which is a 5x5x16 tensor). Nonetheless, I have made the model's weights public along with a baseline LSTM that uses the compressed tof data. \nThis is the model ![Model's architecutre.png](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2Fe6216f66e774fa919fb6521f53b2d07b%2FScreenshot%202025-08-09%20104729.png?generation=1754725671165057&alt=media)",
      "votes": null
    },
    {
      "id": "3266481",
      "postDate": "08/09/2025 07:46:31",
      "content": "<p><a href=\"Tof-autoencoder\" target=\"_blank\">https://www.kaggle.com/datasets/moetasimrady/tof-autoencoder</a>; This dataset houses the model's weights.<br>\nAnd this is the model's training data: <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2F788504447506025b5a902550421d4585%2Fautoencoder%20training.png?generation=1754725564321130&amp;alt=media\" alt=\"Model_training.jpg\"></p>\n<p>And this is the NB of the baseline LSTM with the compression preprocessing steps <a href=\"https://www.kaggle.com/code/moetasimrady/cmi-lstm-classifier-with-tof-and-thm\" target=\"_blank\">Baseline LSTM with compressed tof</a></p>",
      "rawMarkdown": "[https://www.kaggle.com/datasets/moetasimrady/tof-autoencoder](Tof-autoencoder); This dataset houses the model's weights.\nAnd this is the model's training data: ![Model_training.jpg](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2F788504447506025b5a902550421d4585%2Fautoencoder%20training.png?generation=1754725564321130&alt=media)\n\nAnd this is the NB of the baseline LSTM with the compression preprocessing steps [Baseline LSTM with compressed tof](https://www.kaggle.com/code/moetasimrady/cmi-lstm-classifier-with-tof-and-thm)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3266481,
      "author_name": "moetasimrady",
      "author_url": "",
      "post_date": "08/09/2025 07:46:31",
      "content": "<p><a href=\"Tof-autoencoder\" target=\"_blank\">https://www.kaggle.com/datasets/moetasimrady/tof-autoencoder</a>; This dataset houses the model's weights.<br>\nAnd this is the model's training data: <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2F788504447506025b5a902550421d4585%2Fautoencoder%20training.png?generation=1754725564321130&amp;alt=media\" alt=\"Model_training.jpg\"></p>\n<p>And this is the NB of the baseline LSTM with the compression preprocessing steps <a href=\"https://www.kaggle.com/code/moetasimrady/cmi-lstm-classifier-with-tof-and-thm\" target=\"_blank\">Baseline LSTM with compressed tof</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3266480": "Hello Peeps,\n\nI was kind of bothered by the high-dimensionality and the sparisty of the tof sensor data. And I didn't encounter a solution that addressed this concern. So I built a convolutional autoencoder to make the data more NN friendly, and also to be able to train models that are much smaller in size. I am currently experimenting with different ways of post processing the output of the encoder (which is a 5x5x16 tensor). Nonetheless, I have made the model's weights public along with a baseline LSTM that uses the compressed tof data. \nThis is the model ![Model's architecutre.png](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2Fe6216f66e774fa919fb6521f53b2d07b%2FScreenshot%202025-08-09%20104729.png?generation=1754725671165057&alt=media)",
    "3266481": "[https://www.kaggle.com/datasets/moetasimrady/tof-autoencoder](Tof-autoencoder); This dataset houses the model's weights.\nAnd this is the model's training data: ![Model_training.jpg](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13549016%2F788504447506025b5a902550421d4585%2Fautoencoder%20training.png?generation=1754725564321130&alt=media)\n\nAnd this is the NB of the baseline LSTM with the compression preprocessing steps [Baseline LSTM with compressed tof](https://www.kaggle.com/code/moetasimrady/cmi-lstm-classifier-with-tof-and-thm)"
  },
  "source": "meta"
}