I thank all those who organized this competition. I believe that I and many other competitors learned a lot.
\nYou can find my solution source code here.
\nMy solution is to create an image with a height of 384 and a width of 2880 and perform object detection. My solution is a little long and may be wasteful😥
\nanglez is divided by 90 so that it falls within the range [-1, 1]. enmo is normalized using mean and std. and I used log1p for raw emno.
\nFor these three features, I performed basic feature engineering of time series data and created 79 features.
To accommodate the object detection task, make the label a rectangle. The label is 120 points before and after the event point. For example, if there is an event point at position 4992, the label would be (0, 4872, 384, 5112).
\nrandomly sampled a chunk of 2880 steps (4 hours) and used it as input. The input shape is now (Batch_size, 79 +1, 2880). +1 indicates hour feature. This was converted into a pseudo-image for object detection.
\nModels can be divided into two main categories.
\nThe first one looks like this. The shape is channel first.
\n| block name | \ninput shape | \noutput shape | \n
|---|---|---|
| hour_emb | \n(1, 2880) | \n(32, 2880) | \n
| inter_conv | \n(79+32, 2880) | \n(2048, 2880) | \n
| upsample_resblocks * 8 | \n(2048, 1, 2880) | \n(3, 384, 2880) | \n
In addition, resblock consists of the following.
\n| operation | \n- | \nskip | \n
|---|---|---|
| BN | \n\n | \n |
| LReLU(0.2) | \n\n | \n |
| TransposedConv2d | \n\n | TransposedConv2d | \n
| BN | \n\n | \n |
| LReLU(0.2) | \n\n | \n |
| Conv2d | \n\n | \n |
| 👉 | \nelement-wise sum | \n👈 | \n
The second is the general Faster R-CNN. I used timm's convnextv2_base.fcmae_ft_in22k_in1k_384 as the backbone.
\nAll data was used as training data, as there was not enough time to ensemble this model within the submission.
\nI used RAdam optimizer. lr=Batch_size * 1.25e-5, weight_decay=1e-6.
\nThe learning rate was set to decay exponentially by a factor of 0.95 every epoch.
\nI use TorchVision's helper functions for the training loop to make it easier.
Only a single model is used because inference is time consuming.
\nFor each sequence, a window of 2880 points and a stride of 1440 points were used for inference. Only the center 1440 points of the window are used.
\nTake the average of the time axis of the rectangle detected by inference. Simply divide by 2. No other kind of post-processing was done.
When folded into 5 folds, the CV was about 0.770.
\nAnd in the final submission model, Public LB was 0.730, and Private LB was 0.781.
If you have any concerns or questions, feel free to write me!
\nThanks for reading this far! Cheers!🍻
I thank all those who organized this competition. I believe that I and many other competitors learned a lot.
\nYou can find my solution source code here.
\nMy solution is to create an image with a height of 384 and a width of 2880 and perform object detection. My solution is a little long and may be wasteful😥
\nanglez is divided by 90 so that it falls within the range [-1, 1]. enmo is normalized using mean and std. and I used log1p for raw emno.
\nFor these three features, I performed basic feature engineering of time series data and created 79 features.
To accommodate the object detection task, make the label a rectangle. The label is 120 points before and after the event point. For example, if there is an event point at position 4992, the label would be (0, 4872, 384, 5112).
\nrandomly sampled a chunk of 2880 steps (4 hours) and used it as input. The input shape is now (Batch_size, 79 +1, 2880). +1 indicates hour feature. This was converted into a pseudo-image for object detection.
\nModels can be divided into two main categories.
\nThe first one looks like this. The shape is channel first.
\n| block name | \ninput shape | \noutput shape | \n
|---|---|---|
| hour_emb | \n(1, 2880) | \n(32, 2880) | \n
| inter_conv | \n(79+32, 2880) | \n(2048, 2880) | \n
| upsample_resblocks * 8 | \n(2048, 1, 2880) | \n(3, 384, 2880) | \n
In addition, resblock consists of the following.
\n| operation | \n- | \nskip | \n
|---|---|---|
| BN | \n\n | \n |
| LReLU(0.2) | \n\n | \n |
| TransposedConv2d | \n\n | TransposedConv2d | \n
| BN | \n\n | \n |
| LReLU(0.2) | \n\n | \n |
| Conv2d | \n\n | \n |
| 👉 | \nelement-wise sum | \n👈 | \n
The second is the general Faster R-CNN. I used timm's convnextv2_base.fcmae_ft_in22k_in1k_384 as the backbone.
\nAll data was used as training data, as there was not enough time to ensemble this model within the submission.
\nI used RAdam optimizer. lr=Batch_size * 1.25e-5, weight_decay=1e-6.
\nThe learning rate was set to decay exponentially by a factor of 0.95 every epoch.
\nI use TorchVision's helper functions for the training loop to make it easier.
Only a single model is used because inference is time consuming.
\nFor each sequence, a window of 2880 points and a stride of 1440 points were used for inference. Only the center 1440 points of the window are used.
\nTake the average of the time axis of the rectangle detected by inference. Simply divide by 2. No other kind of post-processing was done.
When folded into 5 folds, the CV was about 0.770.
\nAnd in the final submission model, Public LB was 0.730, and Private LB was 0.781.
If you have any concerns or questions, feel free to write me!
\nThanks for reading this far! Cheers!🍻
An interesting and impressive solution yet again! I was glad to compete with you and see you shake up! You got me this time! Hahaha
", "rawMarkdown": "An interesting and impressive solution yet again! I was glad to compete with you and see you shake up! You got me this time! Hahaha", "votes": null } ], "comments": [ { "id": 2551261, "author_name": "cody11null", "author_url": "", "post_date": "12/06/2023 15:35:40", "content": "An interesting and impressive solution yet again! I was glad to compete with you and see you shake up! You got me this time! Hahaha
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2550492": "I thank all those who organized this competition. I believe that I and many other competitors learned a lot.\n\n# source codes\n[You can find my solution source code here](https://github.com/itsuki8914/Child-Mind-Institute---Detect-Sleep-States-87th-Place-Codes).\n\n# TL;DR\n- My solution is **2D object detection**.\n- **Creating pseudo images** using transposed convolutions.\n- Detecting events with **Faster R-CNN**.\n\nMy solution is to create an image with a height of 384 and a width of 2880 and perform object detection. My solution is a little long and may be wasteful😥\n\n\n# Data preparing\n1. Extract anglez, enmo, and hour features for each step and output to csv.\n2. Output events to csv for each sample.\n\n# Feature Engineering\nanglez is divided by 90 so that it falls within the range [-1, 1]. enmo is normalized using mean and std. and I used **log1p for raw emno**.\nFor these three features, I performed basic feature engineering of time series data and created 79 features.\n\n# Labelling\nTo accommodate the object detection task, make the label a rectangle. The label is **120 points before and after the event point**. For example, if there is an event point at position 4992, the label would be (0, 4872, 384, 5112).\n\n# Basic strategy\nrandomly sampled a chunk of 2880 steps (4 hours) and used it as input. The input shape is now (Batch_size, 79 **+1**, 2880). **+1** indicates hour feature. This was converted into a pseudo-image for object detection.\n\n# Model architecture\nModels can be divided into two main categories. \n- The first is a network that creates pseudo-images\n- The second is a network that performs object detection on them. I used Faster R-CNN for this.\n\nThe first one looks like this. The shape is channel first.\n\n| block name | input shape | output shape |\n| --- | --- | --- |\n| hour_emb | (1, 2880) | (32, 2880) |\n| inter_conv | (79+32, 2880) | (2048, 2880) |\n| upsample_resblocks * 8 | (2048, 1, 2880) | (3, 384, 2880) |\n\nIn addition, resblock consists of the following.\n\n| operation | - | skip |\n| --- | --- | --- |\n| BN | | |\n| LReLU(0.2) | | |\n| TransposedConv2d| | TransposedConv2d |\n| BN | | |\n| LReLU(0.2) | | |\n| Conv2d| | |\n| 👉 | element-wise sum | 👈 |\n\n\nThe second is the general Faster R-CNN. I used timm's **convnextv2_base.fcmae_ft_in22k_in1k_384** as the backbone.\n\n# Training\nAll data was used as training data, as there was not enough time to ensemble this model within the submission.\nI used RAdam optimizer. lr=Batch_size * 1.25e-5, weight_decay=1e-6.\nThe learning rate was set to decay exponentially by a factor of 0.95 every epoch.\nI use [TorchVision's helper functions](https://github.com/pytorch/vision/tree/main/references/detection) for the training loop to make it easier.\n\n# Inference and Postprocess\nOnly a single model is used because inference is time consuming.\nFor each sequence, a window of 2880 points and a stride of 1440 points were used for inference. **Only the center 1440 points of the window are used.** \nTake the average of the time axis of the rectangle detected by inference. Simply divide by 2. No other kind of post-processing was done.\n\nWhen folded into 5 folds, the CV was about 0.770.\nAnd in the final submission model, Public LB was 0.730, and Private LB was 0.781.\n\nIf you have any concerns or questions, feel free to write me!\nThanks for reading this far! Cheers!🍻", "2551261": "An interesting and impressive solution yet again! I was glad to compete with you and see you shake up! You got me this time! Hahaha" }, "source": "meta" }