{ "id": 459761, "title": "27th solution", "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/459761", "author_name": "tenten", "post_date": "2023-12-06T14:36:51.535000", "votes": 5, "comment_count": 0, "views": 0, "content": "

I implemented this solution based on @tubotubo's source code. Thank you for sharing!

\n

Input

\n\n

Model

\n\n
month = self.month_embedding(month) \nhour = self.hour_embedding(hour) \nminute = self.minute_embedding(minute) \nx = x[:, :, :] \nx1 = self.feature_extractor1(x[:, :x.shape[]//, :])  \nx2 = self.feature_extractor2(x[:, x.shape[]//:, :])  \nx = torch.cat([x1, x2], dim=) \n\nx = self.encoder(x).squeeze()  \nx = torch.cat([x, month, hour, minute], dim=) \nlogits = self.decoder(x)  \n
\n

PostProcess

\n\n

Ensemble

\n", "messages": [ { "id": 2551188, "postDate": "2023-12-06T14:36:51.537Z", "content": "

I implemented this solution based on @tubotubo's source code. Thank you for sharing!

\n

Input

\n\n

Model

\n\n
month = self.month_embedding(month) \nhour = self.hour_embedding(hour) \nminute = self.minute_embedding(minute) \nx = x[:, :, :] \nx1 = self.feature_extractor1(x[:, :x.shape[]//, :])  \nx2 = self.feature_extractor2(x[:, x.shape[]//:, :])  \nx = torch.cat([x1, x2], dim=) \n\nx = self.encoder(x).squeeze()  \nx = torch.cat([x, month, hour, minute], dim=) \nlogits = self.decoder(x)  \n
\n

PostProcess

\n\n

Ensemble

\n", "rawMarkdown": "I implemented this solution based on @tubotubo's source code. Thank you for sharing!\n\n## Input\n- anglez（normalized）\n- enmo（normalized）\n- hour_sin\n- hour_cos\n- minute_sin\n- minute_cos\n\n## Model\n- Label\n 1. awake\n - BCEWithLogitsLoss\n 2. onset, wakeup\n - BCEWithLogitsLoss\n - loss = (awake loss) * (1 - 0.88) + (onset, wakeup loss) * 0.88\n\n- Data Augmentation\n - GaussianNoise\n - min_amplitude: 0.01\n - max_amplitude: 0.1\n - [Zebra Mixup](https://www.kaggle.com/competitions/g2net-gravitational-wave-detection/discussion/275335)\n\n- encoder: UNet\n| feature extractor | decoder | downsample | CV |\n| --- | --- | --- | --- |\n| LSTM | UNet1D | 2 | 0.7611 |\n| CNN | UNet1D | 2 | 0.7628 |\n| Spectrogram | UNet1D | 2 | 0.7463 |\n| CNN | Transformer | 6 | 0.7480 |\n| Spectrogram | Transformer | 6 | 0.7725 |\n| 2CNN+ time feature embedding | UNet1D | 2 | 0.7705 |\n\n- 2CNN+time feature embedding\n - anglez and enmo are input separately into CNN feature extractors\n - In this model, month, hour, and minute are input as categorical variables.\n - month, hour and minute are converted into embeddings and concatenated with the encoder output before being input to the decoder.\n\n```python\nmonth = self.month_embedding(month) # (batch_size, 4, n_timesteps)\nhour = self.hour_embedding(hour) # (batch_size, 8, n_timesteps)\nminute = self.minute_embedding(minute) # (batch_size, 16, n_timesteps)\nx = x[:, 3:, :] # (batch_size, n_channels, n_timesteps)\nx1 = self.feature_extractor1(x[:, :x.shape[1]//2, :]) # (batch_size, n_kernel, height, n_timesteps)\nx2 = self.feature_extractor2(x[:, x.shape[1]//2:, :]) # (batch_size, n_kernel, height, n_timesteps)\nx = torch.cat([x1, x2], dim=2) # (batch_size, n_kernel, height*2, n_timesteps)\n\nx = self.encoder(x).squeeze(1) # (batch_size, height, n_timesteps)\nx = torch.cat([x, month, hour, minute], dim=1) # (batch_size, height+28, n_timesteps)\nlogits = self.decoder(x) # (batch_size, n_timesteps, n_classes)\n```\n\n## PostProcess\n- scipy.signal find peaks (after ensemble)\n - Chose the smallest possible value for height (0.003)\n - distance optimized with optuna\n- overlap inference\n\n## Ensemble\n- 4fold * 6model\n- Ensemble with weighted output heatmaps\n- Weights optimized with optuna\n - ensemble (mean) -> cv: 0.795\n - ensemble (weighted mean) -> cv: 0.7981 (public: 0.753 private: 0.809)\n\n", "votes": 5 } ], "comments": [], "raw_markdown_by_id": { "2551188": "I implemented this solution based on @tubotubo's source code. Thank you for sharing!\n\n## Input\n- anglez（normalized）\n- enmo（normalized）\n- hour_sin\n- hour_cos\n- minute_sin\n- minute_cos\n\n## Model\n- Label\n 1. awake\n - BCEWithLogitsLoss\n 2. onset, wakeup\n - BCEWithLogitsLoss\n - loss = (awake loss) * (1 - 0.88) + (onset, wakeup loss) * 0.88\n\n- Data Augmentation\n - GaussianNoise\n - min_amplitude: 0.01\n - max_amplitude: 0.1\n - [Zebra Mixup](https://www.kaggle.com/competitions/g2net-gravitational-wave-detection/discussion/275335)\n\n- encoder: UNet\n| feature extractor | decoder | downsample | CV |\n| --- | --- | --- | --- |\n| LSTM | UNet1D | 2 | 0.7611 |\n| CNN | UNet1D | 2 | 0.7628 |\n| Spectrogram | UNet1D | 2 | 0.7463 |\n| CNN | Transformer | 6 | 0.7480 |\n| Spectrogram | Transformer | 6 | 0.7725 |\n| 2CNN+ time feature embedding | UNet1D | 2 | 0.7705 |\n\n- 2CNN+time feature embedding\n - anglez and enmo are input separately into CNN feature extractors\n - In this model, month, hour, and minute are input as categorical variables.\n - month, hour and minute are converted into embeddings and concatenated with the encoder output before being input to the decoder.\n\n```python\nmonth = self.month_embedding(month) # (batch_size, 4, n_timesteps)\nhour = self.hour_embedding(hour) # (batch_size, 8, n_timesteps)\nminute = self.minute_embedding(minute) # (batch_size, 16, n_timesteps)\nx = x[:, 3:, :] # (batch_size, n_channels, n_timesteps)\nx1 = self.feature_extractor1(x[:, :x.shape[1]//2, :]) # (batch_size, n_kernel, height, n_timesteps)\nx2 = self.feature_extractor2(x[:, x.shape[1]//2:, :]) # (batch_size, n_kernel, height, n_timesteps)\nx = torch.cat([x1, x2], dim=2) # (batch_size, n_kernel, height*2, n_timesteps)\n\nx = self.encoder(x).squeeze(1) # (batch_size, height, n_timesteps)\nx = torch.cat([x, month, hour, minute], dim=1) # (batch_size, height+28, n_timesteps)\nlogits = self.decoder(x) # (batch_size, n_timesteps, n_classes)\n```\n\n## PostProcess\n- scipy.signal find peaks (after ensemble)\n - Chose the smallest possible value for height (0.003)\n - distance optimized with optuna\n- overlap inference\n\n## Ensemble\n- 4fold * 6model\n- Ensemble with weighted output heatmaps\n- Weights optimized with optuna\n - ensemble (mean) -> cv: 0.795\n - ensemble (weighted mean) -> cv: 0.7981 (public: 0.753 private: 0.809)\n\n" } }