I share my approach in this discussion.
\nMy model consists of three components: feature extractor, encoder, and decoder.
\nThe feature extractor takes the raw waveform(anglez, enmo, hour) and creates a wavegram (like an image).
\nThe encoder (UNet) takes the wavegrame and creates a two-dimensional heatmap.
\ndecoder takes the heatmap and outputs the probability(sleep, onset, wakeup) of an event occurring at each time.
\nFinally, post-processing is performed to create a submission file.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2Ffa7f5ecfa0f290e1f44e3d2e7dbc4f06%2F(1).jpg?generation=1699079676016847&alt=media\")
Each component has the following already implemented.
\nLB: 0.714
\nfeature extractor=CNN, encoder=UNet, decoder=UNet1D, downsample_rate=2, seq_len=5760
Because the code has changed since the last submission, the experiment may not be accurately reproduced with the above settings.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2Fca7e96b19b87610f029c9b42be3b1bea%2FScreenshot%20from%202023-11-04%2016-39-08.png?generation=1699083570125882&alt=media\")
If you want to make a contribution to this repository, please write a PR or issue summary and link to it in this thread as well to avoid private sharing.
", "messages": [ { "id": 2512088, "postDate": "2023-11-04T06:38:05.810Z", "content": "I share my approach in this discussion.
\nMy model consists of three components: feature extractor, encoder, and decoder.
\nThe feature extractor takes the raw waveform(anglez, enmo, hour) and creates a wavegram (like an image).
\nThe encoder (UNet) takes the wavegrame and creates a two-dimensional heatmap.
\ndecoder takes the heatmap and outputs the probability(sleep, onset, wakeup) of an event occurring at each time.
\nFinally, post-processing is performed to create a submission file.
\n
Each component has the following already implemented.
\nLB: 0.714
\nfeature extractor=CNN, encoder=UNet, decoder=UNet1D, downsample_rate=2, seq_len=5760
Because the code has changed since the last submission, the experiment may not be accurately reproduced with the above settings.
\n
If you want to make a contribution to this repository, please write a PR or issue summary and link to it in this thread as well to avoid private sharing.
", "rawMarkdown": "I share my approach in this discussion.\n- The code is also available on github\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/tree/main\n- submission notebook\nhttps://www.kaggle.com/code/tubotubo/cmi-submit\n\n# Model Structure\nMy model consists of three components: feature extractor, encoder, and decoder.\n\nThe feature extractor takes the raw waveform(anglez, enmo, hour) and creates a wavegram (like an image).\nThe encoder (UNet) takes the wavegrame and creates a two-dimensional heatmap.\ndecoder takes the heatmap and outputs the probability(sleep, onset, wakeup) of an event occurring at each time.\nFinally, post-processing is performed to create a submission file.\n.jpg?generation=1699079676016847&alt=media)\n\nEach component has the following already implemented.\n\n- feature extractor\n - CNN\n - LSTM\n - Spectrogram\n - [PANNs](https://arxiv.org/abs/1912.10211)\n- encoder\n - UNet\n- decoder\n - UNet1D\n - LSTM\n - MLP\n - Transformer\n\n# Result\nLB: 0.714\nfeature extractor=CNN, encoder=UNet, decoder=UNet1D, downsample_rate=2, seq_len=5760\n\nBecause the code has changed since the last submission, the experiment may not be accurately reproduced with the above settings.\n\n# Tips\n1. The evaluation function can be sped up using a binary search. When the number of detections is large, the evaluation index can be calculated tens to hundreds of times faster than the original implementation.\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/blob/8bbc09aff9c34f12259d536fd496deaade8ec087/src/utils/metrics.py#L179\n2. Outputting a file other than submission.csv and submitting it will result in an error. In the inference notebook, it is better to leave only the final submission.csv.\n\n\n\n# How to contribute\n\nIf you want to make a contribution to this repository, please write a PR or issue summary and link to it in [this thread](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940#2530545) as well to avoid private sharing.", "votes": 206 }, { "id": 2512328, "postDate": "2023-11-04T11:24:25.490Z", "content": "I do not believe that publicizing this approach will spoil the efforts of the other participants.
\nThere are two reasons.
\nThe first is that my approach does not encompass all other approaches. I believe that a better model can be created by combining my approach with your approach.
\nThe second is that the code I have published has room for many improvements and I believe there is ample time to make improvements.
\nFor these two reasons, I thought it was worth releasing the code and model to the public.
Temporarily, LB may be full of people who just forked it, but it will be weeded out by those who are serious about experimenting.
", "rawMarkdown": "I do not believe that publicizing this approach will spoil the efforts of the other participants.\nThere are two reasons.\nThe first is that my approach does not encompass all other approaches. I believe that a better model can be created by combining my approach with your approach.\nThe second is that the code I have published has room for many improvements and I believe there is ample time to make improvements.\nFor these two reasons, I thought it was worth releasing the code and model to the public.\n\nTemporarily, LB may be full of people who just forked it, but it will be weeded out by those who are serious about experimenting.", "votes": 48, "replies": [ { "id": 2512971, "postDate": "2023-11-05T04:21:46.723Z", "content": "I appreciate you sharing your approach. But do I appreciate you publishing the \"CMI-submit\" notebook? I'm not sure.
\nJust look at the LB now. So many people have 0.707 score (without putting in any effort).
\nYour action will lead to less \"variety of solutions\". People who were working on creative approaches will give up, and focus on replicating your approach. In the end, this will reduce the variety of solutions the community develops for this competition.
\nI don't know if this is a good thing or a bad thing.
", "rawMarkdown": "I appreciate you sharing your approach. But do I appreciate you publishing the \"CMI-submit\" notebook? I'm not sure.\n\nJust look at the LB now. So many people have 0.707 score (without putting in any effort).\n\nYour action will lead to less \"variety of solutions\". People who were working on creative approaches will give up, and focus on replicating your approach. In the end, this will reduce the variety of solutions the community develops for this competition.\n\nI don't know if this is a good thing or a bad thing.", "votes": 7, "replies": [ { "id": 2512979, "postDate": "2023-11-05T04:46:56.670Z", "content": "I think people need to add his approach to their own notebooks. So, solutions are getting more various and progressive.
", "rawMarkdown": "I think people need to add his approach to their own notebooks. So, solutions are getting more various and progressive." }, { "id": 2512980, "postDate": "2023-11-05T04:51:32.840Z", "content": "I have no problem with him sharing the approach.
\nMy concern is about sharing the \"CMI-submit\" notebook.
", "rawMarkdown": "I have no problem with him sharing the approach.\n\nMy concern is about sharing the \"CMI-submit\" notebook.", "votes": 3 } ] }, { "id": 2514892, "postDate": "2023-11-06T15:38:48.740Z", "content": "Much advovated for your kind sharing @tubotubo
", "rawMarkdown": "Much advovated for your kind sharing @tubotubo ", "replies": [ { "id": 2522105, "postDate": "2023-11-12T11:48:23.423Z", "content": "Can you share more informations about post_process.score_th and post_process.distance?
\nAnd Why use this? @tubotubo
\nThanks
The find_peak function of scipy is used to detect events.
\nSee the following page for details
\nhttps://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.find_peaks.html
\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/blob/main/src/utils/post_process.py
In the epochs,the valid_loss decrease,but the valid_score not increase.
\nlike this:
\nEpoch 0/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:48 • 0:00:00 2.44it/s v_num: 3vs7Epoch 0, global step 119: 'val_loss' reached 0.06178 (best 0.06178), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/checkpoints/epoch=0-step=119.ckpt' as top 1
\nSaved best model 0.061714774133353835 -> 0.040320251738368874
\nEpoch 1/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.66it/s v_num: 3vs7 val_loss: 0.062 val_score: 0.000 train_loss: 0.244
\nEpoch 1, global step 238: 'val_loss' reached 0.04033 (best 0.04033), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check
\nEpoch 2/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.85it/s v_num: 3vs7 val_loss: 0.040 val_score: 0.000 train_loss : 0.175
\nSaved best model 0.040320251738368874 -> 0.03305796951684632
\nEpoch 3/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.79it/s v_num: 3vs7 val_loss: 0.043 val_score: 0.000 train_loss:0.172
\nEpoch 3, global step 476: 'val_loss' reached 0.03304 (best 0.03304), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check
\nEpoch 4/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.89it/s v_num: 3vs7 val_loss: 0.033 val_score: 0.000 train_loss: 0.170
\nEpoch 5/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.81it/s v_num: 3vs7 val_loss: 0.034 val_score: 0.000 train_loss:0.166
It was fine the first time I ran it, and then this happened
", "rawMarkdown": "In the epochs,the valid_loss decrease,but the valid_score not increase.\nlike this:\nEpoch 0/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:48 • 0:00:00 2.44it/s v_num: 3vs7Epoch 0, global step 119: 'val_loss' reached 0.06178 (best 0.06178), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/checkpoints/epoch=0-step=119.ckpt' as top 1\nSaved best model 0.061714774133353835 -> 0.040320251738368874\nEpoch 1/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.66it/s v_num: 3vs7 val_loss: 0.062 val_score: 0.000 train_loss: 0.244 \nEpoch 1, global step 238: 'val_loss' reached 0.04033 (best 0.04033), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check\nEpoch 2/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.85it/s v_num: 3vs7 val_loss: 0.040 val_score: 0.000 train_loss : 0.175 \nSaved best model 0.040320251738368874 -> 0.03305796951684632\nEpoch 3/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.79it/s v_num: 3vs7 val_loss: 0.043 val_score: 0.000 train_loss:0.172 \nEpoch 3, global step 476: 'val_loss' reached 0.03304 (best 0.03304), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check\nEpoch 4/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.89it/s v_num: 3vs7 val_loss: 0.033 val_score: 0.000 train_loss: 0.170 \nEpoch 5/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.81it/s v_num: 3vs7 val_loss: 0.034 val_score: 0.000 train_loss:0.166 \n\nIt was fine the first time I ran it, and then this happened", "votes": 1 }, { "id": 2522885, "postDate": "2023-11-13T06:16:09.313Z", "content": "\nPlease let me know what commands you have executed
", "rawMarkdown": "@lhchina \n\nPlease let me know what commands you have executed" }, { "id": 2522897, "postDate": "2023-11-13T06:26:19.390Z", "content": "python -m run.train exp_name='exp001' num_workers=2 batch_size=32 duration=5760 downsample_rate=2
", "rawMarkdown": "python -m run.train exp_name='exp001' num_workers=2 batch_size=32 duration=5760 downsample_rate=2" }, { "id": 2522901, "postDate": "2023-11-13T06:33:21.183Z", "content": "\nThe learning is going well in my environment.
\nPlease try git pull and then run the command again.
I'm trying it,maybe I change some code in the modelmodule.py.like this:
\nclass PLSleepModel(LightningModule):
\n def init(
\n self,
\n cfg: TrainConfig,
\n val_event_df: pl.DataFrame,
\n feature_dim: int,
\n num_classes: int,
\n duration: int,
\n ):
\n super().init()
\n self.cfg = cfg
\n self.val_event_df = val_event_df
\n num_timesteps = nearest_valid_size(int(duration * cfg.upsample_rate), cfg.downsample_rate)
\n self.model = get_model(
\n cfg,
\n feature_dim=feature_dim,
\n n_classes=num_classes,
\n num_timesteps=num_timesteps // cfg.downsample_rate,
\n )
\n self.duration = duration
\n self.validation_step_outputs: list = []
\n self.__best_loss = np.inf
\n self.__score = 0
def forward(\n self,\n x: torch.Tensor,\n labels: Optional[torch.Tensor] = None,\n do_mixup: bool = ,\n do_cutmix: bool = ,\n) -> ModelOutput:\n return self.model(x, labels, do_mixup, do_cutmix)\n\ndef training_step(self, batch, batch_idx):\n do_mixup = np.random.rand() < self.cfg.aug.mixup_prob\n do_cutmix = np.random.rand() < self.cfg.aug.cutmix_prob\n output = self.model(batch[], batch[], do_mixup, do_cutmix)\n\n self.log(\n ,\n output.loss.detach().item(),\n =,\n =,\n =,\n =,\n )\n return output.loss\n\ndef validation_step(self, batch, batch_idx):\n output = self.model.predict(batch[], self.duration, batch[])\n self.validation_step_outputs.append(\n (\n batch[],\n output.labels.detach().cpu().numpy(),\n output.preds.detach().cpu().numpy(),\n output.loss.detach().item(),\n )\n )\n self.log(\n ,\n output.loss.detach().item(),\n =,\n =,\n =,\n =,\n )\n\n return output.loss\n\ndef on_validation_epoch_end(self):\n keys = []\n x self.validation_step_outputs:\n keys.extend(x[0])\n labels = np.concatenate([x[1] x self.validation_step_outputs])\n preds = np.concatenate([x[2] x self.validation_step_outputs])\n losses = np.array([x[3] x self.validation_step_outputs])\n loss = losses.mean()\n\n val_pred_df = post_process_for_seg(\n =keys,\n =preds,\n =self.cfg.pp.score_th,\n =self.cfg.pp.distance,\n )\n score = event_detection_ap(self.val_event_df.to_pandas(), val_pred_df.to_pandas())\n self.log(, score, =, =, =, =)\n\n loss < self.__best_loss:\n #np.save(, np.array(keys))\n #np.save(, labels)\n #np.save(, preds)\n val_pred_df.write_csv()\n torch.save(self.model.state_dict(), )\n (f)\n self.__best_loss = loss\n\n score > self.__score:\n torch.save(self.model.state_dict(), )\n (f)\n self.__score = score\n\n self.validation_step_outputs.clear()\nBut I changed it back, and it still had a problem
\nWhen I try git pull and then run the command again.it works OK.why？
I do think sharing the strategy and showing the potential was helpful but I do wish that the inference notebook was not shared. I think the leaderboard is filled with forks of this inference notebook. I would think that this was still ok if there was a training notebook that would actually work on kaggle so the masses would be able to play and learn from it. I was able to do so but I fear most will not have.
", "rawMarkdown": "I do think sharing the strategy and showing the potential was helpful but I do wish that the inference notebook was not shared. I think the leaderboard is filled with forks of this inference notebook. I would think that this was still ok if there was a training notebook that would actually work on kaggle so the masses would be able to play and learn from it. I was able to do so but I fear most will not have. ", "votes": 2 } ] } ] }, { "id": 2530561, "postDate": "2023-11-19T09:04:18.357Z", "content": "I've implemented a few additional models, so if you're interested, give it a try!
\nThe model is built with two components: feature_extractor and decoder.
\nThe feature_extractor and decoder that can be specified are as follows
\nThe correspondence table between each model and dataset is as follows.
\n| model | \ndataset | \n
|---|---|
| Spec1D | \nseg | \n
| Spec2DCNN | \nseg | \n
| DETR2DCNN | \ndetr | \n
| CenterNet | \ncenternet | \n
| TransformerAutoModel | \nseg | \n
The command to train CenterNet with feature_extractor=CNNSpectrogram, decoder=UNet1DDecoder is as follows
\nrye run python run/train.py model=CenterNet dataset=centernet feature_extractor=CNNSpectrogram decoder=UNet1DDecoder\nThat's a incredible sharing, appreciate your consistent yet high-quality work !
", "rawMarkdown": "That's a incredible sharing, appreciate your consistent yet high-quality work !", "votes": 1 }, { "id": 2542049, "postDate": "2023-11-29T03:19:39.230Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2542055, "postDate": "2023-11-29T03:29:25.887Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2542056, "postDate": "2023-11-29T03:30:13.360Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2512132, "postDate": "2023-11-04T07:40:28.203Z", "content": "I don't know whether I should be happy or sad now
", "rawMarkdown": "I don't know whether I should be happy or sad now", "votes": 17, "replies": [ { "id": 2512135, "postDate": "2023-11-04T07:47:24.280Z", "content": "I feel exactly the same, I was working very hardly to climb up the ranks and I just got a top50 submission few hours ago which I was very happy about. Then I find this lol. I just see it as a way to ruin the lb and it's not considerate of people's efforts, I mean it's a top 20 submission with a month until the competition ends?. But then I had to hop on the train since everyone is going to hop on it now.
", "rawMarkdown": "I feel exactly the same, I was working very hardly to climb up the ranks and I just got a top50 submission few hours ago which I was very happy about. Then I find this lol. I just see it as a way to ruin the lb and it's not considerate of people's efforts, I mean it's a top 20 submission with a month until the competition ends?. But then I had to hop on the train since everyone is going to hop on it now. ", "votes": 8, "replies": [ { "id": 2512186, "postDate": "2023-11-04T08:17:23.507Z", "rawMarkdown": "", "votes": 3, "isDeleted": true }, { "id": 2512267, "postDate": "2023-11-04T10:05:07.753Z", "rawMarkdown": "", "votes": 14, "isDeleted": true }, { "id": 2512432, "postDate": "2023-11-04T14:10:48.743Z", "content": "100% agree, I think sharing methods and approaches is perfectly fine (up to a certain point in the competition). But what is the point in enabling others to just submit a pre-trained model? At least the people who actually replicate the methods put in some effort to do that.
", "rawMarkdown": "100% agree, I think sharing methods and approaches is perfectly fine (up to a certain point in the competition). But what is the point in enabling others to just submit a pre-trained model? At least the people who actually replicate the methods put in **some** effort to do that.", "votes": 2 }, { "id": 2514052, "postDate": "2023-11-06T01:58:47.770Z", "content": "The rule \"Don't share the trained weight that scores high and may ruin the LB after a certain point in a competition\" is very obscure. As you know, in the competition's rule there are no explicit threshold in terms of score, the definition of ruining the LB, and when the certain point starts. If you hate it, you need to choose a competition that has such restrictions in the rule explicitly. (However, in kaggle at this moment, you can find no such competitions, I suppose. )
", "rawMarkdown": "The rule \"Don't share the trained weight that scores high and may ruin the LB after a certain point in a competition\" is very obscure. As you know, in the competition's rule there are no explicit threshold in terms of score, the definition of ruining the LB, and when the certain point starts. If you hate it, you need to choose a competition that has such restrictions in the rule explicitly. (However, in kaggle at this moment, you can find no such competitions, I suppose. ) " } ] }, { "id": 2525323, "postDate": "2023-11-15T03:24:25.870Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 2513149, "postDate": "2023-11-05T08:19:30.167Z", "content": "If you want to install Rye in a Kaggle notebook, use this command:
\ncurl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash
\ninstead of
\ncurl -sSf https://rye-up.com/get | bash
(see https://github.com/mitsuhiko/rye/issues/246 for details)
", "rawMarkdown": "If you want to install Rye in a Kaggle notebook, use this command:\n```curl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash```\ninstead of\n```curl -sSf https://rye-up.com/get | bash```\n\n(see https://github.com/mitsuhiko/rye/issues/246 for details)", "votes": 11, "replies": [ { "id": 2514292, "postDate": "2023-11-06T07:03:59.203Z", "content": "Another note: if you want to run @tubotubo code in a Kaggle notebook, you should change this line in pyproject.toml:
\nurl = \"https://download.pytorch.org/whl/cu120\"
\nto
\nurl = \"https://download.pytorch.org/whl/cu118\"
\nsince Kaggle environment currently has CUDA 11.4 installed.
In the kaggle,I meet this question in trainning phase，and how to deal with it?
\nOriginal Traceback (most recent call last):
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py\", line 308, in _worker_loop
\n data = fetcher.fetch(index)
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py\", line 51, in fetch
\n data = [self.dataset[idx] for idx in possibly_batched_index]
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py\", line 51, in
\n data = [self.dataset[idx] for idx in possibly_batched_index]
\n File \"/kaggle/input/cmi-code/cmi-code/src/dataset/seg.py\", line 106, in getitem
\n label = get_seg_label(this_event_df, num_frames, self.cfg.duration, start, end)
\n File \"/kaggle/input/cmi-code/cmi-code/src/dataset/seg.py\", line 21, in get_seg_label
\n this_event_df = this_event_df.query(\"@start <= wakeup & onset <= @end\")
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/frame.py\", line 4440, in query
\n res = self.eval(expr, **kwargs)
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/frame.py\", line 4566, in eval
\n return _eval(expr, inplace=inplace, **kwargs)
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/eval.py\", line 357, in eval
\n ret = eng_inst.evaluate()
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/engines.py\", line 81, in evaluate
\n res = self._evaluate()
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/engines.py\", line 121, in _evaluate
\n return ne.evaluate(s, local_dict=scope)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 943, in evaluate
\n raise e
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 851, in validate
\n _names_cache[expr_key] = getExprNames(ex, context)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 714, in getExprNames
\n ex = stringToExpression(text, {}, context)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 274, in stringToExpression
\n raise ValueError(f'Expression {s} has forbidden control characters.')
\nValueError: Expression ((__pd_eval_local_start) <= (wakeup)) & ((onset) <= (__pd_eval_local_end)) has forbidden control characters.
In the query method of pandas, local variables are referenced in strings using the @ symbol. However, this method can sometimes lead to errors. Instead, consider a method that directly computes and filters data frame conditions.
\nThe epochs is set to 50, but in my case, the epochs went out of memory at 23. Please let me know if you have any solutions.
", "rawMarkdown": "In the query method of pandas, local variables are referenced in strings using the @ symbol. However, this method can sometimes lead to errors. Instead, consider a method that directly computes and filters data frame conditions.\n\nThe epochs is set to 50, but in my case, the epochs went out of memory at 23. Please let me know if you have any solutions." }, { "id": 2520118, "postDate": "2023-11-10T15:31:02.497Z", "content": "The version of the code may be different.
\nTry git pull to use the latest code.
I noticed that you recently had updated the code - it now includes detr2D. Can you specify a source of this architecture?
", "rawMarkdown": "I noticed that you recently had updated the code - it now includes detr2D. Can you specify a source of this architecture?" }, { "id": 2520145, "postDate": "2023-11-10T15:53:24.340Z", "content": "\nThis architecture is called DETR.
\nI have implemented it for study purposes, but have not yet achieved good accuracy.
https://arxiv.org/abs/2005.12872
", "rawMarkdown": "@atamazian \n\nThis architecture is called DETR.\nI have implemented it for study purposes, but have not yet achieved good accuracy.\n \nhttps://arxiv.org/abs/2005.12872", "votes": 3 }, { "id": 2520174, "postDate": "2023-11-10T16:19:20.107Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2520759, "postDate": "2023-11-11T06:42:15.040Z", "content": "When I install rye in the kaggle, something wrong. and how to create virtual environment?
\n@tubotubo
!cp -r /kaggle/input/cmi-code/cmi-code cmi-code
\n!pip install hydra-core segmentation_models_pytorch==0.3.3 --no-index --find-links=/kaggle/input/ex-library
\n!curl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash
\n!echo 'source \"$HOME/.rye/env\"' >> ~/.bashrc
\n!source ~/.bashrc
This installer will install rye to /root/.rye
\nThis path can be changed by exporting the RYE_HOME environment variable.
\nDetails:
\n Rye Version: 0.15.2
\n Platform: linux (x86_64)
\nInstalled binary to /root/.rye/shims/rye
\nBootstrapping rye internals
\nDownloading cpython@3.11.6
\nChecking checksum██████████████████████████████░░░░ 49.16 MiB/52.29 MiB
\nsuccess: Downloaded cpython@3.11.6
\nUpgrading pip
\nInstalling internal dependencies
\nUpdated self-python installation at /root/.rye/self
The rye directory /root/.rye/shims was not detected on PATH.
\nIt is highly recommended that you add it.
\nAdd this at the end of your .profile, .zprofile or similar:
\n source \"$HOME/.rye/env\"
\nNote: after adding rye to your path, restart your shell for it to take effect.
!rye sync
\n/bin/bash: line 1: rye: command not found
try !/root/.rye/shims/rye sync instead
It's a great share but may not at a right time.
", "rawMarkdown": "It's a great share but may not at a right time.", "votes": 9 }, { "id": 2514836, "postDate": "2023-11-06T15:03:16.080Z", "content": "\nThank you for sharing gold-standard solution.
\n\n\nThe evaluation function can be sped up using a binary search. When the number of detections is large, the evaluation index can be calculated tens to hundreds of times faster than the original implementation.
\n
FYI, about the bisect search part, I rewrite the original logic. Although computation time is not boosted much, I think the logic became simpler and clearer.
\n ():\n \n \n idx = bisect_left(gt_times, det_time)\n\n best_idx = \n best_error = ()\n\n range_min = (, idx - )\n range_max = ((gt_times), idx + )\n check_idx (range_min, range_max): \n check_idx excluded_indices:\n error = (gt_times[check_idx] - det_time)\n error < best_error:\n best_error = error\n best_idx = check_idx\n\n best_idx, best_error\nThanks for the good refactoring!
", "rawMarkdown": "Thanks for the good refactoring!", "votes": 2 }, { "id": 2516831, "postDate": "2023-11-08T02:24:23.530Z", "content": "The above code doesn't handle the case if the closest time_step is already matched.
\nWe should remove already matched gt indices from list of candidates.
Since my own prediction code predicts at most 1 event pair per night, it produces the same result, but it should be problematic in general cases.
\n ():\n \n idx = np.searchsorted(xs, value, side=)\n best_idx = \n best_error = ()\n\n range_min = (, idx - )\n range_max = ((xs), idx + )\n check_idx (\n range_min, range_max\n ): \n error = (xs[check_idx] - value)\n error < best_error:\n best_error = error\n best_idx = check_idx\n\n best_idx, best_error\n\n\n ():\n \n \n sorted_gt_times = np.asarray(sorted_gt_times)\n available_indices = np.asarray(\n ((((sorted_gt_times))) - excluded_indices), dtype=\n )\n sorted_gt_times = sorted_gt_times[available_indices]\n idx, error = find_nearest(sorted_gt_times, det_time)\n\n idx :\n , ()\n\n available_indices[idx], error\n\n\nI do not believe that publicizing this approach will spoil the efforts of the other participants.
\n
I have to disagree. If you would have just published this post, or maybe even the code, you could maybe argue that. But why publish the weights of a model that scores this high? Especially after there is just one month left. I really don't see the point in that…
", "rawMarkdown": ">I do not believe that publicizing this approach will spoil the efforts of the other participants.\n\nI have to disagree. If you would have just published this post, or maybe even the code, you could maybe argue that. But why publish the weights of a model that scores this high? Especially after there is just one month left. I really don't see the point in that...", "votes": 9, "replies": [ { "id": 2512469, "postDate": "2023-11-04T14:48:10.367Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2512489, "postDate": "2023-11-04T15:08:00.163Z", "content": "Assume that the training code is allowed to be published.
\nIf the training code is not publicly available, I do not think it is permissible to publish the model weights. If you don't publish the training code, you are not providing any value to the platform.
\nHowever, if the training code is to be made public, the weights of the model may or may not be made public.
\nAs long as the training code is publicly available, I think you are providing value to the platform.
\nI also think there is little difference between just forking the training code and submitting and using the published weights.
I think this discussion is a difference of ideas, so I will not discuss it further.
", "rawMarkdown": "Assume that the training code is allowed to be published.\n\nIf the training code is not publicly available, I do not think it is permissible to publish the model weights. If you don't publish the training code, you are not providing any value to the platform.\nHowever, if the training code is to be made public, the weights of the model may or may not be made public.\nAs long as the training code is publicly available, I think you are providing value to the platform.\nI also think there is little difference between just forking the training code and submitting and using the published weights.\n\nI think this discussion is a difference of ideas, so I will not discuss it further.", "replies": [ { "id": 2512548, "postDate": "2023-11-04T16:20:11.083Z", "content": "\n\nHowever, if the training code is to be made public, the weights of the model may or may not be made public.
\n
Exactly, but one results in much more \"forkmittings\" than the other. Much fewer people actually go out of their way and train the model and then submit it. And I know that it can be frustrating for someone that put a lot of effort into making a 50th place submission, to be 300th place the next day. That's why I also think you should be careful with publishing training code, especially 1 month before end, which I think is relatively late for the impact it has. I believe a lot of people love the competitive aspect of it, but realistically, that is gone for many people when anything past the top 2% might come from someone who has never touched a machine learning model in their life.
\nAlso, I wonder why you last submission is one month ago? Couldn't you have published the code back then?
", "rawMarkdown": "> However, if the training code is to be made public, the weights of the model may or may not be made public.\n\nExactly, but one results in much more \"forkmittings\" than the other. Much fewer people actually go out of their way and train the model and then submit it. And I know that it can be frustrating for someone that put a lot of effort into making a 50th place submission, to be 300th place the next day. That's why I also think you should be careful with publishing training code, especially 1 month before end, which I think is relatively late for the impact it has. I believe a lot of people love the competitive aspect of it, but realistically, that is gone for many people when anything past the top 2% might come from someone who has never touched a machine learning model in their life.\n\nAlso, I wonder why you last submission is one month ago? Couldn't you have published the code back then?", "votes": 13 }, { "id": 2512558, "postDate": "2023-11-04T16:35:14.577Z", "content": "\n\nMuch fewer people actually go out of their way and train the model and then submit it.
\n
I am skeptical on this point.
\nSimply using published weights is difficult to maintain a high ranking. For a time, more people may use the published weights, but I believe that gradually more people will use the weights from the improved model using the training code.
\n\nAlso, I wonder why you last submission is one month ago? Couldn't you have published the code back then?
\n
The reason is that a month ago my goal was to win this competition. However, I got busy and decided to withdraw from this competition. Therefore, I decided to publish the code I wrote a month ago.
", "rawMarkdown": "> Much fewer people actually go out of their way and train the model and then submit it. \n\nI am skeptical on this point.\nSimply using published weights is difficult to maintain a high ranking. For a time, more people may use the published weights, but I believe that gradually more people will use the weights from the improved model using the training code.\n\n> Also, I wonder why you last submission is one month ago? Couldn't you have published the code back then?\n\nThe reason is that a month ago my goal was to win this competition. However, I got busy and decided to withdraw from this competition. Therefore, I decided to publish the code I wrote a month ago.", "votes": 2 } ] }, { "id": 2513068, "postDate": "2023-11-05T06:48:47.603Z", "content": "Dude, the remained one month is just a start for the competitions. There still a lot people just began their work just now. If you wish to keep your efforts, you should ask hosts to close the \"join button\".
", "rawMarkdown": "Dude, the remained one month is just a start for the competitions. There still a lot people just began their work just now. If you wish to keep your efforts, you should ask hosts to close the \"join button\".", "votes": -2 } ] }, { "id": 2512388, "postDate": "2023-11-04T13:23:18.197Z", "content": "This is my impression from the experiments I have done, the 2D approach is not that important.
\nIt would be more accurate to capture more detailed information in the time direction rather than frequency.
\nThis means that the 1D approach may be better than the 2D approach.
It's hard to say. For example, I'm using 1d approach, but my best result is just LB 0.639 for now.
", "rawMarkdown": "It's hard to say. For example, I'm using 1d approach, but my best result is just LB 0.639 for now.", "replies": [ { "id": 2512566, "postDate": "2023-11-04T16:45:12.567Z", "content": "Look at rank #19 , he's using 1D approach at LB 0.718
", "rawMarkdown": "Look at rank #19 , he's using 1D approach at LB 0.718", "votes": 1 } ] } ] }, { "id": 2538861, "postDate": "2023-11-26T14:01:36.833Z", "content": "Thank you for your insightful posts, they have been very educational. I have a question regarding the 'duration' parameter. Have you ever tried setting it to a shorter value than what you currently have, and made a submission?
\nI am currently trying to shorten the 'duration', but I keep encountering a 'Submission Scoring Error'. If you have any knowledge on this issue, could you please advise me on how to resolve it?
\nThank you in advance.
", "rawMarkdown": "Thank you for your insightful posts, they have been very educational. I have a question regarding the 'duration' parameter. Have you ever tried setting it to a shorter value than what you currently have, and made a submission?\n\nI am currently trying to shorten the 'duration', but I keep encountering a 'Submission Scoring Error'. If you have any knowledge on this issue, could you please advise me on how to resolve it?\n\nThank you in advance.", "votes": 3, "replies": [ { "id": 2538944, "postDate": "2023-11-26T14:56:33Z", "content": "\nEven if the \"duration\" is small, it should still be possible to run.
\nThe following commands infer for train.
\nYou can use it for debugging.
rye python run/inference.py =train\nBecause you reduced the distance back then, you will achieve more results. The scoring system will limit the number of results input for each ID, and if exceeded, an error will be reported. You can increase the threshold while reducing the distance
", "rawMarkdown": "Because you reduced the distance back then, you will achieve more results. The scoring system will limit the number of results input for each ID, and if exceeded, an error will be reported. You can increase the threshold while reducing the distance", "votes": 5, "replies": [ { "id": 2539652, "postDate": "2023-11-27T08:36:48.653Z", "content": "How did you confirm that?
", "rawMarkdown": "How did you confirm that?", "votes": 1 }, { "id": 2547273, "postDate": "2023-12-03T11:15:23.347Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2547276, "postDate": "2023-12-03T11:16:53.547Z", "content": "I took the wrong sub and just changed the threshold for getting the step in the sub to get the score
", "rawMarkdown": "I took the wrong sub and just changed the threshold for getting the step in the sub to get the score", "votes": 1 } ] } ] }, { "id": 2513614, "postDate": "2023-11-05T15:08:29.703Z", "content": "That's a way to make your life harder by revealing the secrets of your solution.
\nHope you have more tricks and time to get into the top 6. (:
\nBy the way, do you know how to do the ModelCheckpoint in lightning with a metric computed in on_validation_epoch_end?
", "rawMarkdown": "That's a way to make your life harder by revealing the secrets of your solution.\n\nHope you have more tricks and time to get into the top 6. (:\n\nBy the way, do you know how to do the ModelCheckpoint in lightning with a metric computed in on_validation_epoch_end?", "votes": 3, "replies": [ { "id": 2513680, "postDate": "2023-11-05T16:04:48.507Z", "content": "Saving the best weights based on the evaluation index of the competition can be done with the following code.
\nrye run python run/train.py monitor=val_score monitor_mode=max
I meant in general not in your specific code.
\nWhen I compute the mAP in on_validation_epoch_end and log it, somehow ModelCheckpoint does not see the mAP to monitor it.
", "rawMarkdown": "I meant in general not in your specific code.\n\nWhen I compute the mAP in on_validation_epoch_end and log it, somehow ModelCheckpoint does not see the mAP to monitor it.", "votes": -2 }, { "id": 2514124, "postDate": "2023-11-06T04:31:40.150Z", "rawMarkdown": "", "votes": -2, "isDeleted": true }, { "id": 2514423, "postDate": "2023-11-06T09:34:56.900Z", "content": "Is this issue the one you encountered? https://github.com/Lightning-AI/lightning/issues/17131
", "rawMarkdown": "Is this issue the one you encountered? https://github.com/Lightning-AI/lightning/issues/17131", "votes": 2 }, { "id": 2514581, "postDate": "2023-11-06T11:53:27.960Z", "content": "Chatgpt doesn't know about it.
\nThank you for the link, @atamazian. It looks relevant to my problem because I'm using v2.0.
", "rawMarkdown": "Chatgpt doesn't know about it.\n\nThank you for the link, @atamazian. It looks relevant to my problem because I'm using v2.0.", "votes": -1 } ] } ] }, { "id": 2515757, "postDate": "2023-11-07T08:02:16.323Z", "content": "I am trying to train my own version of the model on kaggle and I am having a bit of difficulty. The scripting commands are not exactly intuitive for this effort. Does anyone know the right sequence to get the command to work? I have this
\n ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14900174%2Fb7b2da20ddeb4f4e8755a2e7c75c2c36%2FCapture.PNG?generation=1699344130927775&alt=media\")
The problem is that wandb waits for a user input, which is not possible in Jupyter notebook. I solved this issue by removing wandb logging and switching to local Tensorboard logging.
", "rawMarkdown": "The problem is that wandb waits for a user input, which is not possible in Jupyter notebook. I solved this issue by removing wandb logging and switching to local Tensorboard logging.", "replies": [ { "id": 2515835, "postDate": "2023-11-07T09:17:54.363Z", "content": "You can change original files within Jupyter notebook by inserting modified code to a single cell with putting %%writefile <filename> command on top.
There is a cmd button on Kaggle page, you can run command there.
Where exactly is that? That would help me a lot as I was looking to try a spin-off of this strategy and have been struggling to get the training script to function through several alterations
", "rawMarkdown": "Where exactly is that? That would help me a lot as I was looking to try a spin-off of this strategy and have been struggling to get the training script to function through several alterations", "votes": 1 }, { "id": 2521358, "postDate": "2023-11-11T16:24:05.700Z", "content": "Don't know it will help you or not, I haven't try it. It is actually a Console, maybe it is quite different from Terminal or shell on PC.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13000097%2F76e4c6a684664be69815cf3643e3ebe5%2FConsole_kaggle.png?generation=1699719745001624&alt=media\")
Yeah, the code is really interesting, it is a shame that is in the least readable format available to us. It is a lot of work to even be able to adjust the training for yourself
", "rawMarkdown": "Yeah, the code is really interesting, it is a shame that is in the least readable format available to us. It is a lot of work to even be able to adjust the training for yourself", "votes": -2 }, { "id": 2531776, "postDate": "2023-11-20T14:17:38.827Z", "content": "I'm in the exact same situation. Reading the comments below doesn't help, so if anyone has a cool solution, I'd appreciate it
", "rawMarkdown": "I'm in the exact same situation. Reading the comments below doesn't help, so if anyone has a cool solution, I'd appreciate it" } ] }, { "id": 2530545, "postDate": "2023-11-19T08:35:46.677Z", "content": "If you want to make a contribution to this repository, please write a PR or issue summary and link to it in this thread as well to avoid private sharing.
\ntemplate
\nMy contributions/issues submitted:
\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/pull/30
\nWhen you intend to modify the model, you may want to note that the annotation of the model output shap in spec2Dcnn is different from that in unet1ddecoder (with the addition of transpose).
My issues submitted:
\nBTW, how could it be private sharing if the GitHub repo's URL has been shared publicly beforehand in Kaggle? I thought any information sharing at an external website that had been shared publicly beforehand in Kaggle was not considered as private sharing.
", "rawMarkdown": "My issues submitted:\n\n# Cannot run run/prepare_data.py\n- summary: The command instructed on the README was wrong\n- link: https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/issues/39\n\n# Running run/prepare_data.py killed\n- summary: It may require > 32 GB RAM\n- link: https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/issues/45\n\nBTW, how could it be private sharing if the GitHub repo's URL has been shared publicly beforehand in Kaggle? I thought any information sharing at an external website that had been shared publicly beforehand in Kaggle was not considered as private sharing.", "votes": 2, "replies": [ { "id": 2531517, "postDate": "2023-11-20T09:25:33.687Z", "content": "@riow1983
\nThank you for your response.
It may not fall under private sharing, but I think many people don't even look through the issues and PR.
\nI asked you to post the content in the thread as well, as I think it is unfair to disadvantage people by not looking at the issues and PRs.
Thank you for sharing the code.
\nDid you execute the training code using a Kaggle notebook, or did you do it on your local PC? I'd like to try the training you published on Git in a Kaggle notebook, but I'm concerned that it might not be feasible due to GPU and memory issues.
", "rawMarkdown": "@tubotubo \n\nThank you for sharing the code.\n\nDid you execute the training code using a Kaggle notebook, or did you do it on your local PC? I'd like to try the training you published on Git in a Kaggle notebook, but I'm concerned that it might not be feasible due to GPU and memory issues.", "votes": 3, "replies": [ { "id": 2513083, "postDate": "2023-11-05T07:04:11.407Z", "content": "I experimented in a local environment, but this code works in a kaggle environment as well.
\nTry running it with dir=kaggle.
Or you can change local.yaml content.
I think publishing notebook like this is not good for the people who are serious to join and have score lower than this notebook.
", "rawMarkdown": "I think publishing notebook like this is not good for the people who are serious to join and have score lower than this notebook.", "votes": 4 }, { "id": 2513035, "postDate": "2023-11-05T05:54:29.170Z", "content": "I am interested in Gaussian label width (sigma). How width do you set the value? In your notebook it is set to 10, how many seconds does this correspond to? In the distributed correct answer label, the number of significant digits is about 1 minute, so it seems more reasonable to look at it with a sigma of about 1 minute.
", "rawMarkdown": "I am interested in Gaussian label width (sigma). How width do you set the value? In your notebook it is set to 10, how many seconds does this correspond to? In the distributed correct answer label, the number of significant digits is about 1 minute, so it seems more reasonable to look at it with a sigma of about 1 minute.", "votes": 4, "replies": [ { "id": 2513078, "postDate": "2023-11-05T07:00:38.667Z", "content": "\n\nhow many seconds does this correspond to?
\n
10 * 5 = 50 [sec]
\n\n\nI am interested in Gaussian label width (sigma). How width do you set the value?
\n
Good point.
\nI am training the model using a sigma that I set up at the beginning.
However, this sigma is very important.
\nBy setting the sigma and duration properly, the accuracy (local cv) can be improved by about 0.01-0.02.
The following command can be used for grid search, so try it.
\nrye run python run/train.py -m duration=100,1000,10000 sigma=5,10,15,20
Shall we define sigma value only as command line parameter?
\nI did not find it in any *.py aor *.yaml file.
The default parameters are written under run/conf/.
\nsigma is set in run/conf/dataset/seg.yaml.
\nConfig is specified in src/conf.py.
thank you very much! ))
", "rawMarkdown": "thank you very much! ))" }, { "id": 2526825, "postDate": "2023-11-16T04:41:05.570Z", "content": "what is the best criteria at experiments or grid search?
\nminimum val_loss or maximum val_score?
Better to evaluate with val_score
", "rawMarkdown": "Better to evaluate with val_score" } ] } ] }, { "id": 2518901, "postDate": "2023-11-09T15:43:17.463Z", "content": "@tubotubo if you are working with architectures like UNet please take a look about usage the links between the same levels of encoder and decoder (the left and right parts of UNet) - this usually helps to increase the score for such a kind of models
", "rawMarkdown": "@tubotubo if you are working with architectures like UNet please take a look about usage the links between the same levels of encoder and decoder (the left and right parts of UNet) - this usually helps to increase the score for such a kind of models", "votes": 3, "replies": [ { "id": 2519620, "postDate": "2023-11-10T07:46:47.703Z", "content": "I think he already has skip connection in his code. What else can we try
", "rawMarkdown": "I think he already has skip connection in his code. What else can we try", "replies": [ { "id": 2520686, "postDate": "2023-11-11T04:54:22.047Z", "content": "It seems like he's talking about the seqence length.
", "rawMarkdown": "It seems like he's talking about the seqence length." } ] } ] }, { "id": 2514077, "postDate": "2023-11-06T02:43:31.800Z", "content": "I don't think it's a selfish behavior. Deciding to withdraw oneself but not withdraw models that one has created so far letting them be available to the public is understandable. The more the models are adopted by other participants, the more the models are getting seminal. That's another way of participating in a kaggle competition to the end in a busy period.
", "rawMarkdown": "I don't think it's a selfish behavior. Deciding to withdraw oneself but not withdraw models that one has created so far letting them be available to the public is understandable. The more the models are adopted by other participants, the more the models are getting seminal. That's another way of participating in a kaggle competition to the end in a busy period.", "votes": 2, "replies": [ { "id": 2515671, "postDate": "2023-11-07T05:25:36.597Z", "content": "Totally agree, the goal is to train the best model possible, maybe a lot of people will fine tune this model and get even higher scores then the top5, which would be amazing. This would probably make the competition hosts happy too.
", "rawMarkdown": "Totally agree, the goal is to train the best model possible, maybe a lot of people will fine tune this model and get even higher scores then the top5, which would be amazing. This would probably make the competition hosts happy too. ", "votes": -5 } ] }, { "id": 2560628, "postDate": "2023-12-13T20:35:02.843Z", "content": "Wow!!! Thanks for this! So my thinking of using spectrograms was not a bad one. This is encouraging. Thanks again.
", "rawMarkdown": "Wow!!! Thanks for this! So my thinking of using spectrograms was not a bad one. This is encouraging. Thanks again.", "votes": 1 }, { "id": 2546713, "postDate": "2023-12-02T18:23:17.787Z", "content": "I am wondering where the idea for this architecture comes from. The Unet is by itself an encoder/decoder architecture, and the encoder is a feature extractor. But instead of just using a single unet (1d) as both encoder and feature extractor, you also first extract features using multiple kernel sizes. Then you decode the 2d output into 1d using a separate unet style model. Why not just use a single 1d unet?
\nI'm curious what the 2d heatmap represents. Since this is a 1d problem, why is representing the data as 2d a good idea?
", "rawMarkdown": "I am wondering where the idea for this architecture comes from. The Unet is by itself an encoder/decoder architecture, and the encoder is a feature extractor. But instead of just using a single unet (1d) as both encoder and feature extractor, you also first extract features using multiple kernel sizes. Then you decode the 2d output into 1d using a separate unet style model. Why not just use a single 1d unet?\n\nI'm curious what the 2d heatmap represents. Since this is a 1d problem, why is representing the data as 2d a good idea?", "votes": 1, "replies": [ { "id": 2547056, "postDate": "2023-12-03T06:08:58.140Z", "content": "Why is hour converted to degrees using
\ndef to_coord(x: pl.Expr, max_: int, : ) -> [pl.Expr]:\n = * np. * (x % max_) / max_\n x_sin = .()\n x_cos = .()\n\n return [x_sin.alias(f), x_cos.alias(f)]\nand how important is that? Why not just convert to the range [0, 1] ?
\nIt looks like it makes it smoother. Instead of a sharp peak from 23 to 0, it is smoother.
\nWhy are both hour_sin and hour_cos used? Isn't 1 enough?
When using only the sine function to represent a 24-hour cycle, there is an overlap in the values for AM and PM times, which can lead to ambiguity.
", "rawMarkdown": "When using only the sine function to represent a 24-hour cycle, there is an overlap in the values for AM and PM times, which can lead to ambiguity.", "replies": [ { "id": 2547611, "postDate": "2023-12-03T16:41:29.373Z", "content": "Thanks. I found this article helpful
", "rawMarkdown": "Thanks. I found [this article](https://ianlondon.github.io/blog/encoding-cyclical-features-24hour-time/) helpful" } ] } ] }, { "id": 2544143, "postDate": "2023-11-30T15:53:40.683Z", "content": "Thank you for your project, it's really great and boosted my knowledge a lot! May I ask how do you debug your code if something went wrong? Do you use special tools for that or just use basic things like python debugger or print statements?
", "rawMarkdown": "Thank you for your project, it's really great and boosted my knowledge a lot! May I ask how do you debug your code if something went wrong? Do you use special tools for that or just use basic things like python debugger or print statements?", "votes": 1, "replies": [ { "id": 2544321, "postDate": "2023-11-30T18:39:47.650Z", "content": "I use the \"pdb\" for debugging
", "rawMarkdown": "I use the \"pdb\" for debugging", "votes": 1 } ] }, { "id": 2538567, "postDate": "2023-11-26T09:13:48.983Z", "content": "Did anyone try to run the training script in kaggle and it just crashes the notebook after maybe 20+ epochs?
", "rawMarkdown": "Did anyone try to run the training script in kaggle and it just crashes the notebook after maybe 20+ epochs?", "votes": 1 }, { "id": 2512326, "postDate": "2023-11-04T11:21:17.243Z", "content": "Js, you just ruined the competition
", "rawMarkdown": "Js, you just ruined the competition", "votes": -3 }, { "id": 2547808, "postDate": "2023-12-03T21:50:15.383Z", "content": "Just so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels? In theory this would allow for predicting both an onset and wakeup event at the same timestep.
\nI'm also wondering about the loss function. You use BCEWithLogitsLoss with no weights. Since onset/wakeup are rare, wouldn't this loss function bias towards the background class?
", "rawMarkdown": "Just so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels? In theory this would allow for predicting both an onset and wakeup event at the same timestep.\n\nI'm also wondering about the loss function. You use BCEWithLogitsLoss with no weights. Since onset/wakeup are rare, wouldn't this loss function bias towards the background class?\n\n", "votes": 2, "replies": [ { "id": 2548219, "postDate": "2023-12-04T08:17:47.787Z", "content": "\n\nJust so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels?
\n
Yes. There is a possibility of predicting multiple labels in 1step, although the number may be small.
\n\n\nwouldn't this loss function bias towards the background class?
\n
Since ap depends only on the order of the scores, it does not matter if bg scores higher.
", "rawMarkdown": "> Just so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels?\n\nYes. There is a possibility of predicting multiple labels in 1step, although the number may be small.\n\n> wouldn't this loss function bias towards the background class?\n\nSince ap depends only on the order of the scores, it does not matter if bg scores higher.", "votes": 2, "replies": [ { "id": 2548675, "postDate": "2023-12-04T15:46:55.007Z", "content": "To clarify, I’m surprised the model is able to learn the signal for onset/wakeup. There are 2880 timesteps, and only ~10/2880 > 0 for onset/wakeup. So I’m surprised the model is able to learn this signal.
", "rawMarkdown": "To clarify, I’m surprised the model is able to learn the signal for onset/wakeup. There are 2880 timesteps, and only ~10/2880 > 0 for onset/wakeup. So I’m surprised the model is able to learn this signal. ", "votes": 2 } ] } ] }, { "id": 2519326, "postDate": "2023-11-09T23:09:45.827Z", "content": "What is the best way to further tune this? I am thinking either:
\nDoes anyone know other ways (better ways) to do it?
", "rawMarkdown": "What is the best way to further tune this? I am thinking either:\n1. clone the code to my own github repo, then change the code there.\n2. create a new kaggle notebook, copy the code over from github (but was there a limitation that this was not done like that in the first place?)\n\nDoes anyone know other ways (better ways) to do it?", "votes": 2, "replies": [ { "id": 2519332, "postDate": "2023-11-09T23:18:40.117Z", "content": "Just take the ideas, apply it in your own code. You will benefit more in the long run.
\nThe most critical addition in this code to the existing ideas is the use of 3d outputs and the loss function.
Yeah, I think you have a good point here. Thanks
", "rawMarkdown": "Yeah, I think you have a good point here. Thanks", "votes": 1 }, { "id": 2536185, "postDate": "2023-11-24T02:11:28.440Z", "content": "Did 2. Because ive got no gpu locally
", "rawMarkdown": "Did 2. Because ive got no gpu locally" } ] } ] }, { "id": 2512458, "postDate": "2023-11-04T14:36:00.230Z", "content": "Thanks for the post @tubotubo , you just saved me a lot of time and submission attempts with your comment about having multiple files in the output directory. Kudos!
", "rawMarkdown": "Thanks for the post @tubotubo , you just saved me a lot of time and submission attempts with your comment about having multiple files in the output directory. Kudos!", "votes": 2 }, { "id": 2512216, "postDate": "2023-11-04T08:40:00.367Z", "content": "It's a nice idea to use 2d heatmaps and UNet encoder-decoder scheme. I noticed that you use quite low downsampling (just 2). I was using 12 until now.
", "rawMarkdown": "It's a nice idea to use 2d heatmaps and UNet encoder-decoder scheme. I noticed that you use quite low downsampling (just 2). I was using 12 until now.", "votes": 2 }, { "id": 2512437, "postDate": "2023-11-04T14:16:49.850Z", "content": "@tubotubo very nice approach. Similar thing, that was discussed on the other thread recently. Could you clarify, in terms of feature extractor, you implemented 4 ways to do this. Do you use only one method at a time in your pipeline? Or do you use them somehow one by one, for example, CNN -> LSTM etc?
\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?
", "rawMarkdown": "@tubotubo very nice approach. Similar thing, that was discussed on the other thread recently. Could you clarify, in terms of feature extractor, you implemented 4 ways to do this. Do you use only one method at a time in your pipeline? Or do you use them somehow one by one, for example, CNN -> LSTM etc? \n\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?", "replies": [ { "id": 2512495, "postDate": "2023-11-04T15:15:01.557Z", "content": "\n\nDo you use only one method at a time in your pipeline?
\n
Pipelines use one method at a time.
\n\n\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?
\n
This task is very similar to soud event detection (SED).
\nFor this reason, I tried the 2D approach often used in SED.
\nSound Event Detection: A Tutorial
Thank you. I agree, that it looks exactly like SED. Could you also tell, which loss function did you use? I could not find it in your git code.
", "rawMarkdown": "Thank you. I agree, that it looks exactly like SED. Could you also tell, which loss function did you use? I could not find it in your git code. " }, { "id": 2512802, "postDate": "2023-11-04T20:58:00.940Z", "content": "It uses BCEWithLogitsLoss (it's in the models files, e.g. src/models/spec1D.py)
", "rawMarkdown": "It uses BCEWithLogitsLoss (it's in the models files, e.g. src/models/spec1D.py)", "votes": 2 } ] } ] }, { "id": 2512984, "postDate": "2023-11-05T05:01:18.083Z", "content": "You ruined the LB in the short term
", "rawMarkdown": "You ruined the LB in the short term", "votes": -3, "replies": [ { "id": 2512987, "postDate": "2023-11-05T05:03:33.270Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2512989, "postDate": "2023-11-05T05:04:32.403Z", "content": "Even among the top 100 individuals, some have only submitted once, is this fair for those who have been involved and innovative from the beginning
", "rawMarkdown": "Even among the top 100 individuals, some have only submitted once, is this fair for those who have been involved and innovative from the beginning", "votes": 1 } ] }, { "id": 2512915, "postDate": "2023-11-05T02:44:33.097Z", "content": "Nice sharings!
\nps: I'd always prefer a one-notebook training and inference version of code for competitions. If it is possible to combine them into one notebook, it would be nice. 😂
", "rawMarkdown": "Nice sharings!\n\nps: I'd always prefer a one-notebook training and inference version of code for competitions. If it is possible to combine them into one notebook, it would be nice. 😂", "votes": -7 }, { "id": 2544149, "postDate": "2023-11-30T16:07:49.420Z", "content": "let me ask you a question
\nin config files we can find:
\noptimizer:\n lr: \nand in logger we can see:
\nlr-AdamW ████████▇▇▇▇▇▆▆▆▆▅▅▅▄▄▄▄▃▃▃▃▂▂▂▂▂▂▁▁▁▁▁▁
is the config lr the final value of learning rate or starting value?
", "rawMarkdown": "let me ask you a question\nin config files we can find:\n```python\n# optimizer\noptimizer:\n lr: 0.0005\n```\n\nand in logger we can see:\nlr-AdamW ████████▇▇▇▇▇▆▆▆▆▅▅▅▄▄▄▄▃▃▃▃▂▂▂▂▂▂▁▁▁▁▁▁\n\nis the config lr the final value of learning rate or starting value?", "replies": [ { "id": 2544322, "postDate": "2023-11-30T18:40:52.897Z", "content": "Because of the learning rate scheduler, lr varies with each step.
", "rawMarkdown": "Because of the learning rate scheduler, lr varies with each step.", "replies": [ { "id": 2544334, "postDate": "2023-11-30T18:47:21.273Z", "content": "But at wich epoch do we have lr=0.0005?
", "rawMarkdown": "But at wich epoch do we have lr=0.0005?" }, { "id": 2544339, "postDate": "2023-11-30T18:56:30.160Z", "content": "lr is 0.0005 at step=0.
", "rawMarkdown": "lr is 0.0005 at step=0.", "votes": 1 }, { "id": 2545653, "postDate": "2023-12-01T16:18:18.840Z", "content": "thank you very much!
", "rawMarkdown": "thank you very much!" } ] }, { "id": 2546851, "postDate": "2023-12-02T23:35:32.517Z", "content": "Here's how **lr ** changes:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8763601%2F8340060a52a73a6cbc87fd3dfcba52e3%2Flr.PNG?generation=1701560088302844&alt=media\")
@tubotubo can some one explain me how you created the fold0.yaml config. is it Group Fold or StratifiedGroupKFold?..is it a 5 fold or 4fold. how did you do that. could you please explain?
", "rawMarkdown": "@tubotubo can some one explain me how you created the fold0.yaml config. is it Group Fold or StratifiedGroupKFold?..is it a 5 fold or 4fold. how did you do that. could you please explain?", "replies": [ { "id": 2541993, "postDate": "2023-11-29T01:49:52.683Z", "content": "It seems that fold_0.yaml puts first 80% of series_ids to train and the last 20% to val. The github repo contains config only for fold_0, but, you can add fold_1.yaml, fold_N.yaml, … and split them as you want.
", "rawMarkdown": "It seems that fold_0.yaml puts first 80% of series_ids to train and the last 20% to val. The github repo contains config only for fold_0, but, you can add fold_1.yaml, fold_N.yaml, ... and split them as you want.", "votes": 3 } ] }, { "id": 2530429, "postDate": "2023-11-19T06:19:37.363Z", "content": "Has anyone encountered the same problem as me? I averaged three 729 models (two with different architectures and one with a slightly tweaked architecture) and the other 72+ models, but the score was still lower than the previous 729 model and several 72+ models. I don’t know if it’s because of the diversity. . .
", "rawMarkdown": "Has anyone encountered the same problem as me? I averaged three 729 models (two with different architectures and one with a slightly tweaked architecture) and the other 72+ models, but the score was still lower than the previous 729 model and several 72+ models. I don’t know if it’s because of the diversity. . .", "replies": [ { "id": 2530472, "postDate": "2023-11-19T07:30:28.920Z", "content": "You may overfit on the training set.
", "rawMarkdown": "You may overfit on the training set.", "replies": [ { "id": 2530491, "postDate": "2023-11-19T07:49:08.283Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2528464, "postDate": "2023-11-17T11:44:15.900Z", "content": "nicework,really helpful
", "rawMarkdown": "nicework,really helpful" }, { "id": 2524468, "postDate": "2023-11-14T09:07:50.843Z", "content": "May I ask you how long it takes to train a round（just one epoch）, and what kind of environment is it based on (what kind of GPU is used)? It's my first time to participate in this kind of competition, so I don't know if I have successfully run the code.
", "rawMarkdown": "May I ask you how long it takes to train a round（just one epoch）, and what kind of environment is it based on (what kind of GPU is used)? It's my first time to participate in this kind of competition, so I don't know if I have successfully run the code.", "replies": [ { "id": 2524654, "postDate": "2023-11-14T12:28:25.050Z", "content": "RTX3090 takes about 45min for full train with 50 epochs
\nI use wsl2 @windows11
Thank you for the reply
", "rawMarkdown": "Thank you for the reply" } ] }, { "id": 2525324, "postDate": "2023-11-15T03:28:48.060Z", "content": "On Colab with T4 (high RAM) it takes about 50 seconds per epoch.
", "rawMarkdown": "On Colab with T4 (high RAM) it takes about 50 seconds per epoch." }, { "id": 2525523, "postDate": "2023-11-15T07:04:21.383Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2523609, "postDate": "2023-11-13T15:44:56.230Z", "content": "Nice working and moreover approach, awesome all @tubotubo
", "rawMarkdown": "Nice working and moreover approach, awesome all @tubotubo " }, { "id": 2522477, "postDate": "2023-11-12T17:40:59.383Z", "content": "I notice that you use 24 workers for your dataloaders. Do you mind me asking what specifications your computer has? I would expect with that many workers your RAM would fill up quickly? Or is there a trick I'm not aware of to avoid that problem?
", "rawMarkdown": "I notice that you use 24 workers for your dataloaders. Do you mind me asking what specifications your computer has? I would expect with that many workers your RAM would fill up quickly? Or is there a trick I'm not aware of to avoid that problem?", "replies": [ { "id": 2522499, "postDate": "2023-11-12T17:59:50.420Z", "content": "\n\nDo you mind me asking what specifications your computer has?
\n
The number of threads on my computer is 24.
\n\n\nI would expect with that many workers your RAM would fill up quickly? Or is there a trick I'm not aware of to avoid that problem?
\n
num_workers is the number of parallelism and has nothing to do with RAM.
\nRAM used depends on batch_size
Greetings and thanks for sharing such a nice solution!
\nAnswer my question, please.
\nHere You wrote that wavwgram is made from 3 features - anglez, enmo and hour.
\nBut prepare_data.py file contains code for 8 features:
FEATURE_NAMES = [\n ,\n ,\n ,\n ,\n ,\n ,\n ,\n ,\n]\nHow should we understand this?
\nUPDATE
\nI found it:
\nGreetings and thanks for sharing such a nice solution!
Answer my question, please.
\nHere You wrote that wavwgram is made from 3 features - anglez, enmo and hour.
\nBut prepare_data.py file contains code for 8 features:
FEATURE_NAMES = [\n ,\n ,\n ,\n ,\n ,\n ,\n ,\n ,\n]\nHow should we understand this?
\nUPDATE
\nI found it in conf/train.yaml:
\n\nfeatures:\n - \n - \n \n \n - \n - \n \n \nAnyway, it seems to be 4 features :)
", "rawMarkdown": "Greetings and thanks for sharing such a nice solution!\n\nAnswer my question, please.\nHere You wrote that wavwgram is made from 3 features - `anglez, enmo and hour`.\nBut prepare_data.py file contains code for 8 features:\n```python\nFEATURE_NAMES = [\n \"anglez\",\n \"enmo\",\n \"hour_sin\",\n \"hour_cos\",\n \"month_sin\",\n \"month_cos\",\n \"minute_sin\",\n \"minute_cos\",\n]\n```\n\nHow should we understand this?\n\nUPDATE\n\nI found it:\nGreetings and thanks for sharing such a nice solution!\n\nAnswer my question, please.\nHere You wrote that wavwgram is made from 3 features - `anglez, enmo and hour`.\nBut prepare_data.py file contains code for 8 features:\n```python\nFEATURE_NAMES = [\n \"anglez\",\n \"enmo\",\n \"hour_sin\",\n \"hour_cos\",\n \"month_sin\",\n \"month_cos\",\n \"minute_sin\",\n \"minute_cos\",\n]\n```\n\nHow should we understand this?\n\nUPDATE\n\nI found it in conf/train.yaml:\n\n```python\n# feature\nfeatures:\n - \"anglez\"\n - \"enmo\"\n # - \"month_sin\"\n # - \"month_cos\"\n - \"hour_sin\"\n - \"hour_cos\"\n # - \"minute_sin\"\n # - \"minute_cos\"\n```\n\nAnyway, it seems to be 4 features :)", "replies": [ { "id": 2520116, "postDate": "2023-11-10T15:29:23.753Z", "content": "Sorry, Disscussion is wrong.
\nMy model uses 4 features: anglez, emmo, hour_sin, hour_cos
", "rawMarkdown": "Sorry, Disscussion is wrong.\n\nMy model uses 4 features: anglez, emmo, hour_sin, hour_cos", "votes": 4 } ] }, { "id": 2513057, "postDate": "2023-11-05T06:30:37.227Z", "content": "Thank you for sharing. Keep up the good work.
", "rawMarkdown": "Thank you for sharing. Keep up the good work. " }, { "id": 2547382, "postDate": "2023-12-03T13:03:58.253Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2547401, "postDate": "2023-12-03T13:26:43.210Z", "content": "I think the hidden data set is more diverse than the public data set. Maybe you can try changing the original training/validation ratio and model fusion of tubo to enhance robustness.
", "rawMarkdown": "I think the hidden data set is more diverse than the public data set. Maybe you can try changing the original training/validation ratio and model fusion of tubo to enhance robustness.", "replies": [ { "id": 2547405, "postDate": "2023-12-03T13:30:49.587Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2547412, "postDate": "2023-12-03T13:38:39.140Z", "rawMarkdown": "", "isDeleted": true } ] } ] }, { "id": 2537312, "postDate": "2023-11-25T03:38:23.210Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2537741, "postDate": "2023-11-25T12:59:48.783Z", "content": "He does, in the prepare_data pipeline.
", "rawMarkdown": "He does, in the prepare_data pipeline.", "votes": 1 } ] }, { "id": 2531446, "postDate": "2023-11-20T08:02:15.757Z", "rawMarkdown": "", "votes": 5, "isDeleted": true, "replies": [ { "id": 2531537, "postDate": "2023-11-20T09:49:24.973Z", "content": "\nThanks for the compliment on my code. I am very happy.
\nFirst of all, the answer to your question is to create a prototype and then a github repo.
\nAI/ML projects (especially kaggle) change a lot. Therefore, I believe you need to prototype first to get a deep understanding of the problem.
\nI will use the following process to mature the code.
\nI have not studied software engineering, so please read this as an opinion.
This repo was created as follows.
\n1st iteration
\n2nd iteration
\nThanks for reading to the end.
\nI hope you find it useful.
Done, see below.
", "rawMarkdown": "Done, see below.", "votes": 1 } ] } ] }, { "id": 2530425, "postDate": "2023-11-19T06:16:22.430Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2530420, "postDate": "2023-11-19T06:13:17.827Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2526087, "postDate": "2023-11-15T15:15:03.637Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2523739, "postDate": "2023-11-13T17:35:51.610Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2523786, "postDate": "2023-11-13T18:28:24.923Z", "content": "In my environment, learning has been done successfully.
\nPlease check your environment
I'm also unsure about this
\nWhy don't we take some time and have a closer look at the code? :)
Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" }, { "id": 2543292, "postDate": "2023-11-30T00:29:34.453Z", "content": "Thanks for your sharing
", "rawMarkdown": "Thanks for your sharing" }, { "id": 2534871, "postDate": "2023-11-23T00:52:23.577Z", "content": "Thanks for your sharing
", "rawMarkdown": "Thanks for your sharing" }, { "id": 2515514, "postDate": "2023-11-07T01:59:32.543Z", "content": "thanks for sharing!
", "rawMarkdown": "thanks for sharing! " } ], "comments": [ { "id": 2512328, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-04T11:24:25.490000", "content": "I do not believe that publicizing this approach will spoil the efforts of the other participants.
\nThere are two reasons.
\nThe first is that my approach does not encompass all other approaches. I believe that a better model can be created by combining my approach with your approach.
\nThe second is that the code I have published has room for many improvements and I believe there is ample time to make improvements.
\nFor these two reasons, I thought it was worth releasing the code and model to the public.
Temporarily, LB may be full of people who just forked it, but it will be weeded out by those who are serious about experimenting.
", "votes": 48, "replies": [ { "id": 2512971, "author_name": "Sadhaklal", "author_url": "", "post_date": "2023-11-05T04:21:46.723000", "content": "I appreciate you sharing your approach. But do I appreciate you publishing the \"CMI-submit\" notebook? I'm not sure.
\nJust look at the LB now. So many people have 0.707 score (without putting in any effort).
\nYour action will lead to less \"variety of solutions\". People who were working on creative approaches will give up, and focus on replicating your approach. In the end, this will reduce the variety of solutions the community develops for this competition.
\nI don't know if this is a good thing or a bad thing.
", "votes": 7, "replies": [ { "id": 2512979, "author_name": "DongYK", "author_url": "", "post_date": "2023-11-05T04:46:56.670000", "content": "I think people need to add his approach to their own notebooks. So, solutions are getting more various and progressive.
", "votes": 0, "replies": [] }, { "id": 2512980, "author_name": "Sadhaklal", "author_url": "", "post_date": "2023-11-05T04:51:32.840000", "content": "I have no problem with him sharing the approach.
\nMy concern is about sharing the \"CMI-submit\" notebook.
", "votes": 3, "replies": [] } ] }, { "id": 2514892, "author_name": "Lamal", "author_url": "", "post_date": "2023-11-06T15:38:48.740000", "content": "Much advovated for your kind sharing @tubotubo
", "votes": 0, "replies": [ { "id": 2522105, "author_name": "lhchina", "author_url": "", "post_date": "2023-11-12T11:48:23.423000", "content": "Can you share more informations about post_process.score_th and post_process.distance?
\nAnd Why use this? @tubotubo
\nThanks
The find_peak function of scipy is used to detect events.
\nSee the following page for details
\nhttps://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.find_peaks.html
\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/blob/main/src/utils/post_process.py
In the epochs,the valid_loss decrease,but the valid_score not increase.
\nlike this:
\nEpoch 0/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:48 • 0:00:00 2.44it/s v_num: 3vs7Epoch 0, global step 119: 'val_loss' reached 0.06178 (best 0.06178), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/checkpoints/epoch=0-step=119.ckpt' as top 1
\nSaved best model 0.061714774133353835 -> 0.040320251738368874
\nEpoch 1/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.66it/s v_num: 3vs7 val_loss: 0.062 val_score: 0.000 train_loss: 0.244
\nEpoch 1, global step 238: 'val_loss' reached 0.04033 (best 0.04033), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check
\nEpoch 2/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.85it/s v_num: 3vs7 val_loss: 0.040 val_score: 0.000 train_loss : 0.175
\nSaved best model 0.040320251738368874 -> 0.03305796951684632
\nEpoch 3/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.79it/s v_num: 3vs7 val_loss: 0.043 val_score: 0.000 train_loss:0.172
\nEpoch 3, global step 476: 'val_loss' reached 0.03304 (best 0.03304), saving model to './child-mind-institute-detect-sleep-states/5vzv3vs7/check
\nEpoch 4/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:43 • 0:00:00 2.89it/s v_num: 3vs7 val_loss: 0.033 val_score: 0.000 train_loss: 0.170
\nEpoch 5/49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119/119 0:00:44 • 0:00:00 2.81it/s v_num: 3vs7 val_loss: 0.034 val_score: 0.000 train_loss:0.166
It was fine the first time I ran it, and then this happened
", "votes": 1, "replies": [] }, { "id": 2522885, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-13T06:16:09.313000", "content": "\nPlease let me know what commands you have executed
", "votes": 0, "replies": [] }, { "id": 2522897, "author_name": "lhchina", "author_url": "", "post_date": "2023-11-13T06:26:19.390000", "content": "python -m run.train exp_name='exp001' num_workers=2 batch_size=32 duration=5760 downsample_rate=2
", "votes": 0, "replies": [] }, { "id": 2522901, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-13T06:33:21.183000", "content": "\nThe learning is going well in my environment.
\nPlease try git pull and then run the command again.
I'm trying it,maybe I change some code in the modelmodule.py.like this:
\nclass PLSleepModel(LightningModule):
\n def init(
\n self,
\n cfg: TrainConfig,
\n val_event_df: pl.DataFrame,
\n feature_dim: int,
\n num_classes: int,
\n duration: int,
\n ):
\n super().init()
\n self.cfg = cfg
\n self.val_event_df = val_event_df
\n num_timesteps = nearest_valid_size(int(duration * cfg.upsample_rate), cfg.downsample_rate)
\n self.model = get_model(
\n cfg,
\n feature_dim=feature_dim,
\n n_classes=num_classes,
\n num_timesteps=num_timesteps // cfg.downsample_rate,
\n )
\n self.duration = duration
\n self.validation_step_outputs: list = []
\n self.__best_loss = np.inf
\n self.__score = 0
def forward(\n self,\n x: torch.Tensor,\n labels: Optional[torch.Tensor] = None,\n do_mixup: bool = ,\n do_cutmix: bool = ,\n) -> ModelOutput:\n return self.model(x, labels, do_mixup, do_cutmix)\n\ndef training_step(self, batch, batch_idx):\n do_mixup = np.random.rand() < self.cfg.aug.mixup_prob\n do_cutmix = np.random.rand() < self.cfg.aug.cutmix_prob\n output = self.model(batch[], batch[], do_mixup, do_cutmix)\n\n self.log(\n ,\n output.loss.detach().item(),\n =,\n =,\n =,\n =,\n )\n return output.loss\n\ndef validation_step(self, batch, batch_idx):\n output = self.model.predict(batch[], self.duration, batch[])\n self.validation_step_outputs.append(\n (\n batch[],\n output.labels.detach().cpu().numpy(),\n output.preds.detach().cpu().numpy(),\n output.loss.detach().item(),\n )\n )\n self.log(\n ,\n output.loss.detach().item(),\n =,\n =,\n =,\n =,\n )\n\n return output.loss\n\ndef on_validation_epoch_end(self):\n keys = []\n x self.validation_step_outputs:\n keys.extend(x[0])\n labels = np.concatenate([x[1] x self.validation_step_outputs])\n preds = np.concatenate([x[2] x self.validation_step_outputs])\n losses = np.array([x[3] x self.validation_step_outputs])\n loss = losses.mean()\n\n val_pred_df = post_process_for_seg(\n =keys,\n =preds,\n =self.cfg.pp.score_th,\n =self.cfg.pp.distance,\n )\n score = event_detection_ap(self.val_event_df.to_pandas(), val_pred_df.to_pandas())\n self.log(, score, =, =, =, =)\n\n loss < self.__best_loss:\n #np.save(, np.array(keys))\n #np.save(, labels)\n #np.save(, preds)\n val_pred_df.write_csv()\n torch.save(self.model.state_dict(), )\n (f)\n self.__best_loss = loss\n\n score > self.__score:\n torch.save(self.model.state_dict(), )\n (f)\n self.__score = score\n\n self.validation_step_outputs.clear()\nBut I changed it back, and it still had a problem
\nWhen I try git pull and then run the command again.it works OK.why？
I do think sharing the strategy and showing the potential was helpful but I do wish that the inference notebook was not shared. I think the leaderboard is filled with forks of this inference notebook. I would think that this was still ok if there was a training notebook that would actually work on kaggle so the masses would be able to play and learn from it. I was able to do so but I fear most will not have.
", "votes": 2, "replies": [] } ] } ] }, { "id": 2530561, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-19T09:04:18.357000", "content": "I've implemented a few additional models, so if you're interested, give it a try!
\nThe model is built with two components: feature_extractor and decoder.
\nThe feature_extractor and decoder that can be specified are as follows
\nThe correspondence table between each model and dataset is as follows.
\n| model | \ndataset | \n
|---|---|
| Spec1D | \nseg | \n
| Spec2DCNN | \nseg | \n
| DETR2DCNN | \ndetr | \n
| CenterNet | \ncenternet | \n
| TransformerAutoModel | \nseg | \n
The command to train CenterNet with feature_extractor=CNNSpectrogram, decoder=UNet1DDecoder is as follows
\nrye run python run/train.py model=CenterNet dataset=centernet feature_extractor=CNNSpectrogram decoder=UNet1DDecoder\nThat's a incredible sharing, appreciate your consistent yet high-quality work !
", "votes": 1, "replies": [] }, { "id": 2542049, "author_name": "", "author_url": "", "post_date": "2023-11-29T03:19:39.230000", "content": "", "votes": 0, "replies": [ { "id": 2542055, "author_name": "", "author_url": "", "post_date": "2023-11-29T03:29:25.887000", "content": "", "votes": 0, "replies": [] }, { "id": 2542056, "author_name": "", "author_url": "", "post_date": "2023-11-29T03:30:13.360000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2512132, "author_name": "Donghui Zhang", "author_url": "", "post_date": "2023-11-04T07:40:28.203000", "content": "I don't know whether I should be happy or sad now
", "votes": 17, "replies": [ { "id": 2512135, "author_name": "Catharis", "author_url": "", "post_date": "2023-11-04T07:47:24.280000", "content": "I feel exactly the same, I was working very hardly to climb up the ranks and I just got a top50 submission few hours ago which I was very happy about. Then I find this lol. I just see it as a way to ruin the lb and it's not considerate of people's efforts, I mean it's a top 20 submission with a month until the competition ends?. But then I had to hop on the train since everyone is going to hop on it now.
", "votes": 8, "replies": [ { "id": 2512186, "author_name": "", "author_url": "", "post_date": "2023-11-04T08:17:23.507000", "content": "", "votes": 3, "replies": [] }, { "id": 2512267, "author_name": "", "author_url": "", "post_date": "2023-11-04T10:05:07.753000", "content": "", "votes": 14, "replies": [] }, { "id": 2512432, "author_name": "Fritz Cremer", "author_url": "", "post_date": "2023-11-04T14:10:48.743000", "content": "100% agree, I think sharing methods and approaches is perfectly fine (up to a certain point in the competition). But what is the point in enabling others to just submit a pre-trained model? At least the people who actually replicate the methods put in some effort to do that.
", "votes": 2, "replies": [] }, { "id": 2514052, "author_name": "Ryosuke Horiuchi", "author_url": "", "post_date": "2023-11-06T01:58:47.770000", "content": "The rule \"Don't share the trained weight that scores high and may ruin the LB after a certain point in a competition\" is very obscure. As you know, in the competition's rule there are no explicit threshold in terms of score, the definition of ruining the LB, and when the certain point starts. If you hate it, you need to choose a competition that has such restrictions in the rule explicitly. (However, in kaggle at this moment, you can find no such competitions, I suppose. )
", "votes": 0, "replies": [] } ] }, { "id": 2525323, "author_name": "", "author_url": "", "post_date": "2023-11-15T03:24:25.870000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 2513149, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-05T08:19:30.167000", "content": "If you want to install Rye in a Kaggle notebook, use this command:
\ncurl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash
\ninstead of
\ncurl -sSf https://rye-up.com/get | bash
(see https://github.com/mitsuhiko/rye/issues/246 for details)
", "votes": 11, "replies": [ { "id": 2514292, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-06T07:03:59.203000", "content": "Another note: if you want to run @tubotubo code in a Kaggle notebook, you should change this line in pyproject.toml:
\nurl = \"https://download.pytorch.org/whl/cu120\"
\nto
\nurl = \"https://download.pytorch.org/whl/cu118\"
\nsince Kaggle environment currently has CUDA 11.4 installed.
In the kaggle,I meet this question in trainning phase，and how to deal with it?
\nOriginal Traceback (most recent call last):
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py\", line 308, in _worker_loop
\n data = fetcher.fetch(index)
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py\", line 51, in fetch
\n data = [self.dataset[idx] for idx in possibly_batched_index]
\n File \"/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py\", line 51, in
\n data = [self.dataset[idx] for idx in possibly_batched_index]
\n File \"/kaggle/input/cmi-code/cmi-code/src/dataset/seg.py\", line 106, in getitem
\n label = get_seg_label(this_event_df, num_frames, self.cfg.duration, start, end)
\n File \"/kaggle/input/cmi-code/cmi-code/src/dataset/seg.py\", line 21, in get_seg_label
\n this_event_df = this_event_df.query(\"@start <= wakeup & onset <= @end\")
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/frame.py\", line 4440, in query
\n res = self.eval(expr, **kwargs)
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/frame.py\", line 4566, in eval
\n return _eval(expr, inplace=inplace, **kwargs)
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/eval.py\", line 357, in eval
\n ret = eng_inst.evaluate()
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/engines.py\", line 81, in evaluate
\n res = self._evaluate()
\n File \"/opt/conda/lib/python3.10/site-packages/pandas/core/computation/engines.py\", line 121, in _evaluate
\n return ne.evaluate(s, local_dict=scope)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 943, in evaluate
\n raise e
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 851, in validate
\n _names_cache[expr_key] = getExprNames(ex, context)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 714, in getExprNames
\n ex = stringToExpression(text, {}, context)
\n File \"/opt/conda/lib/python3.10/site-packages/numexpr/necompiler.py\", line 274, in stringToExpression
\n raise ValueError(f'Expression {s} has forbidden control characters.')
\nValueError: Expression ((__pd_eval_local_start) <= (wakeup)) & ((onset) <= (__pd_eval_local_end)) has forbidden control characters.
In the query method of pandas, local variables are referenced in strings using the @ symbol. However, this method can sometimes lead to errors. Instead, consider a method that directly computes and filters data frame conditions.
\nThe epochs is set to 50, but in my case, the epochs went out of memory at 23. Please let me know if you have any solutions.
", "votes": 0, "replies": [] }, { "id": 2520118, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-10T15:31:02.497000", "content": "The version of the code may be different.
\nTry git pull to use the latest code.
I noticed that you recently had updated the code - it now includes detr2D. Can you specify a source of this architecture?
", "votes": 0, "replies": [] }, { "id": 2520145, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-10T15:53:24.340000", "content": "\nThis architecture is called DETR.
\nI have implemented it for study purposes, but have not yet achieved good accuracy.
https://arxiv.org/abs/2005.12872
", "votes": 3, "replies": [] }, { "id": 2520174, "author_name": "", "author_url": "", "post_date": "2023-11-10T16:19:20.107000", "content": "", "votes": 0, "replies": [] }, { "id": 2520759, "author_name": "sdhgkdbvj", "author_url": "", "post_date": "2023-11-11T06:42:15.040000", "content": "When I install rye in the kaggle, something wrong. and how to create virtual environment?
\n@tubotubo
!cp -r /kaggle/input/cmi-code/cmi-code cmi-code
\n!pip install hydra-core segmentation_models_pytorch==0.3.3 --no-index --find-links=/kaggle/input/ex-library
\n!curl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash
\n!echo 'source \"$HOME/.rye/env\"' >> ~/.bashrc
\n!source ~/.bashrc
This installer will install rye to /root/.rye
\nThis path can be changed by exporting the RYE_HOME environment variable.
\nDetails:
\n Rye Version: 0.15.2
\n Platform: linux (x86_64)
\nInstalled binary to /root/.rye/shims/rye
\nBootstrapping rye internals
\nDownloading cpython@3.11.6
\nChecking checksum██████████████████████████████░░░░ 49.16 MiB/52.29 MiB
\nsuccess: Downloaded cpython@3.11.6
\nUpgrading pip
\nInstalling internal dependencies
\nUpdated self-python installation at /root/.rye/self
The rye directory /root/.rye/shims was not detected on PATH.
\nIt is highly recommended that you add it.
\nAdd this at the end of your .profile, .zprofile or similar:
\n source \"$HOME/.rye/env\"
\nNote: after adding rye to your path, restart your shell for it to take effect.
!rye sync
\n/bin/bash: line 1: rye: command not found
try !/root/.rye/shims/rye sync instead
It's a great share but may not at a right time.
", "votes": 9, "replies": [] }, { "id": 2514836, "author_name": "Bilzard", "author_url": "", "post_date": "2023-11-06T15:03:16.080000", "content": "\nThank you for sharing gold-standard solution.
\n\n\nThe evaluation function can be sped up using a binary search. When the number of detections is large, the evaluation index can be calculated tens to hundreds of times faster than the original implementation.
\n
FYI, about the bisect search part, I rewrite the original logic. Although computation time is not boosted much, I think the logic became simpler and clearer.
\n ():\n \n \n idx = bisect_left(gt_times, det_time)\n\n best_idx = \n best_error = ()\n\n range_min = (, idx - )\n range_max = ((gt_times), idx + )\n check_idx (range_min, range_max): \n check_idx excluded_indices:\n error = (gt_times[check_idx] - det_time)\n error < best_error:\n best_error = error\n best_idx = check_idx\n\n best_idx, best_error\nThanks for the good refactoring!
", "votes": 2, "replies": [] }, { "id": 2516831, "author_name": "Bilzard", "author_url": "", "post_date": "2023-11-08T02:24:23.530000", "content": "The above code doesn't handle the case if the closest time_step is already matched.
\nWe should remove already matched gt indices from list of candidates.
Since my own prediction code predicts at most 1 event pair per night, it produces the same result, but it should be problematic in general cases.
\n ():\n \n idx = np.searchsorted(xs, value, side=)\n best_idx = \n best_error = ()\n\n range_min = (, idx - )\n range_max = ((xs), idx + )\n check_idx (\n range_min, range_max\n ): \n error = (xs[check_idx] - value)\n error < best_error:\n best_error = error\n best_idx = check_idx\n\n best_idx, best_error\n\n\n ():\n \n \n sorted_gt_times = np.asarray(sorted_gt_times)\n available_indices = np.asarray(\n ((((sorted_gt_times))) - excluded_indices), dtype=\n )\n sorted_gt_times = sorted_gt_times[available_indices]\n idx, error = find_nearest(sorted_gt_times, det_time)\n\n idx :\n , ()\n\n available_indices[idx], error\n\n\nI do not believe that publicizing this approach will spoil the efforts of the other participants.
\n
I have to disagree. If you would have just published this post, or maybe even the code, you could maybe argue that. But why publish the weights of a model that scores this high? Especially after there is just one month left. I really don't see the point in that…
", "votes": 9, "replies": [ { "id": 2512469, "author_name": "", "author_url": "", "post_date": "2023-11-04T14:48:10.367000", "content": "", "votes": 0, "replies": [] }, { "id": 2512489, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-04T15:08:00.163000", "content": "Assume that the training code is allowed to be published.
\nIf the training code is not publicly available, I do not think it is permissible to publish the model weights. If you don't publish the training code, you are not providing any value to the platform.
\nHowever, if the training code is to be made public, the weights of the model may or may not be made public.
\nAs long as the training code is publicly available, I think you are providing value to the platform.
\nI also think there is little difference between just forking the training code and submitting and using the published weights.
I think this discussion is a difference of ideas, so I will not discuss it further.
", "votes": 0, "replies": [ { "id": 2512548, "author_name": "Fritz Cremer", "author_url": "", "post_date": "2023-11-04T16:20:11.083000", "content": "\n\nHowever, if the training code is to be made public, the weights of the model may or may not be made public.
\n
Exactly, but one results in much more \"forkmittings\" than the other. Much fewer people actually go out of their way and train the model and then submit it. And I know that it can be frustrating for someone that put a lot of effort into making a 50th place submission, to be 300th place the next day. That's why I also think you should be careful with publishing training code, especially 1 month before end, which I think is relatively late for the impact it has. I believe a lot of people love the competitive aspect of it, but realistically, that is gone for many people when anything past the top 2% might come from someone who has never touched a machine learning model in their life.
\nAlso, I wonder why you last submission is one month ago? Couldn't you have published the code back then?
", "votes": 13, "replies": [] }, { "id": 2512558, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-04T16:35:14.577000", "content": "\n\nMuch fewer people actually go out of their way and train the model and then submit it.
\n
I am skeptical on this point.
\nSimply using published weights is difficult to maintain a high ranking. For a time, more people may use the published weights, but I believe that gradually more people will use the weights from the improved model using the training code.
\n\nAlso, I wonder why you last submission is one month ago? Couldn't you have published the code back then?
\n
The reason is that a month ago my goal was to win this competition. However, I got busy and decided to withdraw from this competition. Therefore, I decided to publish the code I wrote a month ago.
", "votes": 2, "replies": [] } ] }, { "id": 2513068, "author_name": "Shiner", "author_url": "", "post_date": "2023-11-05T06:48:47.603000", "content": "Dude, the remained one month is just a start for the competitions. There still a lot people just began their work just now. If you wish to keep your efforts, you should ask hosts to close the \"join button\".
", "votes": -2, "replies": [] } ] }, { "id": 2512388, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-04T13:23:18.197000", "content": "This is my impression from the experiments I have done, the 2D approach is not that important.
\nIt would be more accurate to capture more detailed information in the time direction rather than frequency.
\nThis means that the 1D approach may be better than the 2D approach.
It's hard to say. For example, I'm using 1d approach, but my best result is just LB 0.639 for now.
", "votes": 0, "replies": [ { "id": 2512566, "author_name": "yukiya", "author_url": "", "post_date": "2023-11-04T16:45:12.567000", "content": "Look at rank #19 , he's using 1D approach at LB 0.718
", "votes": 1, "replies": [] } ] } ] }, { "id": 2538861, "author_name": "mizoo", "author_url": "", "post_date": "2023-11-26T14:01:36.833000", "content": "Thank you for your insightful posts, they have been very educational. I have a question regarding the 'duration' parameter. Have you ever tried setting it to a shorter value than what you currently have, and made a submission?
\nI am currently trying to shorten the 'duration', but I keep encountering a 'Submission Scoring Error'. If you have any knowledge on this issue, could you please advise me on how to resolve it?
\nThank you in advance.
", "votes": 3, "replies": [ { "id": 2538944, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-26T14:56:33", "content": "\nEven if the \"duration\" is small, it should still be possible to run.
\nThe following commands infer for train.
\nYou can use it for debugging.
rye python run/inference.py =train\nBecause you reduced the distance back then, you will achieve more results. The scoring system will limit the number of results input for each ID, and if exceeded, an error will be reported. You can increase the threshold while reducing the distance
", "votes": 5, "replies": [ { "id": 2539652, "author_name": "mizoo", "author_url": "", "post_date": "2023-11-27T08:36:48.653000", "content": "How did you confirm that?
", "votes": 1, "replies": [] }, { "id": 2547273, "author_name": "", "author_url": "", "post_date": "2023-12-03T11:15:23.347000", "content": "", "votes": 0, "replies": [] }, { "id": 2547276, "author_name": "wpl", "author_url": "", "post_date": "2023-12-03T11:16:53.547000", "content": "I took the wrong sub and just changed the threshold for getting the step in the sub to get the score
", "votes": 1, "replies": [] } ] } ] }, { "id": 2513614, "author_name": "Tolga", "author_url": "", "post_date": "2023-11-05T15:08:29.703000", "content": "That's a way to make your life harder by revealing the secrets of your solution.
\nHope you have more tricks and time to get into the top 6. (:
\nBy the way, do you know how to do the ModelCheckpoint in lightning with a metric computed in on_validation_epoch_end?
", "votes": 3, "replies": [ { "id": 2513680, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-05T16:04:48.507000", "content": "Saving the best weights based on the evaluation index of the competition can be done with the following code.
\nrye run python run/train.py monitor=val_score monitor_mode=max
I meant in general not in your specific code.
\nWhen I compute the mAP in on_validation_epoch_end and log it, somehow ModelCheckpoint does not see the mAP to monitor it.
", "votes": -2, "replies": [] }, { "id": 2514124, "author_name": "", "author_url": "", "post_date": "2023-11-06T04:31:40.150000", "content": "", "votes": -2, "replies": [] }, { "id": 2514423, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-06T09:34:56.900000", "content": "Is this issue the one you encountered? https://github.com/Lightning-AI/lightning/issues/17131
", "votes": 2, "replies": [] }, { "id": 2514581, "author_name": "Tolga", "author_url": "", "post_date": "2023-11-06T11:53:27.960000", "content": "Chatgpt doesn't know about it.
\nThank you for the link, @atamazian. It looks relevant to my problem because I'm using v2.0.
", "votes": -1, "replies": [] } ] } ] }, { "id": 2515757, "author_name": "RyanB23", "author_url": "", "post_date": "2023-11-07T08:02:16.323000", "content": "I am trying to train my own version of the model on kaggle and I am having a bit of difficulty. The scripting commands are not exactly intuitive for this effort. Does anyone know the right sequence to get the command to work? I have this
\n
The problem is that wandb waits for a user input, which is not possible in Jupyter notebook. I solved this issue by removing wandb logging and switching to local Tensorboard logging.
", "votes": 0, "replies": [ { "id": 2515835, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-07T09:17:54.363000", "content": "You can change original files within Jupyter notebook by inserting modified code to a single cell with putting %%writefile <filename> command on top.
There is a cmd button on Kaggle page, you can run command there.
Where exactly is that? That would help me a lot as I was looking to try a spin-off of this strategy and have been struggling to get the training script to function through several alterations
", "votes": 1, "replies": [] }, { "id": 2521358, "author_name": "Shiner", "author_url": "", "post_date": "2023-11-11T16:24:05.700000", "content": "Don't know it will help you or not, I haven't try it. It is actually a Console, maybe it is quite different from Terminal or shell on PC.
\n
Yeah, the code is really interesting, it is a shame that is in the least readable format available to us. It is a lot of work to even be able to adjust the training for yourself
", "votes": -2, "replies": [] }, { "id": 2531776, "author_name": "Min Young Joo", "author_url": "", "post_date": "2023-11-20T14:17:38.827000", "content": "I'm in the exact same situation. Reading the comments below doesn't help, so if anyone has a cool solution, I'd appreciate it
", "votes": 0, "replies": [] } ] }, { "id": 2530545, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-19T08:35:46.677000", "content": "If you want to make a contribution to this repository, please write a PR or issue summary and link to it in this thread as well to avoid private sharing.
\ntemplate
\nMy contributions/issues submitted:
\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/pull/30
\nWhen you intend to modify the model, you may want to note that the annotation of the model output shap in spec2Dcnn is different from that in unet1ddecoder (with the addition of transpose).
My issues submitted:
\nBTW, how could it be private sharing if the GitHub repo's URL has been shared publicly beforehand in Kaggle? I thought any information sharing at an external website that had been shared publicly beforehand in Kaggle was not considered as private sharing.
", "votes": 2, "replies": [ { "id": 2531517, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-20T09:25:33.687000", "content": "@riow1983
\nThank you for your response.
It may not fall under private sharing, but I think many people don't even look through the issues and PR.
\nI asked you to post the content in the thread as well, as I think it is unfair to disadvantage people by not looking at the issues and PRs.
Thank you for sharing the code.
\nDid you execute the training code using a Kaggle notebook, or did you do it on your local PC? I'd like to try the training you published on Git in a Kaggle notebook, but I'm concerned that it might not be feasible due to GPU and memory issues.
", "votes": 3, "replies": [ { "id": 2513083, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-05T07:04:11.407000", "content": "I experimented in a local environment, but this code works in a kaggle environment as well.
\nTry running it with dir=kaggle.
Or you can change local.yaml content.
I think publishing notebook like this is not good for the people who are serious to join and have score lower than this notebook.
", "votes": 4, "replies": [] }, { "id": 2513035, "author_name": "furu-nag", "author_url": "", "post_date": "2023-11-05T05:54:29.170000", "content": "I am interested in Gaussian label width (sigma). How width do you set the value? In your notebook it is set to 10, how many seconds does this correspond to? In the distributed correct answer label, the number of significant digits is about 1 minute, so it seems more reasonable to look at it with a sigma of about 1 minute.
", "votes": 4, "replies": [ { "id": 2513078, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-05T07:00:38.667000", "content": "\n\nhow many seconds does this correspond to?
\n
10 * 5 = 50 [sec]
\n\n\nI am interested in Gaussian label width (sigma). How width do you set the value?
\n
Good point.
\nI am training the model using a sigma that I set up at the beginning.
However, this sigma is very important.
\nBy setting the sigma and duration properly, the accuracy (local cv) can be improved by about 0.01-0.02.
The following command can be used for grid search, so try it.
\nrye run python run/train.py -m duration=100,1000,10000 sigma=5,10,15,20
Shall we define sigma value only as command line parameter?
\nI did not find it in any *.py aor *.yaml file.
The default parameters are written under run/conf/.
\nsigma is set in run/conf/dataset/seg.yaml.
\nConfig is specified in src/conf.py.
thank you very much! ))
", "votes": 0, "replies": [] }, { "id": 2526825, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2023-11-16T04:41:05.570000", "content": "what is the best criteria at experiments or grid search?
\nminimum val_loss or maximum val_score?
Better to evaluate with val_score
", "votes": 0, "replies": [] } ] } ] }, { "id": 2518901, "author_name": "Alexander Ryzhkov", "author_url": "", "post_date": "2023-11-09T15:43:17.463000", "content": "@tubotubo if you are working with architectures like UNet please take a look about usage the links between the same levels of encoder and decoder (the left and right parts of UNet) - this usually helps to increase the score for such a kind of models
", "votes": 3, "replies": [ { "id": 2519620, "author_name": "yukiya", "author_url": "", "post_date": "2023-11-10T07:46:47.703000", "content": "I think he already has skip connection in his code. What else can we try
", "votes": 0, "replies": [ { "id": 2520686, "author_name": "DongYK", "author_url": "", "post_date": "2023-11-11T04:54:22.047000", "content": "It seems like he's talking about the seqence length.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2514077, "author_name": "Ryosuke Horiuchi", "author_url": "", "post_date": "2023-11-06T02:43:31.800000", "content": "I don't think it's a selfish behavior. Deciding to withdraw oneself but not withdraw models that one has created so far letting them be available to the public is understandable. The more the models are adopted by other participants, the more the models are getting seminal. That's another way of participating in a kaggle competition to the end in a busy period.
", "votes": 2, "replies": [ { "id": 2515671, "author_name": "n00b", "author_url": "", "post_date": "2023-11-07T05:25:36.597000", "content": "Totally agree, the goal is to train the best model possible, maybe a lot of people will fine tune this model and get even higher scores then the top5, which would be amazing. This would probably make the competition hosts happy too.
", "votes": -5, "replies": [] } ] }, { "id": 2560628, "author_name": "AIDLRE001", "author_url": "", "post_date": "2023-12-13T20:35:02.843000", "content": "Wow!!! Thanks for this! So my thinking of using spectrograms was not a bad one. This is encouraging. Thanks again.
", "votes": 1, "replies": [] }, { "id": 2546713, "author_name": "cosmosaa", "author_url": "", "post_date": "2023-12-02T18:23:17.787000", "content": "I am wondering where the idea for this architecture comes from. The Unet is by itself an encoder/decoder architecture, and the encoder is a feature extractor. But instead of just using a single unet (1d) as both encoder and feature extractor, you also first extract features using multiple kernel sizes. Then you decode the 2d output into 1d using a separate unet style model. Why not just use a single 1d unet?
\nI'm curious what the 2d heatmap represents. Since this is a 1d problem, why is representing the data as 2d a good idea?
", "votes": 1, "replies": [ { "id": 2547056, "author_name": "213tubo", "author_url": "", "post_date": "2023-12-03T06:08:58.140000", "content": "Why is hour converted to degrees using
\ndef to_coord(x: pl.Expr, max_: int, : ) -> [pl.Expr]:\n = * np. * (x % max_) / max_\n x_sin = .()\n x_cos = .()\n\n return [x_sin.alias(f), x_cos.alias(f)]\nand how important is that? Why not just convert to the range [0, 1] ?
\nIt looks like it makes it smoother. Instead of a sharp peak from 23 to 0, it is smoother.
\nWhy are both hour_sin and hour_cos used? Isn't 1 enough?
When using only the sine function to represent a 24-hour cycle, there is an overlap in the values for AM and PM times, which can lead to ambiguity.
", "votes": 0, "replies": [ { "id": 2547611, "author_name": "cosmosaa", "author_url": "", "post_date": "2023-12-03T16:41:29.373000", "content": "Thanks. I found this article helpful
", "votes": 0, "replies": [] } ] } ] }, { "id": 2544143, "author_name": "Alex N.", "author_url": "", "post_date": "2023-11-30T15:53:40.683000", "content": "Thank you for your project, it's really great and boosted my knowledge a lot! May I ask how do you debug your code if something went wrong? Do you use special tools for that or just use basic things like python debugger or print statements?
", "votes": 1, "replies": [ { "id": 2544321, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-30T18:39:47.650000", "content": "I use the \"pdb\" for debugging
", "votes": 1, "replies": [] } ] }, { "id": 2538567, "author_name": "Yutao", "author_url": "", "post_date": "2023-11-26T09:13:48.983000", "content": "Did anyone try to run the training script in kaggle and it just crashes the notebook after maybe 20+ epochs?
", "votes": 1, "replies": [] }, { "id": 2512326, "author_name": "Pavlo Semchyshyn", "author_url": "", "post_date": "2023-11-04T11:21:17.243000", "content": "Js, you just ruined the competition
", "votes": -3, "replies": [] }, { "id": 2547808, "author_name": "cosmosaa", "author_url": "", "post_date": "2023-12-03T21:50:15.383000", "content": "Just so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels? In theory this would allow for predicting both an onset and wakeup event at the same timestep.
\nI'm also wondering about the loss function. You use BCEWithLogitsLoss with no weights. Since onset/wakeup are rare, wouldn't this loss function bias towards the background class?
", "votes": 2, "replies": [ { "id": 2548219, "author_name": "213tubo", "author_url": "", "post_date": "2023-12-04T08:17:47.787000", "content": "\n\nJust so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels?
\n
Yes. There is a possibility of predicting multiple labels in 1step, although the number may be small.
\n\n\nwouldn't this loss function bias towards the background class?
\n
Since ap depends only on the order of the scores, it does not matter if bg scores higher.
", "votes": 2, "replies": [ { "id": 2548675, "author_name": "cosmosaa", "author_url": "", "post_date": "2023-12-04T15:46:55.007000", "content": "To clarify, I’m surprised the model is able to learn the signal for onset/wakeup. There are 2880 timesteps, and only ~10/2880 > 0 for onset/wakeup. So I’m surprised the model is able to learn this signal.
", "votes": 2, "replies": [] } ] } ] }, { "id": 2519326, "author_name": "Yutao", "author_url": "", "post_date": "2023-11-09T23:09:45.827000", "content": "What is the best way to further tune this? I am thinking either:
\nDoes anyone know other ways (better ways) to do it?
", "votes": 2, "replies": [ { "id": 2519332, "author_name": "Tolga", "author_url": "", "post_date": "2023-11-09T23:18:40.117000", "content": "Just take the ideas, apply it in your own code. You will benefit more in the long run.
\nThe most critical addition in this code to the existing ideas is the use of 3d outputs and the loss function.
Yeah, I think you have a good point here. Thanks
", "votes": 1, "replies": [] }, { "id": 2536185, "author_name": "n00b", "author_url": "", "post_date": "2023-11-24T02:11:28.440000", "content": "Did 2. Because ive got no gpu locally
", "votes": 0, "replies": [] } ] } ] }, { "id": 2512458, "author_name": "Koen van den Brink", "author_url": "", "post_date": "2023-11-04T14:36:00.230000", "content": "Thanks for the post @tubotubo , you just saved me a lot of time and submission attempts with your comment about having multiple files in the output directory. Kudos!
", "votes": 2, "replies": [] }, { "id": 2512216, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-04T08:40:00.367000", "content": "It's a nice idea to use 2d heatmaps and UNet encoder-decoder scheme. I noticed that you use quite low downsampling (just 2). I was using 12 until now.
", "votes": 2, "replies": [] }, { "id": 2512437, "author_name": "Man of the year", "author_url": "", "post_date": "2023-11-04T14:16:49.850000", "content": "@tubotubo very nice approach. Similar thing, that was discussed on the other thread recently. Could you clarify, in terms of feature extractor, you implemented 4 ways to do this. Do you use only one method at a time in your pipeline? Or do you use them somehow one by one, for example, CNN -> LSTM etc?
\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?
", "votes": 0, "replies": [ { "id": 2512495, "author_name": "213tubo", "author_url": "", "post_date": "2023-11-04T15:15:01.557000", "content": "\n\nDo you use only one method at a time in your pipeline?
\n
Pipelines use one method at a time.
\n\n\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?
\n
This task is very similar to soud event detection (SED).
\nFor this reason, I tried the 2D approach often used in SED.
\nSound Event Detection: A Tutorial
Thank you. I agree, that it looks exactly like SED. Could you also tell, which loss function did you use? I could not find it in your git code.
", "votes": 0, "replies": [] }, { "id": 2512802, "author_name": "chris", "author_url": "", "post_date": "2023-11-04T20:58:00.940000", "content": "It uses BCEWithLogitsLoss (it's in the models files, e.g. src/models/spec1D.py)
", "votes": 2, "replies": [] } ] } ] }, { "id": 2512984, "author_name": "wpl", "author_url": "", "post_date": "2023-11-05T05:01:18.083000", "content": "You ruined the LB in the short term
", "votes": -3, "replies": [ { "id": 2512987, "author_name": "", "author_url": "", "post_date": "2023-11-05T05:03:33.270000", "content": "", "votes": 0, "replies": [] }, { "id": 2512989, "author_name": "wpl", "author_url": "", "post_date": "2023-11-05T05:04:32.403000", "content": "Even among the top 100 individuals, some have only submitted once, is this fair for those who have been involved and innovative from the beginning
", "votes": 1, "replies": [] } ] }, { "id": 2512915, "author_name": "", "author_url": "", "post_date": "2023-11-05T02:44:33.097000", "content": "", "votes": -7, "replies": [] }, { "id": 2544149, "author_name": "", "author_url": "", "post_date": "2023-11-30T16:07:49.420000", "content": "", "votes": 0, "replies": [ { "id": 2544322, "author_name": "", "author_url": "", "post_date": "2023-11-30T18:40:52.897000", "content": "", "votes": 0, "replies": [ { "id": 2544334, "author_name": "", "author_url": "", "post_date": "2023-11-30T18:47:21.273000", "content": "", "votes": 0, "replies": [] }, { "id": 2544339, "author_name": "", "author_url": "", "post_date": "2023-11-30T18:56:30.160000", "content": "", "votes": 1, "replies": [] }, { "id": 2545653, "author_name": "", "author_url": "", "post_date": "2023-12-01T16:18:18.840000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2546851, "author_name": "", "author_url": "", "post_date": "2023-12-02T23:35:32.517000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2541783, "author_name": "", "author_url": "", "post_date": "2023-11-28T19:29:48.070000", "content": "", "votes": 0, "replies": [ { "id": 2541993, "author_name": "", "author_url": "", "post_date": "2023-11-29T01:49:52.683000", "content": "", "votes": 3, "replies": [] } ] }, { "id": 2530429, "author_name": "", "author_url": "", "post_date": "2023-11-19T06:19:37.363000", "content": "", "votes": 0, "replies": [ { "id": 2530472, "author_name": "", "author_url": "", "post_date": "2023-11-19T07:30:28.920000", "content": "", "votes": 0, "replies": [ { "id": 2530491, "author_name": "", "author_url": "", "post_date": "2023-11-19T07:49:08.283000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2528464, "author_name": "", "author_url": "", "post_date": "2023-11-17T11:44:15.900000", "content": "", "votes": 0, "replies": [] }, { "id": 2524468, "author_name": "", "author_url": "", "post_date": "2023-11-14T09:07:50.843000", "content": "", "votes": 0, "replies": [ { "id": 2524654, "author_name": "", "author_url": "", "post_date": "2023-11-14T12:28:25.050000", "content": "", "votes": 0, "replies": [ { "id": 2524847, "author_name": "", "author_url": "", "post_date": "2023-11-14T15:02:46.233000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2525324, "author_name": "", "author_url": "", "post_date": "2023-11-15T03:28:48.060000", "content": "", "votes": 0, "replies": [] }, { "id": 2525523, "author_name": "", "author_url": "", "post_date": "2023-11-15T07:04:21.383000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2523609, "author_name": "", "author_url": "", "post_date": "2023-11-13T15:44:56.230000", "content": "", "votes": 0, "replies": [] }, { "id": 2522477, "author_name": "", "author_url": "", "post_date": "2023-11-12T17:40:59.383000", "content": "", "votes": 0, "replies": [ { "id": 2522499, "author_name": "", "author_url": "", "post_date": "2023-11-12T17:59:50.420000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 2519513, "author_name": "", "author_url": "", "post_date": "2023-11-10T05:39:08.487000", "content": "", "votes": 0, "replies": [ { "id": 2520116, "author_name": "", "author_url": "", "post_date": "2023-11-10T15:29:23.753000", "content": "", "votes": 4, "replies": [] } ] }, { "id": 2513057, "author_name": "", "author_url": "", "post_date": "2023-11-05T06:30:37.227000", "content": "", "votes": 0, "replies": [] }, { "id": 2547382, "author_name": "", "author_url": "", "post_date": "2023-12-03T13:03:58.253000", "content": "", "votes": 0, "replies": [ { "id": 2547401, "author_name": "", "author_url": "", "post_date": "2023-12-03T13:26:43.210000", "content": "", "votes": 0, "replies": [ { "id": 2547405, "author_name": "", "author_url": "", "post_date": "2023-12-03T13:30:49.587000", "content": "", "votes": 0, "replies": [] }, { "id": 2547412, "author_name": "", "author_url": "", "post_date": "2023-12-03T13:38:39.140000", "content": "", "votes": 0, "replies": [] } ] } ] }, { "id": 2537312, "author_name": "", "author_url": "", "post_date": "2023-11-25T03:38:23.210000", "content": "", "votes": 0, "replies": [ { "id": 2537741, "author_name": "", "author_url": "", "post_date": "2023-11-25T12:59:48.783000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 2531446, "author_name": "", "author_url": "", "post_date": "2023-11-20T08:02:15.757000", "content": "", "votes": 5, "replies": [ { "id": 2531537, "author_name": "", "author_url": "", "post_date": "2023-11-20T09:49:24.973000", "content": "", "votes": 15, "replies": [ { "id": 2531687, "author_name": "", "author_url": "", "post_date": "2023-11-20T12:35:57.843000", "content": "", "votes": 3, "replies": [] } ] } ] }, { "id": 2530546, "author_name": "", "author_url": "", "post_date": "2023-11-19T08:39:30.533000", "content": "", "votes": 0, "replies": [ { "id": 2530548, "author_name": "", "author_url": "", "post_date": "2023-11-19T08:43:02.727000", "content": "", "votes": 0, "replies": [ { "id": 2530558, "author_name": "", "author_url": "", "post_date": "2023-11-19T08:56:49.677000", "content": "", "votes": 1, "replies": [] } ] } ] }, { "id": 2530425, "author_name": "", "author_url": "", "post_date": "2023-11-19T06:16:22.430000", "content": "", "votes": 0, "replies": [] }, { "id": 2530420, "author_name": "", "author_url": "", "post_date": "2023-11-19T06:13:17.827000", "content": "", "votes": 0, "replies": [] }, { "id": 2526087, "author_name": "", "author_url": "", "post_date": "2023-11-15T15:15:03.637000", "content": "", "votes": 0, "replies": [] }, { "id": 2523739, "author_name": "", "author_url": "", "post_date": "2023-11-13T17:35:51.610000", "content": "", "votes": 0, "replies": [ { "id": 2523786, "author_name": "", "author_url": "", "post_date": "2023-11-13T18:28:24.923000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2516294, "author_name": "", "author_url": "", "post_date": "2023-11-07T15:51:17.703000", "content": "", "votes": 0, "replies": [ { "id": 2516313, "author_name": "", "author_url": "", "post_date": "2023-11-07T16:08:21.150000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2513364, "author_name": "", "author_url": "", "post_date": "2023-11-05T11:59:02.960000", "content": "", "votes": 2, "replies": [] }, { "id": 2513262, "author_name": "", "author_url": "", "post_date": "2023-11-05T10:32:20.117000", "content": "", "votes": 2, "replies": [] }, { "id": 2512888, "author_name": "", "author_url": "", "post_date": "2023-11-05T01:30:49.630000", "content": "", "votes": -6, "replies": [] }, { "id": 2512478, "author_name": "", "author_url": "", "post_date": "2023-11-04T14:57:44.937000", "content": "", "votes": -9, "replies": [ { "id": 2512485, "author_name": "", "author_url": "", "post_date": "2023-11-04T15:06:50.317000", "content": "", "votes": 1, "replies": [] }, { "id": 2512487, "author_name": "", "author_url": "", "post_date": "2023-11-04T15:07:12.490000", "content": "", "votes": -3, "replies": [] } ] }, { "id": 2512275, "author_name": "", "author_url": "", "post_date": "2023-11-04T10:12:09.223000", "content": "", "votes": -2, "replies": [] }, { "id": 2512422, "author_name": "", "author_url": "", "post_date": "2023-11-04T13:59:47.380000", "content": "", "votes": 0, "replies": [] }, { "id": 2543292, "author_name": "", "author_url": "", "post_date": "2023-11-30T00:29:34.453000", "content": "", "votes": 0, "replies": [] }, { "id": 2534871, "author_name": "", "author_url": "", "post_date": "2023-11-23T00:52:23.577000", "content": "", "votes": 0, "replies": [] }, { "id": 2515514, "author_name": "", "author_url": "", "post_date": "2023-11-07T01:59:32.543000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2512088": "I share my approach in this discussion.\n- The code is also available on github\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/tree/main\n- submission notebook\nhttps://www.kaggle.com/code/tubotubo/cmi-submit\n\n# Model Structure\nMy model consists of three components: feature extractor, encoder, and decoder.\n\nThe feature extractor takes the raw waveform(anglez, enmo, hour) and creates a wavegram (like an image).\nThe encoder (UNet) takes the wavegrame and creates a two-dimensional heatmap.\ndecoder takes the heatmap and outputs the probability(sleep, onset, wakeup) of an event occurring at each time.\nFinally, post-processing is performed to create a submission file.\n.jpg?generation=1699079676016847&alt=media)\n\nEach component has the following already implemented.\n\n- feature extractor\n - CNN\n - LSTM\n - Spectrogram\n - [PANNs](https://arxiv.org/abs/1912.10211)\n- encoder\n - UNet\n- decoder\n - UNet1D\n - LSTM\n - MLP\n - Transformer\n\n# Result\nLB: 0.714\nfeature extractor=CNN, encoder=UNet, decoder=UNet1D, downsample_rate=2, seq_len=5760\n\nBecause the code has changed since the last submission, the experiment may not be accurately reproduced with the above settings.\n\n# Tips\n1. The evaluation function can be sped up using a binary search. When the number of detections is large, the evaluation index can be calculated tens to hundreds of times faster than the original implementation.\nhttps://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states/blob/8bbc09aff9c34f12259d536fd496deaade8ec087/src/utils/metrics.py#L179\n2. Outputting a file other than submission.csv and submitting it will result in an error. In the inference notebook, it is better to leave only the final submission.csv.\n\n\n\n# How to contribute\n\nIf you want to make a contribution to this repository, please write a PR or issue summary and link to it in [this thread](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940#2530545) as well to avoid private sharing.", "2512328": "I do not believe that publicizing this approach will spoil the efforts of the other participants.\nThere are two reasons.\nThe first is that my approach does not encompass all other approaches. I believe that a better model can be created by combining my approach with your approach.\nThe second is that the code I have published has room for many improvements and I believe there is ample time to make improvements.\nFor these two reasons, I thought it was worth releasing the code and model to the public.\n\nTemporarily, LB may be full of people who just forked it, but it will be weeded out by those who are serious about experimenting.", "2530561": "I've implemented a few additional models, so if you're interested, give it a try!\n\n# Implemented models\n\nThe model is built with two components: feature_extractor and decoder.\n\nThe feature_extractor and decoder that can be specified are as follows\n\n### Feature Extractor\n\n- [CNNSpectrogram](https://github.com/analokmaus/kaggle-g2net-public/tree/main/models1d_pytorch)\n- LSTMFeatureExtractor\n- [PANNsFeatureExtractor](https://arxiv.org/abs/1912.10211)\n- SpecFeatureExtractor\n\n### Decoder\n\n- MLPDecoder\n- LSTMDecoder\n- TransformerDecoder\n- TransformerCNNDecoder\n- [UNet1DDecoder](https://github.com/bamps53/kaggle-dfl-3rd-place-solution/blob/master/models/cnn_3d.py)\n- MLPDecoder\n\n### Model\n\n- Spec2DCNN: Segmentation through UNet.\n- Spec1D: Segmentation without UNet\n- DETR2DCNN: Use UNet to detect sleep as in [DETR](https://arxiv.org/abs/2005.12872). This model is still under development.\n- CenterNet: Detect onset and offset, respectively, like [CenterNet](https://arxiv.org/abs/1904.07850) using UNet\n- TransformerAutoModel: \n - Segmentation using huggingface's [AutoModel](https://huggingface.co/transformers/v3.0.2/model_doc/auto.html). don't use feature_extractor and decoder.\n - Since the Internet is not available during inference, it is necessary to create a config dataset and specify the path in the model_name.\n\nThe correspondence table between each model and dataset is as follows.\n\n| model | dataset | \n| --------- | --------- | \n| Spec1D | seg | \n| Spec2DCNN | seg | \n| DETR2DCNN | detr | \n| CenterNet | centernet | \n| TransformerAutoModel | seg |\n\nThe command to train CenterNet with feature_extractor=CNNSpectrogram, decoder=UNet1DDecoder is as follows\n\n```bash\nrye run python run/train.py model=CenterNet dataset=centernet feature_extractor=CNNSpectrogram decoder=UNet1DDecoder\n```", "2512132": "I don't know whether I should be happy or sad now", "2513149": "If you want to install Rye in a Kaggle notebook, use this command:\n```curl -sSf https://rye-up.com/get | RYE_INSTALL_OPTION=\"--yes\" bash```\ninstead of\n```curl -sSf https://rye-up.com/get | bash```\n\n(see https://github.com/mitsuhiko/rye/issues/246 for details)", "2512169": "It's a great share but may not at a right time.", "2514836": "@tubo\n\nThank you for sharing gold-standard solution.\n\n> The evaluation function can be sped up using a binary search. When the number of detections is large, the evaluation index can be calculated tens to hundreds of times faster than the original implementation.\n\nFYI, about the bisect search part, I rewrite the original logic. Although computation time is not boosted much, I think the logic became simpler and clearer.\n\n```\ndef find_nearest_time_idx(gt_times, det_time, excluded_indices: set):\n \"\"\"\n search index of gt_times closest to det_time.\n\n assumes gt_times is sorted in ascending order.\n \"\"\"\n # e.g. if gt_times = [0, 1, 2, 3, 4, 5] and det_time = 2.5, then idx = 3\n idx = bisect_left(gt_times, det_time)\n\n best_idx = None\n best_error = float(\"inf\")\n\n range_min = max(0, idx - 1)\n range_max = min(len(gt_times), idx + 2)\n for check_idx in range(range_min, range_max): # Check the exact, one before, and one after\n if check_idx not in excluded_indices:\n error = abs(gt_times[check_idx] - det_time)\n if error < best_error:\n best_error = error\n best_idx = check_idx\n\n return best_idx, best_error\n```", "2512434": ">I do not believe that publicizing this approach will spoil the efforts of the other participants.\n\nI have to disagree. If you would have just published this post, or maybe even the code, you could maybe argue that. But why publish the weights of a model that scores this high? Especially after there is just one month left. I really don't see the point in that...", "2512388": "This is my impression from the experiments I have done, the 2D approach is not that important.\nIt would be more accurate to capture more detailed information in the time direction rather than frequency.\nThis means that the 1D approach may be better than the 2D approach.", "2538861": "Thank you for your insightful posts, they have been very educational. I have a question regarding the 'duration' parameter. Have you ever tried setting it to a shorter value than what you currently have, and made a submission?\n\nI am currently trying to shorten the 'duration', but I keep encountering a 'Submission Scoring Error'. If you have any knowledge on this issue, could you please advise me on how to resolve it?\n\nThank you in advance.", "2513614": "That's a way to make your life harder by revealing the secrets of your solution.\n\nHope you have more tricks and time to get into the top 6. (:\n\nBy the way, do you know how to do the ModelCheckpoint in lightning with a metric computed in on_validation_epoch_end?", "2515757": "I am trying to train my own version of the model on kaggle and I am having a bit of difficulty. The scripting commands are not exactly intuitive for this effort. Does anyone know the right sequence to get the command to work? I have this\n ", "2530545": "If you want to make a contribution to this repository, please write a PR or issue summary and link to it in this thread as well to avoid private sharing.\n\n------\ntemplate\n# [PR or Issue]title\n\n- summary\n- link", "2512555": "@tubotubo \n\nThank you for sharing the code.\n\nDid you execute the training code using a Kaggle notebook, or did you do it on your local PC? I'd like to try the training you published on Git in a Kaggle notebook, but I'm concerned that it might not be feasible due to GPU and memory issues.", "2514036": "I think publishing notebook like this is not good for the people who are serious to join and have score lower than this notebook.", "2513035": "I am interested in Gaussian label width (sigma). How width do you set the value? In your notebook it is set to 10, how many seconds does this correspond to? In the distributed correct answer label, the number of significant digits is about 1 minute, so it seems more reasonable to look at it with a sigma of about 1 minute.", "2518901": "@tubotubo if you are working with architectures like UNet please take a look about usage the links between the same levels of encoder and decoder (the left and right parts of UNet) - this usually helps to increase the score for such a kind of models", "2514077": "I don't think it's a selfish behavior. Deciding to withdraw oneself but not withdraw models that one has created so far letting them be available to the public is understandable. The more the models are adopted by other participants, the more the models are getting seminal. That's another way of participating in a kaggle competition to the end in a busy period.", "2560628": "Wow!!! Thanks for this! So my thinking of using spectrograms was not a bad one. This is encouraging. Thanks again.", "2546713": "I am wondering where the idea for this architecture comes from. The Unet is by itself an encoder/decoder architecture, and the encoder is a feature extractor. But instead of just using a single unet (1d) as both encoder and feature extractor, you also first extract features using multiple kernel sizes. Then you decode the 2d output into 1d using a separate unet style model. Why not just use a single 1d unet?\n\nI'm curious what the 2d heatmap represents. Since this is a 1d problem, why is representing the data as 2d a good idea?", "2545528": "Why is hour converted to degrees using \n\n```\ndef to_coord(x: pl.Expr, max_: int, name: str) -> list[pl.Expr]:\n rad = 2 * np.pi * (x % max_) / max_\n x_sin = rad.sin()\n x_cos = rad.cos()\n\n return [x_sin.alias(f\"{name}_sin\"), x_cos.alias(f\"{name}_cos\")]\n```\n\nand how important is that? Why not just convert to the range [0, 1] ?\n\nIt looks like it makes it smoother. Instead of a sharp peak from 23 to 0, it is smoother.\n\nWhy are both `hour_sin` and `hour_cos` used? Isn't 1 enough?", "2544143": "Thank you for your project, it's really great and boosted my knowledge a lot! May I ask how do you debug your code if something went wrong? Do you use special tools for that or just use basic things like python debugger or print statements?", "2538567": "Did anyone try to run the training script in kaggle and it just crashes the notebook after maybe 20+ epochs?", "2512326": "Js, you just ruined the competition", "2547808": "Just so I'm sure, this is treating the problem as a multi-label problem where for each timestep we predict 0 or more labels? In theory this would allow for predicting both an onset and wakeup event at the same timestep.\n\nI'm also wondering about the loss function. You use BCEWithLogitsLoss with no weights. Since onset/wakeup are rare, wouldn't this loss function bias towards the background class?\n\n", "2519326": "What is the best way to further tune this? I am thinking either:\n1. clone the code to my own github repo, then change the code there.\n2. create a new kaggle notebook, copy the code over from github (but was there a limitation that this was not done like that in the first place?)\n\nDoes anyone know other ways (better ways) to do it?", "2512458": "Thanks for the post @tubotubo , you just saved me a lot of time and submission attempts with your comment about having multiple files in the output directory. Kudos!", "2512216": "It's a nice idea to use 2d heatmaps and UNet encoder-decoder scheme. I noticed that you use quite low downsampling (just 2). I was using 12 until now.", "2512437": "@tubotubo very nice approach. Similar thing, that was discussed on the other thread recently. Could you clarify, in terms of feature extractor, you implemented 4 ways to do this. Do you use only one method at a time in your pipeline? Or do you use them somehow one by one, for example, CNN -> LSTM etc? \n\nAlso, any logic or thoughts why Unet encoder was used here? Not the ResNet, for example?", "2512984": "You ruined the LB in the short term", "2512915": "Nice sharings!\n\nps: I'd always prefer a one-notebook training and inference version of code for competitions. If it is possible to combine them into one notebook, it would be nice. 😂", "2544149": "let me ask you a question\nin config files we can find:\n```python\n# optimizer\noptimizer:\n lr: 0.0005\n```\n\nand in logger we can see:\nlr-AdamW ████████▇▇▇▇▇▆▆▆▆▅▅▅▄▄▄▄▃▃▃▃▂▂▂▂▂▂▁▁▁▁▁▁\n\nis the config lr the final value of learning rate or starting value?", "2541783": "@tubotubo can some one explain me how you created the fold0.yaml config. is it Group Fold or StratifiedGroupKFold?..is it a 5 fold or 4fold. how did you do that. could you please explain?", "2530429": "Has anyone encountered the same problem as me? I averaged three 729 models (two with different architectures and one with a slightly tweaked architecture) and the other 72+ models, but the score was still lower than the previous 729 model and several 72+ models. I don’t know if it’s because of the diversity. . .", "2528464": "nicework,really helpful", "2524468": "May I ask you how long it takes to train a round（just one epoch）, and what kind of environment is it based on (what kind of GPU is used)? It's my first time to participate in this kind of competition, so I don't know if I have successfully run the code.", "2523609": "Nice working and moreover approach, awesome all @tubotubo ", "2522477": "I notice that you use 24 workers for your dataloaders. Do you mind me asking what specifications your computer has? I would expect with that many workers your RAM would fill up quickly? Or is there a trick I'm not aware of to avoid that problem?", "2519513": "Greetings and thanks for sharing such a nice solution!\n\nAnswer my question, please.\nHere You wrote that wavwgram is made from 3 features - `anglez, enmo and hour`.\nBut prepare_data.py file contains code for 8 features:\n```python\nFEATURE_NAMES = [\n \"anglez\",\n \"enmo\",\n \"hour_sin\",\n \"hour_cos\",\n \"month_sin\",\n \"month_cos\",\n \"minute_sin\",\n \"minute_cos\",\n]\n```\n\nHow should we understand this?\n\nUPDATE\n\nI found it:\nGreetings and thanks for sharing such a nice solution!\n\nAnswer my question, please.\nHere You wrote that wavwgram is made from 3 features - `anglez, enmo and hour`.\nBut prepare_data.py file contains code for 8 features:\n```python\nFEATURE_NAMES = [\n \"anglez\",\n \"enmo\",\n \"hour_sin\",\n \"hour_cos\",\n \"month_sin\",\n \"month_cos\",\n \"minute_sin\",\n \"minute_cos\",\n]\n```\n\nHow should we understand this?\n\nUPDATE\n\nI found it in conf/train.yaml:\n\n```python\n# feature\nfeatures:\n - \"anglez\"\n - \"enmo\"\n # - \"month_sin\"\n # - \"month_cos\"\n - \"hour_sin\"\n - \"hour_cos\"\n # - \"minute_sin\"\n # - \"minute_cos\"\n```\n\nAnyway, it seems to be 4 features :)", "2513057": "Thank you for sharing. Keep up the good work. ", "2547382": "", "2537312": "", "2531446": "", "2530546": "", "2530425": "", "2530420": "", "2526087": "", "2523739": "", "2516294": "", "2513364": "", "2513262": "", "2512888": "", "2512478": "", "2512275": "", "2512422": "Thanks for sharing!", "2543292": "Thanks for your sharing", "2534871": "Thanks for your sharing", "2515514": "thanks for sharing! " } }