{
  "id": 586031,
  "title": "[Notebook Inference Server Prediction Timed Out] Watch out for unwanted parallelism issues in the submission's predict function",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/586031",
  "author_name": "Márk Antal Csizmadia",
  "post_date": "2025-06-24T16:40:35.352000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I had a nasty bug in one of my recent submissions, and I wanted to share what it was in the hope that it can help others avoid it.</p>\n<p>I used a lot of preprocessing steps in the <code>predict</code> function and when I ran the code on kaggle with calling the test inference server, my code woud get stuck after some sequences. Not always the same number of sequences, but usually after 3-7. I got the warining <code>Other threads are currently calling into gRPC, skipping fork() handlers</code> in the code, and when I submitted the notebook, I got <code>Notebook Inference Server Prediction Timed Out</code> which resulted in failed submissions. </p>\n<p>This error message, as far as I could see, is not that well documented on Kaggle, so here is what I found. After a lot of debugging, I figured out that the issue was that seemingly we cannot use parallelism / threading within the predict function becasue the inference server uses gRPC which in turn uses background threads (e.g. for networking for the submission). In particular, the bug was that a <a href=\"https://tsfresh.readthedocs.io/en/latest/\" target=\"_blank\">tsfresh</a> feature extracion function by default used parallelisation with its <code>n_jobs</code> argument (see <a href=\"https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html#module-tsfresh.feature_extraction.extraction\" target=\"_blank\">here</a>). This bug could happen with many other packages that aim to speed up data processing through parallelism</p>\n<p>I am wondering if others experienced this issue too?</p>\n<p>Happy competing!</p>",
  "messages": [
    {
      "id": 3231619,
      "postDate": "2025-06-24T16:40:35.353Z",
      "content": "<p>I had a nasty bug in one of my recent submissions, and I wanted to share what it was in the hope that it can help others avoid it.</p>\n<p>I used a lot of preprocessing steps in the <code>predict</code> function and when I ran the code on kaggle with calling the test inference server, my code woud get stuck after some sequences. Not always the same number of sequences, but usually after 3-7. I got the warining <code>Other threads are currently calling into gRPC, skipping fork() handlers</code> in the code, and when I submitted the notebook, I got <code>Notebook Inference Server Prediction Timed Out</code> which resulted in failed submissions. </p>\n<p>This error message, as far as I could see, is not that well documented on Kaggle, so here is what I found. After a lot of debugging, I figured out that the issue was that seemingly we cannot use parallelism / threading within the predict function becasue the inference server uses gRPC which in turn uses background threads (e.g. for networking for the submission). In particular, the bug was that a <a href=\"https://tsfresh.readthedocs.io/en/latest/\" target=\"_blank\">tsfresh</a> feature extracion function by default used parallelisation with its <code>n_jobs</code> argument (see <a href=\"https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html#module-tsfresh.feature_extraction.extraction\" target=\"_blank\">here</a>). This bug could happen with many other packages that aim to speed up data processing through parallelism</p>\n<p>I am wondering if others experienced this issue too?</p>\n<p>Happy competing!</p>",
      "rawMarkdown": "I had a nasty bug in one of my recent submissions, and I wanted to share what it was in the hope that it can help others avoid it.\n\nI used a lot of preprocessing steps in the `predict` function and when I ran the code on kaggle with calling the test inference server, my code woud get stuck after some sequences. Not always the same number of sequences, but usually after 3-7. I got the warining `Other threads are currently calling into gRPC, skipping fork() handlers` in the code, and when I submitted the notebook, I got `Notebook Inference Server Prediction Timed Out` which resulted in failed submissions. \n\nThis error message, as far as I could see, is not that well documented on Kaggle, so here is what I found. After a lot of debugging, I figured out that the issue was that seemingly we cannot use parallelism / threading within the predict function becasue the inference server uses gRPC which in turn uses background threads (e.g. for networking for the submission). In particular, the bug was that a [tsfresh](https://tsfresh.readthedocs.io/en/latest/) feature extracion function by default used parallelisation with its `n_jobs` argument (see [here](https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html#module-tsfresh.feature_extraction.extraction)). This bug could happen with many other packages that aim to speed up data processing through parallelism\n\nI am wondering if others experienced this issue too?\n\nHappy competing!",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3231619": "I had a nasty bug in one of my recent submissions, and I wanted to share what it was in the hope that it can help others avoid it.\n\nI used a lot of preprocessing steps in the `predict` function and when I ran the code on kaggle with calling the test inference server, my code woud get stuck after some sequences. Not always the same number of sequences, but usually after 3-7. I got the warining `Other threads are currently calling into gRPC, skipping fork() handlers` in the code, and when I submitted the notebook, I got `Notebook Inference Server Prediction Timed Out` which resulted in failed submissions. \n\nThis error message, as far as I could see, is not that well documented on Kaggle, so here is what I found. After a lot of debugging, I figured out that the issue was that seemingly we cannot use parallelism / threading within the predict function becasue the inference server uses gRPC which in turn uses background threads (e.g. for networking for the submission). In particular, the bug was that a [tsfresh](https://tsfresh.readthedocs.io/en/latest/) feature extracion function by default used parallelisation with its `n_jobs` argument (see [here](https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html#module-tsfresh.feature_extraction.extraction)). This bug could happen with many other packages that aim to speed up data processing through parallelism\n\nI am wondering if others experienced this issue too?\n\nHappy competing!"
  }
}