{
  "id": 469427,
  "title": "Other models?",
  "url": "/competitions/bench-tab-v1/discussion/469427",
  "author_name": "",
  "post_date": "2024-01-20T13:40:01.241796300Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Of course, despite its title, this competition is not limited to gbdts and NNs. I would be very curious to see if models outside of those two categories get competitive. Below are some ideas:</p>\n<ul>\n<li>Known approaches: <strong>knn or svm</strong> seems a bit outdated, but its not rare to see some GM using one of those as baseline. I get they are a bit difficult to handle as each feature needs to be scaled idependently.</li>\n<li><strong>Tsetlin machines</strong>: <a href=\"https://en.wikipedia.org/wiki/Tsetlin_machine\" target=\"_blank\">https://en.wikipedia.org/wiki/Tsetlin_machine</a> I have seen some claims about those being competitive but no real performance test. Any way to make those works here?</li>\n<li><strong>Topological Data Analysis</strong>: highly speculative, but I know some of highly efficient unsupervised appraoches (<a href=\"https://arxiv.org/abs/1802.03426\" target=\"_blank\">UMAP</a>), now working in a supervised way. Can this become competitive?</li>\n<li><strong>Quantum ML</strong>: <a href=\"https://arxiv.org/pdf/2212.03223.pdf\" target=\"_blank\">Financial Risk Management on a Neutral Atom Quantum Processor</a> implements quantum ML model and claims to outperform a RF baseline. I'd really like to know if there is a path forward (or if they chose a bad baseline on purpose :-)).</li>\n</ul>\n<p>Please let us know if you have heard about other approaches / if you want (and know how to) test one of those alterntive approaches. </p>",
  "messages": [
    {
      "id": "2611004",
      "postDate": "01/20/2024 13:40:01",
      "content": "<p>Of course, despite its title, this competition is not limited to gbdts and NNs. I would be very curious to see if models outside of those two categories get competitive. Below are some ideas:</p>\n<ul>\n<li>Known approaches: <strong>knn or svm</strong> seems a bit outdated, but its not rare to see some GM using one of those as baseline. I get they are a bit difficult to handle as each feature needs to be scaled idependently.</li>\n<li><strong>Tsetlin machines</strong>: <a href=\"https://en.wikipedia.org/wiki/Tsetlin_machine\" target=\"_blank\">https://en.wikipedia.org/wiki/Tsetlin_machine</a> I have seen some claims about those being competitive but no real performance test. Any way to make those works here?</li>\n<li><strong>Topological Data Analysis</strong>: highly speculative, but I know some of highly efficient unsupervised appraoches (<a href=\"https://arxiv.org/abs/1802.03426\" target=\"_blank\">UMAP</a>), now working in a supervised way. Can this become competitive?</li>\n<li><strong>Quantum ML</strong>: <a href=\"https://arxiv.org/pdf/2212.03223.pdf\" target=\"_blank\">Financial Risk Management on a Neutral Atom Quantum Processor</a> implements quantum ML model and claims to outperform a RF baseline. I'd really like to know if there is a path forward (or if they chose a bad baseline on purpose :-)).</li>\n</ul>\n<p>Please let us know if you have heard about other approaches / if you want (and know how to) test one of those alterntive approaches. </p>",
      "rawMarkdown": "Of course, despite its title, this competition is not limited to gbdts and NNs. I would be very curious to see if models outside of those two categories get competitive. Below are some ideas:\n\n- Known approaches: **knn or svm** seems a bit outdated, but its not rare to see some GM using one of those as baseline. I get they are a bit difficult to handle as each feature needs to be scaled idependently.\n- **Tsetlin machines**: https://en.wikipedia.org/wiki/Tsetlin_machine I have seen some claims about those being competitive but no real performance test. Any way to make those works here?\n- **Topological Data Analysis**: highly speculative, but I know some of highly efficient unsupervised appraoches ([UMAP](https://arxiv.org/abs/1802.03426)), now working in a supervised way. Can this become competitive?\n- **Quantum ML**: [Financial Risk Management on a Neutral Atom Quantum Processor](https://arxiv.org/pdf/2212.03223.pdf) implements quantum ML model and claims to outperform a RF baseline. I'd really like to know if there is a path forward (or if they chose a bad baseline on purpose :-)).\n\nPlease let us know if you have heard about other approaches / if you want (and know how to) test one of those alterntive approaches.",
      "votes": null
    },
    {
      "id": "2611130",
      "postDate": "01/20/2024 15:14:26",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> ,</p>\n<p>This is a very interesting subject.  Plenty of research and work has gone into it.<br>\nThere is a updated blog by Sebastian Raschka regarding the chronology of deep learning of Tabular data which summarises most relevant, varied and interesting approaches and study comparing various techniques/models on tabular data. The list of paper includes the paper mentioned in the competition. </p>\n<p>Link : <a href=\"url\" target=\"_blank\">https://sebastianraschka.com/blog/2022/deep-learning-for-tabular-data.html#gate-gated-additive-tree-ensemble-for-tabular-classification-and-regression-2022-07</a></p>\n<p>Rather than conclusive this blog to be more factual based on research. I hope you find this helpful.</p>\n<p>thanks!</p>",
      "rawMarkdown": "Hi @lucasmorin ,\n\nThis is a very interesting subject.  Plenty of research and work has gone into it.\nThere is a updated blog by Sebastian Raschka regarding the chronology of deep learning of Tabular data which summarises most relevant, varied and interesting approaches and study comparing various techniques/models on tabular data. The list of paper includes the paper mentioned in the competition. \n\nLink : [https://sebastianraschka.com/blog/2022/deep-learning-for-tabular-data.html#gate-gated-additive-tree-ensemble-for-tabular-classification-and-regression-2022-07](url)\n\nRather than conclusive this blog to be more factual based on research. I hope you find this helpful.\n\nthanks!",
      "votes": null
    },
    {
      "id": "2611220",
      "postDate": "01/20/2024 16:10:42",
      "content": "<p>I tried Ngboost, but it runs into LinAlg error. If someone has a guess, I would be glad to hear.</p>",
      "rawMarkdown": "I tried Ngboost, but it runs into LinAlg error. If someone has a guess, I would be glad to hear.",
      "votes": null
    },
    {
      "id": "2611706",
      "postDate": "01/20/2024 22:40:26",
      "content": "<p>I think the kkn approach is the closest to a proper statistical solution. I have a hunch that it may outperform tree-based models in the near future. Trees are just playing on the statistical approach. I added a symbolic regression model, it is close in terms of score to the sklearn implementation of RandomForest. A notebook with a fixed number of iterations, and with a fixed time of 1s for the fit task. </p>",
      "rawMarkdown": "I think the kkn approach is the closest to a proper statistical solution. I have a hunch that it may outperform tree-based models in the near future. Trees are just playing on the statistical approach. I added a symbolic regression model, it is close in terms of score to the sklearn implementation of RandomForest. A notebook with a fixed number of iterations, and with a fixed time of 1s for the fit task.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2611130,
      "author_name": "supplejade",
      "author_url": "",
      "post_date": "01/20/2024 15:14:26",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/lucasmorin\" target=\"_blank\">@lucasmorin</a> ,</p>\n<p>This is a very interesting subject.  Plenty of research and work has gone into it.<br>\nThere is a updated blog by Sebastian Raschka regarding the chronology of deep learning of Tabular data which summarises most relevant, varied and interesting approaches and study comparing various techniques/models on tabular data. The list of paper includes the paper mentioned in the competition. </p>\n<p>Link : <a href=\"url\" target=\"_blank\">https://sebastianraschka.com/blog/2022/deep-learning-for-tabular-data.html#gate-gated-additive-tree-ensemble-for-tabular-classification-and-regression-2022-07</a></p>\n<p>Rather than conclusive this blog to be more factual based on research. I hope you find this helpful.</p>\n<p>thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2611220,
      "author_name": "thomasmeiner",
      "author_url": "",
      "post_date": "01/20/2024 16:10:42",
      "content": "<p>I tried Ngboost, but it runs into LinAlg error. If someone has a guess, I would be glad to hear.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2611706,
      "author_name": "jano123",
      "author_url": "",
      "post_date": "01/20/2024 22:40:26",
      "content": "<p>I think the kkn approach is the closest to a proper statistical solution. I have a hunch that it may outperform tree-based models in the near future. Trees are just playing on the statistical approach. I added a symbolic regression model, it is close in terms of score to the sklearn implementation of RandomForest. A notebook with a fixed number of iterations, and with a fixed time of 1s for the fit task. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2611004": "Of course, despite its title, this competition is not limited to gbdts and NNs. I would be very curious to see if models outside of those two categories get competitive. Below are some ideas:\n\n- Known approaches: **knn or svm** seems a bit outdated, but its not rare to see some GM using one of those as baseline. I get they are a bit difficult to handle as each feature needs to be scaled idependently.\n- **Tsetlin machines**: https://en.wikipedia.org/wiki/Tsetlin_machine I have seen some claims about those being competitive but no real performance test. Any way to make those works here?\n- **Topological Data Analysis**: highly speculative, but I know some of highly efficient unsupervised appraoches ([UMAP](https://arxiv.org/abs/1802.03426)), now working in a supervised way. Can this become competitive?\n- **Quantum ML**: [Financial Risk Management on a Neutral Atom Quantum Processor](https://arxiv.org/pdf/2212.03223.pdf) implements quantum ML model and claims to outperform a RF baseline. I'd really like to know if there is a path forward (or if they chose a bad baseline on purpose :-)).\n\nPlease let us know if you have heard about other approaches / if you want (and know how to) test one of those alterntive approaches.",
    "2611130": "Hi @lucasmorin ,\n\nThis is a very interesting subject.  Plenty of research and work has gone into it.\nThere is a updated blog by Sebastian Raschka regarding the chronology of deep learning of Tabular data which summarises most relevant, varied and interesting approaches and study comparing various techniques/models on tabular data. The list of paper includes the paper mentioned in the competition. \n\nLink : [https://sebastianraschka.com/blog/2022/deep-learning-for-tabular-data.html#gate-gated-additive-tree-ensemble-for-tabular-classification-and-regression-2022-07](url)\n\nRather than conclusive this blog to be more factual based on research. I hope you find this helpful.\n\nthanks!",
    "2611220": "I tried Ngboost, but it runs into LinAlg error. If someone has a guess, I would be glad to hear.",
    "2611706": "I think the kkn approach is the closest to a proper statistical solution. I have a hunch that it may outperform tree-based models in the near future. Trees are just playing on the statistical approach. I added a symbolic regression model, it is close in terms of score to the sklearn implementation of RandomForest. A notebook with a fixed number of iterations, and with a fixed time of 1s for the fit task."
  },
  "source": "meta"
}