The goal for participation in this Kaggle competition was to see how far one could get with what I would consider a standard molecular descriptor based QSAR approach using AutoML.
\nThe workflow is as follows:
\nFirst three steps were wrapped into a KNIME workflow and the processed train and test data files were saved in feather file format (mainly to save disk space and speed up reading the datasets later on).
\nThe AutoML process in FLAML uses a custom cohen_kappa_score loss function during the training process. This was done to train with a metric that is similar to the metric used in scoring the predictions. Class weights are also used to account for the imbalance in the dataset classes.
\nSteps 1-4 achieved public leaderboard scores within the top 20 and I wanted to look for additional descriptors outside the \"usual stuff\". The idea I had was to use other public solubility datasets and create regression models with them, then predict solubility for the compounds of this competition in the training and test dataset and use the predictions as additional descriptors.
\nFor the regression models, I used the cleaned datasets from https://github.com/Mengjintao/SolCuration and followed a similar approach with calculating RDKit 2D, Mogen FP, and Mol2Vec descriptors, followed by feature selection and AutoML regression building using FLAML.
\nAdding the solubility predictions to the standard features helped to increase the score of the final prediction a little bit, just enough to make it into the top 10.
", "comments": [ { "id": 2154074, "author_name": "Mr. Maniac", "author_url": "", "post_date": "2023-02-21T19:52:36.917000", "content": "I have created a notebook (my first) with the FLAML AutoML code and uploaded the training and test data files I have used for the final model (dataset now made public). https://www.kaggle.com/code/mrmanic/automl-with-flaml
\nNote that the notebook has the AutoML run for only 3 minutes but I ran this for around 4 hours.
I also intended to share the final (pickled) model file but I cannot load it into my notebook here at Kaggle because of having used a newer Python version. If anyone is interested in the model, please let me know and I upload it into the public dataset of the notebook.
", "votes": 1, "replies": [] }, { "id": 2165113, "author_name": "RSESHA", "author_url": "", "post_date": "2023-03-02T02:07:34.427000", "content": "Hi @mrmanic :
\nGlad to know you used feature selection. How did the featurewiz library work out for you? Did it hurt or help? How many features were reduced? Just curious.
\nThanks
\nRam
Hi Ram,
\nthe use of feature selection was mainly driven by my old hardware setup at the beginning of the competition. The 1024 fingerprint bits plus RDKit 2D properties plus ~ 300 Mol2Vec descriptors were frequently consuming too much memory during the AutoML runs. Therefore I experimented a bit with feature selection. The number of features was reduced to ~590 and this reduced feature set produced better scores on the leader board than the full set of features. But to be honest, I did not explore this very thoroughly or did any systematic exploration due to time constraints. It could well be that the reduced dataset simply allowed for more or more complex models to be fitted to the training data and therefore a better score was obtained.
Thanks - that was helpful to know.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2157834, "author_name": "Bernhard Rohde", "author_url": "", "post_date": "2023-02-24T11:14:51.783000", "content": "You might want to connect your solution description to the Leaderboard entry.
", "votes": 0, "replies": [ { "id": 2157844, "author_name": "Mr. Maniac", "author_url": "", "post_date": "2023-02-24T11:33:45.963000", "content": "Hi Bernhard,
\nI tried to do that but couldn't figure out how to do it. Can you briefly explain it or point me to a resource where I read it up?
\nThanks
It's on the 'Team' tab (https://www.kaggle.com/competitions/euos-slas/team). I also had a hard time finding it, but considering access control, this looks like the 'logical' place to put it ;-) as it contains the link to the description for the team's entry.
", "votes": 0, "replies": [] }, { "id": 2158009, "author_name": "Mr. Maniac", "author_url": "", "post_date": "2023-02-24T14:40:23.677000", "content": "Thank you so much. Added the link to the leaderboard.
", "votes": 0, "replies": [] } ] } ] } ], "comment_count": 8, "messages": [ { "id": 2153992, "postDate": "2023-02-21T18:51:19.790Z", "content": "The goal for participation in this Kaggle competition was to see how far one could get with what I would consider a standard molecular descriptor based QSAR approach using AutoML.
\nThe workflow is as follows:
\nFirst three steps were wrapped into a KNIME workflow and the processed train and test data files were saved in feather file format (mainly to save disk space and speed up reading the datasets later on).
\nThe AutoML process in FLAML uses a custom cohen_kappa_score loss function during the training process. This was done to train with a metric that is similar to the metric used in scoring the predictions. Class weights are also used to account for the imbalance in the dataset classes.
\nSteps 1-4 achieved public leaderboard scores within the top 20 and I wanted to look for additional descriptors outside the \"usual stuff\". The idea I had was to use other public solubility datasets and create regression models with them, then predict solubility for the compounds of this competition in the training and test dataset and use the predictions as additional descriptors.
\nFor the regression models, I used the cleaned datasets from https://github.com/Mengjintao/SolCuration and followed a similar approach with calculating RDKit 2D, Mogen FP, and Mol2Vec descriptors, followed by feature selection and AutoML regression building using FLAML.
\nAdding the solubility predictions to the standard features helped to increase the score of the final prediction a little bit, just enough to make it into the top 10.
", "rawMarkdown": "The goal for participation in this Kaggle competition was to see how far one could get with what I would consider a standard molecular descriptor based QSAR approach using AutoML.\n\nThe workflow is as follows:\n\n1. Input data cleaning (salt stripping, SMILES canonicalization)\n2. Descriptor calculation (RDKit 2D, Feature Morgen, Mol2Vec)\n3. Feature selection (using featurewiz library) to remove features with a correlation > 0.80\n4. AutoML (using FLAML)\n5. AutoML (using FLAML) with four additional descriptors based on solubility regression models\n\nFirst three steps were wrapped into a KNIME workflow and the processed train and test data files were saved in feather file format (mainly to save disk space and speed up reading the datasets later on). \n\nThe AutoML process in FLAML uses a custom cohen_kappa_score loss function during the training process. This was done to train with a metric that is similar to the metric used in scoring the predictions. Class weights are also used to account for the imbalance in the dataset classes.\n\nSteps 1-4 achieved public leaderboard scores within the top 20 and I wanted to look for additional descriptors outside the \"usual stuff\". The idea I had was to use other public solubility datasets and create regression models with them, then predict solubility for the compounds of this competition in the training and test dataset and use the predictions as additional descriptors. \n\nFor the regression models, I used the cleaned datasets from https://github.com/Mengjintao/SolCuration and followed a similar approach with calculating RDKit 2D, Mogen FP, and Mol2Vec descriptors, followed by feature selection and AutoML regression building using FLAML. \n\nAdding the solubility predictions to the standard features helped to increase the score of the final prediction a little bit, just enough to make it into the top 10.\n\n", "votes": 4 }, { "id": 2154074, "postDate": "2023-02-21T19:52:36.917Z", "content": "I have created a notebook (my first) with the FLAML AutoML code and uploaded the training and test data files I have used for the final model (dataset now made public). https://www.kaggle.com/code/mrmanic/automl-with-flaml
\nNote that the notebook has the AutoML run for only 3 minutes but I ran this for around 4 hours.
I also intended to share the final (pickled) model file but I cannot load it into my notebook here at Kaggle because of having used a newer Python version. If anyone is interested in the model, please let me know and I upload it into the public dataset of the notebook.
", "rawMarkdown": "I have created a notebook (my first) with the FLAML AutoML code and uploaded the training and test data files I have used for the final model (dataset now made public). [https://www.kaggle.com/code/mrmanic/automl-with-flaml](https://www.kaggle.com/code/mrmanic/automl-with-flaml)\nNote that the notebook has the AutoML run for only 3 minutes but I ran this for around 4 hours.\n\nI also intended to share the final (pickled) model file but I cannot load it into my notebook here at Kaggle because of having used a newer Python version. If anyone is interested in the model, please let me know and I upload it into the public dataset of the notebook.", "votes": 1 }, { "id": 2165113, "postDate": "2023-03-02T02:07:34.427Z", "content": "Hi @mrmanic :
\nGlad to know you used feature selection. How did the featurewiz library work out for you? Did it hurt or help? How many features were reduced? Just curious.
\nThanks
\nRam
Hi Ram,
\nthe use of feature selection was mainly driven by my old hardware setup at the beginning of the competition. The 1024 fingerprint bits plus RDKit 2D properties plus ~ 300 Mol2Vec descriptors were frequently consuming too much memory during the AutoML runs. Therefore I experimented a bit with feature selection. The number of features was reduced to ~590 and this reduced feature set produced better scores on the leader board than the full set of features. But to be honest, I did not explore this very thoroughly or did any systematic exploration due to time constraints. It could well be that the reduced dataset simply allowed for more or more complex models to be fitted to the training data and therefore a better score was obtained.
Thanks - that was helpful to know.
", "rawMarkdown": "Thanks - that was helpful to know." } ] } ] }, { "id": 2157834, "postDate": "2023-02-24T11:14:51.783Z", "content": "You might want to connect your solution description to the Leaderboard entry.
", "rawMarkdown": "You might want to connect your solution description to the Leaderboard entry.", "replies": [ { "id": 2157844, "postDate": "2023-02-24T11:33:45.963Z", "content": "Hi Bernhard,
\nI tried to do that but couldn't figure out how to do it. Can you briefly explain it or point me to a resource where I read it up?
\nThanks
It's on the 'Team' tab (https://www.kaggle.com/competitions/euos-slas/team). I also had a hard time finding it, but considering access control, this looks like the 'logical' place to put it ;-) as it contains the link to the description for the team's entry.
", "rawMarkdown": "It's on the 'Team' tab (https://www.kaggle.com/competitions/euos-slas/team). I also had a hard time finding it, but considering access control, this looks like the 'logical' place to put it ;-) as it contains the link to the description for the team's entry." }, { "id": 2158009, "postDate": "2023-02-24T14:40:23.677Z", "content": "Thank you so much. Added the link to the leaderboard.
", "rawMarkdown": "Thank you so much. Added the link to the leaderboard." } ] } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T16:45:40Z", "competition_id": 38158, "competition_slug": "euos-slas", "team_id": 9194019, "solution_url": "/competitions/euos-slas/writeups/mr-maniac-no-eos-id-was-harmed-in-this-prediction-" }