I am sure many people have already been using QM9 to get deeper insight. You can now access the dataset here on Kaggle, and I added a kernel which discusses what data we have there, extracts features and runs a basic LightGBM on them.
\n\nIn summary, QM9 contains full structural information, mulliken partial charges, frequencies for degrees of freedom and many characteristics on the level of molecule. What is vital is that it contains this data not only for the train but also for the test molecules. Please take a look at the kernel and the dataset for more details.
\n\nThe kernel generates a dataframe which contains the features extracted from QM9 for train + test data. You can download the output file and add these columns to your features to check their impact. I tried to do exactly that in this different kernel, as base for my experiment I selected a public kernel which runs fast (it has only one model trained on 90% of the train data, with 10% for early stopping). Simply adding the QM9 features and rerunning improved the LB score from -0.462 to -0.585.
\n\nPlease note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions. My motivation for exploring what can be done with it lies in my fears that it can actually be abused and make this competition really problematic to participate in. These are my points:
\n\nThe features from this forbidden dataset improve public LB scores, and therefore you will not see \"real\" scores at the top, but rather scores obtained with QM9. Moreover it will never be clear if someone used the dataset or not on the public LB.
More importantly, models which use QM9 will score higher on the private LB as well. It is not allowed to use it directly, but one can choose models and random seeds which generate predictions closer to it. Not sure how important it is, but it looks like it can be important, therefore not making it fun to anyone. Am I being paranoiac here? I shared my concerns with the organizers here, no answer yet.
AFAIK, Kaggle usually does not check models for people outside of the gold medals range (too much work), or even only for teams in the money, therefore wreaking havoc for the fair silver/bronze medals competitors.
UPDATE: This post has received multiple downvotes, apparently for sharing QM9. I am honestly surprised, haven't seen such attitude to knowledge and asking questions from the times of Giordano Bruno. This dataset contains a wealth of knowledge on the problem; one can learn what are the relevant characteristics, interactions, dependencies, what latent variables explain the target. Either every serious participant downloads it silently and study, or it becomes a public discussion. Even just the knowledge how much it can improve the LB score is important for the public, given the setting. If, for example, we know that the dataset can improve the score only slightly, I am sure everyone will feel much more at ease.
\n\nUPDATE 2: I removed the dataset and the kernels, apparently it is not allowed to study it in the public domain.
\n\nUPDATE 3: Following the rules update I uploaded the dataset back and re-shared the kernels, hope it saves you time.
", "messages": [ { "id": 548541, "postDate": "2019-06-09T14:50:15.630Z", "content": "I am sure many people have already been using QM9 to get deeper insight. You can now access the dataset here on Kaggle, and I added a kernel which discusses what data we have there, extracts features and runs a basic LightGBM on them.
\n\nIn summary, QM9 contains full structural information, mulliken partial charges, frequencies for degrees of freedom and many characteristics on the level of molecule. What is vital is that it contains this data not only for the train but also for the test molecules. Please take a look at the kernel and the dataset for more details.
\n\nThe kernel generates a dataframe which contains the features extracted from QM9 for train + test data. You can download the output file and add these columns to your features to check their impact. I tried to do exactly that in this different kernel, as base for my experiment I selected a public kernel which runs fast (it has only one model trained on 90% of the train data, with 10% for early stopping). Simply adding the QM9 features and rerunning improved the LB score from -0.462 to -0.585.
\n\nPlease note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions. My motivation for exploring what can be done with it lies in my fears that it can actually be abused and make this competition really problematic to participate in. These are my points:
\n\nThe features from this forbidden dataset improve public LB scores, and therefore you will not see \"real\" scores at the top, but rather scores obtained with QM9. Moreover it will never be clear if someone used the dataset or not on the public LB.
More importantly, models which use QM9 will score higher on the private LB as well. It is not allowed to use it directly, but one can choose models and random seeds which generate predictions closer to it. Not sure how important it is, but it looks like it can be important, therefore not making it fun to anyone. Am I being paranoiac here? I shared my concerns with the organizers here, no answer yet.
AFAIK, Kaggle usually does not check models for people outside of the gold medals range (too much work), or even only for teams in the money, therefore wreaking havoc for the fair silver/bronze medals competitors.
UPDATE: This post has received multiple downvotes, apparently for sharing QM9. I am honestly surprised, haven't seen such attitude to knowledge and asking questions from the times of Giordano Bruno. This dataset contains a wealth of knowledge on the problem; one can learn what are the relevant characteristics, interactions, dependencies, what latent variables explain the target. Either every serious participant downloads it silently and study, or it becomes a public discussion. Even just the knowledge how much it can improve the LB score is important for the public, given the setting. If, for example, we know that the dataset can improve the score only slightly, I am sure everyone will feel much more at ease.
\n\nUPDATE 2: I removed the dataset and the kernels, apparently it is not allowed to study it in the public domain.
\n\nUPDATE 3: Following the rules update I uploaded the dataset back and re-shared the kernels, hope it saves you time.
", "rawMarkdown": "I am sure many people have already been using QM9 to get deeper insight. You can now [access the dataset here on Kaggle](https://www.kaggle.com/zaharch/quantum-machine-9-aka-qm9), and I [added a kernel](https://www.kaggle.com/zaharch/quantum-machine-9-qm9) which discusses what data we have there, extracts features and runs a basic LightGBM on them.\n\nIn summary, QM9 contains full structural information, mulliken partial charges, frequencies for degrees of freedom and many characteristics on the level of molecule. What is vital is that it contains this data not only for the train but also for the test molecules. Please take a look at the kernel and the dataset for more details.\n\nThe kernel generates a dataframe which contains the features extracted from QM9 for train + test data. You can download the output file and add these columns to your features to check their impact. I tried to do exactly that in [this different kernel](https://www.kaggle.com/zaharch/lightgbm-full-pipeline-model-qm9), as base for my experiment I selected [a public kernel](https://www.kaggle.com/kabure/lightgbm-full-pipeline-model) which runs fast (it has only one model trained on 90% of the train data, with 10% for early stopping). Simply adding the QM9 features and rerunning improved the LB score from -0.462 to -0.585. \n\nPlease note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions. My motivation for exploring what can be done with it lies in my fears that it can actually be abused and make this competition really problematic to participate in. These are my points:\n\n1. The features from this forbidden dataset improve public LB scores, and therefore you will not see \"real\" scores at the top, but rather scores obtained with QM9. Moreover it will never be clear if someone used the dataset or not on the public LB. \n\n2. More importantly, models which use QM9 will score higher on the private LB as well. It is not allowed to use it directly, but one can choose models and random seeds which generate predictions closer to it. Not sure how important it is, but it looks like it can be important, therefore not making it fun to anyone. Am I being paranoiac here? I shared my concerns [with the organizers here](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93858#546053), no answer yet. \n\n3. AFAIK, Kaggle usually does not check models for people outside of the gold medals range (too much work), or even only for teams in the money, therefore wreaking havoc for the fair silver/bronze medals competitors. \n\nUPDATE: This post has received multiple downvotes, apparently for sharing QM9. I am honestly surprised, haven't seen such attitude to knowledge and asking questions from the times of Giordano Bruno. This dataset contains a wealth of knowledge on the problem; one can learn what are the relevant characteristics, interactions, dependencies, what latent variables explain the target. Either every serious participant downloads it silently and study, or it becomes a public discussion. Even just the knowledge how much it can improve the LB score is important for the public, given the setting. If, for example, we know that the dataset can improve the score only slightly, I am sure everyone will feel much more at ease.\n\nUPDATE 2: I removed the dataset and the kernels, apparently it is not allowed to study it in the public domain. \n\nUPDATE 3: Following the rules update I uploaded the dataset back and re-shared the kernels, hope it saves you time.", "votes": 47 }, { "id": 549471, "postDate": "2019-06-10T17:31:18.550Z", "content": "This is a very important post. If @nosound didn't share it, then someone else would have. If no one had shared it then I'm sure that certain individuals would use it to leverage their leaderboard position.
\n\nWhat we need is a comment from Kaggle on how they might enforce the rule around QM9 for positions outside of the money so that medals are not awarded unfairly.
", "rawMarkdown": "This is a very important post. If @nosound didn't share it, then someone else would have. If no one had shared it then I'm sure that certain individuals would use it to leverage their leaderboard position.\n\nWhat we need is a comment from Kaggle on how they might enforce the rule around QM9 for positions outside of the money so that medals are not awarded unfairly.", "votes": 14 }, { "id": 550628, "postDate": "2019-06-11T21:00:08Z", "content": "Let's not shoot the messenger. The best news is that QM9 issue came up 3 months instead of 3 days before the deadline.
", "rawMarkdown": "Let's not shoot the messenger. The best news is that QM9 issue came up 3 months instead of 3 days before the deadline.", "votes": 10 }, { "id": 548565, "postDate": "2019-06-09T15:22:23.273Z", "content": "Is this not a violation of the contest rules? You write, \"Please note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions.\" This sentence makes it sound as if the ban is just on using QM9 for your final submission, but it looks like the rules have a broader ban on the use of QM9 for any model development or testing:
\n\n\"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set, including the QM7 and QM9 data sets.\"
", "rawMarkdown": "Is this not a violation of the contest rules? You write, \"Please note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions.\" This sentence makes it sound as if the ban is just on using QM9 for your *final* submission, but it looks like the rules have a broader ban on the use of QM9 for any model development or testing: \n\n\"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set, including the QM7 and QM9 data sets.\"", "votes": 4, "replies": [ { "id": 548577, "postDate": "2019-06-09T15:35:22.897Z", "content": "QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.
\n\nMoreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? Or you just blindly believe that everyone will play by the rules? With so many participants in the competition, everything will be tried.
", "rawMarkdown": "QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.\n\nMoreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? Or you just blindly believe that everyone will play by the rules? With so many participants in the competition, everything will be tried.", "votes": 2 }, { "id": 548582, "postDate": "2019-06-09T15:39:45.653Z", "content": "\"QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.\"
\n\nI'm not sure what the distinction is between \"studying\" QM9 vs. using QM9 for development and testing of one's models.
\n\n\"Moreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? \"
\n\nI share your concern. I'm not sure there's anything the organizers can do at this point. It's a flaw in the contest design, IMO. One of the main reasons I'm thinking of dropping out.
", "rawMarkdown": "*\"QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.\"*\n\nI'm not sure what the distinction is between \"studying\" QM9 vs. using QM9 for development and testing of one's models.\n\n*\"Moreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? \"*\n\nI share your concern. I'm not sure there's anything the organizers can do at this point. It's a flaw in the contest design, IMO. One of the main reasons I'm thinking of dropping out.", "votes": 7 } ] }, { "id": 549445, "postDate": "2019-06-10T16:46:47.077Z", "content": "I have no clue why people down voted this topic. I got plenty of downvotes today. People should be more tolerant towards others point of views. Our approaches should not be limited to the competition but should address to solve bigger challenges. Keep it up man 👍
\n\nThere's a difference between expressing a point of view, and knowingly violating the Rules that every contestant agreed to when entering this competition, don't you think?
", "rawMarkdown": "There's a difference between expressing a point of view, and knowingly violating the Rules that every contestant agreed to when entering this competition, don't you think?", "votes": 6 }, { "id": 549538, "postDate": "2019-06-10T19:16:55.173Z", "content": "@inversion - Can you add a pinned entry to state this dataset is not allowed please?
", "rawMarkdown": "@inversion - Can you add a pinned entry to state this dataset is not allowed please?", "votes": 2 }, { "id": 549549, "postDate": "2019-06-10T19:44:22.717Z", "content": "@scirpus Great suggestion. Done.
", "rawMarkdown": "@scirpus Great suggestion. [Done](https://www.kaggle.com/c/champs-scalar-coupling/discussion/95211#latest-549548).", "votes": 2 }, { "id": 549607, "postDate": "2019-06-10T21:16:05.850Z", "content": "Thanks, @inversion , for the clarification. In your reaction I clearly see that you don't want this dataset to be shared and studied in the public domain, so I deleted it and made the two kernels of mine that use it private. I want to apologize and say that I did not intend to use it in testing or development of my models, nor I tried to encourage others to do so. Instead, I tried to understand how much extra information it contains, learn about the problem and ask the questions that worried me. Sincerely, from the competition rules it was not clear to me that this last part was not allowed. I see that you promised some follow-up, looking forward to it.
", "rawMarkdown": "Thanks, @inversion , for the clarification. In your reaction I clearly see that you don't want this dataset to be shared and studied in the public domain, so I deleted it and made the two kernels of mine that use it private. I want to apologize and say that I did not intend to use it in testing or development of my models, nor I tried to encourage others to do so. Instead, I tried to understand how much extra information it contains, learn about the problem and ask the questions that worried me. Sincerely, from the competition rules it was not clear to me that this last part was not allowed. I see that you promised some follow-up, looking forward to it.", "votes": 4 }, { "id": 550087, "postDate": "2019-06-11T09:36:00.103Z", "content": "Yeah. I would agree with you, it would be fair for everyone if we all use the same data. Anyone following this thread follow inversion's advice.
", "rawMarkdown": "Yeah. I would agree with you, it would be fair for everyone if we all use the same data. Anyone following this thread follow inversion's advice. ", "votes": 1 } ] }, { "id": 550461, "postDate": "2019-06-11T16:39:20.730Z", "content": "Thank you for your effortas, and sharing this knowledge. It is unfair that if a way of geting further information exists and has been made available, it should be only found by the people who entered early on, because they decide to take it down. It does not look like a fair competition anymore.
", "rawMarkdown": "Thank you for your effortas, and sharing this knowledge. It is unfair that if a way of geting further information exists and has been made available, it should be only found by the people who entered early on, because they decide to take it down. It does not look like a fair competition anymore.", "votes": 1 }, { "id": 550629, "postDate": "2019-06-11T21:00:22.123Z", "content": "It makes no sense to forbid the study of this dataset. The people will study secretly and use it making it difficult to everyone to use all the knowledge.
\n\nForbid the dataset its definitely the worse thing the organizers can do.
\n\nIMHO the only possible way to proceed is to share all the knowledge about this dataset in order to avoid unfairness.
", "rawMarkdown": "It makes no sense to forbid the study of this dataset. The people will study secretly and use it making it difficult to everyone to use all the knowledge.\n\nForbid the dataset its definitely the worse thing the organizers can do.\n\nIMHO the only possible way to proceed is to share all the knowledge about this dataset in order to avoid unfairness. " }, { "id": 2086049, "postDate": "2023-01-04T14:28:01.797Z", "content": "Good resource.
", "rawMarkdown": "Good resource. " }, { "id": 568638, "postDate": "2019-07-05T08:53:18.943Z", "content": "So i am kinda confused are we allowed to use to use this dataset and the original dataset from the compitition in combination..?
", "rawMarkdown": "So i am kinda confused are we allowed to use to use this dataset and the original dataset from the compitition in combination..?", "replies": [ { "id": 568774, "postDate": "2019-07-05T12:35:14.143Z", "content": "Hi, @harshitkhandelwal , yes it is allowed. The rules were updated to allow unrestricted use of the dataset. The discussion here is outdated.
", "rawMarkdown": "Hi, @harshitkhandelwal , yes it is allowed. The rules were updated to allow unrestricted use of the dataset. The discussion here is outdated.", "votes": 1 } ] }, { "id": 549504, "postDate": "2019-06-10T18:18:04.257Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 548578, "postDate": "2019-06-09T15:35:31.923Z", "content": "Hello! I understand your frustration. However posting a public kernel that uses QM9 is both against the rules, and in my humble opinion has exactly the opposite effect of what you would like to accomplish. Perhaps it is not to late to take it down. Let's just hope that everyone will play by the rules, and just concentrate on our own models .
", "rawMarkdown": "Hello! I understand your frustration. However posting a public kernel that uses QM9 is both against the rules, and in my humble opinion has exactly the opposite effect of what you would like to accomplish. Perhaps it is not to late to take it down. Let's just hope that everyone will play by the rules, and just concentrate on our own models .", "votes": 4, "isDeleted": true, "replies": [ { "id": 548581, "postDate": "2019-06-09T15:38:48.950Z", "content": "I want to accomplish an answer from the organizers how using QM9 will be prevented in the medals range.
", "rawMarkdown": "I want to accomplish an answer from the organizers how using QM9 will be prevented in the medals range.", "votes": 4 }, { "id": 548587, "postDate": "2019-06-09T15:43:56.577Z", "content": "I certainly understand that. What I don't understand is why at the same time you are also trying help others to potentially bend the rules. Please consider taking down your kernels that are explicitly prohibited by the organizers. Thanks
", "rawMarkdown": "I certainly understand that. What I don't understand is why at the same time you are also trying help others to potentially bend the rules. Please consider taking down your kernels that are explicitly prohibited by the organizers. Thanks", "votes": 2, "isDeleted": true }, { "id": 548595, "postDate": "2019-06-09T15:53:00.243Z", "content": "I don't agree with you position, let's agree to disagree. The dataset is right there online, whoever wants to use to cheat will use it. Hiding from the problem will not solve it.
", "rawMarkdown": "I don't agree with you position, let's agree to disagree. The dataset is right there online, whoever wants to use to cheat will use it. Hiding from the problem will not solve it.", "votes": 10 }, { "id": 548607, "postDate": "2019-06-09T16:09:28.753Z", "content": "We are only responsible for our own actions. Right now you personally are clearly violating the rules and at the same time you are doing everything in your power to potentially help others to do the same (posting the public kernel, making the QM9 dataset available on Kaggle). It really puzzles me, but perhaps we said everything about this that can said. Best of luck with the competition, and please at least think about stopping what you are doing. Zoltan
", "rawMarkdown": "We are only responsible for our own actions. Right now you personally are clearly violating the rules and at the same time you are doing everything in your power to potentially help others to do the same (posting the public kernel, making the QM9 dataset available on Kaggle). It really puzzles me, but perhaps we said everything about this that can said. Best of luck with the competition, and please at least think about stopping what you are doing. Zoltan", "votes": 2, "isDeleted": true }, { "id": 548611, "postDate": "2019-06-09T16:17:10.227Z", "content": "Well, I am not going to participate in the competition if my current understanding of this problem is correct, and I hope this post facilitates the discussion on this early competition stage. If the organizers think that the dataset and the kernel are inappropriate they will remove it, up to them. If this is indeed the case, a statement from the organizers that study QM9 with Kaggle datasets and kernels is not allowed and will be removed can help others in the future.
", "rawMarkdown": "Well, I am not going to participate in the competition if my current understanding of this problem is correct, and I hope this post facilitates the discussion on this early competition stage. If the organizers think that the dataset and the kernel are inappropriate they will remove it, up to them. If this is indeed the case, a statement from the organizers that study QM9 with Kaggle datasets and kernels is not allowed and will be removed can help others in the future. ", "votes": 3 }, { "id": 549543, "postDate": "2019-06-10T19:24:49.777Z", "content": "@nosound - What you just did is in extremely poor form.
\n\n1) You knowingly violated the competition Rules.\n2) You did so with the poor justification that you didn't get an answer as fast as you thought you should.
\n\nYou made the initial request to the competition sponsor 4 days ago. I was out of the office for the last 3.5 days, and thus had no time to coordinate a plan with the sponsor. So you thought you would take it upon yourself to break the rules you agreed to in order to \"facilitate discussion\"?
\n\nIf you ever (mistakenly) believe you need \"break a competition to save the competition\" again, you should at least have the courtesy to give more than 2 business days for a response.
", "rawMarkdown": "@nosound - What you just did is in extremely poor form.\n\n1) You knowingly violated the competition Rules.\n2) You did so with the poor justification that you didn't get an answer as fast as you thought you should. \n\nYou made the initial request to the competition sponsor 4 days ago. I was out of the office for the last 3.5 days, and thus had no time to coordinate a plan with the sponsor. So you thought you would take it upon yourself to break the rules **you agreed to** in order to \"facilitate discussion\"?\n\nIf you ever (mistakenly) believe you need \"break a competition to save the competition\" again, you should at least have the courtesy to give more than 2 business days for a response.", "votes": 7 }, { "id": 549622, "postDate": "2019-06-10T21:49:10.163Z", "content": "I did not understand from the rules that it was not allowed to study this dataset. Now it clear, - you don't want this dataset and kernels to be shared. I removed it.
", "rawMarkdown": "I did not understand from the rules that it was not allowed to study this dataset. Now it clear, - you don't want this dataset and kernels to be shared. I removed it.", "votes": 4 }, { "id": 550631, "postDate": "2019-06-11T21:03:08.530Z", "content": "\n\n\nLet's just hope that everyone will play by the rules, and just concentrate on our own models .
\n
Definitely the people will NOT play by the rules. We have seen it in multiples competitions.
", "rawMarkdown": "> Let's just hope that everyone will play by the rules, and just concentrate on our own models .\n\nDefinitely the people will NOT play by the rules. We have seen it in multiples competitions.", "votes": 1 }, { "id": 551342, "postDate": "2019-06-12T15:50:30.253Z", "content": "I am actually glad that he posted 3 months in advance, not 3 days. If he (or anyone) secretly uses this information, you will never know about it. You can of course remove QM9 from the kernel, but you cannot remove it from the internet, and people will know it and use it. You cannot just cover your eyes and pretend people will follow all the rules.
", "rawMarkdown": "I am actually glad that he posted 3 months in advance, not 3 days. If he (or anyone) secretly uses this information, you will never know about it. You can of course remove QM9 from the kernel, but you cannot remove it from the internet, and people will know it and use it. You cannot just cover your eyes and pretend people will follow all the rules.", "votes": 4 } ] }, { "id": 1064442, "postDate": "2020-10-30T05:58:38.353Z", "content": "Thanks for sharing
", "rawMarkdown": "Thanks for sharing" } ], "comments": [ { "id": 549471, "author_name": "datasaurus", "author_url": "", "post_date": "2019-06-10T17:31:18.550000", "content": "This is a very important post. If @nosound didn't share it, then someone else would have. If no one had shared it then I'm sure that certain individuals would use it to leverage their leaderboard position.
\n\nWhat we need is a comment from Kaggle on how they might enforce the rule around QM9 for positions outside of the money so that medals are not awarded unfairly.
", "votes": 14, "replies": [] }, { "id": 550628, "author_name": "Mycroft Holmes", "author_url": "", "post_date": "2019-06-11T21:00:08", "content": "Let's not shoot the messenger. The best news is that QM9 issue came up 3 months instead of 3 days before the deadline.
", "votes": 10, "replies": [] }, { "id": 548565, "author_name": "fafafa fafafafa", "author_url": "", "post_date": "2019-06-09T15:22:23.273000", "content": "Is this not a violation of the contest rules? You write, \"Please note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions.\" This sentence makes it sound as if the ban is just on using QM9 for your final submission, but it looks like the rules have a broader ban on the use of QM9 for any model development or testing:
\n\n\"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set, including the QM7 and QM9 data sets.\"
", "votes": 4, "replies": [ { "id": 548577, "author_name": "nosound", "author_url": "", "post_date": "2019-06-09T15:35:22.897000", "content": "QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.
\n\nMoreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? Or you just blindly believe that everyone will play by the rules? With so many participants in the competition, everything will be tried.
", "votes": 2, "replies": [] }, { "id": 548582, "author_name": "fafafa fafafafa", "author_url": "", "post_date": "2019-06-09T15:39:45.653000", "content": "\"QM9 contains important insight into the problem and its important characteristics. It is foolish not to study what it contains if one wants to understand the problem, without using it for development and testing of ones models.\"
\n\nI'm not sure what the distinction is between \"studying\" QM9 vs. using QM9 for development and testing of one's models.
\n\n\"Moreover, I raise concerns that someone else could potentially use it to steal medals from fair participants. I think this is a valid concern that requires an answer from the organizers, do you agree? \"
\n\nI share your concern. I'm not sure there's anything the organizers can do at this point. It's a flaw in the contest design, IMO. One of the main reasons I'm thinking of dropping out.
", "votes": 7, "replies": [] } ] }, { "id": 549445, "author_name": "cyberia", "author_url": "", "post_date": "2019-06-10T16:46:47.077000", "content": "I have no clue why people down voted this topic. I got plenty of downvotes today. People should be more tolerant towards others point of views. Our approaches should not be limited to the competition but should address to solve bigger challenges. Keep it up man 👍
\n\nThere's a difference between expressing a point of view, and knowingly violating the Rules that every contestant agreed to when entering this competition, don't you think?
", "votes": 6, "replies": [] }, { "id": 549538, "author_name": "Scirpus", "author_url": "", "post_date": "2019-06-10T19:16:55.173000", "content": "@inversion - Can you add a pinned entry to state this dataset is not allowed please?
", "votes": 2, "replies": [] }, { "id": 549549, "author_name": "inversion", "author_url": "", "post_date": "2019-06-10T19:44:22.717000", "content": "@scirpus Great suggestion. Done.
", "votes": 2, "replies": [] }, { "id": 549607, "author_name": "nosound", "author_url": "", "post_date": "2019-06-10T21:16:05.850000", "content": "Thanks, @inversion , for the clarification. In your reaction I clearly see that you don't want this dataset to be shared and studied in the public domain, so I deleted it and made the two kernels of mine that use it private. I want to apologize and say that I did not intend to use it in testing or development of my models, nor I tried to encourage others to do so. Instead, I tried to understand how much extra information it contains, learn about the problem and ask the questions that worried me. Sincerely, from the competition rules it was not clear to me that this last part was not allowed. I see that you promised some follow-up, looking forward to it.
", "votes": 4, "replies": [] }, { "id": 550087, "author_name": "cyberia", "author_url": "", "post_date": "2019-06-11T09:36:00.103000", "content": "Yeah. I would agree with you, it would be fair for everyone if we all use the same data. Anyone following this thread follow inversion's advice.
", "votes": 1, "replies": [] } ] }, { "id": 550461, "author_name": "Felipe Bivort Haiek", "author_url": "", "post_date": "2019-06-11T16:39:20.730000", "content": "Thank you for your effortas, and sharing this knowledge. It is unfair that if a way of geting further information exists and has been made available, it should be only found by the people who entered early on, because they decide to take it down. It does not look like a fair competition anymore.
", "votes": 1, "replies": [] }, { "id": 550629, "author_name": "Ricardo Hortelano", "author_url": "", "post_date": "2019-06-11T21:00:22.123000", "content": "It makes no sense to forbid the study of this dataset. The people will study secretly and use it making it difficult to everyone to use all the knowledge.
\n\nForbid the dataset its definitely the worse thing the organizers can do.
\n\nIMHO the only possible way to proceed is to share all the knowledge about this dataset in order to avoid unfairness.
", "votes": 0, "replies": [] }, { "id": 2086049, "author_name": "Dr. Asif Khan", "author_url": "", "post_date": "2023-01-04T14:28:01.797000", "content": "Good resource.
", "votes": 0, "replies": [] }, { "id": 568638, "author_name": "black_panther", "author_url": "", "post_date": "2019-07-05T08:53:18.943000", "content": "So i am kinda confused are we allowed to use to use this dataset and the original dataset from the compitition in combination..?
", "votes": 0, "replies": [ { "id": 568774, "author_name": "nosound", "author_url": "", "post_date": "2019-07-05T12:35:14.143000", "content": "Hi, @harshitkhandelwal , yes it is allowed. The rules were updated to allow unrestricted use of the dataset. The discussion here is outdated.
", "votes": 1, "replies": [] } ] }, { "id": 549504, "author_name": "", "author_url": "", "post_date": "2019-06-10T18:18:04.257000", "content": "", "votes": 1, "replies": [] }, { "id": 548578, "author_name": "", "author_url": "", "post_date": "2019-06-09T15:35:31.923000", "content": "Hello! I understand your frustration. However posting a public kernel that uses QM9 is both against the rules, and in my humble opinion has exactly the opposite effect of what you would like to accomplish. Perhaps it is not to late to take it down. Let's just hope that everyone will play by the rules, and just concentrate on our own models .
", "votes": 4, "replies": [ { "id": 548581, "author_name": "nosound", "author_url": "", "post_date": "2019-06-09T15:38:48.950000", "content": "I want to accomplish an answer from the organizers how using QM9 will be prevented in the medals range.
", "votes": 4, "replies": [] }, { "id": 548587, "author_name": "", "author_url": "", "post_date": "2019-06-09T15:43:56.577000", "content": "I certainly understand that. What I don't understand is why at the same time you are also trying help others to potentially bend the rules. Please consider taking down your kernels that are explicitly prohibited by the organizers. Thanks
", "votes": 2, "replies": [] }, { "id": 548595, "author_name": "nosound", "author_url": "", "post_date": "2019-06-09T15:53:00.243000", "content": "I don't agree with you position, let's agree to disagree. The dataset is right there online, whoever wants to use to cheat will use it. Hiding from the problem will not solve it.
", "votes": 10, "replies": [] }, { "id": 548607, "author_name": "", "author_url": "", "post_date": "2019-06-09T16:09:28.753000", "content": "We are only responsible for our own actions. Right now you personally are clearly violating the rules and at the same time you are doing everything in your power to potentially help others to do the same (posting the public kernel, making the QM9 dataset available on Kaggle). It really puzzles me, but perhaps we said everything about this that can said. Best of luck with the competition, and please at least think about stopping what you are doing. Zoltan
", "votes": 2, "replies": [] }, { "id": 548611, "author_name": "nosound", "author_url": "", "post_date": "2019-06-09T16:17:10.227000", "content": "Well, I am not going to participate in the competition if my current understanding of this problem is correct, and I hope this post facilitates the discussion on this early competition stage. If the organizers think that the dataset and the kernel are inappropriate they will remove it, up to them. If this is indeed the case, a statement from the organizers that study QM9 with Kaggle datasets and kernels is not allowed and will be removed can help others in the future.
", "votes": 3, "replies": [] }, { "id": 549543, "author_name": "inversion", "author_url": "", "post_date": "2019-06-10T19:24:49.777000", "content": "@nosound - What you just did is in extremely poor form.
\n\n1) You knowingly violated the competition Rules.\n2) You did so with the poor justification that you didn't get an answer as fast as you thought you should.
\n\nYou made the initial request to the competition sponsor 4 days ago. I was out of the office for the last 3.5 days, and thus had no time to coordinate a plan with the sponsor. So you thought you would take it upon yourself to break the rules you agreed to in order to \"facilitate discussion\"?
\n\nIf you ever (mistakenly) believe you need \"break a competition to save the competition\" again, you should at least have the courtesy to give more than 2 business days for a response.
", "votes": 7, "replies": [] }, { "id": 549622, "author_name": "nosound", "author_url": "", "post_date": "2019-06-10T21:49:10.163000", "content": "I did not understand from the rules that it was not allowed to study this dataset. Now it clear, - you don't want this dataset and kernels to be shared. I removed it.
", "votes": 4, "replies": [] }, { "id": 550631, "author_name": "Ricardo Hortelano", "author_url": "", "post_date": "2019-06-11T21:03:08.530000", "content": "\n\n\nLet's just hope that everyone will play by the rules, and just concentrate on our own models .
\n
Definitely the people will NOT play by the rules. We have seen it in multiples competitions.
", "votes": 1, "replies": [] }, { "id": 551342, "author_name": "Josef K", "author_url": "", "post_date": "2019-06-12T15:50:30.253000", "content": "I am actually glad that he posted 3 months in advance, not 3 days. If he (or anyone) secretly uses this information, you will never know about it. You can of course remove QM9 from the kernel, but you cannot remove it from the internet, and people will know it and use it. You cannot just cover your eyes and pretend people will follow all the rules.
", "votes": 4, "replies": [] } ] }, { "id": 1064442, "author_name": "Manajit", "author_url": "", "post_date": "2020-10-30T05:58:38.353000", "content": "Thanks for sharing
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "548541": "I am sure many people have already been using QM9 to get deeper insight. You can now [access the dataset here on Kaggle](https://www.kaggle.com/zaharch/quantum-machine-9-aka-qm9), and I [added a kernel](https://www.kaggle.com/zaharch/quantum-machine-9-qm9) which discusses what data we have there, extracts features and runs a basic LightGBM on them.\n\nIn summary, QM9 contains full structural information, mulliken partial charges, frequencies for degrees of freedom and many characteristics on the level of molecule. What is vital is that it contains this data not only for the train but also for the test molecules. Please take a look at the kernel and the dataset for more details.\n\nThe kernel generates a dataframe which contains the features extracted from QM9 for train + test data. You can download the output file and add these columns to your features to check their impact. I tried to do exactly that in [this different kernel](https://www.kaggle.com/zaharch/lightgbm-full-pipeline-model-qm9), as base for my experiment I selected [a public kernel](https://www.kaggle.com/kabure/lightgbm-full-pipeline-model) which runs fast (it has only one model trained on 90% of the train data, with 10% for early stopping). Simply adding the QM9 features and rerunning improved the LB score from -0.462 to -0.585. \n\nPlease note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions. My motivation for exploring what can be done with it lies in my fears that it can actually be abused and make this competition really problematic to participate in. These are my points:\n\n1. The features from this forbidden dataset improve public LB scores, and therefore you will not see \"real\" scores at the top, but rather scores obtained with QM9. Moreover it will never be clear if someone used the dataset or not on the public LB. \n\n2. More importantly, models which use QM9 will score higher on the private LB as well. It is not allowed to use it directly, but one can choose models and random seeds which generate predictions closer to it. Not sure how important it is, but it looks like it can be important, therefore not making it fun to anyone. Am I being paranoiac here? I shared my concerns [with the organizers here](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93858#546053), no answer yet. \n\n3. AFAIK, Kaggle usually does not check models for people outside of the gold medals range (too much work), or even only for teams in the money, therefore wreaking havoc for the fair silver/bronze medals competitors. \n\nUPDATE: This post has received multiple downvotes, apparently for sharing QM9. I am honestly surprised, haven't seen such attitude to knowledge and asking questions from the times of Giordano Bruno. This dataset contains a wealth of knowledge on the problem; one can learn what are the relevant characteristics, interactions, dependencies, what latent variables explain the target. Either every serious participant downloads it silently and study, or it becomes a public discussion. Even just the knowledge how much it can improve the LB score is important for the public, given the setting. If, for example, we know that the dataset can improve the score only slightly, I am sure everyone will feel much more at ease.\n\nUPDATE 2: I removed the dataset and the kernels, apparently it is not allowed to study it in the public domain. \n\nUPDATE 3: Following the rules update I uploaded the dataset back and re-shared the kernels, hope it saves you time.", "549471": "This is a very important post. If @nosound didn't share it, then someone else would have. If no one had shared it then I'm sure that certain individuals would use it to leverage their leaderboard position.\n\nWhat we need is a comment from Kaggle on how they might enforce the rule around QM9 for positions outside of the money so that medals are not awarded unfairly.", "550628": "Let's not shoot the messenger. The best news is that QM9 issue came up 3 months instead of 3 days before the deadline.", "548565": "Is this not a violation of the contest rules? You write, \"Please note that QM9 dataset is not allowed by the competition rules to be used to generate one's final submissions.\" This sentence makes it sound as if the ban is just on using QM9 for your *final* submission, but it looks like the rules have a broader ban on the use of QM9 for any model development or testing: \n\n\"You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set, including the QM7 and QM9 data sets.\"", "549445": "I have no clue why people down voted this topic. I got plenty of downvotes today. People should be more tolerant towards others point of views. Our approaches should not be limited to the competition but should address to solve bigger challenges. Keep it up man 👍 \n\n", "550461": "Thank you for your effortas, and sharing this knowledge. It is unfair that if a way of geting further information exists and has been made available, it should be only found by the people who entered early on, because they decide to take it down. It does not look like a fair competition anymore.", "550629": "It makes no sense to forbid the study of this dataset. The people will study secretly and use it making it difficult to everyone to use all the knowledge.\n\nForbid the dataset its definitely the worse thing the organizers can do.\n\nIMHO the only possible way to proceed is to share all the knowledge about this dataset in order to avoid unfairness. ", "2086049": "Good resource. ", "568638": "So i am kinda confused are we allowed to use to use this dataset and the original dataset from the compitition in combination..?", "549504": "", "548578": "Hello! I understand your frustration. However posting a public kernel that uses QM9 is both against the rules, and in my humble opinion has exactly the opposite effect of what you would like to accomplish. Perhaps it is not to late to take it down. Let's just hope that everyone will play by the rules, and just concentrate on our own models .", "1064442": "Thanks for sharing" } }