{ "id": 94457, "title": "What constitutes a forbidden \"quantum mechanics computation\"?", "url": "/competitions/champs-scalar-coupling/discussion/94457", "author_name": "Georgy Frolov", "post_date": "2019-06-04T16:10:33.852000", "votes": 24, "comment_count": 16, "views": 0, "content": "

The rules say:

\n\n
\n

You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set

\n
\n\n

This obviously forbids submitting results of a direct DFT computation, but does it mean that we cannot use any physically informed heuristics? For example, one could imagine a model that constructs molecular orbitals by something like the Hückel method and uses them to generate additional features, or one that quickly calculates properties of some simple structures and combines them etc. Would such solutions be admissible?

", "messages": [ { "id": 543598, "postDate": "2019-06-04T16:10:33.853Z", "content": "

The rules say:

\n\n
\n

You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set

\n
\n\n

This obviously forbids submitting results of a direct DFT computation, but does it mean that we cannot use any physically informed heuristics? For example, one could imagine a model that constructs molecular orbitals by something like the Hückel method and uses them to generate additional features, or one that quickly calculates properties of some simple structures and combines them etc. Would such solutions be admissible?

", "rawMarkdown": "The rules say:\n> You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set\n\nThis obviously forbids submitting results of a direct DFT computation, but does it mean that we cannot use any physically informed heuristics? For example, one could imagine a model that constructs molecular orbitals by something like [the Hückel method](https://en.wikipedia.org/wiki/H%C3%BCckel_method) and uses them to generate additional features, or one that quickly calculates properties of some simple structures and combines them etc. Would such solutions be admissible?", "votes": 24 }, { "id": 543635, "postDate": "2019-06-04T16:44:13.103Z", "content": "

The Hückel method would not be allowed in this competition. It's very vague where classical stops and quantum begins, so I'm glad that you asked.\nThings like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine, as long as it doesn't prohibit academic or commercial use. Running semi-empirical or ab initio quantum mechanics computations is not, including implementing these yourself in your code.

", "rawMarkdown": "The Hückel method would not be allowed in this competition. It's very vague where classical stops and quantum begins, so I'm glad that you asked.\nThings like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine, as long as it doesn't prohibit academic or commercial use. Running semi-empirical or ab initio quantum mechanics computations is not, including implementing these yourself in your code. ", "votes": 10, "replies": [ { "id": 543659, "postDate": "2019-06-04T17:16:47.840Z", "content": "

Thanks for the reply. So any electronic structure data should be prohibited -- or else one could concoct a regression model that uses atomic orbitals as external data and essentially replicates the Hückel method.\nWhat about the Karplus equation (discussed in another thread)?\nI understand it's a thin line, that's why it may be good to just mention such specific examples.

", "rawMarkdown": "Thanks for the reply. So any electronic structure data should be prohibited -- or else one could concoct a regression model that uses atomic orbitals as external data and essentially replicates the Hückel method.\nWhat about the Karplus equation (discussed in another thread)?\nI understand it's a thin line, that's why it may be good to just mention such specific examples.", "votes": 1 }, { "id": 543782, "postDate": "2019-06-04T20:18:55.990Z", "content": "

Great point about using any electronic structure data... I guess once you're at the level of modelling electronic positions then you might as well go full ab initio rather than use ML regression.

\n\n

Hopefully the Karplus equation is allowed because it's a model based on dihedral angles rather than encoding where the electrons are.

", "rawMarkdown": "Great point about using any electronic structure data... I guess once you're at the level of modelling electronic positions then you might as well go full ab initio rather than use ML regression.\n\nHopefully the Karplus equation is allowed because it's a model based on dihedral angles rather than encoding where the electrons are.", "votes": 1 }, { "id": 543797, "postDate": "2019-06-04T20:47:36.753Z", "content": "

The Karplus equations would be fine as they are some empirical fit to the observed data and would likely have to be refit to the training data anyways to match the observations.

", "rawMarkdown": "The Karplus equations would be fine as they are some empirical fit to the observed data and would likely have to be refit to the training data anyways to match the observations.", "votes": 3 }, { "id": 544701, "postDate": "2019-06-05T19:34:15.667Z", "content": "

@larsbratholm

\n\n

I am very concerned about the fact that these methods exist. When you have thousands of resourceful Kagglers working on your data you will find they get creative. People don't have to use QM calculations in their submissions or kernels; it is sufficient to compute the answers elsewhere and simply select the model that predicted them best. What is to stop users from making legitimate models on Kaggle, testing them offline against the 'real' answers and then using those results to choose their final submission?

", "rawMarkdown": "@larsbratholm \n\nI am very concerned about the fact that these methods exist. When you have thousands of resourceful Kagglers working on your data you will find they get creative. People don't have to use QM calculations in their submissions or kernels; it is sufficient to compute the answers elsewhere and simply select the model that predicted them best. What is to stop users from making legitimate models on Kaggle, testing them offline against the 'real' answers and then using those results to choose their final submission?", "votes": 6 }, { "id": 544719, "postDate": "2019-06-05T20:05:05.130Z", "content": "

1) it would take a significant amount of time (maybe more than 3 months) to compute accurate J-couplings for all the molecules.\n2) the competition models are being compared against the test data set, which are not \"accurate\"... So any models trained using \"illegal\" calculations wouldn't do so well on the test data anyway.

", "rawMarkdown": "1) it would take a significant amount of time (maybe more than 3 months) to compute accurate J-couplings for all the molecules.\n2) the competition models are being compared against the test data set, which are not \"accurate\"... So any models trained using \"illegal\" calculations wouldn't do so well on the test data anyway.", "votes": 6 }, { "id": 544756, "postDate": "2019-06-05T21:45:09.233Z", "content": "

@phucco

\n\n

Thank you, that puts my mind at ease. I've been reading some conflicting answers about the time it would take to brute-force the test predictions this way, but if there's no guarantee they'd be the right answers anyway then it's not a problem. It makes me a little confused about the test data though - how have the researchers obtained their values for the J-coupling strength? Are we dealing with empirical results that differ from established theory, hence why methods like DFT wouldn't be reliable?

", "rawMarkdown": "@phucco \n\nThank you, that puts my mind at ease. I've been reading some conflicting answers about the time it would take to brute-force the test predictions this way, but if there's no guarantee they'd be the right answers anyway then it's not a problem. It makes me a little confused about the test data though - how have the researchers obtained their values for the J-coupling strength? Are we dealing with empirical results that differ from established theory, hence why methods like DFT wouldn't be reliable?", "votes": 4 }, { "id": 570457, "postDate": "2019-07-08T10:14:13.893Z", "content": "

As a non chemist, this discussion is fascinating. I see hints to things that could lead to good data addition:

\n\n
\n

Things like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine,

\n \n

The Karplus equations would be fine

\n
\n\n

Some more reading to come for sure!

", "rawMarkdown": "As a non chemist, this discussion is fascinating. I see hints to things that could lead to good data addition:\n\n> Things like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine,\n\n> The Karplus equations would be fine \n\nSome more reading to come for sure!", "votes": 2 } ] }, { "id": 546076, "postDate": "2019-06-06T08:24:53.037Z", "content": "

Closely related:\nIn the mol2, which can pre produced by OpenBabel (see this thread), we have not only bonding data, but also some atom configurations and charges. Are these data allowed? I am not sure and find it hard to google how OpenBabel actually determines them.

", "rawMarkdown": "Closely related:\nIn the `mol2`, which can pre produced by `OpenBabel` (see [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93928#latest-544754)), we have not only bonding data, but also some atom configurations and charges. Are these data allowed? I am not sure and find it hard to google how OpenBabel actually determines them.", "votes": 3, "replies": [ { "id": 552720, "postDate": "2019-06-14T12:07:03.623Z", "content": "

This would be fine.

", "rawMarkdown": "This would be fine.", "votes": 2 }, { "id": 565703, "postDate": "2019-07-01T09:41:35.727Z", "content": "

@larsbratholm\nIs also computing Mulliken charges with OpenBabel ok?

", "rawMarkdown": "@larsbratholm\nIs also computing Mulliken charges with OpenBabel ok?" }, { "id": 568184, "postDate": "2019-07-04T14:27:57.337Z", "content": "

Mulliken charges with OpenBabel are fine, since it's entirely empirical.

", "rawMarkdown": "Mulliken charges with OpenBabel are fine, since it's entirely empirical." } ] }, { "id": 554381, "postDate": "2019-06-17T10:50:23.653Z", "content": "

@larsbratholm Would using Psi4 for feature generation (e.g. deriving rotational constants etc.) be against competition rules in a similar manner? http://www.psicode.org/

", "rawMarkdown": "@larsbratholm Would using `Psi4` for feature generation (e.g. deriving rotational constants etc.) be against competition rules in a similar manner? http://www.psicode.org/", "votes": 1, "replies": [ { "id": 559066, "postDate": "2019-06-23T12:43:24.083Z", "content": "

I would assume that most information you could get from Psi4 would be QM and hence not allowed. Rotational constants are already part of the QM9 dataset, which you can find discussions about in other threads.

", "rawMarkdown": "I would assume that most information you could get from `Psi4` would be QM and hence not allowed. Rotational constants are already part of the QM9 dataset, which you can find discussions about in other threads.", "votes": 1 } ] }, { "id": 552191, "postDate": "2019-06-13T15:54:16.530Z", "content": "

Please update the competition description to be clear for everyone.

", "rawMarkdown": "Please update the competition description to be clear for everyone.", "votes": 1 }, { "id": 551993, "postDate": "2019-06-13T11:49:58.520Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 543635, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-06-04T16:44:13.103000", "content": "

The Hückel method would not be allowed in this competition. It's very vague where classical stops and quantum begins, so I'm glad that you asked.\nThings like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine, as long as it doesn't prohibit academic or commercial use. Running semi-empirical or ab initio quantum mechanics computations is not, including implementing these yourself in your code.

", "votes": 10, "replies": [ { "id": 543659, "author_name": "Georgy Frolov", "author_url": "", "post_date": "2019-06-04T17:16:47.840000", "content": "

Thanks for the reply. So any electronic structure data should be prohibited -- or else one could concoct a regression model that uses atomic orbitals as external data and essentially replicates the Hückel method.\nWhat about the Karplus equation (discussed in another thread)?\nI understand it's a thin line, that's why it may be good to just mention such specific examples.

", "votes": 1, "replies": [] }, { "id": 543782, "author_name": "phucco", "author_url": "", "post_date": "2019-06-04T20:18:55.990000", "content": "

Great point about using any electronic structure data... I guess once you're at the level of modelling electronic positions then you might as well go full ab initio rather than use ML regression.

\n\n

Hopefully the Karplus equation is allowed because it's a model based on dihedral angles rather than encoding where the electrons are.

", "votes": 1, "replies": [] }, { "id": 543797, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-06-04T20:47:36.753000", "content": "

The Karplus equations would be fine as they are some empirical fit to the observed data and would likely have to be refit to the training data anyways to match the observations.

", "votes": 3, "replies": [] }, { "id": 544701, "author_name": "RNA", "author_url": "", "post_date": "2019-06-05T19:34:15.667000", "content": "

@larsbratholm

\n\n

I am very concerned about the fact that these methods exist. When you have thousands of resourceful Kagglers working on your data you will find they get creative. People don't have to use QM calculations in their submissions or kernels; it is sufficient to compute the answers elsewhere and simply select the model that predicted them best. What is to stop users from making legitimate models on Kaggle, testing them offline against the 'real' answers and then using those results to choose their final submission?

", "votes": 6, "replies": [] }, { "id": 544719, "author_name": "phucco", "author_url": "", "post_date": "2019-06-05T20:05:05.130000", "content": "

1) it would take a significant amount of time (maybe more than 3 months) to compute accurate J-couplings for all the molecules.\n2) the competition models are being compared against the test data set, which are not \"accurate\"... So any models trained using \"illegal\" calculations wouldn't do so well on the test data anyway.

", "votes": 6, "replies": [] }, { "id": 544756, "author_name": "RNA", "author_url": "", "post_date": "2019-06-05T21:45:09.233000", "content": "

@phucco

\n\n

Thank you, that puts my mind at ease. I've been reading some conflicting answers about the time it would take to brute-force the test predictions this way, but if there's no guarantee they'd be the right answers anyway then it's not a problem. It makes me a little confused about the test data though - how have the researchers obtained their values for the J-coupling strength? Are we dealing with empirical results that differ from established theory, hence why methods like DFT wouldn't be reliable?

", "votes": 4, "replies": [] }, { "id": 570457, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-08T10:14:13.893000", "content": "

As a non chemist, this discussion is fascinating. I see hints to things that could lead to good data addition:

\n\n
\n

Things like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine,

\n \n

The Karplus equations would be fine

\n
\n\n

Some more reading to come for sure!

", "votes": 2, "replies": [] } ] }, { "id": 546076, "author_name": "Berni", "author_url": "", "post_date": "2019-06-06T08:24:53.037000", "content": "

Closely related:\nIn the mol2, which can pre produced by OpenBabel (see this thread), we have not only bonding data, but also some atom configurations and charges. Are these data allowed? I am not sure and find it hard to google how OpenBabel actually determines them.

", "votes": 3, "replies": [ { "id": 552720, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-06-14T12:07:03.623000", "content": "

This would be fine.

", "votes": 2, "replies": [] }, { "id": 565703, "author_name": "somnambWl", "author_url": "", "post_date": "2019-07-01T09:41:35.727000", "content": "

@larsbratholm\nIs also computing Mulliken charges with OpenBabel ok?

", "votes": 0, "replies": [] }, { "id": 568184, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-07-04T14:27:57.337000", "content": "

Mulliken charges with OpenBabel are fine, since it's entirely empirical.

", "votes": 0, "replies": [] } ] }, { "id": 554381, "author_name": "datasaurus", "author_url": "", "post_date": "2019-06-17T10:50:23.653000", "content": "

@larsbratholm Would using Psi4 for feature generation (e.g. deriving rotational constants etc.) be against competition rules in a similar manner? http://www.psicode.org/

", "votes": 1, "replies": [ { "id": 559066, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-06-23T12:43:24.083000", "content": "

I would assume that most information you could get from Psi4 would be QM and hence not allowed. Rotational constants are already part of the QM9 dataset, which you can find discussions about in other threads.

", "votes": 1, "replies": [] } ] }, { "id": 552191, "author_name": "abiricz", "author_url": "", "post_date": "2019-06-13T15:54:16.530000", "content": "

Please update the competition description to be clear for everyone.

", "votes": 1, "replies": [] }, { "id": 551993, "author_name": "", "author_url": "", "post_date": "2019-06-13T11:49:58.520000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "543598": "The rules say:\n> You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions excluding data from quantum mechanics computations made specifically for the competition, and excluding data from quantum mechanics computations on the molecules from the training and test set\n\nThis obviously forbids submitting results of a direct DFT computation, but does it mean that we cannot use any physically informed heuristics? For example, one could imagine a model that constructs molecular orbitals by something like [the Hückel method](https://en.wikipedia.org/wiki/H%C3%BCckel_method) and uses them to generate additional features, or one that quickly calculates properties of some simple structures and combines them etc. Would such solutions be admissible?", "543635": "The Hückel method would not be allowed in this competition. It's very vague where classical stops and quantum begins, so I'm glad that you asked.\nThings like getting bond orders, aromaticity, electronegativity, van der waal radii or the like from some simple heuristics should be fine, as long as it doesn't prohibit academic or commercial use. Running semi-empirical or ab initio quantum mechanics computations is not, including implementing these yourself in your code. ", "546076": "Closely related:\nIn the `mol2`, which can pre produced by `OpenBabel` (see [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93928#latest-544754)), we have not only bonding data, but also some atom configurations and charges. Are these data allowed? I am not sure and find it hard to google how OpenBabel actually determines them.", "554381": "@larsbratholm Would using `Psi4` for feature generation (e.g. deriving rotational constants etc.) be against competition rules in a similar manner? http://www.psicode.org/", "552191": "Please update the competition description to be clear for everyone.", "551993": "" } }