dsgdb9nsd_096658 also is calculated with a total charge of 1.66 and does not contain a ring. To me it seems to have a pretty straightforward binding structure of single, double, and triple bonds:\n![\"dsgdb9nsd_096658\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/549478/13464/Screenshot%202019-06-10%20at%2019.40.53.png\")
I've calculated molecule bonds with OpenBabel as @matthewmasters has shown in this thread using the mol2 format. This format also spits out atom charges (last column for the ATOMS part). If I sum those up over a molecule, I typically get a whole number as it should be (mostly 0, some molecules are charged). However, for 97 molecules (e.g. dsgdb9nsd_108806 and dsgdb9nsd_007889) I get \\(\\frac{3}{2}\\) and for six molecules (e.g. dsgdb9nsd_024846 and dsgdb9nsd_049333) I even get \\(\\frac{5}{3}\\).
That doesn't make any sense to me as elementary particles always have whole elementary charges -- guess what, that's why they are called like that. Can someone explain what's going on here? Is it a bug of OpenBabel? Are those molecules unphysical (it seems some are at least very special and only are stable at 0 K as discussed here) and OpenBabel can't handle them? Am I interpreting those numbers wrongly?
\n\nAnd besides: I am still not sure whether these charges are allowed to use in the challenge or are coming from forbidden QM calculations.
\n\nEdit:\nThanks to @soerendip comment in this thread, I've learned that OpenBabel outputs Gasteiger(-Marsili) charges. These seem to distribute electron charges according to electron negativities and, hence, are empirical, implying that they should be okay to use. At the same time it makes it even weirder that the total charges are fractions for those 100 molecules...
", "messages": [ { "id": "546546", "postDate": "06/06/2019 17:15:09", "content": "I've calculated molecule bonds with OpenBabel as @matthewmasters has shown in this thread using the mol2 format. This format also spits out atom charges (last column for the ATOMS part). If I sum those up over a molecule, I typically get a whole number as it should be (mostly 0, some molecules are charged). However, for 97 molecules (e.g. dsgdb9nsd_108806 and dsgdb9nsd_007889) I get \\(\\frac{3}{2}\\) and for six molecules (e.g. dsgdb9nsd_024846 and dsgdb9nsd_049333) I even get \\(\\frac{5}{3}\\).
That doesn't make any sense to me as elementary particles always have whole elementary charges -- guess what, that's why they are called like that. Can someone explain what's going on here? Is it a bug of OpenBabel? Are those molecules unphysical (it seems some are at least very special and only are stable at 0 K as discussed here) and OpenBabel can't handle them? Am I interpreting those numbers wrongly?
\n\nAnd besides: I am still not sure whether these charges are allowed to use in the challenge or are coming from forbidden QM calculations.
\n\nEdit:\nThanks to @soerendip comment in this thread, I've learned that OpenBabel outputs Gasteiger(-Marsili) charges. These seem to distribute electron charges according to electron negativities and, hence, are empirical, implying that they should be okay to use. At the same time it makes it even weirder that the total charges are fractions for those 100 molecules...
", "rawMarkdown": "I've calculated molecule bonds with [OpenBabel](http://openbabel.org/wiki/Main_Page) as @matthewmasters has shown in [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93928#latest-546247) using the `mol2` format. This format also spits out atom charges (last column for the ATOMS part). If I sum those up over a molecule, I typically get a whole number as it should be (mostly 0, some molecules are charged). However, for 97 molecules (e.g. `dsgdb9nsd_108806` and `dsgdb9nsd_007889 `) I get \\\\(\\frac{3}{2}\\\\) and for six molecules (e.g. `dsgdb9nsd_024846 ` and `dsgdb9nsd_049333 `) I even get \\\\(\\frac{5}{3}\\\\).\n\nThat doesn't make any sense to me as elementary particles always have whole elementary charges -- guess what, that's why they are called like that. Can someone explain what's going on here? Is it a bug of OpenBabel? Are those molecules unphysical (it seems some are at least very special and only are stable at 0 K as discussed [here](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94117#latest-542356)) and OpenBabel can't handle them? Am I interpreting those numbers wrongly?\n\nAnd besides: I am still not sure whether these charges are allowed to use in the challenge or are coming from forbidden QM calculations.\n\n**Edit:**\nThanks to @soerendip comment in [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93762#546583), I've learned that OpenBabel outputs Gasteiger(-Marsili) charges. These seem to distribute electron charges according to electron negativities and, hence, are empirical, implying that they should be okay to use. At the same time it makes it even weirder that the total charges are fractions for those 100 molecules...", "votes": null }, { "id": "546968", "postDate": "06/07/2019 04:26:48", "content": "For some of the structures the final geometries are potentially wrong. A good example is dsgdb9nsd_024846.xyz This 7-member ring structure has a number of C-N-H bonds, which makes them radicals or at least charged. Now there are stable radicals which are resonance stabilized, so we don't always know if such a molecule could exist. \nSee Factors That Stabilize Free Radicals
\n\nThe source of the CHAMPS dataset is GDB with billions of theoretical isomers and then drilled down to couple 100k\ncompounds that were subjected to geometry analysis with DFT and later NMR predictions. So naturally there are a number of compounds that fall through the filtering grid, we don't know if they potentially exist or could be synthesized.
\n\nFurthermore the bond regeneration of OpenBabel is not always working correctly. Its working for 99% of the molecules but not the rest. Meaning there are always complex cases, where the connectivity regeneration fails. Reason are of bond length, atomic charges, valencies and aromatization issues (in this case with heteroatoms). So in OpenBabel for xyz to mol (not mol2) if the last line contains \"M RAD\" or radicals then this could be indeed radicals or there are issues with the structure.
\n\nFor the purpose of this contest with over 100k training compounds it is OK to throw these few outliers out.
", "rawMarkdown": "For some of the structures the final geometries are potentially wrong. A good example is dsgdb9nsd_024846.xyz This 7-member ring structure has a number of C-N-H bonds, which makes them radicals or at least charged. Now there are stable radicals which are resonance stabilized, so we don't always know if such a molecule could exist. \nSee [Factors That Stabilize Free Radicals](https://www.masterorganicchemistry.com/2013/08/02/3-factors-that-stabilize-free-radicals/)\n\nThe source of the CHAMPS dataset is GDB with billions of theoretical isomers and then drilled down to couple 100k\ncompounds that were subjected to geometry analysis with DFT and later NMR predictions. So naturally there are a number of compounds that fall through the filtering grid, we don't know if they potentially exist or could be synthesized. \n\nFurthermore the bond regeneration of OpenBabel is not always working correctly. Its working for 99% of the molecules but not the rest. Meaning there are always complex cases, where the connectivity regeneration fails. Reason are of bond length, atomic charges, valencies and aromatization issues (in this case with heteroatoms). So in OpenBabel for xyz to mol (not mol2) if the last line contains \"M RAD\" or radicals then this could be indeed radicals or there are issues with the structure. \n\nFor the purpose of this contest with over 100k training compounds it is OK to throw these few outliers out.", "votes": null }, { "id": "547123", "postDate": "06/07/2019 09:57:02", "content": "Thank you for the indepth explanation! 👍
\n\nSo basically it is a (kind of) bug of OpenBabel where its heuristics fail as you say
\n\n\n\n\nFurthermore the bond regeneration of OpenBabel is not always working correctly.
\n
So: be careful with the OpenBabel output.
\n\nHowever, I don't see any general issue with radicals (and also charged molecules of which there are quite a few). Sure, they are typicall not long lived, but still have a scalar coupling constant. They might be difficult examples and outliers in some sense, so it might be useful to exclude them in training. However, there are not wrong per se, whereas broken rational charges of a molecule (radical or not) are just nonsense - and are one indicator of OpenBabel failing to construct the correct bonds of the molecule with its heuristics. And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?
", "rawMarkdown": "Thank you for the indepth explanation! 👍 \n\nSo basically it is a (kind of) bug of OpenBabel where its heuristics fail as you say\n> Furthermore the bond regeneration of OpenBabel is not always working correctly.\n\nSo: be careful with the OpenBabel output.\n\nHowever, I don't see any general issue with radicals (and also charged molecules of which there are quite a few). Sure, they are typicall not long lived, but still have a scalar coupling constant. They might be difficult examples and outliers in some sense, so it might be useful to exclude them in training. However, there are not wrong per se, whereas broken rational charges of a molecule (radical or not) are just nonsense - and are one indicator of OpenBabel failing to construct the correct bonds of the molecule with its heuristics. And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?", "votes": null }, { "id": "547174", "postDate": "06/07/2019 11:16:50", "content": "In all the examples you provide, they contain aromatic rings. Though I don't have an exact reason this happens, I suspect this has something to do with it.
", "rawMarkdown": "In all the examples you provide, they contain aromatic rings. Though I don't have an exact reason this happens, I suspect this has something to do with it.", "votes": null }, { "id": "547386", "postDate": "06/07/2019 16:37:45", "content": "> And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?
\n\nYes, plus level of uncertainty if it is an empirical model.
", "rawMarkdown": "> And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?\n\nYes, plus level of uncertainty if it is an empirical model.", "votes": null }, { "id": "547722", "postDate": "06/08/2019 07:15:12", "content": "Well sure. Some uncertainty. For many molecules they sum up to something like 0.001 or 0.997. However, 1.500 is more than uncertainty.
", "rawMarkdown": "Well sure. *Some* uncertainty. For many molecules they sum up to something like 0.001 or 0.997. However, 1.500 is more than uncertainty.", "votes": null }, { "id": "549478", "postDate": "06/10/2019 17:43:10", "content": "Well, that might well be the most frequent reason why the heuristics of OpenBabel fail, but there are counter examples: dsgdb9nsd_096658 also is calculated with a total charge of 1.66 and does not contain a ring. To me it seems to have a pretty straightforward binding structure of single, double, and triple bonds:\n
Edit: There is one H atom hidden behind the rightmost N atom. Sorry for the bad angle of the picture.
", "rawMarkdown": "Well, that might well be the most frequent reason why the heuristics of OpenBabel fail, but there are counter examples: `dsgdb9nsd_096658` also is calculated with a total charge of `1.66` and does not contain a ring. To me it seems to have a pretty straightforward binding structure of single, double, and triple bonds:\n\n\n*Edit:* There is one H atom hidden behind the rightmost N atom. Sorry for the bad angle of the picture.", "votes": null } ], "comments": [ { "id": 546968, "author_name": "tobikaggle", "author_url": "", "post_date": "06/07/2019 04:26:48", "content": "For some of the structures the final geometries are potentially wrong. A good example is dsgdb9nsd_024846.xyz This 7-member ring structure has a number of C-N-H bonds, which makes them radicals or at least charged. Now there are stable radicals which are resonance stabilized, so we don't always know if such a molecule could exist. \nSee Factors That Stabilize Free Radicals
\n\nThe source of the CHAMPS dataset is GDB with billions of theoretical isomers and then drilled down to couple 100k\ncompounds that were subjected to geometry analysis with DFT and later NMR predictions. So naturally there are a number of compounds that fall through the filtering grid, we don't know if they potentially exist or could be synthesized.
\n\nFurthermore the bond regeneration of OpenBabel is not always working correctly. Its working for 99% of the molecules but not the rest. Meaning there are always complex cases, where the connectivity regeneration fails. Reason are of bond length, atomic charges, valencies and aromatization issues (in this case with heteroatoms). So in OpenBabel for xyz to mol (not mol2) if the last line contains \"M RAD\" or radicals then this could be indeed radicals or there are issues with the structure.
\n\nFor the purpose of this contest with over 100k training compounds it is OK to throw these few outliers out.
", "votes": null, "replies": [ { "id": 547123, "author_name": "bernir", "author_url": "", "post_date": "06/07/2019 09:57:02", "content": "Thank you for the indepth explanation! 👍
\n\nSo basically it is a (kind of) bug of OpenBabel where its heuristics fail as you say
\n\n\n\n\nFurthermore the bond regeneration of OpenBabel is not always working correctly.
\n
So: be careful with the OpenBabel output.
\n\nHowever, I don't see any general issue with radicals (and also charged molecules of which there are quite a few). Sure, they are typicall not long lived, but still have a scalar coupling constant. They might be difficult examples and outliers in some sense, so it might be useful to exclude them in training. However, there are not wrong per se, whereas broken rational charges of a molecule (radical or not) are just nonsense - and are one indicator of OpenBabel failing to construct the correct bonds of the molecule with its heuristics. And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?
", "votes": null, "replies": [] }, { "id": 547386, "author_name": "tobikaggle", "author_url": "", "post_date": "06/07/2019 16:37:45", "content": "> And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?
\n\nYes, plus level of uncertainty if it is an empirical model.
", "votes": null, "replies": [] }, { "id": 547722, "author_name": "bernir", "author_url": "", "post_date": "06/08/2019 07:15:12", "content": "Well sure. Some uncertainty. For many molecules they sum up to something like 0.001 or 0.997. However, 1.500 is more than uncertainty.
", "votes": null, "replies": [] } ] }, { "id": 547174, "author_name": "matthewmasters", "author_url": "", "post_date": "06/07/2019 11:16:50", "content": "In all the examples you provide, they contain aromatic rings. Though I don't have an exact reason this happens, I suspect this has something to do with it.
", "votes": null, "replies": [ { "id": 549478, "author_name": "bernir", "author_url": "", "post_date": "06/10/2019 17:43:10", "content": "Well, that might well be the most frequent reason why the heuristics of OpenBabel fail, but there are counter examples: dsgdb9nsd_096658 also is calculated with a total charge of 1.66 and does not contain a ring. To me it seems to have a pretty straightforward binding structure of single, double, and triple bonds:\n
Edit: There is one H atom hidden behind the rightmost N atom. Sorry for the bad angle of the picture.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "546546": "I've calculated molecule bonds with [OpenBabel](http://openbabel.org/wiki/Main_Page) as @matthewmasters has shown in [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93928#latest-546247) using the `mol2` format. This format also spits out atom charges (last column for the ATOMS part). If I sum those up over a molecule, I typically get a whole number as it should be (mostly 0, some molecules are charged). However, for 97 molecules (e.g. `dsgdb9nsd_108806` and `dsgdb9nsd_007889 `) I get \\\\(\\frac{3}{2}\\\\) and for six molecules (e.g. `dsgdb9nsd_024846 ` and `dsgdb9nsd_049333 `) I even get \\\\(\\frac{5}{3}\\\\).\n\nThat doesn't make any sense to me as elementary particles always have whole elementary charges -- guess what, that's why they are called like that. Can someone explain what's going on here? Is it a bug of OpenBabel? Are those molecules unphysical (it seems some are at least very special and only are stable at 0 K as discussed [here](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94117#latest-542356)) and OpenBabel can't handle them? Am I interpreting those numbers wrongly?\n\nAnd besides: I am still not sure whether these charges are allowed to use in the challenge or are coming from forbidden QM calculations.\n\n**Edit:**\nThanks to @soerendip comment in [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93762#546583), I've learned that OpenBabel outputs Gasteiger(-Marsili) charges. These seem to distribute electron charges according to electron negativities and, hence, are empirical, implying that they should be okay to use. At the same time it makes it even weirder that the total charges are fractions for those 100 molecules...", "546968": "For some of the structures the final geometries are potentially wrong. A good example is dsgdb9nsd_024846.xyz This 7-member ring structure has a number of C-N-H bonds, which makes them radicals or at least charged. Now there are stable radicals which are resonance stabilized, so we don't always know if such a molecule could exist. \nSee [Factors That Stabilize Free Radicals](https://www.masterorganicchemistry.com/2013/08/02/3-factors-that-stabilize-free-radicals/)\n\nThe source of the CHAMPS dataset is GDB with billions of theoretical isomers and then drilled down to couple 100k\ncompounds that were subjected to geometry analysis with DFT and later NMR predictions. So naturally there are a number of compounds that fall through the filtering grid, we don't know if they potentially exist or could be synthesized. \n\nFurthermore the bond regeneration of OpenBabel is not always working correctly. Its working for 99% of the molecules but not the rest. Meaning there are always complex cases, where the connectivity regeneration fails. Reason are of bond length, atomic charges, valencies and aromatization issues (in this case with heteroatoms). So in OpenBabel for xyz to mol (not mol2) if the last line contains \"M RAD\" or radicals then this could be indeed radicals or there are issues with the structure. \n\nFor the purpose of this contest with over 100k training compounds it is OK to throw these few outliers out.", "547123": "Thank you for the indepth explanation! 👍 \n\nSo basically it is a (kind of) bug of OpenBabel where its heuristics fail as you say\n> Furthermore the bond regeneration of OpenBabel is not always working correctly.\n\nSo: be careful with the OpenBabel output.\n\nHowever, I don't see any general issue with radicals (and also charged molecules of which there are quite a few). Sure, they are typicall not long lived, but still have a scalar coupling constant. They might be difficult examples and outliers in some sense, so it might be useful to exclude them in training. However, there are not wrong per se, whereas broken rational charges of a molecule (radical or not) are just nonsense - and are one indicator of OpenBabel failing to construct the correct bonds of the molecule with its heuristics. And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?", "547174": "In all the examples you provide, they contain aromatic rings. Though I don't have an exact reason this happens, I suspect this has something to do with it.", "547386": "> And the sum of the partical charges of the atoms should sum up to the total charge of the molecule. Correct?\n\nYes, plus level of uncertainty if it is an empirical model.", "547722": "Well sure. *Some* uncertainty. For many molecules they sum up to something like 0.001 or 0.997. However, 1.500 is more than uncertainty.", "549478": "Well, that might well be the most frequent reason why the heuristics of OpenBabel fail, but there are counter examples: `dsgdb9nsd_096658` also is calculated with a total charge of `1.66` and does not contain a ring. To me it seems to have a pretty straightforward binding structure of single, double, and triple bonds:\n\n\n*Edit:* There is one H atom hidden behind the rightmost N atom. Sorry for the bad angle of the picture." }, "source": "meta" }