![\"First](\"https://i.imgur.com/QHuIupg.png\")
So, today I thought of working around with new feature, and stuck with an issue.
\n\nTo start with, I found this resource useful.\nhttps://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Supplemental_Modules_(Physical_and_Theoretical_Chemistry)/Physical_Properties_of_Matter/Atomic_and_Molecular_Properties/Dipole_Moments
\n\nIt says water has a dipole moment due to its bent structure, and combined vectors from 2 Hs to O, is the direction of the dipole moment.
\n\nThe details might be wrong but that's the basic idea, that we can reconstruct the dipole moment from the coordinates and charges of atoms.
\n\nAccording to our dipole.csv thingy, dsgdb9nsd_000003 (a.k.a water) has dipole moment of\n\n(0, 0, -1.8511)\n\nWe also know the exact coordinates of the atoms from xyz file too.\n\nO -0.0343604951 0.9775395708 0.0076015923\nH 0.0647664923 0.0205721989 0.0015346341\nH 0.8717903737 1.3007924048 0.0006931336\n\nSo let's start the calculation. Vectors from H to O would be\n\n(-0.0991, 0.9569, 0.0060)\n(-0.9061, -0.3232, 0.0069)\n\nAnd we combine them, which should have the same direction as the dipole moment\n\n(-1.0052, 0.6337, 0.0129)\n\nWait, why the X and Y are not cancelling out?
Charges are not the problem here, because using the eq. (1) from the resource, we end up with same calculation.
\n\nAm I making mistake because I'm too sleepy?
", "messages": [ { "id": 570594, "postDate": "2019-07-08T14:08:43.870Z", "content": "So, today I thought of working around with new feature, and stuck with an issue.
\n\nTo start with, I found this resource useful.\nhttps://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Supplemental_Modules_(Physical_and_Theoretical_Chemistry)/Physical_Properties_of_Matter/Atomic_and_Molecular_Properties/Dipole_Moments
\n\nIt says water has a dipole moment due to its bent structure, and combined vectors from 2 Hs to O, is the direction of the dipole moment.
\n\nThe details might be wrong but that's the basic idea, that we can reconstruct the dipole moment from the coordinates and charges of atoms.
\n\nAccording to our dipole.csv thingy, dsgdb9nsd_000003 (a.k.a water) has dipole moment of\n\n(0, 0, -1.8511)\n\nWe also know the exact coordinates of the atoms from xyz file too.\n\nO -0.0343604951 0.9775395708 0.0076015923\nH 0.0647664923 0.0205721989 0.0015346341\nH 0.8717903737 1.3007924048 0.0006931336\n\nSo let's start the calculation. Vectors from H to O would be\n\n(-0.0991, 0.9569, 0.0060)\n(-0.9061, -0.3232, 0.0069)\n\nAnd we combine them, which should have the same direction as the dipole moment\n\n(-1.0052, 0.6337, 0.0129)\n\nWait, why the X and Y are not cancelling out?
Charges are not the problem here, because using the eq. (1) from the resource, we end up with same calculation.
\n\nAm I making mistake because I'm too sleepy?
", "rawMarkdown": "So, today I thought of working around with new feature, and stuck with an issue.\n\nTo start with, I found this resource useful.\nhttps://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Supplemental_Modules_(Physical_and_Theoretical_Chemistry)/Physical_Properties_of_Matter/Atomic_and_Molecular_Properties/Dipole_Moments\n\nIt says water has a dipole moment due to its bent structure, and combined vectors from 2 Hs to O, is the direction of the dipole moment.\n\nThe details might be wrong but that's the basic idea, that we can reconstruct the dipole moment from the coordinates and charges of atoms.\n\nAccording to our dipole.csv thingy, dsgdb9nsd_000003 (a.k.a water) has dipole moment of\n```\n(0, 0, -1.8511)\n```\nWe also know the exact coordinates of the atoms from xyz file too.\n```\nO -0.0343604951 0.9775395708 0.0076015923\nH 0.0647664923 0.0205721989 0.0015346341\nH 0.8717903737 1.3007924048 0.0006931336\n```\nSo let's start the calculation. Vectors from H to O would be\n```\n(-0.0991, 0.9569, 0.0060)\n(-0.9061, -0.3232, 0.0069)\n```\nAnd we combine them, which should have the same direction as the dipole moment\n```\n(-1.0052, 0.6337, 0.0129)\n```\nWait, why the X and Y are not cancelling out?\n\nCharges are not the problem here, because using the eq. (1) from the resource, we end up with same calculation.\n\nAm I making mistake because I'm too sleepy?", "votes": 18 }, { "id": 571493, "postDate": "2019-07-09T17:03:29.603Z", "content": "It's just because the dipole moment is not expressed in the same cartesian system as the positions (which is stupid, I agree). I have drawn the molecule and the dipole moment, it's in the attached file, and you'll see that it's not oriented in the way it's supposed to be.
", "rawMarkdown": "It's just because the dipole moment is not expressed in the same cartesian system as the positions (which is stupid, I agree). I have drawn the molecule and the dipole moment, it's in the attached file, and you'll see that it's not oriented in the way it's supposed to be.", "votes": 5, "replies": [ { "id": 571713, "postDate": "2019-07-10T02:42:18.567Z", "content": "You mean this particular dipole moment, or generally dipole moments have different coordinates?\nI'm not an expert, but I don't think it's the latter case.
\n\nAssuming it's the first case, that our handed dipole.csv has different universal cartesian coordinate, our calculation on dsgdb9nsd_000005 should have same direction as dsgdb9nsd_000003, but it seems that is not the case.
\n\nI don't mind with the dipole moment now that we may use QM9, however, I'm worried that same thing might be happening on the magnetic shielding tensor, which I have no clue how they are calculated, and some of my calculations seems to be saying that they are symmetrically wrong, using the coordinates from xyz files.
", "rawMarkdown": "You mean this particular dipole moment, or generally dipole moments have different coordinates?\nI'm not an expert, but I don't think it's the latter case.\n\nAssuming it's the first case, that our handed dipole.csv has different universal cartesian coordinate, our calculation on dsgdb9nsd_000005 should have same direction as dsgdb9nsd_000003, but it seems that is not the case.\n\nI don't mind with the dipole moment now that we may use QM9, however, I'm worried that same thing might be happening on the magnetic shielding tensor, which I have no clue how they are calculated, and some of my calculations seems to be saying that they are symmetrically wrong, using the coordinates from xyz files." }, { "id": 571720, "postDate": "2019-07-10T02:55:34.380Z", "content": "How to calculate magnetic shielding tensor? @shahrdar
", "rawMarkdown": "How to calculate magnetic shielding tensor? @shahrdar " }, { "id": 571768, "postDate": "2019-07-10T04:30:19.727Z", "content": "I found the resource from the following discussion is useful. (Please open in new tab, at least my chrome has glitch)\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/96435
\n\nI'm not quite sure, but I think you need 2 vectors to convert this tensor into a scalar. One is the direction of the molecule, and the other is the direction of the external magnetic field.
\n\nLet's think of dsgdb9nsd_000001(CH_4). At least the 4 Hs should be symmetrically equal, however, the tensors are different because the positioning of the atoms does matter.
\n\nThis operation might be wrong, but we can think of a unit vector from Hs to C , and do the multiplication with the tensor, which gives us a vector, and take the norm of it. I feel like these 4 scalars should be equal, but we get following 4 values.\n\n(29.048, 32.422, 28.935, 33.305)\n
Which brings us back to the issues with coordinates.
", "rawMarkdown": "I found the resource from the following discussion is useful. (Please open in new tab, at least my chrome has glitch)\n[https://www.kaggle.com/c/champs-scalar-coupling/discussion/96435](https://www.kaggle.com/c/champs-scalar-coupling/discussion/96435)\n\nI'm not quite sure, but I think you need 2 vectors to convert this tensor into a scalar. One is the direction of the molecule, and the other is the direction of the external magnetic field.\n\nLet's think of dsgdb9nsd_000001(CH_4). At least the 4 Hs should be symmetrically equal, however, the tensors are different because the positioning of the atoms does matter.\n\nThis operation might be wrong, but we can think of a unit vector from Hs to C , and do the multiplication with the tensor, which gives us a vector, and take the norm of it. I feel like these 4 scalars should be equal, but we get following 4 values.\n```\n(29.048, 32.422, 28.935, 33.305)\n```\n\nWhich brings us back to the issues with coordinates." }, { "id": 571851, "postDate": "2019-07-10T06:39:01.630Z", "content": "\n\nRegarding the dipole moments: \n> You mean this particular dipole moment, or generally dipole moments have different coordinates?
\n\nI mean all the dipole moments in the QM9 dataset. They seem to not be expressed in the same cartesian system as the positions. If it were, there wouln't be the inconsistency that you mentioned. In the end I don't know whether it's a problem. I'm not sure that the precise orientation of the dipole moment is important, just its magnitude. But I may be wrong.
\n\nRegarding the magnetic shielding tensor:\n> I feel like these 4 scalars should be equal, but we get following 4 values.
\n\nKeep in mind that the values we have in the QM9 dataset have been calculated using DFT with hybrid exchange-correlation functionals. While these methods are very accurate, they are still approximations to the Schrödinger equation and are therefore not exact. So it's not surprising to see differences in your numbers even if the molecule is symmetric.
\n\nNote that in DFT calculations you have the possibility to force your results to be rigorously symmetric but it is not considered good practice because it may constrain the system. It's better to leave all degrees of freedom and let the DFT calculation finds the symmetries on its own.
", "rawMarkdown": "@shahrdar \n\nRegarding the dipole moments: \n> You mean this particular dipole moment, or generally dipole moments have different coordinates? \n\nI mean all the dipole moments in the QM9 dataset. They seem to not be expressed in the same cartesian system as the positions. If it were, there wouln't be the inconsistency that you mentioned. In the end I don't know whether it's a problem. I'm not sure that the precise orientation of the dipole moment is important, just its magnitude. But I may be wrong.\n\nRegarding the magnetic shielding tensor:\n> I feel like these 4 scalars should be equal, but we get following 4 values.\n\nKeep in mind that the values we have in the QM9 dataset have been calculated using DFT with hybrid exchange-correlation functionals. While these methods are very accurate, they are still approximations to the Schrödinger equation and are therefore not exact. So it's not surprising to see differences in your numbers even if the molecule is symmetric. \n\nNote that in DFT calculations you have the possibility to force your results to be rigorously symmetric but it is not considered good practice because it may constrain the system. It's better to leave all degrees of freedom and let the DFT calculation finds the symmetries on its own.", "votes": 5 }, { "id": 571896, "postDate": "2019-07-10T07:58:49.880Z", "content": "@borisdee \nThank you for the full explanation!
\n\nIt's good to officially know that all the coordinates for dipoles are magically twisted.
\n\nI shall keep the DFT calculation inaccuracy in my mind as well.
", "rawMarkdown": "@borisdee \nThank you for the full explanation!\n\nIt's good to officially know that all the coordinates for dipoles are magically twisted.\n\nI shall keep the DFT calculation inaccuracy in my mind as well." }, { "id": 571979, "postDate": "2019-07-10T09:52:11.903Z", "content": "@borisdee , from my understanding of QM9 only the magnitude of the dipole moment is given.
", "rawMarkdown": "@borisdee , from my understanding of QM9 only the magnitude of the dipole moment is given. " }, { "id": 571986, "postDate": "2019-07-10T09:57:28.887Z", "content": "@fnands \nI assume he was sleepy as well.
", "rawMarkdown": "@fnands \nI assume he was sleepy as well.", "votes": 1 }, { "id": 572016, "postDate": "2019-07-10T10:53:59.107Z", "content": "@shahrdar , haha! \n(Nice new team name btw!)
", "rawMarkdown": "@shahrdar , haha! \n(Nice new team name btw!)", "votes": 1 }, { "id": 572183, "postDate": "2019-07-10T15:37:09.123Z", "content": "@shahrdar Is it possible for you to show code on how to generate those magnetic shielding tensors? On the reference you cited, I see the following equation:
\n\n
E is the total electronic energy of the molecule (so... does that mean the sum of the Mulliken charges?) B is the external magnetic field (How can we determine if there is an external magnetic field?) and mu is the magnetic moment of the nucleus of interest (we don't know the magnetic moments of our atoms).
Later on, there is a second equation:
\n\n![\"Second](\"https://i.imgur.com/823rSni.png\")
But again this is super confusing! It doesn't say what psi is supposed to mean, and also introduces a random delta that is also undefined. I'm really lost here =)
Is this some obscure joke or a test of shahrdar or are you serious about \"delta\" vs \"del\" in Eq.5?
", "rawMarkdown": "Is this some obscure joke or a test of shahrdar or are you serious about \"delta\" vs \"del\" in Eq.5?" }, { "id": 572234, "postDate": "2019-07-10T16:50:48.090Z", "content": "@returnofsputnik \nThe first equation means that the component $\\alpha\\beta$ is the second derivative of the total energy with respect to (i) a variation of the magnetic moment and (ii) a variation of the external magnetic field. So basically your system has a total energy E and you modify slightly both the magnetic field and the magnetic moment of the nucleus. This will create a slight modification of the total energy, which is called a response. You get the shielding component by making the division of all these slight differences. \nTherefore we say that the shielding tensor is the response of your system wrt a variation of the external magnetic field and magnetic moments.
\n\nThe second equation is the same except that it is expressed in terms of the wavefunction $\\psi$ of your system. The first equation is derived from the second.
\n\nHope this helps :)
", "rawMarkdown": "@returnofsputnik \nThe first equation means that the component $\\alpha\\beta$ is the second derivative of the total energy with respect to (i) a variation of the magnetic moment and (ii) a variation of the external magnetic field. So basically your system has a total energy E and you modify slightly both the magnetic field and the magnetic moment of the nucleus. This will create a slight modification of the total energy, which is called a _response_. You get the shielding component by making the division of all these slight differences. \nTherefore we say that the shielding tensor is the response of your system wrt a variation of the external magnetic field and magnetic moments.\n\nThe second equation is the same except that it is expressed in terms of the wavefunction $\\psi$ of your system. The first equation is derived from the second.\n\nHope this helps :)", "votes": 3 }, { "id": 572785, "postDate": "2019-07-11T12:02:51.347Z", "content": "@returnofsputnik \nOk, I shall share the secret sauce..., just make sure you don't tell anyone...
\n\nYou can download it from here as an additional data.
", "rawMarkdown": "@returnofsputnik \nOk, I shall share the secret sauce..., just make sure you don't tell anyone...\n\nYou can download it from [here](https://www.kaggle.com/c/champs-scalar-coupling/data) as an additional data.", "votes": 2 }, { "id": 572836, "postDate": "2019-07-11T13:02:05.677Z", "content": "I thought you had a calculation to generate it based on charges and xyz coordinates, and therefore you could generate it for test dataset. :-)
", "rawMarkdown": "I thought you had a calculation to generate it based on charges and xyz coordinates, and therefore you could generate it for test dataset. :-)", "votes": 1 } ] }, { "id": 613592, "postDate": "2019-08-30T16:37:31.673Z", "content": "@shahrdar, @cedben Much thanks to this post, I investigated this issue and found that there had been a small error in 77 of the molecule calculations in the training set. This error caused the shielding tensor and dipole moment orientation to be incorrect, however the trace of the shielding tensor as well as the magnitude of the dipole vector was still correct. None of the other values, including the target variable were affected.\nUnfortunately one of these molecules was water as you found out.
\n\nI was only able to figure this out a few weeks before the competition ended, so together with the Kaggle team, we decided to let the competition run it's course without any corrections to the data, since such few molecules were affected, it only affected supplementary data and we didn't want to penalize participants that put their effort in early in the competition.
\n\nApologies to the ones that were discouraged from testing a hypothesis because of this.
", "rawMarkdown": "@shahrdar, @cedben Much thanks to this post, I investigated this issue and found that there had been a small error in 77 of the molecule calculations in the training set. This error caused the shielding tensor and dipole moment orientation to be incorrect, however the trace of the shielding tensor as well as the magnitude of the dipole vector was still correct. None of the other values, including the target variable were affected.\nUnfortunately one of these molecules was water as you found out.\n\nI was only able to figure this out a few weeks before the competition ended, so together with the Kaggle team, we decided to let the competition run it's course without any corrections to the data, since such few molecules were affected, it only affected supplementary data and we didn't want to penalize participants that put their effort in early in the competition.\n\nApologies to the ones that were discouraged from testing a hypothesis because of this.", "votes": 3 }, { "id": 570850, "postDate": "2019-07-08T20:48:48.713Z", "content": "Odd, right magnitude, wrong direction. It seems the value was just copied from QM9, but not pointed in the right direction. Well spotted. \n(I'm the sleepy one)
", "rawMarkdown": "Odd, right magnitude, wrong direction. It seems the value was just copied from QM9, but not pointed in the right direction. Well spotted. \n(*I'm the sleepy one*)", "votes": 2 }, { "id": 570817, "postDate": "2019-07-08T19:48:42.423Z", "rawMarkdown": "", "isDeleted": true }, { "id": 570693, "postDate": "2019-07-08T16:43:26.317Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 570642, "postDate": "2019-07-08T15:14:55.140Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 571493, "author_name": "Boris D.", "author_url": "", "post_date": "2019-07-09T17:03:29.603000", "content": "It's just because the dipole moment is not expressed in the same cartesian system as the positions (which is stupid, I agree). I have drawn the molecule and the dipole moment, it's in the attached file, and you'll see that it's not oriented in the way it's supposed to be.
", "votes": 5, "replies": [ { "id": 571713, "author_name": "shahrdar", "author_url": "", "post_date": "2019-07-10T02:42:18.567000", "content": "You mean this particular dipole moment, or generally dipole moments have different coordinates?\nI'm not an expert, but I don't think it's the latter case.
\n\nAssuming it's the first case, that our handed dipole.csv has different universal cartesian coordinate, our calculation on dsgdb9nsd_000005 should have same direction as dsgdb9nsd_000003, but it seems that is not the case.
\n\nI don't mind with the dipole moment now that we may use QM9, however, I'm worried that same thing might be happening on the magnetic shielding tensor, which I have no clue how they are calculated, and some of my calculations seems to be saying that they are symmetrically wrong, using the coordinates from xyz files.
", "votes": 0, "replies": [] }, { "id": 571720, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-07-10T02:55:34.380000", "content": "How to calculate magnetic shielding tensor? @shahrdar
", "votes": 0, "replies": [] }, { "id": 571768, "author_name": "shahrdar", "author_url": "", "post_date": "2019-07-10T04:30:19.727000", "content": "I found the resource from the following discussion is useful. (Please open in new tab, at least my chrome has glitch)\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/96435
\n\nI'm not quite sure, but I think you need 2 vectors to convert this tensor into a scalar. One is the direction of the molecule, and the other is the direction of the external magnetic field.
\n\nLet's think of dsgdb9nsd_000001(CH_4). At least the 4 Hs should be symmetrically equal, however, the tensors are different because the positioning of the atoms does matter.
\n\nThis operation might be wrong, but we can think of a unit vector from Hs to C , and do the multiplication with the tensor, which gives us a vector, and take the norm of it. I feel like these 4 scalars should be equal, but we get following 4 values.\n\n(29.048, 32.422, 28.935, 33.305)\n
Which brings us back to the issues with coordinates.
", "votes": 0, "replies": [] }, { "id": 571851, "author_name": "Boris D.", "author_url": "", "post_date": "2019-07-10T06:39:01.630000", "content": "\n\nRegarding the dipole moments: \n> You mean this particular dipole moment, or generally dipole moments have different coordinates?
\n\nI mean all the dipole moments in the QM9 dataset. They seem to not be expressed in the same cartesian system as the positions. If it were, there wouln't be the inconsistency that you mentioned. In the end I don't know whether it's a problem. I'm not sure that the precise orientation of the dipole moment is important, just its magnitude. But I may be wrong.
\n\nRegarding the magnetic shielding tensor:\n> I feel like these 4 scalars should be equal, but we get following 4 values.
\n\nKeep in mind that the values we have in the QM9 dataset have been calculated using DFT with hybrid exchange-correlation functionals. While these methods are very accurate, they are still approximations to the Schrödinger equation and are therefore not exact. So it's not surprising to see differences in your numbers even if the molecule is symmetric.
\n\nNote that in DFT calculations you have the possibility to force your results to be rigorously symmetric but it is not considered good practice because it may constrain the system. It's better to leave all degrees of freedom and let the DFT calculation finds the symmetries on its own.
", "votes": 5, "replies": [] }, { "id": 571896, "author_name": "shahrdar", "author_url": "", "post_date": "2019-07-10T07:58:49.880000", "content": "@borisdee \nThank you for the full explanation!
\n\nIt's good to officially know that all the coordinates for dipoles are magically twisted.
\n\nI shall keep the DFT calculation inaccuracy in my mind as well.
", "votes": 0, "replies": [] }, { "id": 571979, "author_name": "fnands", "author_url": "", "post_date": "2019-07-10T09:52:11.903000", "content": "@borisdee , from my understanding of QM9 only the magnitude of the dipole moment is given.
", "votes": 0, "replies": [] }, { "id": 571986, "author_name": "shahrdar", "author_url": "", "post_date": "2019-07-10T09:57:28.887000", "content": "@fnands \nI assume he was sleepy as well.
", "votes": 1, "replies": [] }, { "id": 572016, "author_name": "fnands", "author_url": "", "post_date": "2019-07-10T10:53:59.107000", "content": "@shahrdar , haha! \n(Nice new team name btw!)
", "votes": 1, "replies": [] }, { "id": 572183, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-07-10T15:37:09.123000", "content": "@shahrdar Is it possible for you to show code on how to generate those magnetic shielding tensors? On the reference you cited, I see the following equation:
\n\n
E is the total electronic energy of the molecule (so... does that mean the sum of the Mulliken charges?) B is the external magnetic field (How can we determine if there is an external magnetic field?) and mu is the magnetic moment of the nucleus of interest (we don't know the magnetic moments of our atoms).
Later on, there is a second equation:
\n\n
But again this is super confusing! It doesn't say what psi is supposed to mean, and also introduces a random delta that is also undefined. I'm really lost here =)
Is this some obscure joke or a test of shahrdar or are you serious about \"delta\" vs \"del\" in Eq.5?
", "votes": 0, "replies": [] }, { "id": 572234, "author_name": "Boris D.", "author_url": "", "post_date": "2019-07-10T16:50:48.090000", "content": "@returnofsputnik \nThe first equation means that the component $\\alpha\\beta$ is the second derivative of the total energy with respect to (i) a variation of the magnetic moment and (ii) a variation of the external magnetic field. So basically your system has a total energy E and you modify slightly both the magnetic field and the magnetic moment of the nucleus. This will create a slight modification of the total energy, which is called a response. You get the shielding component by making the division of all these slight differences. \nTherefore we say that the shielding tensor is the response of your system wrt a variation of the external magnetic field and magnetic moments.
\n\nThe second equation is the same except that it is expressed in terms of the wavefunction $\\psi$ of your system. The first equation is derived from the second.
\n\nHope this helps :)
", "votes": 3, "replies": [] }, { "id": 572785, "author_name": "shahrdar", "author_url": "", "post_date": "2019-07-11T12:02:51.347000", "content": "@returnofsputnik \nOk, I shall share the secret sauce..., just make sure you don't tell anyone...
\n\nYou can download it from here as an additional data.
", "votes": 2, "replies": [] }, { "id": 572836, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2019-07-11T13:02:05.677000", "content": "I thought you had a calculation to generate it based on charges and xyz coordinates, and therefore you could generate it for test dataset. :-)
", "votes": 1, "replies": [] } ] }, { "id": 613592, "author_name": "Lars Bratholm", "author_url": "", "post_date": "2019-08-30T16:37:31.673000", "content": "@shahrdar, @cedben Much thanks to this post, I investigated this issue and found that there had been a small error in 77 of the molecule calculations in the training set. This error caused the shielding tensor and dipole moment orientation to be incorrect, however the trace of the shielding tensor as well as the magnitude of the dipole vector was still correct. None of the other values, including the target variable were affected.\nUnfortunately one of these molecules was water as you found out.
\n\nI was only able to figure this out a few weeks before the competition ended, so together with the Kaggle team, we decided to let the competition run it's course without any corrections to the data, since such few molecules were affected, it only affected supplementary data and we didn't want to penalize participants that put their effort in early in the competition.
\n\nApologies to the ones that were discouraged from testing a hypothesis because of this.
", "votes": 3, "replies": [] }, { "id": 570850, "author_name": "fnands", "author_url": "", "post_date": "2019-07-08T20:48:48.713000", "content": "Odd, right magnitude, wrong direction. It seems the value was just copied from QM9, but not pointed in the right direction. Well spotted. \n(I'm the sleepy one)
", "votes": 2, "replies": [] }, { "id": 570817, "author_name": "", "author_url": "", "post_date": "2019-07-08T19:48:42.423000", "content": "", "votes": 0, "replies": [] }, { "id": 570693, "author_name": "", "author_url": "", "post_date": "2019-07-08T16:43:26.317000", "content": "", "votes": -1, "replies": [] }, { "id": 570642, "author_name": "", "author_url": "", "post_date": "2019-07-08T15:14:55.140000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "570594": "So, today I thought of working around with new feature, and stuck with an issue.\n\nTo start with, I found this resource useful.\nhttps://chem.libretexts.org/Bookshelves/Physical_and_Theoretical_Chemistry_Textbook_Maps/Supplemental_Modules_(Physical_and_Theoretical_Chemistry)/Physical_Properties_of_Matter/Atomic_and_Molecular_Properties/Dipole_Moments\n\nIt says water has a dipole moment due to its bent structure, and combined vectors from 2 Hs to O, is the direction of the dipole moment.\n\nThe details might be wrong but that's the basic idea, that we can reconstruct the dipole moment from the coordinates and charges of atoms.\n\nAccording to our dipole.csv thingy, dsgdb9nsd_000003 (a.k.a water) has dipole moment of\n```\n(0, 0, -1.8511)\n```\nWe also know the exact coordinates of the atoms from xyz file too.\n```\nO -0.0343604951 0.9775395708 0.0076015923\nH 0.0647664923 0.0205721989 0.0015346341\nH 0.8717903737 1.3007924048 0.0006931336\n```\nSo let's start the calculation. Vectors from H to O would be\n```\n(-0.0991, 0.9569, 0.0060)\n(-0.9061, -0.3232, 0.0069)\n```\nAnd we combine them, which should have the same direction as the dipole moment\n```\n(-1.0052, 0.6337, 0.0129)\n```\nWait, why the X and Y are not cancelling out?\n\nCharges are not the problem here, because using the eq. (1) from the resource, we end up with same calculation.\n\nAm I making mistake because I'm too sleepy?", "571493": "It's just because the dipole moment is not expressed in the same cartesian system as the positions (which is stupid, I agree). I have drawn the molecule and the dipole moment, it's in the attached file, and you'll see that it's not oriented in the way it's supposed to be.", "613592": "@shahrdar, @cedben Much thanks to this post, I investigated this issue and found that there had been a small error in 77 of the molecule calculations in the training set. This error caused the shielding tensor and dipole moment orientation to be incorrect, however the trace of the shielding tensor as well as the magnitude of the dipole vector was still correct. None of the other values, including the target variable were affected.\nUnfortunately one of these molecules was water as you found out.\n\nI was only able to figure this out a few weeks before the competition ended, so together with the Kaggle team, we decided to let the competition run it's course without any corrections to the data, since such few molecules were affected, it only affected supplementary data and we didn't want to penalize participants that put their effort in early in the competition.\n\nApologies to the ones that were discouraged from testing a hypothesis because of this.", "570850": "Odd, right magnitude, wrong direction. It seems the value was just copied from QM9, but not pointed in the right direction. Well spotted. \n(*I'm the sleepy one*)", "570817": "", "570693": "", "570642": "" } }