We basically used this.
\n\nhttps://arxiv.org/abs/1906.04015
\n\nWe modified the architecture slightly so that the outputs would be on the connections of atoms, but that was pretty much it. For simplicity, we trained one net each on j_1, j_2, and j_3 couplings, and then split it into one net for each coupling type (e.g. 1JHC).
\n\nThe only slightly fancy thing we did was that we pretrained by initially using the Mulliken charges as features, which we removed further on in the training.
\n\nThat's it, no particularly clever tricks :-).
\n\nP.S. If anyone has any questions about the paper, Brandon and I will be happy to address them to the best of our ability.
", "messages": [ { "id": "610752", "postDate": "08/29/2019 00:56:30", "content": "We basically used this.
\n\nhttps://arxiv.org/abs/1906.04015
\n\nWe modified the architecture slightly so that the outputs would be on the connections of atoms, but that was pretty much it. For simplicity, we trained one net each on j_1, j_2, and j_3 couplings, and then split it into one net for each coupling type (e.g. 1JHC).
\n\nThe only slightly fancy thing we did was that we pretrained by initially using the Mulliken charges as features, which we removed further on in the training.
\n\nThat's it, no particularly clever tricks :-).
\n\nP.S. If anyone has any questions about the paper, Brandon and I will be happy to address them to the best of our ability.
", "rawMarkdown": "We basically used this.\n\nhttps://arxiv.org/abs/1906.04015\n\nWe modified the architecture slightly so that the outputs would be on the connections of atoms, but that was pretty much it. For simplicity, we trained one net each on j_1, j_2, and j_3 couplings, and then split it into one net for each coupling type (e.g. 1JHC).\n\nThe only slightly fancy thing we did was that we pretrained by initially using the Mulliken charges as features, which we removed further on in the training.\n\nThat's it, no particularly clever tricks :-).\n\nP.S. If anyone has any questions about the paper, Brandon and I will be happy to address them to the best of our ability.", "votes": null }, { "id": "610763", "postDate": "08/29/2019 00:59:31", "content": "wow, 1st time I see it, are you planning to share the implementation ?\nbtw, spectacular result in such a short time :)
", "rawMarkdown": "wow, 1st time I see it, are you planning to share the implementation ?\nbtw, spectacular result in such a short time :)", "votes": null }, { "id": "610767", "postDate": "08/29/2019 01:02:03", "content": "Absolutely! We're hoping to release a software package soon (in the next few weeks) for public use.
", "rawMarkdown": "Absolutely! We're hoping to release a software package soon (in the next few weeks) for public use.", "votes": null }, { "id": "610768", "postDate": "08/29/2019 01:02:33", "content": "Wow, brand new idea...\nYou had a leg up in this :)\nCongratulations!
", "rawMarkdown": "Wow, brand new idea...\nYou had a leg up in this :)\nCongratulations!", "votes": null }, { "id": "610782", "postDate": "08/29/2019 01:10:26", "content": "Thank you! We're very excited to see how far this architecture can go on point-cloud data.
", "rawMarkdown": "Thank you! We're very excited to see how far this architecture can go on point-cloud data.", "votes": null }, { "id": "610803", "postDate": "08/29/2019 01:29:16", "content": "Elegant solution!! Congrats
", "rawMarkdown": "Elegant solution!! Congrats", "votes": null }, { "id": "610817", "postDate": "08/29/2019 01:42:50", "content": "Congratulations! We're very interested in your solution, hope to see it soon!
", "rawMarkdown": "Congratulations! We're very interested in your solution, hope to see it soon!", "votes": null }, { "id": "610947", "postDate": "08/29/2019 03:21:52", "content": "This is impressive! Using a correct architecture seemed to be the key.
", "rawMarkdown": "This is impressive! Using a correct architecture seemed to be the key.", "votes": null }, { "id": "611324", "postDate": "08/29/2019 08:01:49", "content": "Congratulations and thanks for sharing!
", "rawMarkdown": "Congratulations and thanks for sharing!", "votes": null }, { "id": "611413", "postDate": "08/29/2019 08:45:17", "content": "Thanks for sharing, and congrats on the result!
", "rawMarkdown": "Thanks for sharing, and congrats on the result!", "votes": null }, { "id": "611495", "postDate": "08/29/2019 09:43:32", "content": "Thanks for sharing, if I'm not mistaken, the Supplement sections has not yet added to the paper? Do you have any version of ELI5 of your paper for someone who doesn't have background in this topic?
", "rawMarkdown": "Thanks for sharing, if I'm not mistaken, the Supplement sections has not yet added to the paper? Do you have any version of ELI5 of your paper for someone who doesn't have background in this topic?", "votes": null }, { "id": "611498", "postDate": "08/29/2019 09:45:55", "content": "Thank you for sharing and congratulations!
", "rawMarkdown": "Thank you for sharing and congratulations!", "votes": null }, { "id": "619810", "postDate": "09/06/2019 15:46:58", "content": "Your submission is uncorrelated with others relatively and contributes mega-merge !\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/107661\nI wonder why you submission was different from others. It maybe pretraining.
", "rawMarkdown": "Your submission is uncorrelated with others relatively and contributes mega-merge !\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/107661\nI wonder why you submission was different from others. It maybe pretraining.", "votes": null }, { "id": "619839", "postDate": "09/06/2019 16:37:11", "content": "I am trying to understand the paper, it is opening a whole new world. How did you implement this? Pytorch, Tensorflow, something else? Do you plan to share your code at some point?
", "rawMarkdown": "I am trying to understand the paper, it is opening a whole new world. How did you implement this? Pytorch, Tensorflow, something else? Do you plan to share your code at some point?", "votes": null }, { "id": "619852", "postDate": "09/06/2019 16:54:44", "content": "I found these two repos that seem relevant:
\n\nhttps://github.com/HyTruongSon/GraphFlow/
\n\nhttps://github.com/horacepan/CCN
", "rawMarkdown": "I found these two repos that seem relevant:\n\nhttps://github.com/HyTruongSon/GraphFlow/\n\nhttps://github.com/horacepan/CCN", "votes": null }, { "id": "619866", "postDate": "09/06/2019 17:12:20", "content": "Sorry for the belated answer. The Supplement is coming :-).
\n\nHere is a belated ELI5. The basic approach is that we are building an architecture to generalize the idea of standard convolutional layers in Convolutional Neural Networks (CNNs) to molecules. The key property that convolutional layers are designed to preserve is translational covariance: if you translate the input to the network, the output of the network translates as well. Ultimately, this allows you to learn commonalities across an image: if you learn what an object (e.g. a mug) looks likes in one part of the image, you can detect it in a different position in a different image. It’s also worth discussing how CNN’s work: the covariance comes from a linear transformation (the convolution, hence the name), which is followed by a standard nonlinearity.
\n\nWe want to transfer this to molecules. Let’s consider the linear convolution first. But there’s a complication: now the symmetry that we care about isn’t just translation, but rotation in 3 dimensions. If we rotate a molecule, we want our signal to rotate with it. Rotations are harder than translations. There are several complications, including the fact that it is not clear how to “discretize” rotations to perform a convolution without breaking symmetry in some way.
\n\nOur approach is to use ideas from Fourier analysis. It turns out that it is possible to generalize Fourier analysis not just functions on the line, but in the space of rotations. This is nice, because convolutions in Real space look like multiplications in Fourier space. So if we have a Fourier transformed signal, performing convolutions is easy [1]. So we Fourier transform our data, at which point our linear convolutions are just linear algebra.
\n\nSo this helps us define the linear section: what about the nonlinear bit? It’s actually somewhat nontrivial to build a nonlinear function that doesn’t break covariance, or require transformation back into Real space (which can lead to numerical artifacts). However, one thing we can do is multiply signals together. For regular Fourier analysis, this is like constructing beats out of our Fourier modes [2, 3]. We can then efficiently decompose the beating signals into it’s Fourier components using known relations (these are the famous Clebsch-Gordan coefficients), allowing us to keep everything in Fourier space.
\n\nThe only thing that is left to do is to define the signals. This part is a bit technical, but basically in each layer, we grab signals from neighboring atoms based on their relative orientation of the atom and it’s signal (because our signals are rotationally covariant, they have a notion of orientation) [4]. We then perform the Fourier-space convolution and a few (tensor) multiplications, rinse and repeat!
\n\n[1] See https://arxiv.org/pdf/1802.03690.pdf for further reading\n[2] See https://en.wikipedia.org/wiki/Beat_(acoustics)\n[3] Specifically we actually take tensor products of signals. This is more general, but just thinking about multiplications gets the right intuition across.\n[4] The coefficients for this message passing looks a lot like an edge network, and is what we actually read the signals off of for this Kaggle challenge.
", "rawMarkdown": "Sorry for the belated answer. The Supplement is coming :-). \n\nHere is a belated ELI5. The basic approach is that we are building an architecture to generalize the idea of standard convolutional layers in Convolutional Neural Networks (CNNs) to molecules. The key property that convolutional layers are designed to preserve is *translational covariance*: if you translate the input to the network, the output of the network translates as well. Ultimately, this allows you to learn commonalities across an image: if you learn what an object (e.g. a mug) looks likes in one part of the image, you can detect it in a different position in a different image. It’s also worth discussing how CNN’s work: the covariance comes from a linear transformation (the convolution, hence the name), which is followed by a standard nonlinearity.\n\nWe want to transfer this to molecules. Let’s consider the linear convolution first. But there’s a complication: now the symmetry that we care about isn’t just *translation*, but *rotation in 3 dimensions*. If we rotate a molecule, we want our signal to rotate with it. Rotations are harder than translations. There are several complications, including the fact that it is not clear how to “discretize” rotations to perform a convolution without breaking symmetry in some way.\n\nOur approach is to use ideas from Fourier analysis. It turns out that it is possible to generalize Fourier analysis not just functions on the line, *but in the space of rotations*. This is nice, because convolutions in Real space look like multiplications in Fourier space. So if we have a Fourier transformed signal, performing convolutions is easy [1]. So we Fourier transform our data, at which point our linear convolutions are just linear algebra.\n\nSo this helps us define the linear section: what about the nonlinear bit? It’s actually somewhat nontrivial to build a nonlinear function that doesn’t break covariance, or require transformation back into Real space (which can lead to numerical artifacts). However, one thing we can do is multiply signals together. For regular Fourier analysis, this is like constructing beats out of our Fourier modes [2, 3]. We can then efficiently decompose the beating signals into it’s Fourier components using known relations (these are the famous Clebsch-Gordan coefficients), allowing us to keep everything in Fourier space.\n\nThe only thing that is left to do is to define the signals. This part is a bit technical, but basically in each layer, we grab signals from neighboring atoms based on their relative orientation of the atom and it’s signal (because our signals are rotationally covariant, they have a notion of orientation) [4]. We then perform the Fourier-space convolution and a few (tensor) multiplications, rinse and repeat!\n\n[1] See https://arxiv.org/pdf/1802.03690.pdf for further reading\n[2] See https://en.wikipedia.org/wiki/Beat_(acoustics)\n[3] Specifically we actually take tensor products of signals. This is more general, but just thinking about multiplications gets the right intuition across.\n[4] The coefficients for this message passing looks a lot like an edge network, and is what we actually read the signals off of for this Kaggle challenge.", "votes": null }, { "id": "619877", "postDate": "09/06/2019 17:24:16", "content": "Our implementation is built on PyTorch. We're still working on making it presentable, but hopefully an alpha version should be up soon.
\n\nSon's and Horace's codes are built for graph neural networks. There is a lot of ideological overlap with cormorant, but (unless I'm mistaken) they don't work explicitly with rotational equivariance. The method we are using does, however.
\n\nhttps://github.com/zlin7/CGNet
\n\nUntil we get cormorant up, this code might be informative.
\n\nIn the long-run, we'd really like to a write a custom-library for performing rotational equivariant computations. The problem is that, in general, there is a lot of sparse linear algebra on objects that can look like lists of matrices of different shapes. This means that, to really do the calculations fast on a GPU, one needs custom codes. Hopefully we can this this up soon as well.
", "rawMarkdown": "Our implementation is built on PyTorch. We're still working on making it presentable, but hopefully an alpha version should be up soon.\n\nSon's and Horace's codes are built for graph neural networks. There is a lot of ideological overlap with cormorant, but (unless I'm mistaken) they don't work explicitly with rotational equivariance. The method we are using does, however.\n\nhttps://github.com/zlin7/CGNet\n\nUntil we get cormorant up, this code might be informative.\n\nIn the long-run, we'd really like to a write a custom-library for performing rotational equivariant computations. The problem is that, in general, there is a lot of sparse linear algebra on objects that can look like lists of matrices of different shapes. This means that, to really do the calculations fast on a GPU, one needs custom codes. Hopefully we can this this up soon as well.", "votes": null }, { "id": "619879", "postDate": "09/06/2019 17:26:09", "content": "I suspect it's because our model works on very different principles. Most other models were based on graph neural networks or SchNet. These are perform rotationally invariant calculations. Ours, however, is rotationally equivariant. We also do many of the calculations in rotational Fourier space.
", "rawMarkdown": "I suspect it's because our model works on very different principles. Most other models were based on graph neural networks or SchNet. These are perform rotationally invariant calculations. Ours, however, is rotationally *equivariant*. We also do many of the calculations in rotational Fourier space.", "votes": null }, { "id": "619893", "postDate": "09/06/2019 18:06:04", "content": "Thanks for the answer. I'll dig into that too.
", "rawMarkdown": "Thanks for the answer. I'll dig into that too.", "votes": null } ], "comments": [ { "id": 610763, "author_name": "jesucristo", "author_url": "", "post_date": "08/29/2019 00:59:31", "content": "wow, 1st time I see it, are you planning to share the implementation ?\nbtw, spectacular result in such a short time :)
", "votes": null, "replies": [ { "id": 610767, "author_name": "ehthiede", "author_url": "", "post_date": "08/29/2019 01:02:03", "content": "Absolutely! We're hoping to release a software package soon (in the next few weeks) for public use.
", "votes": null, "replies": [] } ] }, { "id": 610768, "author_name": "antaresnyc", "author_url": "", "post_date": "08/29/2019 01:02:33", "content": "Wow, brand new idea...\nYou had a leg up in this :)\nCongratulations!
", "votes": null, "replies": [ { "id": 610782, "author_name": "ehthiede", "author_url": "", "post_date": "08/29/2019 01:10:26", "content": "Thank you! We're very excited to see how far this architecture can go on point-cloud data.
", "votes": null, "replies": [] } ] }, { "id": 610803, "author_name": "bibek777", "author_url": "", "post_date": "08/29/2019 01:29:16", "content": "Elegant solution!! Congrats
", "votes": null, "replies": [] }, { "id": 610817, "author_name": "jiazhuang", "author_url": "", "post_date": "08/29/2019 01:42:50", "content": "Congratulations! We're very interested in your solution, hope to see it soon!
", "votes": null, "replies": [] }, { "id": 610947, "author_name": "artgor", "author_url": "", "post_date": "08/29/2019 03:21:52", "content": "This is impressive! Using a correct architecture seemed to be the key.
", "votes": null, "replies": [] }, { "id": 611324, "author_name": "stanislavblinov", "author_url": "", "post_date": "08/29/2019 08:01:49", "content": "Congratulations and thanks for sharing!
", "votes": null, "replies": [] }, { "id": 611413, "author_name": "cpmpml", "author_url": "", "post_date": "08/29/2019 08:45:17", "content": "Thanks for sharing, and congrats on the result!
", "votes": null, "replies": [] }, { "id": 611495, "author_name": "vithng", "author_url": "", "post_date": "08/29/2019 09:43:32", "content": "Thanks for sharing, if I'm not mistaken, the Supplement sections has not yet added to the paper? Do you have any version of ELI5 of your paper for someone who doesn't have background in this topic?
", "votes": null, "replies": [ { "id": 619866, "author_name": "ehthiede", "author_url": "", "post_date": "09/06/2019 17:12:20", "content": "Sorry for the belated answer. The Supplement is coming :-).
\n\nHere is a belated ELI5. The basic approach is that we are building an architecture to generalize the idea of standard convolutional layers in Convolutional Neural Networks (CNNs) to molecules. The key property that convolutional layers are designed to preserve is translational covariance: if you translate the input to the network, the output of the network translates as well. Ultimately, this allows you to learn commonalities across an image: if you learn what an object (e.g. a mug) looks likes in one part of the image, you can detect it in a different position in a different image. It’s also worth discussing how CNN’s work: the covariance comes from a linear transformation (the convolution, hence the name), which is followed by a standard nonlinearity.
\n\nWe want to transfer this to molecules. Let’s consider the linear convolution first. But there’s a complication: now the symmetry that we care about isn’t just translation, but rotation in 3 dimensions. If we rotate a molecule, we want our signal to rotate with it. Rotations are harder than translations. There are several complications, including the fact that it is not clear how to “discretize” rotations to perform a convolution without breaking symmetry in some way.
\n\nOur approach is to use ideas from Fourier analysis. It turns out that it is possible to generalize Fourier analysis not just functions on the line, but in the space of rotations. This is nice, because convolutions in Real space look like multiplications in Fourier space. So if we have a Fourier transformed signal, performing convolutions is easy [1]. So we Fourier transform our data, at which point our linear convolutions are just linear algebra.
\n\nSo this helps us define the linear section: what about the nonlinear bit? It’s actually somewhat nontrivial to build a nonlinear function that doesn’t break covariance, or require transformation back into Real space (which can lead to numerical artifacts). However, one thing we can do is multiply signals together. For regular Fourier analysis, this is like constructing beats out of our Fourier modes [2, 3]. We can then efficiently decompose the beating signals into it’s Fourier components using known relations (these are the famous Clebsch-Gordan coefficients), allowing us to keep everything in Fourier space.
\n\nThe only thing that is left to do is to define the signals. This part is a bit technical, but basically in each layer, we grab signals from neighboring atoms based on their relative orientation of the atom and it’s signal (because our signals are rotationally covariant, they have a notion of orientation) [4]. We then perform the Fourier-space convolution and a few (tensor) multiplications, rinse and repeat!
\n\n[1] See https://arxiv.org/pdf/1802.03690.pdf for further reading\n[2] See https://en.wikipedia.org/wiki/Beat_(acoustics)\n[3] Specifically we actually take tensor products of signals. This is more general, but just thinking about multiplications gets the right intuition across.\n[4] The coefficients for this message passing looks a lot like an edge network, and is what we actually read the signals off of for this Kaggle challenge.
", "votes": null, "replies": [] } ] }, { "id": 611498, "author_name": "rakibilly", "author_url": "", "post_date": "08/29/2019 09:45:55", "content": "Thank you for sharing and congratulations!
", "votes": null, "replies": [] }, { "id": 619810, "author_name": "toshik", "author_url": "", "post_date": "09/06/2019 15:46:58", "content": "Your submission is uncorrelated with others relatively and contributes mega-merge !\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/107661\nI wonder why you submission was different from others. It maybe pretraining.
", "votes": null, "replies": [ { "id": 619879, "author_name": "ehthiede", "author_url": "", "post_date": "09/06/2019 17:26:09", "content": "I suspect it's because our model works on very different principles. Most other models were based on graph neural networks or SchNet. These are perform rotationally invariant calculations. Ours, however, is rotationally equivariant. We also do many of the calculations in rotational Fourier space.
", "votes": null, "replies": [] } ] }, { "id": 619839, "author_name": "cpmpml", "author_url": "", "post_date": "09/06/2019 16:37:11", "content": "I am trying to understand the paper, it is opening a whole new world. How did you implement this? Pytorch, Tensorflow, something else? Do you plan to share your code at some point?
", "votes": null, "replies": [ { "id": 619852, "author_name": "cpmpml", "author_url": "", "post_date": "09/06/2019 16:54:44", "content": "I found these two repos that seem relevant:
\n\nhttps://github.com/HyTruongSon/GraphFlow/
\n\nhttps://github.com/horacepan/CCN
", "votes": null, "replies": [] }, { "id": 619877, "author_name": "ehthiede", "author_url": "", "post_date": "09/06/2019 17:24:16", "content": "Our implementation is built on PyTorch. We're still working on making it presentable, but hopefully an alpha version should be up soon.
\n\nSon's and Horace's codes are built for graph neural networks. There is a lot of ideological overlap with cormorant, but (unless I'm mistaken) they don't work explicitly with rotational equivariance. The method we are using does, however.
\n\nhttps://github.com/zlin7/CGNet
\n\nUntil we get cormorant up, this code might be informative.
\n\nIn the long-run, we'd really like to a write a custom-library for performing rotational equivariant computations. The problem is that, in general, there is a lot of sparse linear algebra on objects that can look like lists of matrices of different shapes. This means that, to really do the calculations fast on a GPU, one needs custom codes. Hopefully we can this this up soon as well.
", "votes": null, "replies": [] }, { "id": 619893, "author_name": "cpmpml", "author_url": "", "post_date": "09/06/2019 18:06:04", "content": "Thanks for the answer. I'll dig into that too.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "610752": "We basically used this.\n\nhttps://arxiv.org/abs/1906.04015\n\nWe modified the architecture slightly so that the outputs would be on the connections of atoms, but that was pretty much it. For simplicity, we trained one net each on j_1, j_2, and j_3 couplings, and then split it into one net for each coupling type (e.g. 1JHC).\n\nThe only slightly fancy thing we did was that we pretrained by initially using the Mulliken charges as features, which we removed further on in the training.\n\nThat's it, no particularly clever tricks :-).\n\nP.S. If anyone has any questions about the paper, Brandon and I will be happy to address them to the best of our ability.", "610763": "wow, 1st time I see it, are you planning to share the implementation ?\nbtw, spectacular result in such a short time :)", "610767": "Absolutely! We're hoping to release a software package soon (in the next few weeks) for public use.", "610768": "Wow, brand new idea...\nYou had a leg up in this :)\nCongratulations!", "610782": "Thank you! We're very excited to see how far this architecture can go on point-cloud data.", "610803": "Elegant solution!! Congrats", "610817": "Congratulations! We're very interested in your solution, hope to see it soon!", "610947": "This is impressive! Using a correct architecture seemed to be the key.", "611324": "Congratulations and thanks for sharing!", "611413": "Thanks for sharing, and congrats on the result!", "611495": "Thanks for sharing, if I'm not mistaken, the Supplement sections has not yet added to the paper? Do you have any version of ELI5 of your paper for someone who doesn't have background in this topic?", "611498": "Thank you for sharing and congratulations!", "619810": "Your submission is uncorrelated with others relatively and contributes mega-merge !\nhttps://www.kaggle.com/c/champs-scalar-coupling/discussion/107661\nI wonder why you submission was different from others. It maybe pretraining.", "619839": "I am trying to understand the paper, it is opening a whole new world. How did you implement this? Pytorch, Tensorflow, something else? Do you plan to share your code at some point?", "619852": "I found these two repos that seem relevant:\n\nhttps://github.com/HyTruongSon/GraphFlow/\n\nhttps://github.com/horacepan/CCN", "619866": "Sorry for the belated answer. The Supplement is coming :-). \n\nHere is a belated ELI5. The basic approach is that we are building an architecture to generalize the idea of standard convolutional layers in Convolutional Neural Networks (CNNs) to molecules. The key property that convolutional layers are designed to preserve is *translational covariance*: if you translate the input to the network, the output of the network translates as well. Ultimately, this allows you to learn commonalities across an image: if you learn what an object (e.g. a mug) looks likes in one part of the image, you can detect it in a different position in a different image. It’s also worth discussing how CNN’s work: the covariance comes from a linear transformation (the convolution, hence the name), which is followed by a standard nonlinearity.\n\nWe want to transfer this to molecules. Let’s consider the linear convolution first. But there’s a complication: now the symmetry that we care about isn’t just *translation*, but *rotation in 3 dimensions*. If we rotate a molecule, we want our signal to rotate with it. Rotations are harder than translations. There are several complications, including the fact that it is not clear how to “discretize” rotations to perform a convolution without breaking symmetry in some way.\n\nOur approach is to use ideas from Fourier analysis. It turns out that it is possible to generalize Fourier analysis not just functions on the line, *but in the space of rotations*. This is nice, because convolutions in Real space look like multiplications in Fourier space. So if we have a Fourier transformed signal, performing convolutions is easy [1]. So we Fourier transform our data, at which point our linear convolutions are just linear algebra.\n\nSo this helps us define the linear section: what about the nonlinear bit? It’s actually somewhat nontrivial to build a nonlinear function that doesn’t break covariance, or require transformation back into Real space (which can lead to numerical artifacts). However, one thing we can do is multiply signals together. For regular Fourier analysis, this is like constructing beats out of our Fourier modes [2, 3]. We can then efficiently decompose the beating signals into it’s Fourier components using known relations (these are the famous Clebsch-Gordan coefficients), allowing us to keep everything in Fourier space.\n\nThe only thing that is left to do is to define the signals. This part is a bit technical, but basically in each layer, we grab signals from neighboring atoms based on their relative orientation of the atom and it’s signal (because our signals are rotationally covariant, they have a notion of orientation) [4]. We then perform the Fourier-space convolution and a few (tensor) multiplications, rinse and repeat!\n\n[1] See https://arxiv.org/pdf/1802.03690.pdf for further reading\n[2] See https://en.wikipedia.org/wiki/Beat_(acoustics)\n[3] Specifically we actually take tensor products of signals. This is more general, but just thinking about multiplications gets the right intuition across.\n[4] The coefficients for this message passing looks a lot like an edge network, and is what we actually read the signals off of for this Kaggle challenge.", "619877": "Our implementation is built on PyTorch. We're still working on making it presentable, but hopefully an alpha version should be up soon.\n\nSon's and Horace's codes are built for graph neural networks. There is a lot of ideological overlap with cormorant, but (unless I'm mistaken) they don't work explicitly with rotational equivariance. The method we are using does, however.\n\nhttps://github.com/zlin7/CGNet\n\nUntil we get cormorant up, this code might be informative.\n\nIn the long-run, we'd really like to a write a custom-library for performing rotational equivariant computations. The problem is that, in general, there is a lot of sparse linear algebra on objects that can look like lists of matrices of different shapes. This means that, to really do the calculations fast on a GPU, one needs custom codes. Hopefully we can this this up soon as well.", "619879": "I suspect it's because our model works on very different principles. Most other models were based on graph neural networks or SchNet. These are perform rotationally invariant calculations. Ours, however, is rotationally *equivariant*. We also do many of the calculations in rotational Fourier space.", "619893": "Thanks for the answer. I'll dig into that too." }, "source": "meta" }