{
  "id": 103118,
  "title": "Distance as the main feature and -1.48 LB score",
  "url": "/competitions/champs-scalar-coupling/discussion/103118",
  "author_name": "",
  "post_date": "2019-08-07T08:13:21.378568600Z",
  "votes": 73,
  "comment_count": 8,
  "views": 0,
  "content": "<p>As @cpmpml said, the main challenge of this competition is to find a correct representation.</p>\n\n<p>Luckily for us, organizers left us a great hint:\n<code>\nit is possible to accurately calculate scalar coupling constants given only a 3D molecular structure as input\n</code>\nThat means, having the same set of atoms in the same relative positions we can predict scalar coupling constant looking up for similar combinations of atoms in other molecules.</p>\n\n<p>The main challenge - is to convert X,Y,Z coordinates of each atom into some transition/rotation-invariant representation, so that it can be pattern matched by some KNN or boosted tree algorithm.</p>\n\n<p>The simplest invariant feature - is just a distance between two atoms. But the challenge is to uniquely represent each atom with distances.</p>\n\n<p>If we have just <code>2 atoms</code>, we can represent uniquely the combo with 3 features:\n* distance <code>d_1_0</code> - where 0, 1 are atoms indexes\n* and types for <code>atom_0</code> and <code>atom_1</code></p>\n\n<p>For <code>3 atoms</code> we need 6 features:\n* distances: <code>d_1_0</code>, <code>d_2_0</code>, <code>d_2_1</code>\n* and types for <code>atom_0</code>, <code>atom_1</code>, <code>atom_2</code></p>\n\n<p><code>4 atoms</code> with 10 features:\n* distances: <code>d_1_0</code>, <code>d_2_0</code>, <code>d_2_1</code>, <code>d_3_0</code>, <code>d_3_1</code>, <code>d_3_2</code>\n* and types for <code>atom_0</code>, <code>atom_1</code>, <code>atom_2</code>, <code>atom_3</code></p>\n\n<p>Four core atoms distances are enough to uniquely describe each next atom. So each next atom will have only 4 distances for itself. Why?</p>\n\n<p>Imagine you have 3 non-colinear points(they should create a triangle). By using 3 distances we can represent two points - kinda vertex of a pyramid. To fix this representation to just one possible point we need this 4th non-colinear distance.</p>\n\n<p>So, in the end each coupling pair can be described like this:\n1. Take the center point between two initial atoms\n2. Calculate all the distances to the rest of atoms in the molecule\n3. Take n-closest atoms and sort them by distance\n4. Take 2 closest atoms to create 4 core atoms\n5. Calculate all required distances using 4 core atoms\n6. Final representation is a list of tuples ordered by the distance to the center: <code>[(atom_type, distance_i_0, distance_i_1, distance_i_2, distance_i_3)]</code> </p>\n\n<p>This representation can uniquely describe each combination and is suitable for any KNN or tree-based algorithm.</p>\n\n<p>In this kernel: <a href=\"https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481\">https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481</a>\nI made a POC of this idea. So far, it's the highest score(<code>-1.481</code>) public non-blended solution.</p>\n\n<p>I used LGB and left a lot of room for improvements:\n* Score can be boosted by tuning LGB further\n* Increasing number of folds\n* Experimenting with a number of atoms for each coupling type\n* Adding other features\n* Trying to encode atom type into OHE or using CatBoost\n* Trying other tree libraries (XGB, CatBoost)\n* Try to use some type of NN (dumb FeedForward didn't work well)</p>\n\n<p>Sadly, KNN from sklearn is too slow, to deal with such amount of data. Would be cool if somebody checks the assumption, that using KNN-like approach will score better than LGB. POC can be built with Pandas + Python. Atom sequence can be encoded in one string and can be initial key for a search. And distances will be the second-order keys.</p>\n\n<h2>Caveats</h2>\n\n<p>The representation has one flaw which is obvious when looking at the relatively low cv score for <code>*JHC</code> pairs. If 3rd and 4th atoms are located symmetrically to the first two atoms and have mostly equal distance to the center, we will have jittered representation.</p>\n\n<p>That means, two very similar combinations but with slightly different 3rd and 4th atom distances will end-up with a different set of distances despite they are very similar.</p>\n\n<p>Another flaw is the same issue but in <code>*JHH</code> pairs. Other pairs have a unique direction because they are of different types. But <code>*JHH</code> have the same atoms, so we have again jittered core atoms but with the first two atoms. In practice, looks like the amount of training data compensated this flaw.</p>\n\n<h2>Summary</h2>\n\n<p>Will be cool to see forked kernels and further improvements to this idea.</p>",
  "messages": [
    {
      "id": "593879",
      "postDate": "08/07/2019 08:13:21",
      "content": "<p>As @cpmpml said, the main challenge of this competition is to find a correct representation.</p>\n\n<p>Luckily for us, organizers left us a great hint:\n<code>\nit is possible to accurately calculate scalar coupling constants given only a 3D molecular structure as input\n</code>\nThat means, having the same set of atoms in the same relative positions we can predict scalar coupling constant looking up for similar combinations of atoms in other molecules.</p>\n\n<p>The main challenge - is to convert X,Y,Z coordinates of each atom into some transition/rotation-invariant representation, so that it can be pattern matched by some KNN or boosted tree algorithm.</p>\n\n<p>The simplest invariant feature - is just a distance between two atoms. But the challenge is to uniquely represent each atom with distances.</p>\n\n<p>If we have just <code>2 atoms</code>, we can represent uniquely the combo with 3 features:\n* distance <code>d_1_0</code> - where 0, 1 are atoms indexes\n* and types for <code>atom_0</code> and <code>atom_1</code></p>\n\n<p>For <code>3 atoms</code> we need 6 features:\n* distances: <code>d_1_0</code>, <code>d_2_0</code>, <code>d_2_1</code>\n* and types for <code>atom_0</code>, <code>atom_1</code>, <code>atom_2</code></p>\n\n<p><code>4 atoms</code> with 10 features:\n* distances: <code>d_1_0</code>, <code>d_2_0</code>, <code>d_2_1</code>, <code>d_3_0</code>, <code>d_3_1</code>, <code>d_3_2</code>\n* and types for <code>atom_0</code>, <code>atom_1</code>, <code>atom_2</code>, <code>atom_3</code></p>\n\n<p>Four core atoms distances are enough to uniquely describe each next atom. So each next atom will have only 4 distances for itself. Why?</p>\n\n<p>Imagine you have 3 non-colinear points(they should create a triangle). By using 3 distances we can represent two points - kinda vertex of a pyramid. To fix this representation to just one possible point we need this 4th non-colinear distance.</p>\n\n<p>So, in the end each coupling pair can be described like this:\n1. Take the center point between two initial atoms\n2. Calculate all the distances to the rest of atoms in the molecule\n3. Take n-closest atoms and sort them by distance\n4. Take 2 closest atoms to create 4 core atoms\n5. Calculate all required distances using 4 core atoms\n6. Final representation is a list of tuples ordered by the distance to the center: <code>[(atom_type, distance_i_0, distance_i_1, distance_i_2, distance_i_3)]</code> </p>\n\n<p>This representation can uniquely describe each combination and is suitable for any KNN or tree-based algorithm.</p>\n\n<p>In this kernel: <a href=\"https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481\">https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481</a>\nI made a POC of this idea. So far, it's the highest score(<code>-1.481</code>) public non-blended solution.</p>\n\n<p>I used LGB and left a lot of room for improvements:\n* Score can be boosted by tuning LGB further\n* Increasing number of folds\n* Experimenting with a number of atoms for each coupling type\n* Adding other features\n* Trying to encode atom type into OHE or using CatBoost\n* Trying other tree libraries (XGB, CatBoost)\n* Try to use some type of NN (dumb FeedForward didn't work well)</p>\n\n<p>Sadly, KNN from sklearn is too slow, to deal with such amount of data. Would be cool if somebody checks the assumption, that using KNN-like approach will score better than LGB. POC can be built with Pandas + Python. Atom sequence can be encoded in one string and can be initial key for a search. And distances will be the second-order keys.</p>\n\n<h2>Caveats</h2>\n\n<p>The representation has one flaw which is obvious when looking at the relatively low cv score for <code>*JHC</code> pairs. If 3rd and 4th atoms are located symmetrically to the first two atoms and have mostly equal distance to the center, we will have jittered representation.</p>\n\n<p>That means, two very similar combinations but with slightly different 3rd and 4th atom distances will end-up with a different set of distances despite they are very similar.</p>\n\n<p>Another flaw is the same issue but in <code>*JHH</code> pairs. Other pairs have a unique direction because they are of different types. But <code>*JHH</code> have the same atoms, so we have again jittered core atoms but with the first two atoms. In practice, looks like the amount of training data compensated this flaw.</p>\n\n<h2>Summary</h2>\n\n<p>Will be cool to see forked kernels and further improvements to this idea.</p>",
      "rawMarkdown": "As @cpmpml said, the main challenge of this competition is to find a correct representation.\n\nLuckily for us, organizers left us a great hint:\n```\nit is possible to accurately calculate scalar coupling constants given only a 3D molecular structure as input\n```\nThat means, having the same set of atoms in the same relative positions we can predict scalar coupling constant looking up for similar combinations of atoms in other molecules.\n\nThe main challenge - is to convert X,Y,Z coordinates of each atom into some transition/rotation-invariant representation, so that it can be pattern matched by some KNN or boosted tree algorithm.\n\nThe simplest invariant feature - is just a distance between two atoms. But the challenge is to uniquely represent each atom with distances.\n\nIf we have just `2 atoms`, we can represent uniquely the combo with 3 features:\n* distance `d_1_0` - where 0, 1 are atoms indexes\n* and types for `atom_0` and `atom_1`\n\nFor `3 atoms` we need 6 features:\n* distances: `d_1_0`, `d_2_0`, `d_2_1`\n* and types for `atom_0`, `atom_1`, `atom_2`\n\n`4 atoms` with 10 features:\n* distances: `d_1_0`, `d_2_0`, `d_2_1`, `d_3_0`, `d_3_1`, `d_3_2`\n* and types for `atom_0`, `atom_1`, `atom_2`, `atom_3`\n\nFour core atoms distances are enough to uniquely describe each next atom. So each next atom will have only 4 distances for itself. Why?\n\nImagine you have 3 non-colinear points(they should create a triangle). By using 3 distances we can represent two points - kinda vertex of a pyramid. To fix this representation to just one possible point we need this 4th non-colinear distance.\n\nSo, in the end each coupling pair can be described like this:\n1. Take the center point between two initial atoms\n2. Calculate all the distances to the rest of atoms in the molecule\n3. Take n-closest atoms and sort them by distance\n4. Take 2 closest atoms to create 4 core atoms\n5. Calculate all required distances using 4 core atoms\n6. Final representation is a list of tuples ordered by the distance to the center: `[(atom_type, distance_i_0, distance_i_1, distance_i_2, distance_i_3)]` \n\nThis representation can uniquely describe each combination and is suitable for any KNN or tree-based algorithm.\n\nIn this kernel: https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481\nI made a POC of this idea. So far, it's the highest score(`-1.481`) public non-blended solution.\n\nI used LGB and left a lot of room for improvements:\n* Score can be boosted by tuning LGB further\n* Increasing number of folds\n* Experimenting with a number of atoms for each coupling type\n* Adding other features\n* Trying to encode atom type into OHE or using CatBoost\n* Trying other tree libraries (XGB, CatBoost)\n* Try to use some type of NN (dumb FeedForward didn't work well)\n\nSadly, KNN from sklearn is too slow, to deal with such amount of data. Would be cool if somebody checks the assumption, that using KNN-like approach will score better than LGB. POC can be built with Pandas + Python. Atom sequence can be encoded in one string and can be initial key for a search. And distances will be the second-order keys.\n\n## Caveats\nThe representation has one flaw which is obvious when looking at the relatively low cv score for `*JHC` pairs. If 3rd and 4th atoms are located symmetrically to the first two atoms and have mostly equal distance to the center, we will have jittered representation.\n\nThat means, two very similar combinations but with slightly different 3rd and 4th atom distances will end-up with a different set of distances despite they are very similar.\n\nAnother flaw is the same issue but in `*JHH` pairs. Other pairs have a unique direction because they are of different types. But `*JHH` have the same atoms, so we have again jittered core atoms but with the first two atoms. In practice, looks like the amount of training data compensated this flaw.\n\n## Summary\n\nWill be cool to see forked kernels and further improvements to this idea.",
      "votes": null
    },
    {
      "id": "593944",
      "postDate": "08/07/2019 10:15:24",
      "content": "<p>Thank you for sharing ! Great work</p>",
      "rawMarkdown": "Thank you for sharing ! Great work",
      "votes": null
    },
    {
      "id": "593999",
      "postDate": "08/07/2019 12:14:39",
      "content": "<blockquote>\n  <p>So far, it's the highest score(-1.481) public non-blended solution.</p>\n</blockquote>\n\n<p>Great job, and thanks for your nice write-up! I find those blended kernels quite painful to look at to be honest</p>",
      "rawMarkdown": "&gt; So far, it's the highest score(-1.481) public non-blended solution.\n\nGreat job, and thanks for your nice write-up! I find those blended kernels quite painful to look at to be honest",
      "votes": null
    },
    {
      "id": "594205",
      "postDate": "08/07/2019 17:42:46",
      "content": "<p>Nice! I've literally been setting up something similar to this the past few days too.</p>",
      "rawMarkdown": "Nice! I've literally been setting up something similar to this the past few days too.",
      "votes": null
    },
    {
      "id": "594387",
      "postDate": "08/07/2019 23:49:49",
      "content": "<p>Thanks <a href=\"/criskiev\">@criskiev</a> for sharing this insightful approach demonstrated with a kernel.</p>",
      "rawMarkdown": "Thanks @criskiev for sharing this insightful approach demonstrated with a kernel.",
      "votes": null
    },
    {
      "id": "594723",
      "postDate": "08/08/2019 10:28:27",
      "content": "<p>Beautiful solution! Most useful features I've found from the public kernel!  0.075 boosting on LB!</p>",
      "rawMarkdown": "Beautiful solution! Most useful features I've found from the public kernel!  0.075 boosting on LB!",
      "votes": null
    },
    {
      "id": "598005",
      "postDate": "08/13/2019 02:15:27",
      "content": "<p>Spectacular! Thank you for sharing.</p>",
      "rawMarkdown": "Spectacular! Thank you for sharing.",
      "votes": null
    },
    {
      "id": "598479",
      "postDate": "08/13/2019 16:13:27",
      "content": "<blockquote>\n  <p>Will be cool to see forked kernels</p>\n</blockquote>\n\n<p>Here you go :) <a href=\"https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold\">https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold</a>\nA couple of simple tweaks gain more than -1 on the public leaderboard</p>",
      "rawMarkdown": "&gt; Will be cool to see forked kernels\n\nHere you go :) https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold\nA couple of simple tweaks gain more than -1 on the public leaderboard",
      "votes": null
    },
    {
      "id": "609923",
      "postDate": "08/28/2019 08:32:22",
      "content": "<p>Thank you for sharing! Helped a lot.</p>",
      "rawMarkdown": "Thank you for sharing! Helped a lot.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 593944,
      "author_name": "duykhanh99",
      "author_url": "",
      "post_date": "08/07/2019 10:15:24",
      "content": "<p>Thank you for sharing ! Great work</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 593999,
      "author_name": "marcogorelli",
      "author_url": "",
      "post_date": "08/07/2019 12:14:39",
      "content": "<blockquote>\n  <p>So far, it's the highest score(-1.481) public non-blended solution.</p>\n</blockquote>\n\n<p>Great job, and thanks for your nice write-up! I find those blended kernels quite painful to look at to be honest</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 594205,
      "author_name": "roushrsh",
      "author_url": "",
      "post_date": "08/07/2019 17:42:46",
      "content": "<p>Nice! I've literally been setting up something similar to this the past few days too.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 594387,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "08/07/2019 23:49:49",
      "content": "<p>Thanks <a href=\"/criskiev\">@criskiev</a> for sharing this insightful approach demonstrated with a kernel.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 594723,
      "author_name": "hengzheng",
      "author_url": "",
      "post_date": "08/08/2019 10:28:27",
      "content": "<p>Beautiful solution! Most useful features I've found from the public kernel!  0.075 boosting on LB!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 598005,
      "author_name": "stocks",
      "author_url": "",
      "post_date": "08/13/2019 02:15:27",
      "content": "<p>Spectacular! Thank you for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 598479,
      "author_name": "marcogorelli",
      "author_url": "",
      "post_date": "08/13/2019 16:13:27",
      "content": "<blockquote>\n  <p>Will be cool to see forked kernels</p>\n</blockquote>\n\n<p>Here you go :) <a href=\"https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold\">https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold</a>\nA couple of simple tweaks gain more than -1 on the public leaderboard</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 609923,
      "author_name": "koerstz",
      "author_url": "",
      "post_date": "08/28/2019 08:32:22",
      "content": "<p>Thank you for sharing! Helped a lot.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "593879": "As @cpmpml said, the main challenge of this competition is to find a correct representation.\n\nLuckily for us, organizers left us a great hint:\n```\nit is possible to accurately calculate scalar coupling constants given only a 3D molecular structure as input\n```\nThat means, having the same set of atoms in the same relative positions we can predict scalar coupling constant looking up for similar combinations of atoms in other molecules.\n\nThe main challenge - is to convert X,Y,Z coordinates of each atom into some transition/rotation-invariant representation, so that it can be pattern matched by some KNN or boosted tree algorithm.\n\nThe simplest invariant feature - is just a distance between two atoms. But the challenge is to uniquely represent each atom with distances.\n\nIf we have just `2 atoms`, we can represent uniquely the combo with 3 features:\n* distance `d_1_0` - where 0, 1 are atoms indexes\n* and types for `atom_0` and `atom_1`\n\nFor `3 atoms` we need 6 features:\n* distances: `d_1_0`, `d_2_0`, `d_2_1`\n* and types for `atom_0`, `atom_1`, `atom_2`\n\n`4 atoms` with 10 features:\n* distances: `d_1_0`, `d_2_0`, `d_2_1`, `d_3_0`, `d_3_1`, `d_3_2`\n* and types for `atom_0`, `atom_1`, `atom_2`, `atom_3`\n\nFour core atoms distances are enough to uniquely describe each next atom. So each next atom will have only 4 distances for itself. Why?\n\nImagine you have 3 non-colinear points(they should create a triangle). By using 3 distances we can represent two points - kinda vertex of a pyramid. To fix this representation to just one possible point we need this 4th non-colinear distance.\n\nSo, in the end each coupling pair can be described like this:\n1. Take the center point between two initial atoms\n2. Calculate all the distances to the rest of atoms in the molecule\n3. Take n-closest atoms and sort them by distance\n4. Take 2 closest atoms to create 4 core atoms\n5. Calculate all required distances using 4 core atoms\n6. Final representation is a list of tuples ordered by the distance to the center: `[(atom_type, distance_i_0, distance_i_1, distance_i_2, distance_i_3)]` \n\nThis representation can uniquely describe each combination and is suitable for any KNN or tree-based algorithm.\n\nIn this kernel: https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481\nI made a POC of this idea. So far, it's the highest score(`-1.481`) public non-blended solution.\n\nI used LGB and left a lot of room for improvements:\n* Score can be boosted by tuning LGB further\n* Increasing number of folds\n* Experimenting with a number of atoms for each coupling type\n* Adding other features\n* Trying to encode atom type into OHE or using CatBoost\n* Trying other tree libraries (XGB, CatBoost)\n* Try to use some type of NN (dumb FeedForward didn't work well)\n\nSadly, KNN from sklearn is too slow, to deal with such amount of data. Would be cool if somebody checks the assumption, that using KNN-like approach will score better than LGB. POC can be built with Pandas + Python. Atom sequence can be encoded in one string and can be initial key for a search. And distances will be the second-order keys.\n\n## Caveats\nThe representation has one flaw which is obvious when looking at the relatively low cv score for `*JHC` pairs. If 3rd and 4th atoms are located symmetrically to the first two atoms and have mostly equal distance to the center, we will have jittered representation.\n\nThat means, two very similar combinations but with slightly different 3rd and 4th atom distances will end-up with a different set of distances despite they are very similar.\n\nAnother flaw is the same issue but in `*JHH` pairs. Other pairs have a unique direction because they are of different types. But `*JHH` have the same atoms, so we have again jittered core atoms but with the first two atoms. In practice, looks like the amount of training data compensated this flaw.\n\n## Summary\n\nWill be cool to see forked kernels and further improvements to this idea.",
    "593944": "Thank you for sharing ! Great work",
    "593999": "&gt; So far, it's the highest score(-1.481) public non-blended solution.\n\nGreat job, and thanks for your nice write-up! I find those blended kernels quite painful to look at to be honest",
    "594205": "Nice! I've literally been setting up something similar to this the past few days too.",
    "594387": "Thanks @criskiev for sharing this insightful approach demonstrated with a kernel.",
    "594723": "Beautiful solution! Most useful features I've found from the public kernel!  0.075 boosting on LB!",
    "598005": "Spectacular! Thank you for sharing.",
    "598479": "&gt; Will be cool to see forked kernels\n\nHere you go :) https://www.kaggle.com/marcogorelli/distance-is-all-you-need-groupkfold\nA couple of simple tweaks gain more than -1 on the public leaderboard",
    "609923": "Thank you for sharing! Helped a lot."
  },
  "source": "meta"
}