{
  "id": 97826,
  "title": "Augmentation",
  "url": "/competitions/champs-scalar-coupling/discussion/97826",
  "author_name": "",
  "post_date": "2019-06-29T04:58:25.064917800Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p><img src=\"https://media.springernature.com/lw785/springer-static/image/art%3A10.1007%2Fs00214-016-1951-4/MediaObjects/214_2016_1951_Fig10_HTML.gif\" alt=\"\"></p>\n\n<p>Augmentation like this may be useful. How can we get the ground truth of such geometric perturbed moelcules? Any chemist can suggest here? (i presume using QM software to compute J coupling of augmented sample is illegal here?) </p>\n\n<p><a href=\"https://link.springer.com/article/10.1007/s00214-016-1951-4\">https://link.springer.com/article/10.1007/s00214-016-1951-4</a></p>",
  "messages": [
    {
      "id": "564178",
      "postDate": "06/29/2019 04:58:25",
      "content": "<p><img src=\"https://media.springernature.com/lw785/springer-static/image/art%3A10.1007%2Fs00214-016-1951-4/MediaObjects/214_2016_1951_Fig10_HTML.gif\" alt=\"\"></p>\n\n<p>Augmentation like this may be useful. How can we get the ground truth of such geometric perturbed moelcules? Any chemist can suggest here? (i presume using QM software to compute J coupling of augmented sample is illegal here?) </p>\n\n<p><a href=\"https://link.springer.com/article/10.1007/s00214-016-1951-4\">https://link.springer.com/article/10.1007/s00214-016-1951-4</a></p>",
      "rawMarkdown": "![](https://media.springernature.com/lw785/springer-static/image/art%3A10.1007%2Fs00214-016-1951-4/MediaObjects/214_2016_1951_Fig10_HTML.gif)\n\nAugmentation like this may be useful. How can we get the ground truth of such geometric perturbed moelcules? Any chemist can suggest here? (i presume using QM software to compute J coupling of augmented sample is illegal here?) \n\nhttps://link.springer.com/article/10.1007/s00214-016-1951-4",
      "votes": null
    },
    {
      "id": "564209",
      "postDate": "06/29/2019 06:06:13",
      "content": "<p>I'm not a chemist but your suggestion and the article bring to mind a bunch of stuff that I'll try to elucidate here. </p>\n\n<p>The article describes using kriging to predict atomic properties from molecular geometry. Kriging is also known as Gaussian process regression and is essentially an interpolation method. There is a Python library for this: <a href=\"https://github.com/GPflow/GPflow\">GPFlow</a> that could in theory be applied, but I have found that it doesn't scale well.</p>\n\n<p>From my PhD days, I vaguely remember a regularization technique inspired by kriging that perturbed the features of each training example but kept the label the same. This is locally constant interpolation. It can be shown to be equivalent to a form of Tikhonov regularization, as can weight decay in neural networks, and using l1 and l2 regularization in LightGBM. In contrast to kriging, which unbiased, this method increases the bias but reduces the variance.</p>\n\n<p>This brings me closer to your question, instead of perturbing the molecule and keeping the label, can we estimate the label better. One possibility would be to use weak supervised learning. For example, suppose we've trained a GCN with regression head to predict the scalar coupling constant. We could use another technique, such as LightGBM with ACSFs (kernel coming soon!) to estimate the scalar coupling constant of geometrically perturbed molecules. This is a variation on pseudo-labelling. Since the resulting labels will be noisy, we gain some regularization for free.</p>\n\n<p>This still leaves a question for the chemists: what perturbations make sense?</p>",
      "rawMarkdown": "I'm not a chemist but your suggestion and the article bring to mind a bunch of stuff that I'll try to elucidate here. \n\nThe article describes using kriging to predict atomic properties from molecular geometry. Kriging is also known as Gaussian process regression and is essentially an interpolation method. There is a Python library for this: [GPFlow](https://github.com/GPflow/GPflow) that could in theory be applied, but I have found that it doesn't scale well.\n\nFrom my PhD days, I vaguely remember a regularization technique inspired by kriging that perturbed the features of each training example but kept the label the same. This is locally constant interpolation. It can be shown to be equivalent to a form of Tikhonov regularization, as can weight decay in neural networks, and using l1 and l2 regularization in LightGBM. In contrast to kriging, which unbiased, this method increases the bias but reduces the variance.\n\nThis brings me closer to your question, instead of perturbing the molecule and keeping the label, can we estimate the label better. One possibility would be to use weak supervised learning. For example, suppose we've trained a GCN with regression head to predict the scalar coupling constant. We could use another technique, such as LightGBM with ACSFs (kernel coming soon!) to estimate the scalar coupling constant of geometrically perturbed molecules. This is a variation on pseudo-labelling. Since the resulting labels will be noisy, we gain some regularization for free.\n\nThis still leaves a question for the chemists: what perturbations make sense?",
      "votes": null
    },
    {
      "id": "565748",
      "postDate": "07/01/2019 10:30:24",
      "content": "<blockquote>\n  <p>i presume using QM software to compute J coupling of augmented sample is illegal here?</p>\n</blockquote>\n\n<p>As I understood, the point of the competition is to avoid expensive QM computation :)</p>",
      "rawMarkdown": "&gt; i presume using QM software to compute J coupling of augmented sample is illegal here?\n\nAs I understood, the point of the competition is to avoid expensive QM computation :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 564209,
      "author_name": "robertburbidge",
      "author_url": "",
      "post_date": "06/29/2019 06:06:13",
      "content": "<p>I'm not a chemist but your suggestion and the article bring to mind a bunch of stuff that I'll try to elucidate here. </p>\n\n<p>The article describes using kriging to predict atomic properties from molecular geometry. Kriging is also known as Gaussian process regression and is essentially an interpolation method. There is a Python library for this: <a href=\"https://github.com/GPflow/GPflow\">GPFlow</a> that could in theory be applied, but I have found that it doesn't scale well.</p>\n\n<p>From my PhD days, I vaguely remember a regularization technique inspired by kriging that perturbed the features of each training example but kept the label the same. This is locally constant interpolation. It can be shown to be equivalent to a form of Tikhonov regularization, as can weight decay in neural networks, and using l1 and l2 regularization in LightGBM. In contrast to kriging, which unbiased, this method increases the bias but reduces the variance.</p>\n\n<p>This brings me closer to your question, instead of perturbing the molecule and keeping the label, can we estimate the label better. One possibility would be to use weak supervised learning. For example, suppose we've trained a GCN with regression head to predict the scalar coupling constant. We could use another technique, such as LightGBM with ACSFs (kernel coming soon!) to estimate the scalar coupling constant of geometrically perturbed molecules. This is a variation on pseudo-labelling. Since the resulting labels will be noisy, we gain some regularization for free.</p>\n\n<p>This still leaves a question for the chemists: what perturbations make sense?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 565748,
      "author_name": "raimonds1993",
      "author_url": "",
      "post_date": "07/01/2019 10:30:24",
      "content": "<blockquote>\n  <p>i presume using QM software to compute J coupling of augmented sample is illegal here?</p>\n</blockquote>\n\n<p>As I understood, the point of the competition is to avoid expensive QM computation :)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "564178": "![](https://media.springernature.com/lw785/springer-static/image/art%3A10.1007%2Fs00214-016-1951-4/MediaObjects/214_2016_1951_Fig10_HTML.gif)\n\nAugmentation like this may be useful. How can we get the ground truth of such geometric perturbed moelcules? Any chemist can suggest here? (i presume using QM software to compute J coupling of augmented sample is illegal here?) \n\nhttps://link.springer.com/article/10.1007/s00214-016-1951-4",
    "564209": "I'm not a chemist but your suggestion and the article bring to mind a bunch of stuff that I'll try to elucidate here. \n\nThe article describes using kriging to predict atomic properties from molecular geometry. Kriging is also known as Gaussian process regression and is essentially an interpolation method. There is a Python library for this: [GPFlow](https://github.com/GPflow/GPflow) that could in theory be applied, but I have found that it doesn't scale well.\n\nFrom my PhD days, I vaguely remember a regularization technique inspired by kriging that perturbed the features of each training example but kept the label the same. This is locally constant interpolation. It can be shown to be equivalent to a form of Tikhonov regularization, as can weight decay in neural networks, and using l1 and l2 regularization in LightGBM. In contrast to kriging, which unbiased, this method increases the bias but reduces the variance.\n\nThis brings me closer to your question, instead of perturbing the molecule and keeping the label, can we estimate the label better. One possibility would be to use weak supervised learning. For example, suppose we've trained a GCN with regression head to predict the scalar coupling constant. We could use another technique, such as LightGBM with ACSFs (kernel coming soon!) to estimate the scalar coupling constant of geometrically perturbed molecules. This is a variation on pseudo-labelling. Since the resulting labels will be noisy, we gain some regularization for free.\n\nThis still leaves a question for the chemists: what perturbations make sense?",
    "565748": "&gt; i presume using QM software to compute J coupling of augmented sample is illegal here?\n\nAs I understood, the point of the competition is to avoid expensive QM computation :)"
  },
  "source": "meta"
}