{
  "id": 93928,
  "title": "Helpful Information: How to obtain bonding data",
  "url": "/competitions/champs-scalar-coupling/discussion/93928",
  "author_name": "",
  "post_date": "2019-05-31T06:21:48.250206Z",
  "votes": 76,
  "comment_count": 31,
  "views": 0,
  "content": "<p>Hello, I've wrote about this in several places in the discussion but thought I'd put it here for more eyes to see. Bond information can be derived from the XYZ files in several different ways. Most molecular visualization software is able to deduce what bonds exist through a nearest neighbors approach + some logic about atomic valence.</p>\n\n<p>To automate this processes without writing a custom script, we can use a tool like <a href=\"http://openbabel.org/wiki/Main_Page\">OpenBabel</a> to convert the xyz file into something that contains bond information such as mol2. Once you have it installed, you can convert a molecule as follows:\n<code>obabel -ixyz -omol2 [input file] -O [output file]</code></p>\n\n<p>Now if we look at the mol2 we just generated, we will see two blocks of data: the ATOM block, which contains the information originally encoded in the xyz file and the BOND block which describes each bond. Column 1 is the bond ID, column 2 is the first atom ID, column 3 is the second atom ID, and last column is the bond type.</p>",
  "messages": [
    {
      "id": "540183",
      "postDate": "05/31/2019 06:21:48",
      "content": "<p>Hello, I've wrote about this in several places in the discussion but thought I'd put it here for more eyes to see. Bond information can be derived from the XYZ files in several different ways. Most molecular visualization software is able to deduce what bonds exist through a nearest neighbors approach + some logic about atomic valence.</p>\n\n<p>To automate this processes without writing a custom script, we can use a tool like <a href=\"http://openbabel.org/wiki/Main_Page\">OpenBabel</a> to convert the xyz file into something that contains bond information such as mol2. Once you have it installed, you can convert a molecule as follows:\n<code>obabel -ixyz -omol2 [input file] -O [output file]</code></p>\n\n<p>Now if we look at the mol2 we just generated, we will see two blocks of data: the ATOM block, which contains the information originally encoded in the xyz file and the BOND block which describes each bond. Column 1 is the bond ID, column 2 is the first atom ID, column 3 is the second atom ID, and last column is the bond type.</p>",
      "rawMarkdown": "Hello, I've wrote about this in several places in the discussion but thought I'd put it here for more eyes to see. Bond information can be derived from the XYZ files in several different ways. Most molecular visualization software is able to deduce what bonds exist through a nearest neighbors approach + some logic about atomic valence.\n\nTo automate this processes without writing a custom script, we can use a tool like [OpenBabel](http://openbabel.org/wiki/Main_Page) to convert the xyz file into something that contains bond information such as mol2. Once you have it installed, you can convert a molecule as follows:\n`obabel -ixyz -omol2 [input file] -O [output file]`\n\nNow if we look at the mol2 we just generated, we will see two blocks of data: the ATOM block, which contains the information originally encoded in the xyz file and the BOND block which describes each bond. Column 1 is the bond ID, column 2 is the first atom ID, column 3 is the second atom ID, and last column is the bond type.",
      "votes": null
    },
    {
      "id": "540209",
      "postDate": "05/31/2019 06:40:04",
      "content": "<p>Hi, very cool mind! I like this sharing mind of kagglers. \nLet's solve big problems in nanoscale together.</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "Hi, very cool mind! I like this sharing mind of kagglers. \nLet's solve big problems in nanoscale together.\n\nThanks!",
      "votes": null
    },
    {
      "id": "540274",
      "postDate": "05/31/2019 09:12:22",
      "content": "<p>cdjson format looks easy to process, it also preserves atom ids.</p>",
      "rawMarkdown": "cdjson format looks easy to process, it also preserves atom ids.",
      "votes": null
    },
    {
      "id": "540356",
      "postDate": "05/31/2019 11:58:38",
      "content": "<p>Thanks, MatthewMasters.  Looking forward to checking this out at some point.</p>",
      "rawMarkdown": "Thanks, MatthewMasters.  Looking forward to checking this out at some point.",
      "votes": null
    },
    {
      "id": "540381",
      "postDate": "05/31/2019 12:36:12",
      "content": "<p>Hi, I just want to add that if you would like to convert every .xyz file to a different file type using obabel, you can do so from the windows command prompt using:</p>\n\n<p>obabel *.xyz -oXXX -m         //(where XXX is the desired file type).  </p>\n\n<p>I imagine there is something similar for linux or mac</p>",
      "rawMarkdown": "Hi, I just want to add that if you would like to convert every .xyz file to a different file type using obabel, you can do so from the windows command prompt using:\n\nobabel *.xyz -oXXX -m         //(where XXX is the desired file type).  \n\nI imagine there is something similar for linux or mac",
      "votes": null
    },
    {
      "id": "540385",
      "postDate": "05/31/2019 12:49:15",
      "content": "<p><code>obabel -L formats</code> to get the available file formats.</p>",
      "rawMarkdown": "`obabel -L formats` to get the available file formats.",
      "votes": null
    },
    {
      "id": "540553",
      "postDate": "05/31/2019 16:36:13",
      "content": "<p>For those of you preferring dockerizing everything, there is a pre-built docker image:\n<code>docker run -it informaticsmatters/obabel</code></p>",
      "rawMarkdown": "For those of you preferring dockerizing everything, there is a pre-built docker image:\n`docker run -it informaticsmatters/obabel`",
      "votes": null
    },
    {
      "id": "540677",
      "postDate": "05/31/2019 20:42:19",
      "content": "<p>Will this run in python jupyter notebook?</p>",
      "rawMarkdown": "Will this run in python jupyter notebook?",
      "votes": null
    },
    {
      "id": "540722",
      "postDate": "06/01/2019 00:18:15",
      "content": "<p>Yeah, just use os.system('cmd goes here')</p>",
      "rawMarkdown": "Yeah, just use os.system('cmd goes here')",
      "votes": null
    },
    {
      "id": "544754",
      "postDate": "06/05/2019 21:44:14",
      "content": "<p>This file format contains some additional information like atom configuration, atom charge, and bonding type. Is it by the rules to use these data or does it fall under the <strong>EXTERNAL DATA</strong> of the rules? Things like atom configuration are quantum mechanical data...</p>",
      "rawMarkdown": "This file format contains some additional information like atom configuration, atom charge, and bonding type. Is it by the rules to use these data or does it fall under the **EXTERNAL DATA** of the rules? Things like atom configuration are quantum mechanical data...",
      "votes": null
    },
    {
      "id": "546247",
      "postDate": "06/06/2019 11:46:01",
      "content": "<p>Hi, Thanks for the Tip. Is there an output format reporting distances and bond angles?\nfound... in case someone needs it: report -- Open Babel report format</p>",
      "rawMarkdown": "Hi, Thanks for the Tip. Is there an output format reporting distances and bond angles?\nfound... in case someone needs it: report -- Open Babel report format",
      "votes": null
    },
    {
      "id": "546565",
      "postDate": "06/06/2019 17:30:28",
      "content": "<p>Probably an naive question given I have little background in this area. But doesn't the dataset \"train\" already give the bond with atom_index_0, atom_index_1, type, i.e. atom_index_0 is bound to atom_index_1 with bond type given by column type? Thanks,</p>",
      "rawMarkdown": "Probably an naive question given I have little background in this area. But doesn't the dataset \"train\" already give the bond with atom_index_0, atom_index_1, type, i.e. atom_index_0 is bound to atom_index_1 with bond type given by column type? Thanks,",
      "votes": null
    },
    {
      "id": "546572",
      "postDate": "06/06/2019 17:36:11",
      "content": "<p>Nope. These are simply the two atoms (of the same molecule) for which the coupling is given. They might happen to be at opposite ends of the molecule. An example is <code>dsgdb9nsd_000003</code> (H2O), where (only) the coupling of the two H atoms is in the set.</p>",
      "rawMarkdown": "Nope. These are simply the two atoms (of the same molecule) for which the coupling is given. They might happen to be at opposite ends of the molecule. An example is `dsgdb9nsd_000003` (H2O), where (only) the coupling of the two H atoms is in the set.",
      "votes": null
    },
    {
      "id": "546589",
      "postDate": "06/06/2019 17:51:56",
      "content": "<p>Thanks again for your answer. Very clear :D</p>",
      "rawMarkdown": "Thanks again for your answer. Very clear :D",
      "votes": null
    },
    {
      "id": "547473",
      "postDate": "06/07/2019 19:09:20",
      "content": "<p>Can anyone provide instructions for installing openbabel on Kaggle kernals?  I've got it working fine on my anaconda at home but I am currently mobile and working from my phone.\nCheers\nJames</p>",
      "rawMarkdown": "Can anyone provide instructions for installing openbabel on Kaggle kernals?  I've got it working fine on my anaconda at home but I am currently mobile and working from my phone.\nCheers\nJames",
      "votes": null
    },
    {
      "id": "547512",
      "postDate": "06/07/2019 20:14:06",
      "content": "<p>I don't use the kernels but this seemed to work for me:</p>\n\n<p><code>\n!conda install -c openbabel openbabel -y\n!obabel --help\n</code></p>",
      "rawMarkdown": "I don't use the kernels but this seemed to work for me:\n\n```\n!conda install -c openbabel openbabel -y\n!obabel --help\n```",
      "votes": null
    },
    {
      "id": "548174",
      "postDate": "06/08/2019 23:22:44",
      "content": "<p>I'm glad to come across your post, even since I've already made a custom script... Thanks for the link!</p>",
      "rawMarkdown": "I'm glad to come across your post, even since I've already made a custom script... Thanks for the link!",
      "votes": null
    },
    {
      "id": "553143",
      "postDate": "06/15/2019 07:21:02",
      "content": "<p>Is there are reply in another thread or is this still open?</p>",
      "rawMarkdown": "Is there are reply in another thread or is this still open?",
      "votes": null
    },
    {
      "id": "553167",
      "postDate": "06/15/2019 08:40:47",
      "content": "<p>There actually is a reply in the main thread that handles this issue: <a href=\"https://www.kaggle.com/c/champs-scalar-coupling/discussion/94457\">What constitutes a forbidden \"quantum mechanics computation\"?</a></p>\n\n<p>The answer is: <strong>yes it is allowed.</strong></p>\n\n<p>By now I've learned that OpenBabel uses heuristics to build the bondings and charges, which explains the answer. More on that is in this thread <a href=\"https://www.kaggle.com/c/champs-scalar-coupling/discussion/94755#latest-549478\">Broken charges from OpenBabel</a>. See <a href=\"/tobikaggle\">@tobikaggle</a>'s answer therein.</p>",
      "rawMarkdown": "There actually is a reply in the main thread that handles this issue: [What constitutes a forbidden \"quantum mechanics computation\"?](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94457)\n\nThe answer is: **yes it is allowed.**\n\nBy now I've learned that OpenBabel uses heuristics to build the bondings and charges, which explains the answer. More on that is in this thread [Broken charges from OpenBabel](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94755#latest-549478). See @tobikaggle's answer therein.",
      "votes": null
    },
    {
      "id": "555305",
      "postDate": "06/18/2019 18:07:31",
      "content": "<p>Maybe a trivial addition, but:    </p>\n\n<p>you can read .mol2 files using Biopandas (<a href=\"http://rasbt.github.io/biopandas\">http://rasbt.github.io/biopandas</a> ): </p>\n\n<p><code>\nfrom biopandas.mol2 import PandasMol2\npmol = PandasMol2().read_mol2('./data/xxx.mol2')\n</code>\n(firstly you need to install it, say <br>\npip install biopandas </p>\n\n<p>then .mol2 file can be seen as in usual panda dataframe:</p>\n\n<blockquote>\n  <p>pmol.df.head(5)</p>\n</blockquote>",
      "rawMarkdown": "Maybe a trivial addition, but:    \n\nyou can read .mol2 files using Biopandas (http://rasbt.github.io/biopandas ): \n\n```\nfrom biopandas.mol2 import PandasMol2\npmol = PandasMol2().read_mol2('./data/xxx.mol2')\n```\n(firstly you need to install it, say  \npip install biopandas \n\nthen .mol2 file can be seen as in usual panda dataframe:\n\n&gt; pmol.df.head(5)",
      "votes": null
    },
    {
      "id": "555344",
      "postDate": "06/18/2019 19:27:59",
      "content": "<p>Links to good stuff are always welcome, thanks!</p>",
      "rawMarkdown": "Links to good stuff are always welcome, thanks!",
      "votes": null
    },
    {
      "id": "557694",
      "postDate": "06/21/2019 12:01:09",
      "content": "<p>Hi <a href=\"/bernir\">@bernir</a>, thanks for the clarification. How do you associate molecule names to real molecules <code>dsgdb9nsd_000003</code> -&gt; <code>H2O</code> ??</p>",
      "rawMarkdown": "Hi @bernir, thanks for the clarification. How do you associate molecule names to real molecules `dsgdb9nsd_000003` -&gt; `H2O` ??",
      "votes": null
    },
    {
      "id": "557704",
      "postDate": "06/21/2019 12:11:10",
      "content": "<p>There are some online database for that but the quickest way is to google the raw formula</p>",
      "rawMarkdown": "There are some online database for that but the quickest way is to google the raw formula",
      "votes": null
    },
    {
      "id": "557724",
      "postDate": "06/21/2019 12:31:10",
      "content": "<p>I don't do that on a large scale / for all molecules. I've looked at a few individual molecules for different reasons and some are just obvious (like <code>H2O</code>). It might, however, be useful to add molecule information  like triple points or such to your training data.</p>",
      "rawMarkdown": "I don't do that on a large scale / for all molecules. I've looked at a few individual molecules for different reasons and some are just obvious (like `H2O`). It might, however, be useful to add molecule information  like triple points or such to your training data.",
      "votes": null
    },
    {
      "id": "557730",
      "postDate": "06/21/2019 12:34:01",
      "content": "<p>I see ... by raw formula do you mean <code>dsgdb9nsd_000003</code>?</p>\n\n<p>I believe, there is a lot of information in knowing what molecule we are talking about in our model ...</p>",
      "rawMarkdown": "I see ... by raw formula do you mean `dsgdb9nsd_000003`?\n\nI believe, there is a lot of information in knowing what molecule we are talking about in our model ...",
      "votes": null
    },
    {
      "id": "557745",
      "postDate": "06/21/2019 12:45:41",
      "content": "<p>Well, we have not just the 'name' (which, I think, is just some arbitrary ID), we have the full structure data: type of atoms and their positions. We already have the information that there are two hydrogen atoms and one oxygen atom for <code>dsgdb9nsd_000003</code>. Some very basic chemistry knowledge is enough to know that this can only be water (sometimes there are different configurations, that is different molecules, for a given set of atoms).</p>\n\n<p>Of course, knowing this particular molecule helps... But that's kind of part of the challenge. We also have potential energy for each molecule in the training set. So you can train a model to predict potential energy. Crawling that information from the net, or having some other kind of program to get this information, is probably not a good approach, because it is unlikely that you get it for each and every molecule.</p>",
      "rawMarkdown": "Well, we have not just the 'name' (which, I think, is just some arbitrary ID), we have the full structure data: type of atoms and their positions. We already have the information that there are two hydrogen atoms and one oxygen atom for `dsgdb9nsd_000003`. Some very basic chemistry knowledge is enough to know that this can only be water (sometimes there are different configurations, that is different molecules, for a given set of atoms).\n\nOf course, knowing this particular molecule helps... But that's kind of part of the challenge. We also have potential energy for each molecule in the training set. So you can train a model to predict potential energy. Crawling that information from the net, or having some other kind of program to get this information, is probably not a good approach, because it is unlikely that you get it for each and every molecule.",
      "votes": null
    },
    {
      "id": "562015",
      "postDate": "06/26/2019 23:37:01",
      "content": "<p>thank you for sharing.</p>",
      "rawMarkdown": "thank you for sharing.",
      "votes": null
    },
    {
      "id": "562391",
      "postDate": "06/27/2019 08:04:46",
      "content": "<p>Even easyer to launch a command in  a notebook: <br>\n<code>! cmd goes here</code></p>",
      "rawMarkdown": "Even easyer to launch a command in  a notebook:   \n`! cmd goes here`",
      "votes": null
    },
    {
      "id": "562792",
      "postDate": "06/27/2019 14:40:43",
      "content": "<p>thanks!</p>",
      "rawMarkdown": "thanks!",
      "votes": null
    },
    {
      "id": "563950",
      "postDate": "06/28/2019 20:09:24",
      "content": "<p>Although I do not think that it will help in this competition, the usual way to get the name of a molecule from a structure is by creating the so-called InChI key, which is a (mostly) unique identifier for molecules, e.g. with open babel:</p>\n\n<p>obabel -ixyz dsgdb9nsd_000010.xyz -oinchi -xK</p>\n\n<p>WEVYAHXRMPXWCK-UHFFFAOYSA-N</p>\n\n<p>Numerous webpages are able to convert the inchi key to a name (if there is one...). Acetonitrile in this case.</p>",
      "rawMarkdown": "Although I do not think that it will help in this competition, the usual way to get the name of a molecule from a structure is by creating the so-called InChI key, which is a (mostly) unique identifier for molecules, e.g. with open babel:\n\nobabel -ixyz dsgdb9nsd_000010.xyz -oinchi -xK\n\nWEVYAHXRMPXWCK-UHFFFAOYSA-N\n\nNumerous webpages are able to convert the inchi key to a name (if there is one...). Acetonitrile in this case.",
      "votes": null
    },
    {
      "id": "565218",
      "postDate": "06/30/2019 15:58:27",
      "content": "<p>This is a nice way of getting molecule name, thanks</p>",
      "rawMarkdown": "This is a nice way of getting molecule name, thanks",
      "votes": null
    },
    {
      "id": "588773",
      "postDate": "07/31/2019 02:44:14",
      "content": "<p>atom_index changes if you try to aggregate the .xyz files into one and then try to extract the edge information. Don't do that. Process individual xyz files to get edge data. Also, multiprocessing yields incorrect data (joblib).  </p>",
      "rawMarkdown": "atom_index changes if you try to aggregate the .xyz files into one and then try to extract the edge information. Don't do that. Process individual xyz files to get edge data. Also, multiprocessing yields incorrect data (joblib).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 540209,
      "author_name": "youhanlee",
      "author_url": "",
      "post_date": "05/31/2019 06:40:04",
      "content": "<p>Hi, very cool mind! I like this sharing mind of kagglers. \nLet's solve big problems in nanoscale together.</p>\n\n<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 540274,
      "author_name": "alexfir",
      "author_url": "",
      "post_date": "05/31/2019 09:12:22",
      "content": "<p>cdjson format looks easy to process, it also preserves atom ids.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 540356,
      "author_name": "abendorf",
      "author_url": "",
      "post_date": "05/31/2019 11:58:38",
      "content": "<p>Thanks, MatthewMasters.  Looking forward to checking this out at some point.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 540381,
      "author_name": "tblack",
      "author_url": "",
      "post_date": "05/31/2019 12:36:12",
      "content": "<p>Hi, I just want to add that if you would like to convert every .xyz file to a different file type using obabel, you can do so from the windows command prompt using:</p>\n\n<p>obabel *.xyz -oXXX -m         //(where XXX is the desired file type).  </p>\n\n<p>I imagine there is something similar for linux or mac</p>",
      "votes": null,
      "replies": [
        {
          "id": 540385,
          "author_name": "larsbratholm",
          "author_url": "",
          "post_date": "05/31/2019 12:49:15",
          "content": "<p><code>obabel -L formats</code> to get the available file formats.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 544754,
          "author_name": "bernir",
          "author_url": "",
          "post_date": "06/05/2019 21:44:14",
          "content": "<p>This file format contains some additional information like atom configuration, atom charge, and bonding type. Is it by the rules to use these data or does it fall under the <strong>EXTERNAL DATA</strong> of the rules? Things like atom configuration are quantum mechanical data...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 553143,
          "author_name": "micpie",
          "author_url": "",
          "post_date": "06/15/2019 07:21:02",
          "content": "<p>Is there are reply in another thread or is this still open?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 553167,
          "author_name": "bernir",
          "author_url": "",
          "post_date": "06/15/2019 08:40:47",
          "content": "<p>There actually is a reply in the main thread that handles this issue: <a href=\"https://www.kaggle.com/c/champs-scalar-coupling/discussion/94457\">What constitutes a forbidden \"quantum mechanics computation\"?</a></p>\n\n<p>The answer is: <strong>yes it is allowed.</strong></p>\n\n<p>By now I've learned that OpenBabel uses heuristics to build the bondings and charges, which explains the answer. More on that is in this thread <a href=\"https://www.kaggle.com/c/champs-scalar-coupling/discussion/94755#latest-549478\">Broken charges from OpenBabel</a>. See <a href=\"/tobikaggle\">@tobikaggle</a>'s answer therein.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 540553,
      "author_name": "andris",
      "author_url": "",
      "post_date": "05/31/2019 16:36:13",
      "content": "<p>For those of you preferring dockerizing everything, there is a pre-built docker image:\n<code>docker run -it informaticsmatters/obabel</code></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 540677,
      "author_name": "lujainfl",
      "author_url": "",
      "post_date": "05/31/2019 20:42:19",
      "content": "<p>Will this run in python jupyter notebook?</p>",
      "votes": null,
      "replies": [
        {
          "id": 540722,
          "author_name": "matthewmasters",
          "author_url": "",
          "post_date": "06/01/2019 00:18:15",
          "content": "<p>Yeah, just use os.system('cmd goes here')</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 562391,
          "author_name": "asauve",
          "author_url": "",
          "post_date": "06/27/2019 08:04:46",
          "content": "<p>Even easyer to launch a command in  a notebook: <br>\n<code>! cmd goes here</code></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 546247,
      "author_name": "mtinti",
      "author_url": "",
      "post_date": "06/06/2019 11:46:01",
      "content": "<p>Hi, Thanks for the Tip. Is there an output format reporting distances and bond angles?\nfound... in case someone needs it: report -- Open Babel report format</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 546565,
      "author_name": "zhenlanwang",
      "author_url": "",
      "post_date": "06/06/2019 17:30:28",
      "content": "<p>Probably an naive question given I have little background in this area. But doesn't the dataset \"train\" already give the bond with atom_index_0, atom_index_1, type, i.e. atom_index_0 is bound to atom_index_1 with bond type given by column type? Thanks,</p>",
      "votes": null,
      "replies": [
        {
          "id": 546572,
          "author_name": "bernir",
          "author_url": "",
          "post_date": "06/06/2019 17:36:11",
          "content": "<p>Nope. These are simply the two atoms (of the same molecule) for which the coupling is given. They might happen to be at opposite ends of the molecule. An example is <code>dsgdb9nsd_000003</code> (H2O), where (only) the coupling of the two H atoms is in the set.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 546589,
          "author_name": "zhenlanwang",
          "author_url": "",
          "post_date": "06/06/2019 17:51:56",
          "content": "<p>Thanks again for your answer. Very clear :D</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 557694,
          "author_name": "alvarocalle",
          "author_url": "",
          "post_date": "06/21/2019 12:01:09",
          "content": "<p>Hi <a href=\"/bernir\">@bernir</a>, thanks for the clarification. How do you associate molecule names to real molecules <code>dsgdb9nsd_000003</code> -&gt; <code>H2O</code> ??</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 557704,
          "author_name": "asauve",
          "author_url": "",
          "post_date": "06/21/2019 12:11:10",
          "content": "<p>There are some online database for that but the quickest way is to google the raw formula</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 557724,
          "author_name": "bernir",
          "author_url": "",
          "post_date": "06/21/2019 12:31:10",
          "content": "<p>I don't do that on a large scale / for all molecules. I've looked at a few individual molecules for different reasons and some are just obvious (like <code>H2O</code>). It might, however, be useful to add molecule information  like triple points or such to your training data.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 557730,
          "author_name": "alvarocalle",
          "author_url": "",
          "post_date": "06/21/2019 12:34:01",
          "content": "<p>I see ... by raw formula do you mean <code>dsgdb9nsd_000003</code>?</p>\n\n<p>I believe, there is a lot of information in knowing what molecule we are talking about in our model ...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 557745,
          "author_name": "bernir",
          "author_url": "",
          "post_date": "06/21/2019 12:45:41",
          "content": "<p>Well, we have not just the 'name' (which, I think, is just some arbitrary ID), we have the full structure data: type of atoms and their positions. We already have the information that there are two hydrogen atoms and one oxygen atom for <code>dsgdb9nsd_000003</code>. Some very basic chemistry knowledge is enough to know that this can only be water (sometimes there are different configurations, that is different molecules, for a given set of atoms).</p>\n\n<p>Of course, knowing this particular molecule helps... But that's kind of part of the challenge. We also have potential energy for each molecule in the training set. So you can train a model to predict potential energy. Crawling that information from the net, or having some other kind of program to get this information, is probably not a good approach, because it is unlikely that you get it for each and every molecule.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 547473,
      "author_name": "jjohnsumn",
      "author_url": "",
      "post_date": "06/07/2019 19:09:20",
      "content": "<p>Can anyone provide instructions for installing openbabel on Kaggle kernals?  I've got it working fine on my anaconda at home but I am currently mobile and working from my phone.\nCheers\nJames</p>",
      "votes": null,
      "replies": [
        {
          "id": 547512,
          "author_name": "matthewmasters",
          "author_url": "",
          "post_date": "06/07/2019 20:14:06",
          "content": "<p>I don't use the kernels but this seemed to work for me:</p>\n\n<p><code>\n!conda install -c openbabel openbabel -y\n!obabel --help\n</code></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 548174,
      "author_name": "asauve",
      "author_url": "",
      "post_date": "06/08/2019 23:22:44",
      "content": "<p>I'm glad to come across your post, even since I've already made a custom script... Thanks for the link!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 555305,
      "author_name": "burundukchip",
      "author_url": "",
      "post_date": "06/18/2019 18:07:31",
      "content": "<p>Maybe a trivial addition, but:    </p>\n\n<p>you can read .mol2 files using Biopandas (<a href=\"http://rasbt.github.io/biopandas\">http://rasbt.github.io/biopandas</a> ): </p>\n\n<p><code>\nfrom biopandas.mol2 import PandasMol2\npmol = PandasMol2().read_mol2('./data/xxx.mol2')\n</code>\n(firstly you need to install it, say <br>\npip install biopandas </p>\n\n<p>then .mol2 file can be seen as in usual panda dataframe:</p>\n\n<blockquote>\n  <p>pmol.df.head(5)</p>\n</blockquote>",
      "votes": null,
      "replies": [
        {
          "id": 555344,
          "author_name": "asauve",
          "author_url": "",
          "post_date": "06/18/2019 19:27:59",
          "content": "<p>Links to good stuff are always welcome, thanks!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 562792,
          "author_name": "jameschapman19",
          "author_url": "",
          "post_date": "06/27/2019 14:40:43",
          "content": "<p>thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 562015,
      "author_name": "yslee04",
      "author_url": "",
      "post_date": "06/26/2019 23:37:01",
      "content": "<p>thank you for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 563950,
      "author_name": "chrisslz",
      "author_url": "",
      "post_date": "06/28/2019 20:09:24",
      "content": "<p>Although I do not think that it will help in this competition, the usual way to get the name of a molecule from a structure is by creating the so-called InChI key, which is a (mostly) unique identifier for molecules, e.g. with open babel:</p>\n\n<p>obabel -ixyz dsgdb9nsd_000010.xyz -oinchi -xK</p>\n\n<p>WEVYAHXRMPXWCK-UHFFFAOYSA-N</p>\n\n<p>Numerous webpages are able to convert the inchi key to a name (if there is one...). Acetonitrile in this case.</p>",
      "votes": null,
      "replies": [
        {
          "id": 565218,
          "author_name": "asauve",
          "author_url": "",
          "post_date": "06/30/2019 15:58:27",
          "content": "<p>This is a nice way of getting molecule name, thanks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 588773,
      "author_name": "silence2",
      "author_url": "",
      "post_date": "07/31/2019 02:44:14",
      "content": "<p>atom_index changes if you try to aggregate the .xyz files into one and then try to extract the edge information. Don't do that. Process individual xyz files to get edge data. Also, multiprocessing yields incorrect data (joblib).  </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "540183": "Hello, I've wrote about this in several places in the discussion but thought I'd put it here for more eyes to see. Bond information can be derived from the XYZ files in several different ways. Most molecular visualization software is able to deduce what bonds exist through a nearest neighbors approach + some logic about atomic valence.\n\nTo automate this processes without writing a custom script, we can use a tool like [OpenBabel](http://openbabel.org/wiki/Main_Page) to convert the xyz file into something that contains bond information such as mol2. Once you have it installed, you can convert a molecule as follows:\n`obabel -ixyz -omol2 [input file] -O [output file]`\n\nNow if we look at the mol2 we just generated, we will see two blocks of data: the ATOM block, which contains the information originally encoded in the xyz file and the BOND block which describes each bond. Column 1 is the bond ID, column 2 is the first atom ID, column 3 is the second atom ID, and last column is the bond type.",
    "540209": "Hi, very cool mind! I like this sharing mind of kagglers. \nLet's solve big problems in nanoscale together.\n\nThanks!",
    "540274": "cdjson format looks easy to process, it also preserves atom ids.",
    "540356": "Thanks, MatthewMasters.  Looking forward to checking this out at some point.",
    "540381": "Hi, I just want to add that if you would like to convert every .xyz file to a different file type using obabel, you can do so from the windows command prompt using:\n\nobabel *.xyz -oXXX -m         //(where XXX is the desired file type).  \n\nI imagine there is something similar for linux or mac",
    "540385": "`obabel -L formats` to get the available file formats.",
    "540553": "For those of you preferring dockerizing everything, there is a pre-built docker image:\n`docker run -it informaticsmatters/obabel`",
    "540677": "Will this run in python jupyter notebook?",
    "540722": "Yeah, just use os.system('cmd goes here')",
    "544754": "This file format contains some additional information like atom configuration, atom charge, and bonding type. Is it by the rules to use these data or does it fall under the **EXTERNAL DATA** of the rules? Things like atom configuration are quantum mechanical data...",
    "546247": "Hi, Thanks for the Tip. Is there an output format reporting distances and bond angles?\nfound... in case someone needs it: report -- Open Babel report format",
    "546565": "Probably an naive question given I have little background in this area. But doesn't the dataset \"train\" already give the bond with atom_index_0, atom_index_1, type, i.e. atom_index_0 is bound to atom_index_1 with bond type given by column type? Thanks,",
    "546572": "Nope. These are simply the two atoms (of the same molecule) for which the coupling is given. They might happen to be at opposite ends of the molecule. An example is `dsgdb9nsd_000003` (H2O), where (only) the coupling of the two H atoms is in the set.",
    "546589": "Thanks again for your answer. Very clear :D",
    "547473": "Can anyone provide instructions for installing openbabel on Kaggle kernals?  I've got it working fine on my anaconda at home but I am currently mobile and working from my phone.\nCheers\nJames",
    "547512": "I don't use the kernels but this seemed to work for me:\n\n```\n!conda install -c openbabel openbabel -y\n!obabel --help\n```",
    "548174": "I'm glad to come across your post, even since I've already made a custom script... Thanks for the link!",
    "553143": "Is there are reply in another thread or is this still open?",
    "553167": "There actually is a reply in the main thread that handles this issue: [What constitutes a forbidden \"quantum mechanics computation\"?](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94457)\n\nThe answer is: **yes it is allowed.**\n\nBy now I've learned that OpenBabel uses heuristics to build the bondings and charges, which explains the answer. More on that is in this thread [Broken charges from OpenBabel](https://www.kaggle.com/c/champs-scalar-coupling/discussion/94755#latest-549478). See @tobikaggle's answer therein.",
    "555305": "Maybe a trivial addition, but:    \n\nyou can read .mol2 files using Biopandas (http://rasbt.github.io/biopandas ): \n\n```\nfrom biopandas.mol2 import PandasMol2\npmol = PandasMol2().read_mol2('./data/xxx.mol2')\n```\n(firstly you need to install it, say  \npip install biopandas \n\nthen .mol2 file can be seen as in usual panda dataframe:\n\n&gt; pmol.df.head(5)",
    "555344": "Links to good stuff are always welcome, thanks!",
    "557694": "Hi @bernir, thanks for the clarification. How do you associate molecule names to real molecules `dsgdb9nsd_000003` -&gt; `H2O` ??",
    "557704": "There are some online database for that but the quickest way is to google the raw formula",
    "557724": "I don't do that on a large scale / for all molecules. I've looked at a few individual molecules for different reasons and some are just obvious (like `H2O`). It might, however, be useful to add molecule information  like triple points or such to your training data.",
    "557730": "I see ... by raw formula do you mean `dsgdb9nsd_000003`?\n\nI believe, there is a lot of information in knowing what molecule we are talking about in our model ...",
    "557745": "Well, we have not just the 'name' (which, I think, is just some arbitrary ID), we have the full structure data: type of atoms and their positions. We already have the information that there are two hydrogen atoms and one oxygen atom for `dsgdb9nsd_000003`. Some very basic chemistry knowledge is enough to know that this can only be water (sometimes there are different configurations, that is different molecules, for a given set of atoms).\n\nOf course, knowing this particular molecule helps... But that's kind of part of the challenge. We also have potential energy for each molecule in the training set. So you can train a model to predict potential energy. Crawling that information from the net, or having some other kind of program to get this information, is probably not a good approach, because it is unlikely that you get it for each and every molecule.",
    "562015": "thank you for sharing.",
    "562391": "Even easyer to launch a command in  a notebook:   \n`! cmd goes here`",
    "562792": "thanks!",
    "563950": "Although I do not think that it will help in this competition, the usual way to get the name of a molecule from a structure is by creating the so-called InChI key, which is a (mostly) unique identifier for molecules, e.g. with open babel:\n\nobabel -ixyz dsgdb9nsd_000010.xyz -oinchi -xK\n\nWEVYAHXRMPXWCK-UHFFFAOYSA-N\n\nNumerous webpages are able to convert the inchi key to a name (if there is one...). Acetonitrile in this case.",
    "565218": "This is a nice way of getting molecule name, thanks",
    "588773": "atom_index changes if you try to aggregate the .xyz files into one and then try to extract the edge information. Don't do that. Process individual xyz files to get edge data. Also, multiprocessing yields incorrect data (joblib)."
  },
  "source": "meta"
}