![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F40954%2F3b3f81d749753dce7b6c312cdacd9694%2Fatom_indices2.png?generation=1562490526355229&alt=media\")
I want to preface this post by saying I have nothing against ods.ai in general. I am a part of the slack group and find conversations there are generally well intentioned. I don't post there- but I do read the competition specific channels (through google translate).
\n\nThat being said, I recently read the thread below- found it interesting, and wanted to bring the idea to this forum.
\n\nThe idea is that atom_index may shed some information about the target. My understanding is that the index is just an arbitrary way to identify each atom. Curious to hear if anyone has come across something to show otherwise.
Google translated text:\n```\nmephistopheies [Yesterday at 9:09 AM]\nKaroch, I do not catch up with how at all the indexing of atoms is done, is it deterministic or not?\n14 replies
\n\niggisv9t [1 day ago]\nIs it very important?
\n\nartgor [1 day ago]\nOn the one hand, indexing is generally random — it simply means an index from a file with structures.\nOn the other hand, if you build graphs of the distribution of the target by indices, it turns out that the distributions are very different
\n\nmephistopheies [1 day ago]\nI'm strained by the fact that the features at the ends of the ribs work well, for example, primitive type by byg agg
\n\nif the order is random, it opens up the possibility of almost infinite augmentation, and it looks like something dumb
\n\nmephistopheies [1 day ago]\nI assume that there is some sort of numbering algorithm, and here it is interesting to understand what kind of algorithm it is.
\n\nmephistopheies [1 day ago]\nif it is, then probably somewhere in the paper chemists there is a description
\n\nil_buono [1 day ago]\nI think that the order is not random. It may coincide with the order in the SMILES record. But did not check.
\n\niggisv9t [1 day ago]\nI do everything based on the fact that the order is random. You can think of your order. It’s just the only reasonable assumption that it occurred to me that the order depends on orientation in space, which means that you can rotate the molecule and renumber it differently.
\n\niggisv9t [1 day ago]\nThat is, it seems to me that it will be safer to number them somehow in their own way.
\n\niggisv9t [1 day ago]\ngroup by (atom index) agg - is it on the whole molecule or inside the train and the test? (edited)
vadym.safronov [1 day ago]\nOdd was recalled the famous suitcase master Dmitry Ivanovich Mendeleev with his periodic table and groups of elements with similar properties depending on the sequence number.
\n\nkudasova [1 day ago]\nNot exactly checked, but it seems the indexes are sorted by the number of bonds with other atoms, as in tinker xyz files. Here on the example of ethanol http://chembytes.wikidot.com/tnk-tut00
\n\niggisv9t [1 day ago]\n@ vadym.safronov not looking at these memories is dangerous to voice.
\n\nvadym.safronov [1 day ago]\nNorm - in the local culinary circle, also speaking on the theoretical part
\n\nSergey Zlobin [1 day ago]\nThat is, the atom index makes sense to use as a feature? I don’t really like it, but if there’s some sort of sorting: thinking_face:\n```
", "messages": [ { "id": "569606", "postDate": "07/07/2019 02:25:59", "content": "I want to preface this post by saying I have nothing against ods.ai in general. I am a part of the slack group and find conversations there are generally well intentioned. I don't post there- but I do read the competition specific channels (through google translate).
\n\nThat being said, I recently read the thread below- found it interesting, and wanted to bring the idea to this forum.
\n\nThe idea is that atom_index may shed some information about the target. My understanding is that the index is just an arbitrary way to identify each atom. Curious to hear if anyone has come across something to show otherwise.
Google translated text:\n```\nmephistopheies [Yesterday at 9:09 AM]\nKaroch, I do not catch up with how at all the indexing of atoms is done, is it deterministic or not?\n14 replies
\n\niggisv9t [1 day ago]\nIs it very important?
\n\nartgor [1 day ago]\nOn the one hand, indexing is generally random — it simply means an index from a file with structures.\nOn the other hand, if you build graphs of the distribution of the target by indices, it turns out that the distributions are very different
\n\nmephistopheies [1 day ago]\nI'm strained by the fact that the features at the ends of the ribs work well, for example, primitive type by byg agg
\n\nif the order is random, it opens up the possibility of almost infinite augmentation, and it looks like something dumb
\n\nmephistopheies [1 day ago]\nI assume that there is some sort of numbering algorithm, and here it is interesting to understand what kind of algorithm it is.
\n\nmephistopheies [1 day ago]\nif it is, then probably somewhere in the paper chemists there is a description
\n\nil_buono [1 day ago]\nI think that the order is not random. It may coincide with the order in the SMILES record. But did not check.
\n\niggisv9t [1 day ago]\nI do everything based on the fact that the order is random. You can think of your order. It’s just the only reasonable assumption that it occurred to me that the order depends on orientation in space, which means that you can rotate the molecule and renumber it differently.
\n\niggisv9t [1 day ago]\nThat is, it seems to me that it will be safer to number them somehow in their own way.
\n\niggisv9t [1 day ago]\ngroup by (atom index) agg - is it on the whole molecule or inside the train and the test? (edited)
vadym.safronov [1 day ago]\nOdd was recalled the famous suitcase master Dmitry Ivanovich Mendeleev with his periodic table and groups of elements with similar properties depending on the sequence number.
\n\nkudasova [1 day ago]\nNot exactly checked, but it seems the indexes are sorted by the number of bonds with other atoms, as in tinker xyz files. Here on the example of ethanol http://chembytes.wikidot.com/tnk-tut00
\n\niggisv9t [1 day ago]\n@ vadym.safronov not looking at these memories is dangerous to voice.
\n\nvadym.safronov [1 day ago]\nNorm - in the local culinary circle, also speaking on the theoretical part
\n\nSergey Zlobin [1 day ago]\nThat is, the atom index makes sense to use as a feature? I don’t really like it, but if there’s some sort of sorting: thinking_face:\n```
", "rawMarkdown": "I want to preface this post by saying I have nothing against ods.ai in general. I am a part of the slack group and find conversations there are generally well intentioned. I don't post there- but I do read the competition specific channels (through google translate).\n\nThat being said, I recently read the thread below- found it interesting, and wanted to bring the idea to this forum. \n\nThe idea is that `atom_index` may shed some information about the target. My understanding is that the index is just an arbitrary way to identify each atom. Curious to hear if anyone has come across something to show otherwise.\n\nGoogle translated text:\n```\nmephistopheies [Yesterday at 9:09 AM]\nKaroch, I do not catch up with how at all the indexing of atoms is done, is it deterministic or not?\n14 replies\n\niggisv9t [1 day ago]\nIs it very important?\n\nartgor [1 day ago]\nOn the one hand, indexing is generally random — it simply means an index from a file with structures.\nOn the other hand, if you build graphs of the distribution of the target by indices, it turns out that the distributions are very different\n\nmephistopheies [1 day ago]\nI'm strained by the fact that the features at the ends of the ribs work well, for example, primitive type by byg agg\n\nif the order is random, it opens up the possibility of almost infinite augmentation, and it looks like something dumb\n\nmephistopheies [1 day ago]\nI assume that there is some sort of numbering algorithm, and here it is interesting to understand what kind of algorithm it is.\n\nmephistopheies [1 day ago]\nif it is, then probably somewhere in the paper chemists there is a description\n\nil_buono [1 day ago]\nI think that the order is not random. It may coincide with the order in the SMILES record. But did not check.\n\niggisv9t [1 day ago]\nI do everything based on the fact that the order is random. You can think of your order. It’s just the only reasonable assumption that it occurred to me that the order depends on orientation in space, which means that you can rotate the molecule and renumber it differently.\n\niggisv9t [1 day ago]\nThat is, it seems to me that it will be safer to number them somehow in their own way.\n\niggisv9t [1 day ago]\n`group by (atom index) agg` - is it on the whole molecule or inside the train and the test? (edited)\n\nvadym.safronov [1 day ago]\nOdd was recalled the famous suitcase master Dmitry Ivanovich Mendeleev with his periodic table and groups of elements with similar properties depending on the sequence number.\n\nkudasova [1 day ago]\nNot exactly checked, but it seems the indexes are sorted by the number of bonds with other atoms, as in tinker xyz files. Here on the example of ethanol http://chembytes.wikidot.com/tnk-tut00\n\niggisv9t [1 day ago]\n@ vadym.safronov not looking at these memories is dangerous to voice.\n\nvadym.safronov [1 day ago]\nNorm - in the local culinary circle, also speaking on the theoretical part\n\nSergey Zlobin [1 day ago]\nThat is, the atom index makes sense to use as a feature? I don’t really like it, but if there’s some sort of sorting: thinking_face:\n```", "votes": null }, { "id": "569763", "postDate": "07/07/2019 09:20:05", "content": "In principle, the indices should not be relevant.\nBut I have posted this already here somewhere else - there is a difference between the heavy atom indices (CNOF) and the H atom indices due to the generation algorithm of the structures (probably from SMILES):
\n\n
Usually the heavy atom indices come first and afterwards the H atom indices. Some programs add the H atoms in the order of the heavy atoms they belong to, I have not checked this here.\nThere may also be some order within the heavy atoms themselves, but I am not sure for the reason here, perhaps because new molecules were generated from some base structures ..?
\n\natom_index_0 corresponds to the position of the H atom in the different couplings.\nPossibly there is also some correlation between the atom_index_0 (which is always higher than that for the last heavy atom) and the molecular size. Just some thoughts.
\n\nFurthermore, grouping by atom_index_0 and molecule_name seems to be a useful feature as it collects the different coupling types for each molecule routed in the same H.
", "rawMarkdown": "In principle, the indices should not be relevant.\nBut I have posted this already here somewhere else - there is a difference between the heavy atom indices (CNOF) and the H atom indices due to the generation algorithm of the structures (probably from SMILES):\n\n\n\nUsually the heavy atom indices come first and afterwards the H atom indices. Some programs add the H atoms in the order of the heavy atoms they belong to, I have not checked this here.\nThere may also be some order within the heavy atoms themselves, but I am not sure for the reason here, perhaps because new molecules were generated from some base structures ..?\n\natom_index_0 corresponds to the position of the H atom in the different couplings.\nPossibly there is also some correlation between the atom_index_0 (which is always higher than that for the last heavy atom) and the molecular size. Just some thoughts.\n\nFurthermore, grouping by atom_index_0 and molecule_name seems to be a useful feature as it collects the different coupling types for each molecule routed in the same H.", "votes": null }, { "id": "569849", "postDate": "07/07/2019 12:54:00", "content": "Nice plot! That is helpful. I must have missed your earlier post where this was discussed. Would it serve better to show this plot this as a bar chart instead of a violin plot since atom_index has discrete values and not continuous?
This certainly gives me something to think about. My hypothesis is that atom_index_0 and atom_index_1 are only useful as a proxy for other, more useful features. It would be better to use the other features directly. For instance, if they are ordered by atom weight - using weight as a feature would be better than using atom_index directly.
Here is the screenshot of the conversation. I'll continue to post translations on the official kaggle discussion forum if I think they have useful information.
\n\n![\"\"](\"https://i.imgur.com/HNiI8km.png\")
\n![\"\"](\"https://i.imgur.com/yKWOiKe.png\")
Yes - agreed - there are probably ways to plot this nicer...\nThanks for bringing this topic back to kaggle.
", "rawMarkdown": "Yes - agreed - there are probably ways to plot this nicer...\nThanks for bringing this topic back to kaggle.", "votes": null }, { "id": "570154", "postDate": "07/07/2019 23:00:50", "content": "the hostility towards ods.ai is amazing. people like @onodera can not accept it. it is a pity that kaggle turned into this standard \"i am a grand-m, so fuck off\" meme. Used to be much more of a learning platform. Now turns into much more of marketing and self-promotion platform\n@robikscube you example is very good. Every one can go to ods.ai and read what is written there. no need to complain. it is all open. @onodera you also have an account there. why do you complain then?
", "rawMarkdown": "the hostility towards ods.ai is amazing. people like @onodera can not accept it. it is a pity that kaggle turned into this standard \"i am a grand-m, so fuck off\" meme. Used to be much more of a learning platform. Now turns into much more of marketing and self-promotion platform\n@robikscube you example is very good. Every one can go to ods.ai and read what is written there. no need to complain. it is all open. @onodera you also have an account there. why do you complain then?", "votes": null }, { "id": "570155", "postDate": "07/07/2019 23:01:56", "content": "so ods is running some undercover operations here. lol. they are talking cooking recipes
", "rawMarkdown": "so ods is running some undercover operations here. lol. they are talking cooking recipes", "votes": null }, { "id": "570809", "postDate": "07/08/2019 19:31:41", "content": "\"Now turns into much more of marketing and self-promotion platform\"
\n\nI think even Kaggle is used by people not only for studies but for promoting themselves. I don't understand your complaint. Sorry if I misunderstood you.
", "rawMarkdown": "\"Now turns into much more of marketing and self-promotion platform\"\n\nI think even Kaggle is used by people not only for studies but for promoting themselves. I don't understand your complaint. Sorry if I misunderstood you.", "votes": null }, { "id": "570854", "postDate": "07/08/2019 20:56:07", "content": ":feelsgoodmeme:
", "rawMarkdown": ":feelsgoodmeme:", "votes": null }, { "id": "570962", "postDate": "07/09/2019 01:32:25", "content": "![\"\"](\"https://cdn.the-village.ru/the-village.ru/post_image-image/P6joHkrQzA-CD3Dfhqd1uA.jpg\")
The indices must follow some sort of graph traversal, so the order does contain some information on the molecule topology. But you get way more information if you compute bonds IMHO. The graph I am discussing is where atoms are vertices and bonds are edges.
", "rawMarkdown": "The indices must follow some sort of graph traversal, so the order does contain some information on the molecule topology. But you get way more information if you compute bonds IMHO. The graph I am discussing is where atoms are vertices and bonds are edges.", "votes": null } ], "comments": [ { "id": 569763, "author_name": "chrisslz", "author_url": "", "post_date": "07/07/2019 09:20:05", "content": "In principle, the indices should not be relevant.\nBut I have posted this already here somewhere else - there is a difference between the heavy atom indices (CNOF) and the H atom indices due to the generation algorithm of the structures (probably from SMILES):
\n\n
Usually the heavy atom indices come first and afterwards the H atom indices. Some programs add the H atoms in the order of the heavy atoms they belong to, I have not checked this here.\nThere may also be some order within the heavy atoms themselves, but I am not sure for the reason here, perhaps because new molecules were generated from some base structures ..?
\n\natom_index_0 corresponds to the position of the H atom in the different couplings.\nPossibly there is also some correlation between the atom_index_0 (which is always higher than that for the last heavy atom) and the molecular size. Just some thoughts.
\n\nFurthermore, grouping by atom_index_0 and molecule_name seems to be a useful feature as it collects the different coupling types for each molecule routed in the same H.
", "votes": null, "replies": [ { "id": 569849, "author_name": "robikscube", "author_url": "", "post_date": "07/07/2019 12:54:00", "content": "Nice plot! That is helpful. I must have missed your earlier post where this was discussed. Would it serve better to show this plot this as a bar chart instead of a violin plot since atom_index has discrete values and not continuous?
This certainly gives me something to think about. My hypothesis is that atom_index_0 and atom_index_1 are only useful as a proxy for other, more useful features. It would be better to use the other features directly. For instance, if they are ordered by atom weight - using weight as a feature would be better than using atom_index directly.
Yes - agreed - there are probably ways to plot this nicer...\nThanks for bringing this topic back to kaggle.
", "votes": null, "replies": [] } ] }, { "id": 569853, "author_name": "robikscube", "author_url": "", "post_date": "07/07/2019 12:56:33", "content": "Here is the screenshot of the conversation. I'll continue to post translations on the official kaggle discussion forum if I think they have useful information.
\n\n\n
so ods is running some undercover operations here. lol. they are talking cooking recipes
", "votes": null, "replies": [] } ] }, { "id": 570154, "author_name": "snovik1975", "author_url": "", "post_date": "07/07/2019 23:00:50", "content": "the hostility towards ods.ai is amazing. people like @onodera can not accept it. it is a pity that kaggle turned into this standard \"i am a grand-m, so fuck off\" meme. Used to be much more of a learning platform. Now turns into much more of marketing and self-promotion platform\n@robikscube you example is very good. Every one can go to ods.ai and read what is written there. no need to complain. it is all open. @onodera you also have an account there. why do you complain then?
", "votes": null, "replies": [ { "id": 570809, "author_name": "muhakabartay", "author_url": "", "post_date": "07/08/2019 19:31:41", "content": "\"Now turns into much more of marketing and self-promotion platform\"
\n\nI think even Kaggle is used by people not only for studies but for promoting themselves. I don't understand your complaint. Sorry if I misunderstood you.
", "votes": null, "replies": [] } ] }, { "id": 570854, "author_name": "nesterov", "author_url": "", "post_date": "07/08/2019 20:56:07", "content": ":feelsgoodmeme:
", "votes": null, "replies": [] }, { "id": 570962, "author_name": "ivanpan", "author_url": "", "post_date": "07/09/2019 01:32:25", "content": "
The indices must follow some sort of graph traversal, so the order does contain some information on the molecule topology. But you get way more information if you compute bonds IMHO. The graph I am discussing is where atoms are vertices and bonds are edges.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "569606": "I want to preface this post by saying I have nothing against ods.ai in general. I am a part of the slack group and find conversations there are generally well intentioned. I don't post there- but I do read the competition specific channels (through google translate).\n\nThat being said, I recently read the thread below- found it interesting, and wanted to bring the idea to this forum. \n\nThe idea is that `atom_index` may shed some information about the target. My understanding is that the index is just an arbitrary way to identify each atom. Curious to hear if anyone has come across something to show otherwise.\n\nGoogle translated text:\n```\nmephistopheies [Yesterday at 9:09 AM]\nKaroch, I do not catch up with how at all the indexing of atoms is done, is it deterministic or not?\n14 replies\n\niggisv9t [1 day ago]\nIs it very important?\n\nartgor [1 day ago]\nOn the one hand, indexing is generally random — it simply means an index from a file with structures.\nOn the other hand, if you build graphs of the distribution of the target by indices, it turns out that the distributions are very different\n\nmephistopheies [1 day ago]\nI'm strained by the fact that the features at the ends of the ribs work well, for example, primitive type by byg agg\n\nif the order is random, it opens up the possibility of almost infinite augmentation, and it looks like something dumb\n\nmephistopheies [1 day ago]\nI assume that there is some sort of numbering algorithm, and here it is interesting to understand what kind of algorithm it is.\n\nmephistopheies [1 day ago]\nif it is, then probably somewhere in the paper chemists there is a description\n\nil_buono [1 day ago]\nI think that the order is not random. It may coincide with the order in the SMILES record. But did not check.\n\niggisv9t [1 day ago]\nI do everything based on the fact that the order is random. You can think of your order. It’s just the only reasonable assumption that it occurred to me that the order depends on orientation in space, which means that you can rotate the molecule and renumber it differently.\n\niggisv9t [1 day ago]\nThat is, it seems to me that it will be safer to number them somehow in their own way.\n\niggisv9t [1 day ago]\n`group by (atom index) agg` - is it on the whole molecule or inside the train and the test? (edited)\n\nvadym.safronov [1 day ago]\nOdd was recalled the famous suitcase master Dmitry Ivanovich Mendeleev with his periodic table and groups of elements with similar properties depending on the sequence number.\n\nkudasova [1 day ago]\nNot exactly checked, but it seems the indexes are sorted by the number of bonds with other atoms, as in tinker xyz files. Here on the example of ethanol http://chembytes.wikidot.com/tnk-tut00\n\niggisv9t [1 day ago]\n@ vadym.safronov not looking at these memories is dangerous to voice.\n\nvadym.safronov [1 day ago]\nNorm - in the local culinary circle, also speaking on the theoretical part\n\nSergey Zlobin [1 day ago]\nThat is, the atom index makes sense to use as a feature? I don’t really like it, but if there’s some sort of sorting: thinking_face:\n```", "569763": "In principle, the indices should not be relevant.\nBut I have posted this already here somewhere else - there is a difference between the heavy atom indices (CNOF) and the H atom indices due to the generation algorithm of the structures (probably from SMILES):\n\n\n\nUsually the heavy atom indices come first and afterwards the H atom indices. Some programs add the H atoms in the order of the heavy atoms they belong to, I have not checked this here.\nThere may also be some order within the heavy atoms themselves, but I am not sure for the reason here, perhaps because new molecules were generated from some base structures ..?\n\natom_index_0 corresponds to the position of the H atom in the different couplings.\nPossibly there is also some correlation between the atom_index_0 (which is always higher than that for the last heavy atom) and the molecular size. Just some thoughts.\n\nFurthermore, grouping by atom_index_0 and molecule_name seems to be a useful feature as it collects the different coupling types for each molecule routed in the same H.", "569849": "Nice plot! That is helpful. I must have missed your earlier post where this was discussed. Would it serve better to show this plot this as a bar chart instead of a violin plot since `atom_index` has discrete values and not continuous?\n\nThis certainly gives me something to think about. My hypothesis is that `atom_index_0` and `atom_index_1` are only useful as a proxy for other, more useful features. It would be better to use the other features directly. For instance, if they are ordered by atom weight - using weight as a feature would be better than using `atom_index` directly.", "569853": "Here is the screenshot of the conversation. I'll continue to post translations on the official kaggle discussion forum if I think they have useful information.\n\n\n", "569996": "Yes - agreed - there are probably ways to plot this nicer...\nThanks for bringing this topic back to kaggle.", "570154": "the hostility towards ods.ai is amazing. people like @onodera can not accept it. it is a pity that kaggle turned into this standard \"i am a grand-m, so fuck off\" meme. Used to be much more of a learning platform. Now turns into much more of marketing and self-promotion platform\n@robikscube you example is very good. Every one can go to ods.ai and read what is written there. no need to complain. it is all open. @onodera you also have an account there. why do you complain then?", "570155": "so ods is running some undercover operations here. lol. they are talking cooking recipes", "570809": "\"Now turns into much more of marketing and self-promotion platform\"\n\nI think even Kaggle is used by people not only for studies but for promoting themselves. I don't understand your complaint. Sorry if I misunderstood you.", "570854": ":feelsgoodmeme:", "570962": "", "571268": "The indices must follow some sort of graph traversal, so the order does contain some information on the molecule topology. But you get way more information if you compute bonds IMHO. The graph I am discussing is where atoms are vertices and bonds are edges." }, "source": "meta" }