Can someone explain what the indexes are. I assumed since they are indexes, there would be no pattern.
\n\nThe attached boxplots show patterns with the indexes and scalar_coupling_constants.
", "messages": [ { "id": "547963", "postDate": "06/08/2019 15:12:43", "content": "Can someone explain what the indexes are. I assumed since they are indexes, there would be no pattern.
\n\nThe attached boxplots show patterns with the indexes and scalar_coupling_constants.
", "rawMarkdown": "Can someone explain what the indexes are. I assumed since they are indexes, there would be no pattern.\n\nThe attached boxplots show patterns with the indexes and scalar_coupling_constants.", "votes": null }, { "id": "547965", "postDate": "06/08/2019 15:13:22", "content": "https://www.kaggle.com/lowecoryr/eda-predicting-molecular-properties
", "rawMarkdown": "https://www.kaggle.com/lowecoryr/eda-predicting-molecular-properties", "votes": null }, { "id": "548027", "postDate": "06/08/2019 16:43:42", "content": "Yeah, it seems like the train and test datasets are biased in this way. Ideally, I think they should not show any correlation, but apparently they do. It kind of flaws the competition because the top model will likely take these biases into account and won't be truly generalized. Not much we as competitors can do about it though. Btw, I think the axis labels on your graph are wrong
", "rawMarkdown": "Yeah, it seems like the train and test datasets are biased in this way. Ideally, I think they should not show any correlation, but apparently they do. It kind of flaws the competition because the top model will likely take these biases into account and won't be truly generalized. Not much we as competitors can do about it though. Btw, I think the axis labels on your graph are wrong", "votes": null }, { "id": "548222", "postDate": "06/09/2019 02:19:22", "content": "Inspired by this thread, I looked into this.\nThe analysis is currently in my ad hoc EDA kernel.\nhttps://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-
\n\nIt looks like they selected atom_indices in a different way for different types. There is a very strong relationship there that is manifesting itself in your charts (which I recreate in my kernel).\nThere are some very strange exceptions to the rule, which I have not yet been able to explain/exploit.
", "rawMarkdown": "Inspired by this thread, I looked into this.\nThe analysis is currently in my ad hoc EDA kernel.\n[https://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-](https://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-)\n\nIt looks like they selected atom_indices in a different way for different types. There is a very strong relationship there that is manifesting itself in your charts (which I recreate in my kernel).\nThere are some very strange exceptions to the rule, which I have not yet been able to explain/exploit.", "votes": null }, { "id": "561396", "postDate": "06/26/2019 12:00:44", "content": "Any bias in the indices shouldn't matter as they are just indices. Indices for couplings between atoms of the same element follows slightly different rules than indices for couplings between different elements. Maybe what you're seeing is an artifact of this?
", "rawMarkdown": "Any bias in the indices shouldn't matter as they are just indices. Indices for couplings between atoms of the same element follows slightly different rules than indices for couplings between different elements. Maybe what you're seeing is an artifact of this?", "votes": null }, { "id": "563945", "postDate": "06/28/2019 19:55:00", "content": "The success of the atom_index features is puzzling me as well...\nSome of the unusal correlations in the kernels may be related to the way the structures are created. Often the molecules are stored by cheminformatics programs as rdkit only with heavy atoms (CNOF) (i.e. without H atoms). If later on hydrogens are added or an initial 3D structure is generated, they are added after the heavy atoms, i.e. having a higher index. The first index fro most molecules will never be an H and the last index should nearly always be an H. i.e the JHH couplings will all have higher atom indices.
", "rawMarkdown": "The success of the atom_index features is puzzling me as well...\nSome of the unusal correlations in the kernels may be related to the way the structures are created. Often the molecules are stored by cheminformatics programs as rdkit only with heavy atoms (CNOF) (i.e. without H atoms). If later on hydrogens are added or an initial 3D structure is generated, they are added after the heavy atoms, i.e. having a higher index. The first index fro most molecules will never be an H and the last index should nearly always be an H. i.e the JHH couplings will all have higher atom indices.", "votes": null }, { "id": "564259", "postDate": "06/29/2019 07:55:23", "content": "The atom_index_0 (H atom) usually comes after the all the heay atoms and possibly contains some implicit information on the molecular size.
", "rawMarkdown": "The atom_index_0 (H atom) usually comes after the all the heay atoms and possibly contains some implicit information on the molecular size.", "votes": null } ], "comments": [ { "id": 547965, "author_name": "lowecoryr", "author_url": "", "post_date": "06/08/2019 15:13:22", "content": "https://www.kaggle.com/lowecoryr/eda-predicting-molecular-properties
", "votes": null, "replies": [] }, { "id": 548027, "author_name": "matthewmasters", "author_url": "", "post_date": "06/08/2019 16:43:42", "content": "Yeah, it seems like the train and test datasets are biased in this way. Ideally, I think they should not show any correlation, but apparently they do. It kind of flaws the competition because the top model will likely take these biases into account and won't be truly generalized. Not much we as competitors can do about it though. Btw, I think the axis labels on your graph are wrong
", "votes": null, "replies": [] }, { "id": 548222, "author_name": "alekseynp", "author_url": "", "post_date": "06/09/2019 02:19:22", "content": "Inspired by this thread, I looked into this.\nThe analysis is currently in my ad hoc EDA kernel.\nhttps://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-
\n\nIt looks like they selected atom_indices in a different way for different types. There is a very strong relationship there that is manifesting itself in your charts (which I recreate in my kernel).\nThere are some very strange exceptions to the rule, which I have not yet been able to explain/exploit.
", "votes": null, "replies": [] }, { "id": 561396, "author_name": "larsbratholm", "author_url": "", "post_date": "06/26/2019 12:00:44", "content": "Any bias in the indices shouldn't matter as they are just indices. Indices for couplings between atoms of the same element follows slightly different rules than indices for couplings between different elements. Maybe what you're seeing is an artifact of this?
", "votes": null, "replies": [] }, { "id": 563945, "author_name": "chrisslz", "author_url": "", "post_date": "06/28/2019 19:55:00", "content": "The success of the atom_index features is puzzling me as well...\nSome of the unusal correlations in the kernels may be related to the way the structures are created. Often the molecules are stored by cheminformatics programs as rdkit only with heavy atoms (CNOF) (i.e. without H atoms). If later on hydrogens are added or an initial 3D structure is generated, they are added after the heavy atoms, i.e. having a higher index. The first index fro most molecules will never be an H and the last index should nearly always be an H. i.e the JHH couplings will all have higher atom indices.
", "votes": null, "replies": [] }, { "id": 564259, "author_name": "chrisslz", "author_url": "", "post_date": "06/29/2019 07:55:23", "content": "The atom_index_0 (H atom) usually comes after the all the heay atoms and possibly contains some implicit information on the molecular size.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "547963": "Can someone explain what the indexes are. I assumed since they are indexes, there would be no pattern.\n\nThe attached boxplots show patterns with the indexes and scalar_coupling_constants.", "547965": "https://www.kaggle.com/lowecoryr/eda-predicting-molecular-properties", "548027": "Yeah, it seems like the train and test datasets are biased in this way. Ideally, I think they should not show any correlation, but apparently they do. It kind of flaws the competition because the top model will likely take these biases into account and won't be truly generalized. Not much we as competitors can do about it though. Btw, I think the axis labels on your graph are wrong", "548222": "Inspired by this thread, I looked into this.\nThe analysis is currently in my ad hoc EDA kernel.\n[https://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-](https://www.kaggle.com/alekseynp/answering-my-eda-questions#Worrying-Atom-Index-Funny-Business-)\n\nIt looks like they selected atom_indices in a different way for different types. There is a very strong relationship there that is manifesting itself in your charts (which I recreate in my kernel).\nThere are some very strange exceptions to the rule, which I have not yet been able to explain/exploit.", "561396": "Any bias in the indices shouldn't matter as they are just indices. Indices for couplings between atoms of the same element follows slightly different rules than indices for couplings between different elements. Maybe what you're seeing is an artifact of this?", "563945": "The success of the atom_index features is puzzling me as well...\nSome of the unusal correlations in the kernels may be related to the way the structures are created. Often the molecules are stored by cheminformatics programs as rdkit only with heavy atoms (CNOF) (i.e. without H atoms). If later on hydrogens are added or an initial 3D structure is generated, they are added after the heavy atoms, i.e. having a higher index. The first index fro most molecules will never be an H and the last index should nearly always be an H. i.e the JHH couplings will all have higher atom indices.", "564259": "The atom_index_0 (H atom) usually comes after the all the heay atoms and possibly contains some implicit information on the molecular size." }, "source": "meta" }