From the description: \"... these quantum mechanics calculations are extremely expensive (days or weeks per molecule) ...\" I surmise that we are predicting scalar coupling constants for the structures already computed (possibly long time ago) and likely freely available in the public domain. It seems very likely that the vast majority or even all of several thousand of structures in the test set have been published and publicly available.
\n\nIf I am correct, then this situation opens a backdoor to earning medals by simply locating the data and adding some noise. In fact, even the uncertainty of the situation already diminishes the value of the medals. I see some similarity to what happened in the LANL competition.
\n\nA better way to run these types of competitions is on freshly generated (and unpublished) data in the Private set.
\n\nI am just stating my opinion here. It is possible that I am simply confused. I do hope this will stimulate further discussion of format improvement for similar competitions in the future.
\n\nEdit: according to Iverson's post below the constants will be released after the competition. Looks like my dog followed the wrong trail.
", "messages": [ { "id": "553444", "postDate": "06/15/2019 17:51:21", "content": "From the description: \"... these quantum mechanics calculations are extremely expensive (days or weeks per molecule) ...\" I surmise that we are predicting scalar coupling constants for the structures already computed (possibly long time ago) and likely freely available in the public domain. It seems very likely that the vast majority or even all of several thousand of structures in the test set have been published and publicly available.
\n\nIf I am correct, then this situation opens a backdoor to earning medals by simply locating the data and adding some noise. In fact, even the uncertainty of the situation already diminishes the value of the medals. I see some similarity to what happened in the LANL competition.
\n\nA better way to run these types of competitions is on freshly generated (and unpublished) data in the Private set.
\n\nI am just stating my opinion here. It is possible that I am simply confused. I do hope this will stimulate further discussion of format improvement for similar competitions in the future.
\n\nEdit: according to Iverson's post below the constants will be released after the competition. Looks like my dog followed the wrong trail.
", "rawMarkdown": "From the description: \"*... these quantum mechanics calculations are extremely expensive (days or weeks per molecule) ...*\" I surmise that we are predicting scalar coupling constants for the structures already computed (possibly long time ago) and likely freely available in the public domain. It seems very likely that the vast majority or even all of several thousand of structures in the test set have been published and publicly available.\n\nIf I am correct, then this situation opens a backdoor to earning medals by simply locating the data and adding some noise. In fact, even the uncertainty of the situation already diminishes the value of the medals. I see some similarity to what happened in the LANL competition.\n\nA better way to run these types of competitions is on freshly generated (and **unpublished**) data in the Private set.\n\nI am just stating my opinion here. It is possible that I am simply confused. I do hope this will stimulate further discussion of format improvement for similar competitions in the future.\n\n**Edit: according to Iverson's post below the constants will be released after the competition. Looks like my dog followed the wrong trail.**", "votes": null }, { "id": "553457", "postDate": "06/15/2019 18:19:14", "content": "Seems like I have seen a discussion where it was believed that the values were computed at a non-standard condition not likely to have been published, making your argument not correct.
", "rawMarkdown": "Seems like I have seen a discussion where it was believed that the values were computed at a non-standard condition not likely to have been published, making your argument not correct.", "votes": null }, { "id": "553468", "postDate": "06/15/2019 18:47:25", "content": "Hi Mycroft -
\n\nIt's better to ask about the situation first before making conclusions based on assumptions and conjecture.
", "rawMarkdown": "Hi Mycroft -\n\nIt's better to ask about the situation first before making conclusions based on assumptions and conjecture.", "votes": null }, { "id": "553502", "postDate": "06/15/2019 20:26:10", "content": "@pcjimmmy - You are 100% correct.
", "rawMarkdown": "pcjimmmy - You are 100% correct.", "votes": null }, { "id": "553509", "postDate": "06/15/2019 20:46:14", "content": "This is what I am doing - asking. [I asked a similar question before] (https://www.kaggle.com/c/champs-scalar-coupling/discussion/95496#552830) and got no answer.\nIs it true then that the coupling constants calculated for this particular T for the molecules in the Private Test have never been released to the public?
", "rawMarkdown": "This is what I am doing - asking. [I asked a similar question before] (https://www.kaggle.com/c/champs-scalar-coupling/discussion/95496#552830) and got no answer.\nIs it true then that the coupling constants calculated for this particular T for the molecules in the Private Test have never been released to the public?", "votes": null }, { "id": "553589", "postDate": "06/16/2019 01:55:55", "content": "I tell ya, I get no respect.
", "rawMarkdown": "I tell ya, I get no respect.", "votes": null }, { "id": "553729", "postDate": "06/16/2019 08:30:15", "content": "@stocks lol\nI believe these are legit concerns. It is possible to calculate the results by QM, so a data leak is not required, but then again the rules don't allow that. Its not just a data submission contest. Code for the parametrization is needed to win the contest.
", "rawMarkdown": "stocks lol\nI believe these are legit concerns. It is possible to calculate the results by QM, so a data leak is not required, but then again the rules don't allow that. Its not just a data submission contest. Code for the parametrization is needed to win the contest.", "votes": null }, { "id": "554000", "postDate": "06/16/2019 18:49:18", "content": "Correct. They will be released after the competition.
", "rawMarkdown": "Correct. They will be released after the competition.", "votes": null }, { "id": "554004", "postDate": "06/16/2019 18:59:16", "content": "Then I got it all wrong. This could have been stated clearly in the data description (e.g. \"our group(s) generated the constants, nobody else have them, etc.\").
", "rawMarkdown": "Then I got it all wrong. This could have been stated clearly in the data description (e.g. \"our group(s) generated the constants, nobody else have them, etc.\").", "votes": null }, { "id": "556081", "postDate": "06/19/2019 20:15:00", "content": "I think the better option is to run the competition as a \"kernels only\" contest. With only the given data set being allowed. Then the host would be able to see if people used outside data.
\n\nThe purpose of Kaggle is to determine who can build the best machine learning solution with the information given not to determine who has the most experience with a given scientific or industry problem. Unfortunately, from reading some of the discussion, many people do not understand this.
", "rawMarkdown": "I think the better option is to run the competition as a \"kernels only\" contest. With only the given data set being allowed. Then the host would be able to see if people used outside data. \n\nThe purpose of Kaggle is to determine who can build the best machine learning solution with the information given not to determine who has the most experience with a given scientific or industry problem. Unfortunately, from reading some of the discussion, many people do not understand this.", "votes": null }, { "id": "556172", "postDate": "06/19/2019 23:22:03", "content": "Kernels have some limitations. For example, I cannot fit all the features generated by Andrew Lukyanenko into a new kernel. Maybe so many features are not needed but who knows this early in the competition?
\n\nWe are asked to find heuristics in lieu of heave quantum mechanical calculations. I am guessing the heuristics are likely to be computationally intensive as well.
", "rawMarkdown": "Kernels have some limitations. For example, I cannot fit all the features generated by Andrew Lukyanenko into a new kernel. Maybe so many features are not needed but who knows this early in the competition? \n\nWe are asked to find heuristics in lieu of heave quantum mechanical calculations. I am guessing the heuristics are likely to be computationally intensive as well.", "votes": null } ], "comments": [ { "id": 553457, "author_name": "pcjimmmy", "author_url": "", "post_date": "06/15/2019 18:19:14", "content": "Seems like I have seen a discussion where it was believed that the values were computed at a non-standard condition not likely to have been published, making your argument not correct.
", "votes": null, "replies": [ { "id": 553468, "author_name": "stocks", "author_url": "", "post_date": "06/15/2019 18:47:25", "content": "@pcjimmmy - You are 100% correct.
", "votes": null, "replies": [] } ] }, { "id": 553501, "author_name": "inversion", "author_url": "", "post_date": "06/15/2019 20:25:30", "content": "Hi Mycroft -
\n\nIt's better to ask about the situation first before making conclusions based on assumptions and conjecture.
", "votes": null, "replies": [ { "id": 553509, "author_name": "stocks", "author_url": "", "post_date": "06/15/2019 20:46:14", "content": "This is what I am doing - asking. [I asked a similar question before] (https://www.kaggle.com/c/champs-scalar-coupling/discussion/95496#552830) and got no answer.\nIs it true then that the coupling constants calculated for this particular T for the molecules in the Private Test have never been released to the public?
", "votes": null, "replies": [] }, { "id": 554000, "author_name": "inversion", "author_url": "", "post_date": "06/16/2019 18:49:18", "content": "Correct. They will be released after the competition.
", "votes": null, "replies": [] }, { "id": 554004, "author_name": "stocks", "author_url": "", "post_date": "06/16/2019 18:59:16", "content": "Then I got it all wrong. This could have been stated clearly in the data description (e.g. \"our group(s) generated the constants, nobody else have them, etc.\").
", "votes": null, "replies": [] } ] }, { "id": 553589, "author_name": "stocks", "author_url": "", "post_date": "06/16/2019 01:55:55", "content": "I tell ya, I get no respect.
", "votes": null, "replies": [ { "id": 553729, "author_name": "tobikaggle", "author_url": "", "post_date": "06/16/2019 08:30:15", "content": "@stocks lol\nI believe these are legit concerns. It is possible to calculate the results by QM, so a data leak is not required, but then again the rules don't allow that. Its not just a data submission contest. Code for the parametrization is needed to win the contest.
", "votes": null, "replies": [] } ] }, { "id": 556081, "author_name": "stajh05", "author_url": "", "post_date": "06/19/2019 20:15:00", "content": "I think the better option is to run the competition as a \"kernels only\" contest. With only the given data set being allowed. Then the host would be able to see if people used outside data.
\n\nThe purpose of Kaggle is to determine who can build the best machine learning solution with the information given not to determine who has the most experience with a given scientific or industry problem. Unfortunately, from reading some of the discussion, many people do not understand this.
", "votes": null, "replies": [ { "id": 556172, "author_name": "stocks", "author_url": "", "post_date": "06/19/2019 23:22:03", "content": "Kernels have some limitations. For example, I cannot fit all the features generated by Andrew Lukyanenko into a new kernel. Maybe so many features are not needed but who knows this early in the competition?
\n\nWe are asked to find heuristics in lieu of heave quantum mechanical calculations. I am guessing the heuristics are likely to be computationally intensive as well.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "553444": "From the description: \"*... these quantum mechanics calculations are extremely expensive (days or weeks per molecule) ...*\" I surmise that we are predicting scalar coupling constants for the structures already computed (possibly long time ago) and likely freely available in the public domain. It seems very likely that the vast majority or even all of several thousand of structures in the test set have been published and publicly available.\n\nIf I am correct, then this situation opens a backdoor to earning medals by simply locating the data and adding some noise. In fact, even the uncertainty of the situation already diminishes the value of the medals. I see some similarity to what happened in the LANL competition.\n\nA better way to run these types of competitions is on freshly generated (and **unpublished**) data in the Private set.\n\nI am just stating my opinion here. It is possible that I am simply confused. I do hope this will stimulate further discussion of format improvement for similar competitions in the future.\n\n**Edit: according to Iverson's post below the constants will be released after the competition. Looks like my dog followed the wrong trail.**", "553457": "Seems like I have seen a discussion where it was believed that the values were computed at a non-standard condition not likely to have been published, making your argument not correct.", "553468": "1. I doubt the organizers themselves ran the simulations on several thousand of molecules and are now patiently waiting for this competition to complete before publishing their data. We are talking about years of computing.\n2. I am guessing that quantum calculations even if done at different T are much more predictive than the data allowed for predictions in this contest. In other words, even if only calculations done at different T are publicly available somewhere it will be enough to cheat on the medals.", "553501": "Hi Mycroft -\n\nIt's better to ask about the situation first before making conclusions based on assumptions and conjecture.", "553502": "pcjimmmy - You are 100% correct.", "553509": "This is what I am doing - asking. [I asked a similar question before] (https://www.kaggle.com/c/champs-scalar-coupling/discussion/95496#552830) and got no answer.\nIs it true then that the coupling constants calculated for this particular T for the molecules in the Private Test have never been released to the public?", "553589": "I tell ya, I get no respect.", "553729": "stocks lol\nI believe these are legit concerns. It is possible to calculate the results by QM, so a data leak is not required, but then again the rules don't allow that. Its not just a data submission contest. Code for the parametrization is needed to win the contest.", "554000": "Correct. They will be released after the competition.", "554004": "Then I got it all wrong. This could have been stated clearly in the data description (e.g. \"our group(s) generated the constants, nobody else have them, etc.\").", "556081": "I think the better option is to run the competition as a \"kernels only\" contest. With only the given data set being allowed. Then the host would be able to see if people used outside data. \n\nThe purpose of Kaggle is to determine who can build the best machine learning solution with the information given not to determine who has the most experience with a given scientific or industry problem. Unfortunately, from reading some of the discussion, many people do not understand this.", "556172": "Kernels have some limitations. For example, I cannot fit all the features generated by Andrew Lukyanenko into a new kernel. Maybe so many features are not needed but who knows this early in the competition? \n\nWe are asked to find heuristics in lieu of heave quantum mechanical calculations. I am guessing the heuristics are likely to be computationally intensive as well." }, "source": "meta" }