In the Evaluation section of the Overview, I found the following table
\nP9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
\nP04637 GO:1990837 0.23
\nP04637 GO:0031625 0.989
\nP04637 GO:0043565 .64
\nP04637 GO:0001091 0.49
\netc.
What are the probabilities for the right side of this table? Please express it in a formula.
", "messages": [ { "id": "2288124", "postDate": "06/05/2023 07:28:52", "content": "In the Evaluation section of the Overview, I found the following table
\nP9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
\nP04637 GO:1990837 0.23
\nP04637 GO:0031625 0.989
\nP04637 GO:0043565 .64
\nP04637 GO:0001091 0.49
\netc.
What are the probabilities for the right side of this table? Please express it in a formula.
", "rawMarkdown": "In the Evaluation section of the Overview, I found the following table\n\nP9WHI7 GO:0009274 0.931 \nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\nP04637 GO:1990837 0.23\nP04637 GO:0031625 0.989\nP04637 GO:0043565 .64\nP04637 GO:0001091 0.49\netc.\n\n\nWhat are the probabilities for the right side of this table? Please express it in a formula.", "votes": null }, { "id": "2288875", "postDate": "06/05/2023 18:22:27", "content": "Hey! To answer your question and make it as simple as possible :
\nFor each protein ID, our goal is to determine if each possible GO term in G is contained in its structure. So to make it as a formula, if we denote $$ \\forall i \\in I , \\ \\ \\ G_{i} \\subseteq G $$ the subset of GO terms contained in protein i, we want to compute as well as possible the following probabilities :
\n$$ \\forall g \\in G, \\ \\ \\ p_{i}^{g} = \\mathbb{P}(g \\in G_{i}) $$
\nHope this helps! Dont hesitate if you have any question! š¤
", "rawMarkdown": "Hey! To answer your question and make it as simple as possible : \n1. We dispose of a set of protein IDs (first row), lets write it $$ I = ( I_{1}, \\dots, I_{N} ) $$\n2. We dispose of a set of GO terms (second row) that could be contained in each protein, lets write it $$ G = ( G^{1}, \\dots, G^{K} ) $$\n\nFor each protein ID, our goal is to determine if each possible GO term in G is contained in its structure. So to make it as a formula, if we denote $$ \\forall i \\in I , \\ \\ \\ G_{i} \\subseteq G $$ the subset of GO terms contained in protein i, we want to compute as well as possible the following probabilities : \n\n$$ \\forall g \\in G, \\ \\ \\ p_{i}^{g} = \\mathbb{P}(g \\in G_{i}) $$\n\nHope this helps! Dont hesitate if you have any question! š¤", "votes": null }, { "id": "2290523", "postDate": "06/06/2023 22:35:33", "content": "Hello. Thank you very much for kindly answering my question. So, your equation means
\n$$P(g = g_k^j | protein id = I_k)$$
\nwhere \\(g_k^j\\) is the \\(j\\)-th element of \\(G_k\\), right? I was thinking the same way, but I was confused because when I perform the calculation using the table above, the sum does not equal 1. For example, the sum of the numbers on the right in the table below is not 1.
P9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
Is it correct to say that they should add up to 1 in reality?
", "rawMarkdown": "Hello. Thank you very much for kindly answering my question. So, your equation means \n$$P(g = g_k^j | protein id = I_k)$$\nwhere \\\\(g_k^j\\\\) is the \\\\(j\\\\)-th element of \\\\(G_k\\\\), right? I was thinking the same way, but I was confused because when I perform the calculation using the table above, the sum does not equal 1. For example, the sum of the numbers on the right in the table below is not 1.\n\nP9WHI7 GO:0009274 0.931\nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\n\nIs it correct to say that they should add up to 1 in reality?", "votes": null }, { "id": "2290527", "postDate": "06/06/2023 22:48:35", "content": "No, there is no need for them to add up to one. You don't know in advance how many GO terms correctly describe the function of a test sequence.
", "rawMarkdown": "No, there is no need for them to add up to one. You don't know in advance how many GO terms correctly describe the function of a test sequence.", "votes": null }, { "id": "2290545", "postDate": "06/06/2023 23:42:42", "content": "Indeed we are in the framework of multilabel classification so they don't sum at 1
", "rawMarkdown": "Indeed we are in the framework of multilabel classification so they don't sum at 1", "votes": null }, { "id": "2290617", "postDate": "06/07/2023 01:36:52", "content": "Hmm, I might not fully understand the problem of multi-label classification. Should I interpret the numbers in the right column of the table as the output of the following formula?
\n$$P(g = g_k^j | protein id = I_k)$$
If so, according to a certain threshold, for each protein ID I_k, I am considering writing in the submission file the GO terms (g_k^1,ā¦ ,g_k^l) with high probability in the sense of the formula, and their probabilities (p_k^1,ā¦ ,p_k^l) as follows:
\n$$ (I_k, g_k^1, p_k^1) $$
\nā¦
\n$$ (I_k, g_k^l, p_k^l)$$
Is that a good approach?
", "rawMarkdown": "Hmm, I might not fully understand the problem of multi-label classification. Should I interpret the numbers in the right column of the table as the output of the following formula?\n$$P(g = g_k^j | protein id = I_k)$$\n\nIf so, according to a certain threshold, for each protein ID I_k, I am considering writing in the submission file the GO terms (g_k^1,... ,g_k^l) with high probability in the sense of the formula, and their probabilities (p_k^1,... ,p_k^l) as follows:\n\n$$ (I_k, g_k^1, p_k^1) $$\n...\n$$ (I_k, g_k^l, p_k^l)$$\n\nIs that a good approach?", "votes": null }, { "id": "2290626", "postDate": "06/07/2023 01:46:54", "content": "I was puzzled because the sum of the numbers in the right column of the table exceeded 1 for a certain protein ID, as shown below:
\nP9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
If my interpretation is still incorrect, could you possibly share some reference materials?
", "rawMarkdown": "I was puzzled because the sum of the numbers in the right column of the table exceeded 1 for a certain protein ID, as shown below:\n\nP9WHI7 GO:0009274 0.931\nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\n\nIf my interpretation is still incorrect, could you possibly share some reference materials?", "votes": null }, { "id": "2290929", "postDate": "06/07/2023 08:01:11", "content": "@ariafschneider - I think it's perhaps easier to think about those numbers as a measure of how confident you are in each prediction (of a particular GO term for the specific protein). The way in which the scoring metric is calculated finds the confidence threshold at which your F-score is maximised (for each of MF, BP & CC), and in that context ranking your predictions by confidence is more important than precisely scaling them as probabilities - though they do need to be between 0 and 1. Have another look at the Evaluation section under the Overview tab of this Competition's Home Page. Once you understand how the score is calculated, the requirements for the probability (or confidence, if we prefer to think of it that way) numbers are easier to comprehend.
\nA fun experiment would be to take your best model, divide all the probabilities by 2 and resubmit it. What does the result of that process tell you?
", "rawMarkdown": "ariafschneider - I think it's perhaps easier to think about those numbers as a measure of how confident you are in each prediction (of a particular GO term for the specific protein). The way in which the scoring metric is calculated finds the confidence threshold at which your F-score is maximised (for each of MF, BP & CC), and in that context ranking your predictions by confidence is more important than precisely scaling them as probabilities - though they do need to be between 0 and 1. Have another look at the Evaluation section under the Overview tab of this Competition's Home Page. Once you understand how the score is calculated, the requirements for the probability (or confidence, if we prefer to think of it that way) numbers are easier to comprehend.\n\nA fun experiment would be to take your best model, divide all the probabilities by 2 and resubmit it. What does the result of that process tell you?", "votes": null } ], "comments": [ { "id": 2288875, "author_name": "henriupton", "author_url": "", "post_date": "06/05/2023 18:22:27", "content": "Hey! To answer your question and make it as simple as possible :
\nFor each protein ID, our goal is to determine if each possible GO term in G is contained in its structure. So to make it as a formula, if we denote $$ \\forall i \\in I , \\ \\ \\ G_{i} \\subseteq G $$ the subset of GO terms contained in protein i, we want to compute as well as possible the following probabilities :
\n$$ \\forall g \\in G, \\ \\ \\ p_{i}^{g} = \\mathbb{P}(g \\in G_{i}) $$
\nHope this helps! Dont hesitate if you have any question! š¤
", "votes": null, "replies": [ { "id": 2290523, "author_name": "ariafschneider", "author_url": "", "post_date": "06/06/2023 22:35:33", "content": "Hello. Thank you very much for kindly answering my question. So, your equation means
\n$$P(g = g_k^j | protein id = I_k)$$
\nwhere \\(g_k^j\\) is the \\(j\\)-th element of \\(G_k\\), right? I was thinking the same way, but I was confused because when I perform the calculation using the table above, the sum does not equal 1. For example, the sum of the numbers on the right in the table below is not 1.
P9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
Is it correct to say that they should add up to 1 in reality?
", "votes": null, "replies": [ { "id": 2290527, "author_name": "jbomitchell", "author_url": "", "post_date": "06/06/2023 22:48:35", "content": "No, there is no need for them to add up to one. You don't know in advance how many GO terms correctly describe the function of a test sequence.
", "votes": null, "replies": [] }, { "id": 2290545, "author_name": "henriupton", "author_url": "", "post_date": "06/06/2023 23:42:42", "content": "Indeed we are in the framework of multilabel classification so they don't sum at 1
", "votes": null, "replies": [ { "id": 2290617, "author_name": "ariafschneider", "author_url": "", "post_date": "06/07/2023 01:36:52", "content": "Hmm, I might not fully understand the problem of multi-label classification. Should I interpret the numbers in the right column of the table as the output of the following formula?
\n$$P(g = g_k^j | protein id = I_k)$$
If so, according to a certain threshold, for each protein ID I_k, I am considering writing in the submission file the GO terms (g_k^1,ā¦ ,g_k^l) with high probability in the sense of the formula, and their probabilities (p_k^1,ā¦ ,p_k^l) as follows:
\n$$ (I_k, g_k^1, p_k^1) $$
\nā¦
\n$$ (I_k, g_k^l, p_k^l)$$
Is that a good approach?
", "votes": null, "replies": [ { "id": 2290626, "author_name": "ariafschneider", "author_url": "", "post_date": "06/07/2023 01:46:54", "content": "I was puzzled because the sum of the numbers in the right column of the table exceeded 1 for a certain protein ID, as shown below:
\nP9WHI7 GO:0009274 0.931
\nP9WHI7 GO:0071944 0.540
\nP9WHI7 GO:0005575 0.324
If my interpretation is still incorrect, could you possibly share some reference materials?
", "votes": null, "replies": [ { "id": 2290929, "author_name": "jbomitchell", "author_url": "", "post_date": "06/07/2023 08:01:11", "content": "@ariafschneider - I think it's perhaps easier to think about those numbers as a measure of how confident you are in each prediction (of a particular GO term for the specific protein). The way in which the scoring metric is calculated finds the confidence threshold at which your F-score is maximised (for each of MF, BP & CC), and in that context ranking your predictions by confidence is more important than precisely scaling them as probabilities - though they do need to be between 0 and 1. Have another look at the Evaluation section under the Overview tab of this Competition's Home Page. Once you understand how the score is calculated, the requirements for the probability (or confidence, if we prefer to think of it that way) numbers are easier to comprehend.
\nA fun experiment would be to take your best model, divide all the probabilities by 2 and resubmit it. What does the result of that process tell you?
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2288124": "In the Evaluation section of the Overview, I found the following table\n\nP9WHI7 GO:0009274 0.931 \nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\nP04637 GO:1990837 0.23\nP04637 GO:0031625 0.989\nP04637 GO:0043565 .64\nP04637 GO:0001091 0.49\netc.\n\n\nWhat are the probabilities for the right side of this table? Please express it in a formula.", "2288875": "Hey! To answer your question and make it as simple as possible : \n1. We dispose of a set of protein IDs (first row), lets write it $$ I = ( I_{1}, \\dots, I_{N} ) $$\n2. We dispose of a set of GO terms (second row) that could be contained in each protein, lets write it $$ G = ( G^{1}, \\dots, G^{K} ) $$\n\nFor each protein ID, our goal is to determine if each possible GO term in G is contained in its structure. So to make it as a formula, if we denote $$ \\forall i \\in I , \\ \\ \\ G_{i} \\subseteq G $$ the subset of GO terms contained in protein i, we want to compute as well as possible the following probabilities : \n\n$$ \\forall g \\in G, \\ \\ \\ p_{i}^{g} = \\mathbb{P}(g \\in G_{i}) $$\n\nHope this helps! Dont hesitate if you have any question! š¤", "2290523": "Hello. Thank you very much for kindly answering my question. So, your equation means \n$$P(g = g_k^j | protein id = I_k)$$\nwhere \\\\(g_k^j\\\\) is the \\\\(j\\\\)-th element of \\\\(G_k\\\\), right? I was thinking the same way, but I was confused because when I perform the calculation using the table above, the sum does not equal 1. For example, the sum of the numbers on the right in the table below is not 1.\n\nP9WHI7 GO:0009274 0.931\nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\n\nIs it correct to say that they should add up to 1 in reality?", "2290527": "No, there is no need for them to add up to one. You don't know in advance how many GO terms correctly describe the function of a test sequence.", "2290545": "Indeed we are in the framework of multilabel classification so they don't sum at 1", "2290617": "Hmm, I might not fully understand the problem of multi-label classification. Should I interpret the numbers in the right column of the table as the output of the following formula?\n$$P(g = g_k^j | protein id = I_k)$$\n\nIf so, according to a certain threshold, for each protein ID I_k, I am considering writing in the submission file the GO terms (g_k^1,... ,g_k^l) with high probability in the sense of the formula, and their probabilities (p_k^1,... ,p_k^l) as follows:\n\n$$ (I_k, g_k^1, p_k^1) $$\n...\n$$ (I_k, g_k^l, p_k^l)$$\n\nIs that a good approach?", "2290626": "I was puzzled because the sum of the numbers in the right column of the table exceeded 1 for a certain protein ID, as shown below:\n\nP9WHI7 GO:0009274 0.931\nP9WHI7 GO:0071944 0.540\nP9WHI7 GO:0005575 0.324\n\nIf my interpretation is still incorrect, could you possibly share some reference materials?", "2290929": "ariafschneider - I think it's perhaps easier to think about those numbers as a measure of how confident you are in each prediction (of a particular GO term for the specific protein). The way in which the scoring metric is calculated finds the confidence threshold at which your F-score is maximised (for each of MF, BP & CC), and in that context ranking your predictions by confidence is more important than precisely scaling them as probabilities - though they do need to be between 0 and 1. Have another look at the Evaluation section under the Overview tab of this Competition's Home Page. Once you understand how the score is calculated, the requirements for the probability (or confidence, if we prefer to think of it that way) numbers are easier to comprehend.\n\nA fun experiment would be to take your best model, divide all the probabilities by 2 and resubmit it. What does the result of that process tell you?" }, "source": "meta" }