Hello, @inversion , I am very sorry, I mistakenly predicted too many GO terms, with each protein reaching up to 4k -3w, and forgot to extract the best 1500. May I ask how to calculate this? Will the 1500 GO terms with the highest score be selected for each protein?
", "messages": [ { "id": "2404333", "postDate": "08/23/2023 07:26:45", "content": "Hello, @inversion , I am very sorry, I mistakenly predicted too many GO terms, with each protein reaching up to 4k -3w, and forgot to extract the best 1500. May I ask how to calculate this? Will the 1500 GO terms with the highest score be selected for each protein?
", "rawMarkdown": "Hello, @inversion , I am very sorry, I mistakenly predicted too many GO terms, with each protein reaching up to 4k -3w, and forgot to extract the best 1500. May I ask how to calculate this? Will the 1500 GO terms with the highest score be selected for each protein?", "votes": null }, { "id": "2404364", "postDate": "08/23/2023 07:48:35", "content": "Hi @songqizhou,
\nActually, the limitation (i.e., max_terms) is 500 terms for one protein per subontology; that is, the very first 500 (non-repeating) terms are taken into consideration, all of the latter ones are truncated. As can be seen in the pred_parser function of parser.py, the pred_file is parsed sequentially,
(pred_file) f:\n line f:\n line = line.strip().split()\n line (line) > :\n p_id, term_id, prob = line[:]\n ns = ns_dict.get(term_id)\n ns gts p_id gts[ns].ids:\n i = gts[ns].ids[p_id]\n \n max_terms np.count_nonzero(matrix[ns][i]) <= max_terms:\n \n j = onts[ns].terms_dict.get(term_id)[]\n ids[ns][p_id] = i\n matrix[ns][i, j] = (matrix[ns][i, j], (prob))\nThe main points are:
\nmax_terms is set to 500.max_terms limitation is imposed on each subontology separately.Once the max_terms is reached for one protein in one subontology, all the latter predictions will be discarded.
Hope this helps, thanks.
", "rawMarkdown": "Hi @songqizhou,\n\nActually, the limitation (*i.e.,* `max_terms`) is **500 terms for one protein per subontology**; that is, the very first 500 (non-repeating) terms are taken into consideration, all of the latter ones are truncated. As can be seen in the `pred_parser` function of [`parser.py`](https://github.com/BioComputingUP/CAFA-evaluator/blob/master/src/parser.py), the `pred_file` is parsed sequentially,\n\n```python\nwith open(pred_file) as f:\n for line in f:\n line = line.strip().split()\n if line and len(line) > 2:\n p_id, term_id, prob = line[:3]\n ns = ns_dict.get(term_id)\n if ns in gts and p_id in gts[ns].ids:\n i = gts[ns].ids[p_id]\n # ===\n if max_terms is None or np.count_nonzero(matrix[ns][i]) <= max_terms:\n # ===\n j = onts[ns].terms_dict.get(term_id)['index']\n ids[ns][p_id] = i\n matrix[ns][i, j] = max(matrix[ns][i, j], float(prob))\n```\n\nThe main points are:\n1. `max_terms` is set to 500.\n2. The `max_terms` limitation is imposed on each subontology **separately**.\n3. If there exists any duplicated protein-term pair which has been recorded, then the max proba will be taken.\n\nOnce the `max_terms` is reached for one protein in one subontology, all the latter predictions will be discarded.\n\nHope this helps, thanks.", "votes": null }, { "id": "2404434", "postDate": "08/23/2023 08:54:57", "content": "Thank you very much @abaojiang .It means that it is selected in the order of the list, and the order of each protein term in my results is not in order of probability, which has resulted in a much lower score for me. It is indeed regrettable.😭
", "rawMarkdown": "Thank you very much @abaojiang .It means that it is selected in the order of the list, and the order of each protein term in my results is not in order of probability, which has resulted in a much lower score for me. It is indeed regrettable.😭", "votes": null }, { "id": "2404827", "postDate": "08/23/2023 14:27:21", "content": "It took me much time to figure out how the official evaluator works (it's indeed complicated…) and do some optimization to make it run faster for local CV.
\nGood luck with your final rank on private LB 😉.
Thank you Abao, I also hope good luck accompanies you! 愿好运与你相伴！😄
", "rawMarkdown": "Thank you Abao, I also hope good luck accompanies you! 愿好运与你相伴！😄", "votes": null } ], "comments": [ { "id": 2404364, "author_name": "abaojiang", "author_url": "", "post_date": "08/23/2023 07:48:35", "content": "Hi @songqizhou,
\nActually, the limitation (i.e., max_terms) is 500 terms for one protein per subontology; that is, the very first 500 (non-repeating) terms are taken into consideration, all of the latter ones are truncated. As can be seen in the pred_parser function of parser.py, the pred_file is parsed sequentially,
(pred_file) f:\n line f:\n line = line.strip().split()\n line (line) > :\n p_id, term_id, prob = line[:]\n ns = ns_dict.get(term_id)\n ns gts p_id gts[ns].ids:\n i = gts[ns].ids[p_id]\n \n max_terms np.count_nonzero(matrix[ns][i]) <= max_terms:\n \n j = onts[ns].terms_dict.get(term_id)[]\n ids[ns][p_id] = i\n matrix[ns][i, j] = (matrix[ns][i, j], (prob))\nThe main points are:
\nmax_terms is set to 500.max_terms limitation is imposed on each subontology separately.Once the max_terms is reached for one protein in one subontology, all the latter predictions will be discarded.
Hope this helps, thanks.
", "votes": null, "replies": [ { "id": 2404434, "author_name": "songqizhou", "author_url": "", "post_date": "08/23/2023 08:54:57", "content": "Thank you very much @abaojiang .It means that it is selected in the order of the list, and the order of each protein term in my results is not in order of probability, which has resulted in a much lower score for me. It is indeed regrettable.😭
", "votes": null, "replies": [ { "id": 2404827, "author_name": "abaojiang", "author_url": "", "post_date": "08/23/2023 14:27:21", "content": "It took me much time to figure out how the official evaluator works (it's indeed complicated…) and do some optimization to make it run faster for local CV.
\nGood luck with your final rank on private LB 😉.
Thank you Abao, I also hope good luck accompanies you! 愿好运与你相伴！😄
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "2404333": "Hello, @inversion , I am very sorry, I mistakenly predicted too many GO terms, with each protein reaching up to 4k -3w, and forgot to extract the best 1500. May I ask how to calculate this? Will the 1500 GO terms with the highest score be selected for each protein?", "2404364": "Hi @songqizhou,\n\nActually, the limitation (*i.e.,* `max_terms`) is **500 terms for one protein per subontology**; that is, the very first 500 (non-repeating) terms are taken into consideration, all of the latter ones are truncated. As can be seen in the `pred_parser` function of [`parser.py`](https://github.com/BioComputingUP/CAFA-evaluator/blob/master/src/parser.py), the `pred_file` is parsed sequentially,\n\n```python\nwith open(pred_file) as f:\n for line in f:\n line = line.strip().split()\n if line and len(line) > 2:\n p_id, term_id, prob = line[:3]\n ns = ns_dict.get(term_id)\n if ns in gts and p_id in gts[ns].ids:\n i = gts[ns].ids[p_id]\n # ===\n if max_terms is None or np.count_nonzero(matrix[ns][i]) <= max_terms:\n # ===\n j = onts[ns].terms_dict.get(term_id)['index']\n ids[ns][p_id] = i\n matrix[ns][i, j] = max(matrix[ns][i, j], float(prob))\n```\n\nThe main points are:\n1. `max_terms` is set to 500.\n2. The `max_terms` limitation is imposed on each subontology **separately**.\n3. If there exists any duplicated protein-term pair which has been recorded, then the max proba will be taken.\n\nOnce the `max_terms` is reached for one protein in one subontology, all the latter predictions will be discarded.\n\nHope this helps, thanks.", "2404434": "Thank you very much @abaojiang .It means that it is selected in the order of the list, and the order of each protein term in my results is not in order of probability, which has resulted in a much lower score for me. It is indeed regrettable.😭", "2404827": "It took me much time to figure out how the official evaluator works (it's indeed complicated...) and do some optimization to make it run faster for local CV. \nGood luck with your final rank on private LB 😉.", "2405023": "Thank you Abao, I also hope good luck accompanies you! 愿好运与你相伴！😄" }, "source": "meta" }