The phase II of this competition starts, in this phase all the participants will be able to submit their code through a google form, we will be running the scripts on our end and updating the scores on the workshop site leaderboard. You can submit as many scripts as you want but we will be evaluating the latest script from your side and will update the workshop leaderboard every 5 days.
\nFor full details on code requirements and phase II review Overview and Evaluation page.
\nSubmit code here: https://forms.gle/CwfbtZd4k6FwmEFj7
\nWorkshop Leaderboard: https://sites.google.com/corp/view/rcv2023/leaderboard-workshop-challenges
Hi, can you guys please send a feedback so that we could know our submission is valid or invalid, as for now, the leaderboard has not updated yet.
", "rawMarkdown": "Hi, can you guys please send a feedback so that we could know our submission is valid or invalid, as for now, the leaderboard has not updated yet.", "replies": [ { "id": 2303797, "postDate": "2023-06-15T13:31:20.723Z", "content": "Hi Binh, we were setting up an automated way to evaluate submissions; we will email each participant with failed submissions bo 16 June!
", "rawMarkdown": "Hi Binh, we were setting up an automated way to evaluate submissions; we will email each participant with failed submissions bo 16 June!" } ] }, { "id": 2296598, "postDate": "2023-06-12T01:52:49.807Z", "content": "Hi, @rishabh2001
\nI have a question about the Phase II metric.
\nAccording to the description, phase II metric is as follows.
\nTWE = (accuracy^2) / inference_time (in minutes)
Answering your both questions:
\n1) we consider 86 instead of 0.86
\n2) unit of inference time is in minutes as mentioned. For the baseline accuracy is 35.25% and time is 1.70688 minutes.
Hope this helps.
", "rawMarkdown": "Answering your both questions:\n1) we consider 86 instead of 0.86\n2) unit of inference time is in minutes as mentioned. For the baseline accuracy is 35.25% and time is 1.70688 minutes.\n\nHope this helps.", "votes": 1 } ] }, { "id": 2256461, "postDate": "2023-05-12T14:00:15.220Z", "content": "The phase II of this competition starts, in this phase all the participants will be able to submit their code through a google form, we will be running the scripts on our end and updating the scores on the workshop site leaderboard. You can submit as many scripts as you want but we will be evaluating the latest script from your side and will update the workshop leaderboard every 5 days.
\nFor full details on code requirements and phase II review Overview and Evaluation page.
\nSubmit code here: https://forms.gle/CwfbtZd4k6FwmEFj7
\nWorkshop Leaderboard: https://sites.google.com/corp/view/rcv2023/leaderboard-workshop-challenges
Hi, can you guys please send a feedback so that we could know our submission is valid or invalid, as for now, the leaderboard has not updated yet.
", "votes": 0, "replies": [ { "id": 2303797, "author_name": "Arnav Chavan", "author_url": "", "post_date": "2023-06-15T13:31:20.723000", "content": "Hi Binh, we were setting up an automated way to evaluate submissions; we will email each participant with failed submissions bo 16 June!
", "votes": 0, "replies": [] } ] }, { "id": 2296598, "author_name": "shj", "author_url": "", "post_date": "2023-06-12T01:52:49.807000", "content": "Hi, @rishabh2001
\nI have a question about the Phase II metric.
\nAccording to the description, phase II metric is as follows.
\nTWE = (accuracy^2) / inference_time (in minutes)
Answering your both questions:
\n1) we consider 86 instead of 0.86
\n2) unit of inference time is in minutes as mentioned. For the baseline accuracy is 35.25% and time is 1.70688 minutes.
Hope this helps.
", "votes": 1, "replies": [] } ] }, { "id": 2346726, "author_name": "", "author_url": "", "post_date": "2023-07-16T13:21:53.367000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2302966": "Hi, can you guys please send a feedback so that we could know our submission is valid or invalid, as for now, the leaderboard has not updated yet.", "2296598": "Hi, @rishabh2001 \n\nI have a question about the Phase II metric.\n\nAccording to the description, phase II metric is as follows.\n\n`TWE = (accuracy^2) / inference_time (in minutes)`\n\n1. (Accuracy) For example, if we get 86% accuracy, is it 86 or 0.86 in the metric?\n2. (Inference Time) Is the unit of inference time minutes or hours? The leaderboard's baseline score (727.97) is larger than expected, so I want to double check that the units are correct.", "2256461": "The phase II of this competition starts, in this phase all the participants will be able to submit their code through a google form, we will be running the scripts on our end and updating the scores on the workshop site leaderboard. You can submit as many scripts as you want but we will be evaluating the latest script from your side and will update the workshop leaderboard every 5 days.\n\nFor full details on code requirements and phase II review Overview and Evaluation page.\n\nSubmit code here: [https://forms.gle/CwfbtZd4k6FwmEFj7](https://forms.gle/CwfbtZd4k6FwmEFj7)\nWorkshop Leaderboard: [https://sites.google.com/corp/view/rcv2023/leaderboard-workshop-challenges](https://sites.google.com/corp/view/rcv2023/leaderboard-workshop-challenges)", "2346726": "" } }