\\[IoU(A,B)= \\frac{A \\cap B}{A \\cup B}\\]
\n\nThe metric sweeps over a range of IoU thresholds, at each point calculating an F1 Score. The threshold values range from 0.5 to 0.95 with a step size of 0.05: `(0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95)`. In other words, at a threshold of 0.5, a predicted object is considered a \"hit\" if its intersection over union with a ground truth object is greater than 0.5.\n\nAt each threshold value *t*, the F1 Score value is calculated based on the number of true positives (TP), false negatives (FN), and false positives (FP) resulting from comparing the predicted object to all ground truth objects as follows:\n\n\\[F_1(t)= \\frac{2TP}{2TP + FN + FP}\\]
\n\nA true positive is counted when a single predicted object matches a ground truth object with an IoU above the threshold. A false positive indicates a predicted object had no associated ground truth object. A false negative indicates a ground truth object had no associated predicted object. The average F1 Score of a single image is then calculated as the mean of the above F1 Score values at each IoU threshold:\n\n\\[\\frac{1}{|thresholds|} \\sum_t F_1(t) \\]
\n\nLastly, the score returned by the competition metric is the mean taken over the individual average F1 Scores of each image in the test dataset.\n\n### Submission File\nThe submission file expect two columns: \n\n- ID: The unique sample identifier (image name);\n- Mask: The predicted binary detection mask.\n\nThe metric uses run-length encoding on the pixel values. Thus, instead of submitting an exhaustive list of indices for your detection, you will submit pairs of values that contain a start position and a run length. E.g. '1 3' implies starting at pixel 1 and running a total of 3 pixels (1,2,3).\n\nThe competition format requires a space delimited list of pairs. For example, '1 3 10 5' implies pixels 1,2,3,10,11,12,13,14 are to be included in the mask. The pixels are one-indexed and numbered from top to bottom, then left to right: 1 is pixel (1,1), 2 is pixel (2,1), etc. A prediction of \"no egg in the image\" should have a blank value in the Mask column.\n\nThe metric checks that the pairs are sorted, positive, and the decoded pixel values are not duplicated. It also checks that no two predicted masks for the same image are overlapping.\n\nThe submission file should look like this:\nID,Mask", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": "### One account per participant\n\nYou cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.\n\n### No private sharing outside teams\n\nPrivately sharing code or data outside of teams is not permitted. It's okay to share code if made available to all participants on the forums.\n\n### Submission Limits\n\nYou may submit a maximum of 1 entry per day.\n\nYou may select up to 1 final submission for judging.\n\n### COMPETITION FRAMEWORK\n\nThese are the complete, official rules for the Competition (the 'Competition Rules') and incorporate by reference the contents of the Competition Website listed above.\n\nBy downloading a dataset linked from the Competition Website, submitting an entry to this Competition, or joining a Team in this Competition, you are agreeing to be bound by these Competition Rules which constitute a binding agreement between you and contest organizers.\n\nNote that Competition deadlines are subject to change and additional hurdle deadlines may be introduced during the Competition. Any additional or altered deadlines not already described in these Competition Rules will be publicized to the Participants via the Competition Website. It is the Participant's responsibility in this Competition to check the website regularly throughout the Competition to stay informed of any new or updated deadlines. \n\nEach registered individual is referred to as a Participant. You may only participate using a single, unique Kaggle account registered at http://www.kaggle.com. Participating using more than one Kaggle account per individual Participant is a breach of these Competition Rules and Competition Sponsor reserves the right to disqualify any Participant (or Team including Participant) who is found to breach these Competition Rules.\n\n### ELIGIBILITY\n\nThe Competition is open to all individuals over the age of 18 at the time of entry and to all validly formed legal entities that have not declared or been declared in bankruptcy. \n\n### SUBMISSIONS\n\n'Submission' means the material submitted by you in the manner and format specified on the Website via the Submission form on the Website. You may submit up to the maximum number of Submissions per day as specified above. All Submissions must be uploaded to the Website in the manner and format specified on the Website. Submissions must be received prior to the Competition deadline and adhere to the guidelines for Submissions specified on the Website.\n\nSubmissions may not use or incorporate information from hand labelling o or human prediction of the validation dataset or test data records.\n\n### INTELLECTUAL PROPERTY\n\n#### DATA\n\n'Data' means the Data or Datasets linked from the Competition Website for the purpose of use by Participants in the Competition. For the avoidance of doubt, Data is deemed for the purpose of these Competition Rules to include any prototype or executable code provided to Participants by Kaggle via the Website. Participants must use the Data only as permitted by these Competition Rules and any associated data use rules specified on the Competition Website.\n\nUnless otherwise permitted by the terms of the Competition Website, Participants must use the Data solely for the purpose and duration of the Competition, including but not limited to reading and learning from the Data, analyzing the Data, modifying the Data and generally preparing your Submission and any underlying models and participating in forum discussions on the Website. Participants agree to use suitable measures to prevent persons who have not formally agreed to these Competition Rules from gaining access to the Data and agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Data to any party not participating in the Competition. Participants agree to notify Kaggle immediately upon learning of any possible unauthorized transmission or unauthorized access of the Data and agree to work with Kaggle to rectify any unauthorized transmission. Participants agree that participation in the Competition shall not be construed as having or being granted a license (expressly, by implication, estoppel, or otherwise) under, or any right of ownership in, any of the Data.\n\n#### EXTERNAL DATA\n\nThe use of third-parties publicly available datasets, from the same domain or from different ones, is allowed, but their use must be declared to the challenge organisers. \n\n#### CODE SHARING\n\nParticipants are prohibited from privately sharing source or executable code developed in connection with or based upon the Data, and any such sharing is a breach of these Competition Rules and may result in disqualification.\n\n### WINNERS' OBLIGATIONS\n\n#### PARTICIPANT INTELLECTUAL PROPERTY LICENSING\n\nAs a further condition, each winning Participant thereby licenses their winning Submission and the source code used to generate the Submission according to the Winner License Type specified above.\n\n#### CHEATING\n\nParticipating using more than one Kaggle account is deemed cheating and, if discovered, will result in disqualification from the Competition and any other affected Competitions and may result in banning or deactivation of affected Kaggle accounts.\n\n### WARRANTIES AND OBLIGATIONS\n\n#### PARTICIPANT WARRANTIES AND OBLIGATIONS\n\nBy registering, you agree that (a) your Account is complete, correct and accurate and (b) your registration may be rejected or terminated and all Entries submitted by you and/or your Team may be disqualified if any of the information in your Account is (or Competition Sponsor has reasonable grounds to believe it is) incomplete, incorrect or inaccurate. You are solely responsible for your Account. All registration information is deemed collected in the United States.\n\nParticipation is subject to all federal, state and local laws and regulations. Void where prohibited or restricted by law. You are responsible for checking applicable laws and regulations in your jurisdiction before participating in the Competition to make sure that your participation is legal. You are responsible for all taxes and reporting related to any award that you may receive as part of the Competition. You are responsible for abiding by your employer's policies regarding participation in the Competition. Competition Sponsor disclaims any and all liability or responsibility for disputes arising between you and your employer related to this Competition.\n\nEach Participant is solely responsible for all equipment, including but not necessarily limited to a computer and internet connection necessary to access the Website and to develop and upload any Submission, and any telephone, data, hosting or other service fees associated with such access, as well as all costs incurred by or behalf of the Entrant in participating in the Competition.\n\nBy entering a Submission, you represent and warrant that all information you enter on the Website is true and complete to the best of your knowledge, that you have the right and authority to make the Submission (including any underlying code and model) on your own behalf or on behalf of the persons and entities that you specify within the Submission, and that your Submission:\n\n* is your own original work, or is used by permission, in which case full and proper credit and identify is given and the third party contributions are clearly identified within your Submission;\n* does not contain confidential information or trade secrets and is not the subject of a registered patent or pending patent application;\n* does not violate or infringe upon the patent rights, industrial design rights, copyrights, trademarks, rights of privacy, publicity or other intellectual property or other rights of any person or entity;\n* does not contain malicious code, such as viruses, timebombs, cancelbots, worms, Trojan horses or other potentially harmful programs or other material or information;\n* does not and will not violate any applicable law, statute, ordinance, rule or regulation;\n* does not trigger any reporting or royalty obligation to any third party; and\n* was not previously published and has not won any other prize/award. \n A breach of any of these warranties will result in the corresponding Submission being invalid.\n\n#### LIMITATION OF LIABILITY\n\nBy participating in the Competition, each Participant agrees to release, indemnify and hold harmless Competition Sponsor, Kaggle, their respective affiliates, subsidiaries, advertising and promotions agencies, as applicable, and each of their respective agents, representatives, officers, directors, shareholders, and employees from and against any injuries, losses, damages, claims, actions and any liability of any kind resulting from or arising out of your participation in or association with the Competition. Competition Sponsor is not responsible for any miscommunications such as technical failures related to computer, telephone, cable, and unavailable network or server connections, related technical failures, or other failures related to hardware, software or virus, or incomplete, late or misdirected Submissions. Competition Sponsor reserves the right to cancel, modify or suspend the Competition should any computer virus, bug or other technical difficulty or other causes beyond the control of Competition Sponsor corrupt the administration, security or proper play of the Competition, and to determine winners from among Submission not affected by the corruption, if any, in its sole discretion.\n\nNeither Kaggle nor Competition Sponsor are responsible for (a) late, lost, stolen, damaged, garbled, incomplete, incorrect or misdirected Entries or other communications, (b) errors, omissions, interruptions, deletions, defects, or delays in operations or transmission of information, in each case whether arising by way of technical or other failures or malfunctions of computer hardware, software, communications devices, or transmission lines, or (c) data corruption, theft, destruction, unauthorized access to or alteration of Submission materials, loss or otherwise. Neither Kaggle nor Competition Sponsor are responsible for electronic communications or emails which are undeliverable as a result of any form of active or passive filtering of any kind, or insufficient space in any email account to receive email messages. Competition Sponsor disclaims any liability for damage to any computer system resulting from participation in, or accessing or downloading information in connection with, the Competition.\n\n#### RESERVATION OF RIGHTS\n\nCompetition Sponsor reserves the right to modify the Start Date, End Date, and the right to modify, add or remove additional Competition deadlines for any reason. Any changes will be communicated through the Competition Website. Competition Sponsor also reserves the right to modify, remove or add Data to the Website upon notice via the Competition Website. NEITHER KAGGLE NOR COMPETITION SPONSOR ARE RESPONSIBLE FOR ANY FAILURE OF A PARTICIPANT TO RECEIVE DATA CHANGES.\n\n### MISCELLANEOUS\n\n#### SEVERABILITY\n\nThe invalidity or unenforceability of any provision of these Competition Rules shall not affect the validity or enforceability of any other provision. In the event that any provision is determined to be invalid or otherwise unenforceable or illegal, these Competition Rules shall otherwise remain in effect and be construed in accordance with their terms as if the invalid or illegal provision was not contained herein.", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "The Challenge", "content": "### Challenge periods\nThe challenge is based on images acquired by a KFM from samples collected by using several devices. Each FLOTAC/Mini-FLOTAC sample generates very high-resolution images, split in hundreds of overlapping patches, saved in jpeg or png format. This results in thousands of images made available for the competition, unpublished and not provided to other people before. Each patch may or may not contain parasites’ eggs. For training data, participants are also provided with ROI (region of interest) files containing, for each image, the coordinates of the bounding box of parasites within it, as identified and segmented by an expert operator. Test data is provided without the ROI files. The use of third-parties publicly available datasets, from the same domain or from different ones, is allowed, but their use must be declared to the challenge organizers. \n\nTraining and test datasets belong to different acquisition samples (i.e. different FLOTAC and/or Mini-FLOTAC devices). In particular, the test will happen in two different moments: \n\n1. during the first period of the competition (day 1 up to 1 day before the deadline), a test sample (images coming from a single FLOTAC and/or Mini-FLOTAC device) will be made available for participants to test the performance of their models;\n2. on the last day, a new test dataset (with images coming from more than one FLOTAC and/or Mini-FLOTAC device) will be released. THIS WILL BE THE DATA USED TO DETERMINE THE WINNERS. \n\nA subset of the test data will be used for the public leaderboard, while the remaining part will be only used for the private leaderboard. During the first period, 100% of the data will be used for the public leaderboard, while during the second period (the last day), only 5% of the data will be used for the public leaderboard. Teams will be allowed a maximum of 1 submission per day. This means that they will have a single submission on the last day to submit a prediction for the real test set. \n\nThe submission will consist of a .csv file where the participants have to report, on different lines and for each image, a pixel-encoded binary mask (please refer to the evaluation page for details) identifying all the detected eggs (if any).", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Tracks", "content": "Two are the available tracks (the first mandatory, the second optional). For both tracks the participants are asked to submit a Unix executable (or python code, as preferred). This will be used to reproduce the obtained results (both tracks) and to measure inference speed under the same workload conditions (Track 2). Teams interested in taking part in Track 2 have to write their will on a topic on the discussion board. \n\n### Track 1\n\nDetection of parasites’ eggs. The F1 Score at different intersection over union (IoU) thresholds will be used as performance metric (see the Evaluation page). In case of a draw, the team with the lowest number of submissions made will be declared the winner.\n\n### Track 2\n\nAlso the inference time (on the test set, run on a HPC unix machine equipped with GPU acceleration) will be considered to declare the winner, according to the following formula:\n\n
micro_00_00, 2 45 80 2
micro_00_01, 4 1 8 3 15 10
micro_00_02,
micro_00_03, 1 2
\\[score = 2 * \\frac{F_1 * T_n}{F_1 + T_n}\\]
\n\nwhere *Tn* is the min-max normalized execution time (median value over 10 repetitions) between all the participants.", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Publications", "content": "The two winners will be invited to describe their solutions during the 21st International Conference on Image Analysis and Processing (ICIAP 2021). Authors of top performers, innovative and promising solutions will be invited to submit a scientific paper describing their solution to a special issue in a top-ranked international journal. For this aim, late submissions will be accepted, as the data and the competition website will be kept available after the challenge deadline. The aim is to make the competition a standardized common experimental protocol, providing researchers with a large number of samples collected in a well-known environment, with a set of scores for competitors’ approaches already available.", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }