# Competition Overview “2025 Video and Image Processing Cup Infrared-Visual Fusion for Enhanced Drone Detection, Tracking and Payload Identification in Surveillance Videos” :contentReference[oaicite:19]{index=19} Unmanned Aerial Vehicles (UAVs), commonly referred to as drones, have gained significant prominence in recent years due to their diverse applications in areas such as surveillance, delivery, agriculture, logistics, disaster response, and military operations. However, the proliferation of drones has also introduced critical challenges such as unauthorized aerial activity, potential threats from payload delivery systems including payload delivery for malicious purposes as well as major security and privacy concerns. This necessitates the detection and tracking of drones, along with the identification of their payloads to ensure safety and security, particularly in sensitive or restricted areas. Traditional vision-based approaches for drone detection predominantly rely on RGB images, which are often hindered by environmental factors such as low light, fog, or glare. On the other hand, infrared (IR) imaging provides complementary information by capturing thermal signatures enabling robust detection in challenging conditions such as nighttime or occlusion even in cases where the drones are exceedingly far from the field of view (FoV) of the surveillance cameras. Although it may seem that infrared imaging outperforms RGB imaging in such cases, standalone use of IR data may lack the spatial and textural richness provided by RGB images. This makes it essential to combine the two modalities to leverage their complementary strengths. Alongside drone detection, the ability to detect and identify payloads carried by drones is critical due to the potential risks associated with unauthorized or malicious payload delivery. Payloads may include hazardous materials, surveillance equipment, or contraband, posing threats to public safety, security, and privacy. Identifying payloads in real-time enables proactive measures to mitigate risks making it an essential aspect of drone monitoring systems. However, identifying and analyzing payloads carried by drones poses unique challenges. Payloads may vary in size, shape and thermal properties making it difficult to rely on a single modality for robust detection. RGB imaging provides critical visual cues for recognizing payload shapes and textures while IR imaging can highlight heat signatures of payloads, particularly those carrying heat-generating devices or components. A fusion-based approach is therefore crucial to enhance payload detection accuracy and reliability. :contentReference[oaicite:20]{index=20} ## Timeline 6. Important Dates • Challenge announcement 30 March 2025 • Availability of Training Data – 10th April 2025 • Team Registration Deadline: 30 April 2025 • Availability of Test Data – 20th May 2025 • Final submission due: 15 June 2025 • Finalists announcement: 15 July 2025 • Presentation by top 3 teams and announcement of winners at ICIP 2025: September 14-17, 2025 *(All deadlines follow America/New_York local time as implied by the document context.)* --- # Data Description ## 2.2 Dataset Description **Drone Detection and Tracking Dataset:** Parameter Considered Description Type(s) IR/Thermal, RGB/Visual Modality of the dataset Images and Videos Number of classes 2 (Drone and Bird) Number of images and videos in the train set 45000 IR-RGB image pairs Number of images and videos in the validation set 6500 IR and RGB images + 40 video sequences Number of images and videos in the test set 13000 IR and RGB images + 25 video sequences Frame rate of videos in the dataset 30 fps Resolution of each video 320 x 256 pixels Image size 320 x 256 Duration of each video 10 s Environmental factors considered Hilly regions, thick forest cover, cloudy sky, fog and mist Scenarios Single class in a frame, multiple classes in a frame, objects far off from the FoV of the source camera, swarm of birds and drones, drones captured under extreme environmental conditions as stated above Distortions considered Speckle noise, salt and pepper noise, Gaussian blur, uneven illumination, motion bur, camera instability and AWGN Number of videos with distortions 30 Intensity of distortions (on a scale of 5) - Speckle noise – 2 - Salt and Pepper noise – 2 - Gaussian blur – 4 - Uneven illumination – 4 - Motion blur – 4 - Camera instability – 5 - AWGN – 3 :contentReference[oaicite:21]{index=21} **Payload Identification Dataset:** Parameter Considered Description Type(s) IR/Thermal, RGB/Visual Modality of the dataset Images Number of classes 2 (Harmful payload and Normal payload) Number of images in the train set 25000 IR-RGB image pairs Number of images in the validation set 4000 IR and RGB images Number of images in the test set 8000 IR and RGB images Image size 320 x 256 Environmental factors considered Hilly regions, thick forest cover, cloudy sky, fog and mist Scenarios Single class in a frame, multiple classes in a frame, objects far off from the FoV of the source camera, swarm of birds and drones, drones captured under extreme environmental conditions as stated above Distortions considered Speckle noise, salt and pepper noise, Gaussian blur, uneven illumination, motion bur, camera instability and AWGN Intensity of distortions (on a scale of 5) - Speckle noise – 2 - Salt and Pepper noise – 2 - Gaussian blur – 4 - Uneven illumination – 4 - Motion blur – 4 - Camera instability – 5 - AWGN – 3 :contentReference[oaicite:22]{index=22} ## 2.5 Data organization The data is organized as follows: Main Directory Sub Directory Dataset Split Type Contents Data/ RGB/ Train/ Folder Images/ , Labels/ Validation/ Folder Images/ , Labels/ Test/ Folder Images/ , Labels/ IR/ Train/ Folder Images/ , Labels/ Validation/ Folder Images/ , Labels/ Test/ Folder Images/ , Labels/ Payload/ Train/ Folder Images/ , Labels/ Validation/ Folder Images/ , Labels/ Test/ Folder Images/ , Labels/ Dataset_Statistics.txt - File Describes dataset statistics Challenge_Description.txt - File Describes the challenge :contentReference[oaicite:23]{index=23} *(Use exactly this folder structure when packaging data on Kaggle Datasets.)* --- # Evaluation Metrics ## 3. Evaluation Metrics (from Section 3 and 2.4 Evaluation Protocols) ### Task 1: RGB + IR Drone Detection Final Composite Score Formula: · S_det = ((A + F₁ + P + R) / 4) × R_b × S_d Where: · • A = Accuracy (Normalized as a decimal, e.g., 92% → 0.92) · • F₁ = F1 Score (Normalized similarly to Accuracy) · • P = Precision (Normalized as a decimal) · • R = Recall (Normalized as a decimal) · • R_b = Normalized Robustness Score - Calculated as the average ratio of the performance metric (F1 Score) achieved under each adverse condition to a benchmark maximum for that condition. - Example: R_b = (1/N) * Σ (F1_i / F1_max_i), for N adverse conditions. · • S_d = Normalized Inference Speed - Calculated as the ratio of the model's achieved FPS to a target FPS (e.g., 30 FPS), capped at 1. - Formula: S_d = min(1, FPS_model / FPS_target). :contentReference[oaicite:24]{index=24} ### Task 2: RGB + IR Drone Tracking Final Composite Score Formula: · S_track = ((I + C + D) / 3) × S_t Where: · • I = Average Intersection-over-Union (IoU) between predicted and ground truth trajectories · • C = Tracking Consistency factor - C = 1 - (M / 15), where M is the number of consecutive missed frames (for M < 15; if M ≥ 15, C = 0) · • D = Directional Accuracy - Indicates the accuracy in determining whether the drone is approaching or receding from the camera’s field of view. · • S_t = Normalized Inference Speed for Tracking - Computed similarly to S_d, using the FPS achieved during tracking relative to a target FPS. :contentReference[oaicite:25]{index=25} ### Task 3: Payload Detection Final Composite Score Formula: · S_payload = (C_score + mAP) / 2 Where: · • C_score = (A + F₁ + P + R) / 4 - A, F₁, P, and R are the standard classification metrics for detecting payload types, each normalized to [0, 1]. · • mAP = Mean Average Precision - Assesses the overall detection quality, especially in scenarios with multiple payload instances, normalized to [0, 1]. :contentReference[oaicite:26]{index=26} ### Overall Team Ranking Final Overall Ranking Formula: · S_overall = w₁ * S_det + w₂ * S_track + w₃ * S_payload For equal weighting (w₁ = w₂ = w₃ = 1/3): · S_overall = (S_det + S_track + S_payload) / 3 :contentReference[oaicite:27]{index=27} *(This ensures performance across detection, tracking, and payload identification is integrated exactly as in the Official Document.)* --- # Competition Rules ## 4. Participants submissions (Section 4) 4.1 Results Submission Guidelines: Each participating team must submit their complete solution in a zipped file via the challenge webpage. The submission must adhere to the following specifications: ● File Naming Format: Name your zipped file as: TeamName_IR_RGB_Drone_ICIP2025 ● Contents of the Zipped File: 1. Main Code - containing the complete implementation details with comments. 2. Demo Code: a separate or integrated demo code that can be executed on a test video to evaluate the submitted solution. Note: Incomplete submissions that do not include scores/results for all trials or required components will be disqualified. :contentReference[oaicite:28]{index=28} 4.2 Technical Report Each team must include a short technical report in their zipped submission. This report should provide a clear summary of your approach and must cover the following areas: ● System Architecture: ○ Describe your fusion-based approach that integrates RGB and infrared (IR) data for enhanced drone detection, tracking, and payload identification. ○ Explain the data preprocessing steps, sensor fusion techniques, and overall algorithmic pipeline. ● Algorithmic Implementation: ○ Provide a detailed explanation of the algorithms used for: ■ Detecting drones and differentiating them from birds in images/videos. ■ Tracking drones, including methods for computing the trajectory and determining whether a drone is approaching or receding from the FoV of the camera source. ■ Detecting the type of distortions (e.g., low lighting, fog, motion blur) and handling these challenges. ○ Justify your design choices and any optimization techniques employed. ● Demo Code Description: ○ Explain how the demo code functions, highlighting how detection confidence scores, distortion types, bounding box outputs, and drone trajectories are visualized. Describe how the demo code facilitates real-time evaluation of your solution. ● Performance Metrics: ○ Summarize the performance of your system using the evaluation metrics defined in Section 3 (e.g., accuracy, IoU, mAP, inference speed). ● System Specifications: ○ Detail the hardware and software environment used, including inference times on both CPU and GPU. Submission Deadline: Ensure that your complete submission (zipped file with all required items) is received by the final submission deadline. :contentReference[oaicite:29]{index=29} ## 5. SPS Competition Terms & Conditions (Section 5.1) 5.1.a. Team Composition ⮚ Each team MUST be composed of: (i) One faculty member (the Supervisor) and (ii) At least 3 but no more than 10 undergraduates; Optionally (iii) At most one graduate student (the Tutor). o At least three of the undergraduate team members must be SPS student members at time of team registration, and maintain this status until the end of the competition o Faculty (Supervisor): Postdocs and research associates are not considered as faculty members. o Graduate Student (Tutor): A graduate student is a student having earned at least a 4-year University degree at the time of submission. Please note: Tutors are not eligible to receive travel grants or prize money. o Undergraduate: An undergraduate student is a student without a 4-year degree. ⮚ Team members cannot be changed after the team registration deadline. ⮚ At least one undergraduate team member must be in attendance (in-person) of the final round of the competition held at the respective conference (ICASSP or ICIP) to present the team’s work.* *Important notice: Upon registering a team for the competition, the team must commit to at least one undergraduate team member representing the team by attending the physical competition and participating in the final round of the competition at the physical conference. Should a team not be able to participate physically (in-person) in the final round of the competition held at the respective conference (ICASSP or ICIP) for any reason, at any point in the competition, then the team must notify SPS Staff and organizers immediately. This will result in the team being ineligible to continue in the competition, therefore forfeiting their position in the competition. Teams must make every effort to attend the final round at the conference; visa issues may be an exception. If all team members are unable to obtain visas, please be prepared to present proof of visa process, communication to obtain visa, as well as a visa denial. All eligibility decisions are at the discretion of the SPS Student Services Committee and competition organizers. Should a team be disqualified or forfeit their finalist position for any reason, the next team selected by the organizers may be contacted to compete in the final round, following the same guidelines as above. :contentReference[oaicite:30]{index=30} 5.1.b Team ineligibility (Further clarification) Specific team ineligibility in addition to the above. Any of these criteria will result in the team being disqualified/ineligible to continue in the competition: ⮚ Teams that are composed with 50% or more of its undergraduate or tutor members being students who have previously participated on a finalist team within the last year are not eligible. For example, if two of four undergraduate team members participated as a member of a finalist team during the 2023 SP Cup will not be eligible to participate on a team for the 2023 VIP Cup. ⮚ Any team members who have placed in the top three teams of any SPS competition held during the previous conference, i.e. a member from one of the 3 finalist teams of the 2023 SP Cup (at ICASSP) will not be eligible to participate in the 2023 VIP Cup (at ICIP). Any team member that has received SPS travel support within the previous year would not be eligible to receive travel support again if their team is selected as a finalist. :contentReference[oaicite:31]{index=31} ## 6. Prizes for Finalists (Section 5.4) The [eligible] three teams with the highest performance in the open competition based on the judging criteria will be selected as finalists and invited to participate in the final competition at ICASSP or ICIP. The champion team will receive a Grand Prize of $5,000. The first and the second runner-up teams will receive a prize of $2,500 and $1,500, respectively, in addition to travel grants and complimentary conference registrations for up to three team members. ⮚ Up to three undergraduate student members from each finalist team will be provided travel support to attend the conference in-person. In-person attendance of the physical conference is required for reimbursement. o Travel grant funds are offered on a reimbursement basis of up to $1,200 for continental travel and $1,700 for intercontinental travel. There are no exceptions. o Funds will be issued by way of a bank transfer after the competition via Concur. o Receipts for reimbursement must be uploaded into Concur ⮚ Complimentary conference registration for up to three undergraduate team members. o These complimentary conference registrations cannot be used to cover any papers accepted by the conference. o You must notify Jaqueline Rash, Jaqueline.rash@ieee.org, via email of the three team members who have been chosen to receive complimentary registration. o VISA information: Once registered for the conference, each individual will have the opportunity to request an invitation letter through the conference website to be used for the visa application process. This is the only way to receive a letter for visa purposes. ⮚ The finalist teams will also be invited to join the Conference Banquet and the SPS Student Job Fair, so that they can meet and talk to SPS leaders and global experts. Please note registration for the Conference Banquet and Student Job Fair is limited and based on availability. You must add this event to your registration. If you are unable to add these events, then maximum capacity has been reached. There may be additional availability for the Student Job Fair, but not the Conference Banquet. You can inquire by emailing Jaqueline.Rash@ieee.org. :contentReference[oaicite:32]{index=32} ## 7. Registration and Important Resources (Section 7) 7.1 Official SP Cup Team Registration All teams must be registered through the official competition registration system in order to be considered as a participating team. Official team registration must be submitted via: https://www2.securecms.com/VIPCup/VIPRegistration.asp All teams MUST acknowledge and agree to the ‘SP and VIP Cup Terms & Conditions’ to be eligible for the competition. :contentReference[oaicite:33]{index=33} --- # Contacts Competition Organizers (technical, competition-specific inquiries): Vishnu Ravishankar Email: vishnuravishankar27@gmail.com Dharini Raghavan Email: dhariniraghavan2001@gmail.com Dr S Sethu Selvi Email: selvi@msrit.edu Dr Raghuram S Email: raghuram@msrit.edu Dr Sitaram Ramachandrula Email: NagaVenkata.R@247.ai Shefali Singh Email: singhshefali921@gmail.com Sangeetha Kar Email: sangeeta.you@gmail.com Suman Jangid Email: sumanjangid1782@gmail.com SPS Staff (Terms & Conditions, Travel Grants, Prizes): Jaqueline Rash, SPS Membership Program and Events Manager Email: Jaqueline.Rash@ieee.org SPS Student Services Committee Lucas Thomaz, Chair Email: lucas.thomaz@smt.ufrj.br :contentReference[oaicite:34]{index=34}