Unmanned Aerial Vehicles (UAVs), commonly referred to as drones, have gained significant prominence in recent years due to their diverse applications in areas such as surveillance, delivery, agriculture, logistics, disaster response, and military operations. However, the proliferation of drones has also introduced critical challenges such as unauthorized aerial activity, potential threats from payload delivery systems including payload delivery for malicious purposes as well as major security and privacy concerns. This necessitates the detection and tracking of drones, along with the identification of their payloads to ensure safety and security, particularly in sensitive or restricted areas.

Traditional vision-based approaches for drone detection predominantly rely on RGB images, which are often hindered by environmental factors such as low light, fog, or glare. On the other hand, infrared (IR) imaging provides complementary information by capturing thermal signatures enabling robust detection in challenging conditions such as nighttime or occlusion even in cases where the drones are exceedingly far from the field of view (FoV) of the surveillance cameras. Although it may seem that infrared imaging outperforms RGB imaging in such cases, standalone use of IR data may lack the spatial and textural richness provided by RGB images. This makes it essential to combine the two modalities to leverage their complementary strengths.

Alongside drone detection, the ability to detect and identify payloads carried by drones is critical due to the potential risks associated with unauthorized or malicious payload delivery. Payloads may include hazardous materials, surveillance equipment, or contraband, posing threats to public safety, security, and privacy. Identifying payloads in real-time enables proactive measures to mitigate risks making it an essential aspect of drone monitoring systems. However, identifying and analyzing payloads carried by drones poses unique challenges. Payloads may vary in size, shape and thermal properties making it difficult to rely on a single modality for robust detection. RGB imaging provides critical visual cues for recognizing payload shapes and textures while IR imaging can highlight heat signatures of payloads, particularly those carrying heat-generating devices or components. A fusion-based approach is therefore crucial to enhance payload detection accuracy and reliability.