ScaleFusion: hierarchical feature aggregation for unified multiscale object detection

Detecting objects across a wide range of scales, particularly small ones, remains a significant challenge in computer vision. Existing methods often improve small object detection at the cost of performance on larger objects or introduce significant computational overhead through external techniques like image slicing. This paper introduces ScaleFusion, a novel, unified, end-to-end object detection architecture designed to provide robust performance across all scales within a single model. The core of our approach is a hierarchical feature aggregation strategy structured like a tree. ScaleFusion processes an image by running a shared backbone network only on fine-grained patches at the lowest level and then progressively fuses these features to build rich, multiscale representations for coarser levels. This bottom-up, patch-based fusion strategy fundamentally differs from traditional top-down Feature Pyramid Networks (FPNs) and provides an integrated alternative to multistage wrappers like Slicing Aided Hyper Inference (SAHI). We demonstrate the performance of ScaleFusion on the VisDrone and TJU-DHD Traffic datasets, where it achieves a mean Average Precision (mAP@0.5) of 38.3% on VisDrone and 78% on TJU-DHD Traffic. Our model improves detection on small objects by 1.5 to 3 times compared to YOLO variants, while also performing competitively on medium and large objects. By achieving a superior balance across the entire scale spectrum, ScaleFusion offers an integrated solution that eliminates the need for complex, multistage inference wrappers.