Adversarial Malware Detection on Consumer Devices Using Optimized Image-Based Ensembles

Malware is a constant attack to consumer gadgets, and it targets a wide range of file types and operating systems. To combat this, researchers have put much effort into creating malware detection algorithms that utilize machine learning and deep learning. Despite these advancements, adversarial attacks can still undermine security with specifically designed inputs and avoid detection. The proposed malware detection methodology improves malware classification accuracy and adversarial attack resistance with multistage deep learning. The methodology starts by turning malware binaries into 2D color images for visual inspection. Adversarial examples are generated using the fast gradient sign method, projected gradient descent, and Carlini–Wagner attacks to evaluate model resistance. Genetic algorithms are used to hyperparameter optimize the fundamental classification models, such as Xception, Inception V3, and VGG19, guaranteeing the selection of optimal settings. The models are merged into an ensemble to improve detection accuracy and classify malware families efficiently. The proposed approach achieved high detection accuracy of 97.86% and 94.24% using dumpware10 and MaleVis datasets. Results from this study provide a basis for smart home gadgets and smartphones to incorporate enhanced malware detection capabilities, which have far-reaching implications for the consumer electronics market.