: A lightweight and efficient two-stage video flame detection algorithm is designed to address the high false positive rate, poor adaptability, and low efficiency of flame detection algorithms in complex scenes. In the first stage, an improved Adaptive Gaussian Mixture Model (AGMM) is used to rapidly construct the background model for the video image sequence. By exploiting the flickering and surging characteristics of flames, suspicious candidate regions are extracted from the sequence. In the second stage, a Residual Deep Normalization and Convolutional Neural Network (ResDN) is employed to discriminate the suspicious candidate regions. Additionally, a simplified residual block is introduced to replace the original convolutional layer, enabling lightweight design and achieving accurate flame detection and localization. Compared to traditional classification algorithms, the proposed two-stage video flame detection algorithm can effectively overcome environmental interference in complex scenes, accurately and rapidly identify flames, and exhibit higher detection rates and adaptability.