To enhance the computational efficiency and cross-path feature interaction of the SlowFast network in action recognition tasks, a video dynamic sparse token (VDST) selection module and a bidirectional gated cross-attention module (Bi-CAM) are incorporated into the original dual-path framework. By adaptively selecting key spatiotemporal regions in the fast pathway and establishing fine-grained bidirectional semantic fusion between the slow and fast pathways, the trade-off between computational complexity and recognition performance in the improved SlowFast architecture is investigated. It is shown that redundant feature computation is effectively reduced and model accuracy is maintained by the VDST module while FLOPs(floating point operations) are significantly lowered; semantic interaction across pathways is enhanced and more comprehensive action representations are obtained by the Bi-CAM module; and a top-1 accuracy of 95.5% on the UCF101 dataset with only 33.6 GFLOPs(giga floating point operations) is achieved by the combined model. It is demonstrated that computational efficiency is substantially improved and high recognition accuracy is preserved by the proposed multi-module enhanced SlowFast framework, providing a feasible solution for efficient video understanding.
LUO Jun-xin, WANG Long-ye, ZENG Xiao-li, DENG Xing-yu, LAN Xin.
Dual-path Video Action Recognition Method Fusing Dynamic Token Sparsification and Cross-path Attention[J].
Science Technology and Engineering, 2026, 26(31): 13546-13554 DOI:10.12404/j.issn.1671-1815.2508424
WangZhiguo, ZhangYujin. Anomaly detection in surveillance videos: a survey[J]. Journal of Tsinghua University (Science and Technology), 2020, 60(6): 518-529.
[3]
YurtseverE, LambertJ, CarballoA, et al. A survey of autonomous driving: common practices and emerging technologies[J]. IEEE Access, 2020, 8: 58443-58469.
CaoRuochen, FengXiufang, ZhaoChen. Video action recognition based on sport feature enhancement in two-stream network[J]. Science Technology and Engineering, 2025, 25(4): 1540-1546.
XuanHanyu, ChenQiang, WuZhiliang, et al. A survey on audiovisual multimodal learning[J]. Acta Automatica Sinica, 2025, 51(8): 124.
[10]
LinY Z, SunX T, RenY G. Slow-Fast convolutional attention network for human action recognition[J]. Journal of Imaging Science and Technology, 2025, 69(5): 050505.
[11]
BruneauP, MartinA. A review of transform-based deep learning for time series analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025, 47(3): 1200-1220.
ChenJiangbo, LiTao, ZhuXiaoyan, et al. Video behavior recognition based on spatio-temporal sampling[J]. Image and Signal Processing, 2021, 10(3): 45-52.
[14]
ShenL, GongG, HeT, et al. FastVID: dynamic density pruning for fast video large language models[J]. arXiv Preprint, arXiv: 2503. 11187, 2025.
[15]
TranD, BourdevL, FergusR, et al. Learning Spatiotemporal Features with 3D convolutional networks[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 4489-4497.
[16]
CarreiraJ, ZissermanA. Quo vadis, action recognition? A new model and the kinetics dataset[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4724-4733.
[17]
TranD, WangH, TorresaniL, et al. A closer look at spatiotemporal convolutions for action recognition[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6450-6459.
[18]
ShanC, LiuY, ChenX, et al. Temporal interlacing network for video action recognition[C]//Computer Vision-ECCV 2020: Proceedings of the 16th European Conference. Glasgow: Springer, 2020: 408-424.
[19]
FeichtenhoferC, FanH, MalikJ, et al. SlowFast networks for video recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6202-6211.
[20]
GuC, SunC, RossD A, et al. AVA: a video dataset of spatio-temporally localized atomic visual actions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 6047-6056.
[21]
BertasiusG, WangH, TorralbaL. Is space-time attention all you need for video understanding?[C]//Proceedings of the International Conference on Machine Learning (ICML). Virtual Event: PMLR, 2021: 813-824.
[22]
LiuZ, NingJ, CaoY, et al. Video Swin Transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 3202-3211.
[23]
WuQ, CuiR, LiY, et al. HaltingVT: adaptive token halting Transformer for efficient video recognition[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul: IEEE, 2024: 4305-4309.
[24]
CaoJ, YeP, LiS, et al. MADTP: multimodal alignment-guided dynamic token pruning for accelerating vision-language Transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2024: 15710-15719.
[25]
ShaoK, TaoK, QinC, et al. HoliTom: Holistic token merging for fast video large language models[J/OL]. arXiv Preprint, arXiv: 2505. 21334, 2025.
[26]
KoolW, van HoofH, WellingM. Stochastic beams and where to find them: the Gumbel-Top-k trick for sampling sequences without replacement[C]//Proceedings of the 36th International Conference on Machine Learning. Long Beach: PMLR, 2019: 3479-3487.
[27]
TaoY, TangY, WangY, et al. Saliency-driven dynamic token pruning for large language models[J/OL]. arXiv Preprint, arXiv: 2504. 04514, 2025.
[28]
SoomroK, ZamirA R, ShahM. UCF101: a dataset of 101 human actions classes from videos in the wild[R]. Orlando: University of Central Florida, 2012.
[29]
KuehneH, JhuangH, GarroteE, et al. HMDB: a large video database for human motion recognition[C]//Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV). Barcelona: IEEE, 2011: 2556-2563.
[30]
LinJ, GanC, HanS. TSM: temporal shift module for efficient video understanding[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 7082-7092.
[31]
ZongM, WangR, ChenX, et al. Motion saliency based multi-stream multiplier ResNets for action recognition[J]. Image and Vision Computing, 2021, 112: 104108.
[32]
RanasingheK, NaseerM, KhanS, et al. Self-supervised video Transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 11461-11471.
[33]
LiuY, SunJ, LinY, et al. Keyframe-oriented vision token pruning: enhancing efficiency of large vision language models on long-form video processing[J/OL]. arXiv Preprint, arXiv: 2503. 10742, 2025.