1 School of Intelligence Science and Technology, Beijing University of Civil Engineering and Architecture, Beijing 102616, China
2 Beijing Key Laboratory of Super Intelligent Technology for Urban Architecture, Beijing University of Civil Engineering and Architecture, Beijing 102616, China
With the rapid growth and increasing complexity of malicious software, traditional methods of malware detection have become inadequate to meet current security demands. A deep learning-based ensemble framework was proposed to address limitations of traditional malware detection methods. The framework integrated GPT-4, BERT, and a hybrid CNN-LSTM architecture. First, contextual explanations of API calls were generated by GPT-4 and subsequently encoded into semantic vectors using BERT. The hybrid CNN-LSTM was then employed to capture both local patterns and long-term dependencies within behavioral sequences. Experiments conducted on the mal-api-2019 dataset achieved an accuracy of 97.53%, a precision of 96.80%, a recall of 97.50%, and an F1-score of 97.15%, outperforming state-of-the-art approaches. Further validation on the VirusShare dataset confirmed superior generalization capability. The work provides an innovative solution for combating evolving malware threats through the synergistic integration of natural language processing and deep learning techniques.
QIAN Li-ping, HUANG Nan-nan, CONG Lin, CUI Yu-ting, CHEN Yan-peng.
Application of Deep Learning in Malware Detection: Integrated Framework of GPT-4, BERT, and CNN-LSTM[J].
Science Technology and Engineering, 2026, 26(6): 2502-2413 DOI:10.12404/j.issn.1671-1815.2501355
现有的恶意软件检测方法主要分为静态分析、动态分析和混合分析3种[7]。静态分析通过逆向工程直接解析可执行文件的代码特征(如字节序列、控制流结构),无需运行样本,具有安全性高、覆盖全代码路径的优势,但易受加壳和混淆技术干扰;动态分析则在沙箱环境中监控样本的运行时行为:如应用程序编程接口(application programming interface,API)调用序列、注册表操作,能够有效识别混淆后恶意代码的潜在威胁,但存在资源消耗大、执行路径覆盖不全的局限性;混合分析通过融合静态特征的全局性和动态行为的上下文信息,在降低误报率的同时提升了对复杂变种恶意软件的检测能力。随着深度学习技术在自然语言处理(natural language processing,NLP)领域的成功应用,研究者开始将其应用于恶意软件的动态行为分析,尤其是API序列分析,这已成为研究的焦点[8-10]。
为了克服上述挑战,现提出一种集成框架GBCL(GPT-4, BERT, and CNN-LSTM)。该方法的核心思想是利用GPT-4的强大文本生成能力为API调用序列生成富含上下文信息的解释性文本,再通过BERT模型对这些解释性文本进行深度语义编码,最后结合CNN-LSTM网络进行序列特征学习和分类。以期更深入理解API调用语义、增强对未知恶意软件识别能力,并提升模型可解释性的新方法。
动态分析方法通过在受控的模拟环境中执行测试程序来捕获其实时行为,监控关键活动如注册表操作、API调用和网络通信,以分析程序的行为特征并判断其恶意性。Tobiyama等[17]利用LSTM构建行为模型,将API调用转化为特征向量并生成特征图像,再用CNN进行检测,但面对长序列时性能可能下降,且易过拟合于小规模数据集。Rosenberg等[18]将API序列分割为子序列,使用RNN(recurrent neural network)检测,一旦某个子序列被标记为恶意,整个序列即被报告为恶意。Ndibanje等[19]基于API序列构建特征,通过虚拟环境中的动态API调用和相似性计算区分程序行为,然后应用机器学习和统计方法检测恶意代码。Chen等[20]提出了一种基于深度神经网络DNN(deep neural network)的恶意代码检测方法,该方法巧妙地融合了经验规则和统计规则来分析API调用参数。此外,还采用了高斯混合模型聚类算法GSDMM(collapsed Gibbs sampling algorithm for the dirich-let multinomial mixture model)来对其余的API调用进行分类,从而为这些调用分配敏感性标签。这种方法不仅提高了对API调用模式的理解,而且增强了模型对恶意行为的识别能力。Dabas等[21]开发了MalAnalyser(malware analyzer)系统,从API调用序列中提取频繁子序列,结合全局和局部粒子群优化算法和遗传算法,识别关键特征以有效检测恶意代码。这些方法虽然提高了检测的准确性,但在处理长序列和数据集规模限制时仍面临挑战。
1.3 混合分析检测方法
混合分析方法融合了静态和动态分析技术,以增强恶意代码检测的能力[22]。Zhao等[23]结合可移植可执行文件(Portable Executable,PE)文件的静态特征和程序执行的动态行为,取得了约95%的准确率。Han等[24]提出了MalDAE(malware detection and explanation)框架,分析静态和动态API序列的相关性,基于语义映射融合API调用序列,利用随机森林分类器进行分类。这些研究表明,混合分析方法能有效提高恶意代码检测的准确性,尤其是在处理混淆技术时。
针对恶意软件检测领域的关键问题,提出了一种基于生成可解释文本与深度学习融合的新框架GBCL(GPT-4, BERT, and CNN-LSTM)。现有方法主要存在以下不足:①静态分析对未知恶意软件的适应性较差,难以处理复杂多变的行为模式;②API调用序列的上下文语义难以有效建模,导致特征表达不足;③模型可解释性较低,使分类结果缺乏直观支持;④特征提取与融合方法的局限性限制了检测性能的提升。为解决这些问题,GBCL结合自然语言生成技术与深度学习模型,显著改善了检测的准确性、效率及可解释性。
(1)Prompt设计优化:通过构建针对API调用的特定Prompt(提示文本),精准引导GPT-4生成符合语义的高质量描述。例如,输入“Describe the purpose of CreateFile API call”时,GPT-4会生成“Create or open a file for reading, writing, or both”这样的准确描述。Prompt的明确设计是生成文本针对性与一致性的关键,直接影响生成内容的准确性与可用性。
YanP, TanS, WangM, et al. Prompt engineering-assisted malware dynamic analysis using GPT-4[J]. arXiv Preprint, arXiv: 2312.08317, 2023.
[2]
KhanK, MehmoodA, KhanS, et al. A survey on intrusion detection and prevention in wireless AD-HOC networks[J]. Journal of Systems Architecture, 2020, 105.DOI: 10.1016/j.sysarc.2019.101701.
[3]
MohurleS, PatilM. A brief study of wannacry threat: ransomware attack 2017[J]. International Journal of Advanced Research in Computer Science, 2017, 8(5): 1938-1940.
MaBolin, ZhangZheng, ChenYuan, et al. The defense method for code-injection attacks based on Instruction set randomization[J]. Journal of Cyber Security, 2020, 5(4): 30-43.
ShaoSihao, GaoQin, MaSen, et al. Progress in research on buffer overflow vulnerability analysis technologies[J]. Journal of Software, 2018, 29(5): 1179-1198.
FengXiaomeng, SunQiuye, WangBingyu, et al. The coordinated cyber physical power attack strategy based on worm propagation and false data injection[J]. Acta Automatica Sinica, 2022, 48(10): 2429-2441.
[10]
YadavB, TokekarS. Recent innovations and comparison of deep learning techniques in malware classification: a review[J]. International Journal of Information Security Science, 2021, 9(4): 230-247.
[11]
SinghJ, SinghJ. A survey on machine learning-based malware detection in executable files[J]. Journal of Systems Architecture, 2021, 112.DOI: 10.1016/j.sysarc.2020.101861.
[12]
AthiwaratkunB, StokesJ W. Malware classification with LSTM and GRU language models and a character-level CNN[C]//IEEE international Conference on Acoustics, New Orleans:Speech and Signal Processing (ICASSP). New York: IEEE, 2017: 2482-2486.
[13]
ManiathS, AshokA, PoornachandranP, et al. Deep learning LSTM based ransomware detection[C]// Recent Developments in Control, Automation & Power Engineering (RDCAPE). Noida: IEEE, 2017: 442-446.
WangShuai, HeWenchun, WangFudi, et al. QA system integrating large language models with knowledge graphs and vector retrieval[J]. Science Technology and Engineering, 2024, 24(32): 13902-13910.
FanZhipeng, LiJun, LiuYuqiang, et al. Malware classification based on gray texture fingerprint[J]. Science Technology and Engineering, 2020, 20(29): 12014-12020.
DongXingtong, ChenShihong, ChenShuxin. Optimized algorithm design for text plagiarism detection in natural language processing[J]. Science Technology and Engineering, 2022, 22(3): 1091-1097.
YangWang, GaoMing, JiangTing, et al. A malicious code static detection framework based on multi-feature ensemble learning[J]. Journal of Computer Research and Development, 2021, 58(5): 1021-1034.
[24]
LiS, JiangL, ZhangQ, et al. A malicious mining code detection method based on multi-features fusion[J]. IEEE Transactions on Network Science and Engineering, 2022, 10(5): 2731-2739.
[25]
TobiyamaS, YamaguchiY, ShimadaH, et al.Malware detection with deep neural network using process behavior[C]//IEEE 40th Annual Computer Software and Applications Conference (COMPSAC). Atlanta: IEEE, 2016: 577-582.
[26]
RosenbergI, ShabtaiA, RokachL, et al. Generic black-box end-to-end attack against stateof the art API call based malware classifiers[C]//Research in Attacks, Intrusions, and Defenses: 21st International Symposium, RAID 2018. Atlanta: IEEE, 2018: 490-510.
[27]
NdibanjeB, KimK H, KangY J, et al. Cross-method-based analysis and classification of malicious behavior by API calls extraction[J]. Applied Sciences, 2019, 9(2). DOI:10.3390/app9020239.
[28]
ChenX, HaoZ, LiL, et al. Cruparamer: learning on parameter-augmented API sequences for malware detection[J]. IEEE Transactions on Information Forensics and Security, 2022, 17: 788-803.
[29]
DabasN, SharmaP. MalAnalyser: an effective and efficient Windows malware detection method based on API call sequences[J]. Expert Systems with Applications, 2023, 230.DOI:10.1016/j.eswa.2023.120756.
[30]
DamodaranA, TroiaF D, VisaggioC A, et al. A comparison of static, dynamic, and hybrid analysis for malware detection[J]. Journal of Computer Virology and Hacking Techniques, 2017, 13: 1-12.
[31]
ZhaoJ, ZhangS, LiuB, et al. Malware detection using machine learning based on the combination of dynamic and static features[C]// 27th International Conference on Computer Communication and Networks (ICCCN). Hangzhou: IEEE, 2018: 1-6.
[32]
HanW, XueJ, WangY, et al. MalDAE: detecting and explaining malware based on correlation and fusion of static and dynamic characteristics[J]. Computers & Security, 2019, 83: 208-233.
[33]
KangJ, JangS, LiS, et al. Long short-term memory-based malware classification method for information security[J]. Computers & Electrical Engineering, 2019, 77: 366-375.
[34]
AmerE, ZelinkaI. A dynamic Windows malware detection and prediction method based on contextual understanding of API call sequence[J]. Computers & Security, 2020, 92. DOI: 10.1016/j.cose.2020.101760.
WangXinfeng, HuangKunming, ZhangWenjie. Malware detection based on application programming interface sequence and convolutional neural network[J]. Journal of Information Security Research, 2020, 6(3): 212-219.
[37]
LiuY, WangY. Arobust malware detection system using deep learning on API calls[C]// IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC). Chengdu: IEEE, 2019: 1456-1460.
[38]
WangP, TangZ, WangJ. A novel few-shot malware classification approach for unknown family recognition with multi-prototype modeling[J]. Computers & Security, 2021, 106. DOI: 10.1016/j.cose.2021.102273.
[39]
GreffK, SrivastavaR K, KoutníkJ, et al. LSTM: a search space odyssey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 28(10): 2222-2232.
[40]
CatakF O, YazıA F, ElezajO, et al. Deep learning based sequential model for malware analysis using Windows exe API Calls[J]. PeerJ Computer Science, 2020, 6. DOI: 10.7717/peerj-cs.285.
[41]
AvciC, TekinerdoganB, CatalC. Analyzing the performance of long short-term memory architectures for malware detection models[J]. Concurrency and Computation: Practice and Experience, 2023, 35(6): 1-11.
[42]
PandaB, BisoyiS S, PanigrahyS. An ensemble approach for imbalanced multiclass malware classification using 1D-CNN[J]. PeerJ Computer Science, 2023. DOI: 10.7717/peerj-cs.1677.
[43]
OwohN, AdejohJ, HosseinzadehS, et al. Malware detection based on API call sequence analysis: a gated recurrent unit-generative adversarial network model approach[J]. Future Internet, 2024, 16(10). DOI: 10.3390/fi16100369.
[44]
LiC, ChenZ, ZhengJ. An efficient transformer encoder-based classification of malware using API calls[C]// IEEE 24th International Conference on High Performance Computing & Communications. Haikou: IEEE, 2022: 839-846.
[45]
LiC, LüQ, LiN, et al. A novel deep framework for dynamic malware detection based on API sequence intrinsic features[J]. Computers & Security, 2022, 116. DOI: 10.1016/j.cose.2022.102686.
[46]
CannarileA, CarreraF, GalantucciS, et al. A study on malware detection and classification using the analysis of API calls sequences through shallow learning and recurrent neural networks[C]// Italian Conference on Cybersecurity. Rome: ITASEC, 2022: 124-134.
[47]
TranT K, SatoH. NLP-based approaches for malware classification from API sequences[C]// 21st Asia Pacific Symposium on Intelligent and Evolutionary Systems (IES). Hanoi: IEEE, 2017: 101-105.
[48]
ZhuangJ, Al HasanM. Non-exhaustive learning using gaussian mixture generative adversarial networks[C]// Machine Learning and Knowledge Discovery in Databases. Bilbao: Springer International Publishing, 2021: 3-18.
[49]
JianY, KuangH, RenC, et al. A novel framework for image-based malware detection with a deep neural network[J]. Compu-ters & Security, 2021, 109.DOI:10.1016/j.cose.2021.102400.
[50]
SharmaO, SharmaA, KaliaA. Windowsand IoT malware visuali-zation and classification with deep CNN and Xception CNN using Markov images[J]. Journal of Intelligent Information Systems, 2023, 60(2): 349-375.
[51]
FalanaO J, SodiyaA S, OnashogaS A, et al. Mal-Detect: an intelligent visualization approach for malware detection[J]. Journal of King Saud University-Computer and Information Sciences, 2022, 34(5): 1968-1983.
[52]
VasanD, AlazabM, WassanS, et al. IMCFN: image-based malware classification using fine-tuned convolutional neural network architecture[J]. Computer Networks, 2020, 171.DOI: 10.1016/j.comnet.2020.107138.
PDF
(6512KB)
