With the widespread adoption of large model technologies, the quality of intelligent adversarial information weapons has significantly improved. The “malicious cognitive behavior strategies” embedded within large models may directly impact the ethical norms and security safeguards of information warfare in the intelligent era. To uncover model biases and reveal differences among word vector libraries across diverse cultural contexts, a quantitative analysis of the ideological orientation of word embeddings from static large models was conducted. Five representative word vector libraries were selected, and a quantitative evaluation paradigm based on positive distance, negative distance, and tendency scores was proposed, using carefully curated test words. By computing the Euclidean distances between test words and benchmark terms and applying a normalization method to derive tendency scores, an intuitive quantitative metric was established, enabling precise quantification of ideological tendencies in word vectors. Analysis of both pre-training corpora and resulting embeddings directly exposes the endogenous biases inherent in the models. Experimental results show that word vector libraries trained on different corpora exhibit significant differences in ideological orientation, and the proposed “Euclidean distance-tendency score” evaluation paradigm can objectively differentiate among these libraries. The study concludes that the proposed method provides an operational framework for bias detection and “value alignment” in large models, applicable to word vector analysis across languages and cultural backgrounds, thereby facilitating the identification of potential value deviations prior to model deployment.
LÜ Shu-han, LI Yong, LI Shao-hua, LIN Fang, CHEN Yu-ang.
Evaluation Method for Ideological Tendencies of Large Models Based on “Euclidean Distance-Propensity Score” Paradigm[J].
Science Technology and Engineering, 2026, 26(11): 4732-4740 DOI:10.12404/j.issn.1671-1815.2503083
BuylM, RogiersA, NoelsS, et al. Large language models reflect the ideology of their creators[J/OL]. arXiv Preprint, arXiv: 2410. 18417. 2024.
[2]
RozadoD. Large language models are biased towards the conservative end of the political spectrum[J]. IEEE Transactions on Technology and Society, 2023, 4(3): 215-227.
[3]
GhafouriV, AgarwalV, ZhangY, et al. AI in the gray: exploring moderation policies in dialogic large language models vs human answers in controversial topics[J]. Proceedings of the ACM on Human-Computer Interaction, 2024, 8: 1-31.
[4]
FengS B, ParkC Y, LiuY H, et al. From pretraining data to language models to downstream tasks: tracking the trails of political biases leading to unfair NLP models[J]. Proceedings of the ACM on Human-Computer Interaction, 2023, 7: 1-32.
[5]
RozadoD. The political biases of ChatGPT[J]. Social Sciences, 2023, 12(3): 148.
LüLiyuan, LiYanhao, WangJianxiao, et al. Research on the values of large language models: conceptual framework and empirical evaluation[J]. Electronic Governance, 2025, 265(1): 15-28.
[8]
MotokiF, PinhoN V, RodriguesV. More human than human: measuring ChatGPT political bias[J]. Public Choice, 2024, 198(1): 3-23.
YuanFang, LuWei, ShenHao. Construction and analysis of chinese news event data based on unsupervised techniques[J]. Journal of Communication University of China (Natural Science Edition), 2023, 30(5): 1-9.
YanDuanwu, MeiXirui, YangXiongfei. Research on microblog text topic clustering based on the fusion of topic model and word embedding[J]. Journal of Modern Information, 2021, 41(10): 67-74.
FeiWei, HuQibiao, HouJingrui. Research on the utility evaluation of baidu baike entries from the perspective of information quality[J]. Intelligence Magazine, 2023, 42(4): 195-202.
WangXiaoli, HuKaibao. Research on translator behavior from the perspective of corpus-based critical translation studies: frontiers and prospects[J]. Foreign Languages and Their Teaching, 2024(3): 125-134.
[21]
胡开宝, 李涛, 孟令子. 语料库批评翻译学概论[M]. 北京: 高等教育出版社, 2018.
[22]
HuKaibao, LiTao, MengLingzi. An introduction to corpus-based critical translation studies[M]. Beijing: Higher Education Press, 2018.
LiuPengyuan. Paying attention to the restructuring of value perception by large language models and preventing risks related to national ideology[J]. Language Strategy Research, 2024, 9(5): 49-50.
WangPeng, GuQingshui, BianLongpeng. Large model RAG practical guide: principles, applications, and system construction[M]. Beijing: Mechanical Industry Press, 2024.
LiXiaodong, ZhangYingmin, LiYuqiang, et al. DNS covert channel detection algorithm based on multi-channel convolution neural network and attention mechanism[J]. Science Technology and Engineering, 2024, 24(35): 15137-15144.
MaRongheng, YuChunyu, TongYixin. Image denoising based on residual dense networks and attention mechanism[J]. Science Technology and Engineering, 2025, 25(9): 3795-3805.
WangShuai, HeWenchun, WangFudi, et al. Question answering system based on large language model integrating knowledge graph and vector retrieval[J]. Science Technology and Engineering, 2024, 24(32): 13902-13910.
JiWeidong, NiWanlu. A dynamic population sizing method based on Euclidean distance[J]. Journal of Electronics & Information Technology, 2022, 44(6): 2195-2206.