基于CoSENT和改进K-means的冒犯性评论文本主题识别

陈健飞 ,  卜凡亮 ,  王一帆

科学技术与工程 ›› 2024, Vol. 24 ›› Issue (31) :13442-13449

PDF (2545KB)
科学技术与工程 ›› 2024, Vol. 24 ›› Issue (31) :13442-13449 DOI: 10.12404/j.issn.1671-1815.2309664
自动化技术、计算机技术

基于CoSENT和改进K-means的冒犯性评论文本主题识别

作者信息 +

Text Topic Recognition of Offensive Comments Based on CoSENT and Improved K-means

Author information +
文章历史 +

摘要

为快速识别冒犯性评论文本中的用户热点主题,解决传统主题模型在处理评论文本时语义描述不充分、上下文信息丢失和主题连贯性不强,以及K-means聚类算法对K值和初始中心点敏感的问题。使用CoSENT(cosine sentence)模型获取包含冒犯性语言的评论文本的句子级向量特征,对通过统一流形逼近与投影算法即UMAP(uniform manifold approximation and projection)模型降维后的向量矩阵使用基于Canopy +的改进K-means算法进行类簇划分,用(class term frequency-inverse document frequency, c-TF-IDF)识别各主题簇的主题特征,进行主题建模。通过对比冒犯性评论文本数据集以及普通评论数据集的实验验证了方法有效性。结果表明本文方法能够得到更好的主题一致性。

Abstract

To quickly identify users' hot topics in offensive comment texts and solve the problems of insufficient semantic description, loss of contextual information, and weak topic coherence of traditional topic models when dealing with comment texts, as well as the sensitivity of K-value and initial centroid of K-means clustering algorithm. The CoSENT (cosine sentence) model was used in this paper to obtain sentence-level vector features of comment texts containing offensive language. An improved K-means algorithm based on Canopy +was used for class clustering on the vector-matrix after dimensionality reduction through the UMAP(uniform manifold approximation and projection) model. c-TF-IDF (class term frequency-inverse document frequency) was used to identify the thematic features of each thematic cluster for thematic modeling. The validity of the method is verified through experiments comparing the offensive comment text dataset as well as the ordinary comment dataset. The results show that the method in this paper can get better topic consistency.

关键词

自然语言处理 / 主题模型 / CoSENT / K-means

Key words

natural language processing / topic modelling / CoSENT / K-means

引用本文

引用格式 ▾
陈健飞, 卜凡亮, 王一帆. 基于CoSENT和改进K-means的冒犯性评论文本主题识别[J]. 科学技术与工程, 2024, 24(31): 13442-13449 DOI:10.12404/j.issn.1671-1815.2309664
CHEN Jian-fei, BU Fan-liang, WANG Yi-fan. Text Topic Recognition of Offensive Comments Based on CoSENT and Improved K-means[J]. Science Technology and Engineering, 2024, 24(31): 13442-13449 DOI:10.12404/j.issn.1671-1815.2309664

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

中国人民公安大学安全防范工程双一流专项(2023SYL08)

PDF (2545KB)

39

访问

0

被引

详细

导航
相关文章

/

〈 〉