
科学技术与工程

中国技术经济学会
Patent Classification Method Based on Pre-trained Language Model and TRIZ Inventive Principle
Li-zhen JIA , Xiao-lei BAI
Science Technology and Engineering ›› 2024, Vol. 24 ›› Issue (30) : 13055 -13063.
To fully explore the existing solutions and technical knowledge in patent texts, based on TRIZ (theory of inventive problem solving), a method based on pre-trained language models is proposed for Chinese patent classification research oriented towards TRIZ inventive principles. Based on WWM (whole word masking technology), the Chinese RoBERTa model was further pre-trained with different number of patent datasets (composed of title and abstract of patent), and RoBERTa_patent1.0 and RoBERTa_patent 2.0 models specific to the patent domain were generated. On this basis, a Fully Connected Layer was added to construct three patent classification models based on RoBERTa, RoBERTa_patent1.0 and RoBERTa_patent2.0. Then, the constructed patent datasets based on TRIZ inventive principle was used to train and test the above three patent classification models. The experimental results show that, RoBERTa_patent2.0_IP has higher accuracy, PMacro, RMacro, and F1Macro, reaching 96%, 95.69%, 94%, and 94.84% respectively, achieving automatic classification of Chinese patent texts based on TRIZ inventive principle and helping designers understand and apply TRIZ inventive principle and achieve innovative product design.
pre-trained language model / ROBERTa (robustly optimized BERT pre-training approach) / IP (inventive principle) / WWM (whole word masking) / text classification
/
| 〈 |
|
〉 |