Patent Classification Method Based on Pre-trained Language Model and TRIZ Inventive Principle

Li-zhen JIA , Xiao-lei BAI

Science Technology and Engineering ›› 2024, Vol. 24 ›› Issue (30) : 13055 -13063.

PDF
Science Technology and Engineering ›› 2024, Vol. 24 ›› Issue (30) : 13055-13063 DOI: 10.12404/j.issn.1671-1815.2308844
research-article
Patent Classification Method Based on Pre-trained Language Model and TRIZ Inventive Principle
Author information +
History +
PDF

Abstract

To fully explore the existing solutions and technical knowledge in patent texts, based on TRIZ (theory of inventive problem solving), a method based on pre-trained language models is proposed for Chinese patent classification research oriented towards TRIZ inventive principles. Based on WWM (whole word masking technology), the Chinese RoBERTa model was further pre-trained with different number of patent datasets (composed of title and abstract of patent), and RoBERTa_patent1.0 and RoBERTa_patent 2.0 models specific to the patent domain were generated. On this basis, a Fully Connected Layer was added to construct three patent classification models based on RoBERTa, RoBERTa_patent1.0 and RoBERTa_patent2.0. Then, the constructed patent datasets based on TRIZ inventive principle was used to train and test the above three patent classification models. The experimental results show that, RoBERTa_patent2.0_IP has higher accuracy, PMacro, RMacro, and F1Macro, reaching 96%, 95.69%, 94%, and 94.84% respectively, achieving automatic classification of Chinese patent texts based on TRIZ inventive principle and helping designers understand and apply TRIZ inventive principle and achieve innovative product design.

Keywords

pre-trained language model / ROBERTa (robustly optimized BERT pre-training approach) / IP (inventive principle) / WWM (whole word masking) / text classification

Cite this article

Download citation ▾
Li-zhen JIA, Xiao-lei BAI. Patent Classification Method Based on Pre-trained Language Model and TRIZ Inventive Principle. Science Technology and Engineering, 2024, 24(30): 13055-13063 DOI:10.12404/j.issn.1671-1815.2308844

登录浏览全文

4963

注册一个新账户 忘记密码

References

Funding

PDF (7322KB)

18

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉