
科学技术与工程

中国技术经济学会
Extracting Key Elements of 110 Police Incidents Based on Open Source Large Language Models
Xin-tao XING , Peng CHEN
Science Technology and Engineering ›› 2025, Vol. 25 ›› Issue (29) : 12547 -12559.
With the rapid development of Internet technology and the continuous increase in the number of police cases, relying on a large amount of manpower and traditional natural language processing methods to extract police case elements is no longer applicable. Utilizing advanced technologies such as large language models to improve the extraction effect of police case elements has become an inevitable trend in the development of the field of police data analysis. In view of the characteristics of police cases and the extraction requirements of police case elements, the method of constructing a large language model in the public security field was adopted to extract police case elements. For the base model, a variety of lightweight (6/7b) open-source large language models were tested. To improve the model performance, based on LoRA fine-tuning, combined with data engineering, a zero-shot data augmentation method and a prompt engineering method based on ChatGPT for police cases were proposed to enhance the model performance. The key police case data of Beijing in 2019 were used as examples for testing. Compared with the base model, prompt engineering can increase the F1 by 3% ~ 30%, and fine-tuning can increase the F1 by 10% ~ 50%. After fine-tuning different base models and making comparisons, the Baichuan model shows the best comprehensive performance in the extraction of police case elements. When using the data augmentation method to expand the data by 2 times, the F1 is further increased by 4%, and the model ability reaches the optimal level. Compared with the fine-tuned general information extraction large language of universal information extraction model(UIE), the large language model for police case element extraction constructed increases the F1 of each element extraction by about 5%, and for the "organization" element of police cases, the F1 is increased by 42%.
large language models / police incident / information extraction
/
| 〈 |
|
〉 |