基于ELKB日志管理系统的应用

郭翠娟 ,  李思佳

科学技术与工程 ›› 2024, Vol. 24 ›› Issue (3) :1146-1153

PDF (2099KB)
科学技术与工程 ›› 2024, Vol. 24 ›› Issue (3) :1146-1153 DOI: 10.12404/j.issn.1671-1815.2301502
自动化技术、计算机技术

基于ELKB日志管理系统的应用

作者信息 +

Application Based on ELKB Log Management System

Author information +
文章历史 +

摘要

针对现阶段云平台应用日志运维效率低的问题,研究一种基于ELKB架构的日志管理方案。该方案可实现对日志数据的高效采集和统一存储,并提供日志查询和可视化分析的功能。详细阐述日志的采集和存储方案,提出优化采集器Filebeat基础参数的方法,日志存储过程部署Elasticsearch集群模式,并总结Elasticsearch集群节点的性能提高方法,以此为基础搭建日志可视化管理系统。实验结果表明:参数优化后的Filebeat进行日志采集时,内存占用率与默认配置相比降低47%;Elasticsearch集群模式在进行关键字查询时的数据吞吐率(250 req/s)高于单机服务器,且集群节点的段合并优化有效降低了日志的索引内存占用率,内存优化百分比介于11.9%和22.1%之间。可见日志管理系统部署方式灵活,有效提高了日志采集和检索效率,为云平台日志管理提供了可行的方案。

Abstract

Aiming at the problem of low efficiency of application log operation and maintenance in cloud platform at present, a log management scheme based on ELKB architecture was studied. The scheme can realize the efficient collection and unified storage of log data, and provide the functions of log query and visual analysis. The log collection and storage scheme was described in detail, and the method of optimizing the basic parameters of the collector Filebeat was proposed. The log stored procedure deploys the Elasticsearch cluster mode, and summarizes the performance improvement method of the Elasticsearch cluster node. Based on this, a log visualization management system was built. The experimental results show that the memory occupancy rate is reduced by 47% compared with the default configuration when the Filebeat with optimized parameters is used for log collection. The data throughput of Elasticsearch cluster mode in keyword query (250 req/s) is higher than that of single server, and the segment merging optimization of cluster nodes effectively reduces the index memory occupancy rate of logs, and the memory optimization percentage is between 11.9% and 22.1%. It can be seen that the deployment of log management system is flexible, which effectively improves the efficiency of log collection and retrieval, and provides a feasible solution for log management of cloud platform.

引用本文

引用格式 ▾
郭翠娟, 李思佳. 基于ELKB日志管理系统的应用[J]. 科学技术与工程, 2024, 24(3): 1146-1153 DOI:10.12404/j.issn.1671-1815.2301502
GUO Cui-juan, LI Si-jia. Application Based on ELKB Log Management System[J]. Science Technology and Engineering, 2024, 24(3): 1146-1153 DOI:10.12404/j.issn.1671-1815.2301502

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

基金资助

国家自然科学基金(61903273)

天津市科技计划项目(20YDTPJC01090)

PDF (2099KB)

29

访问

0

被引

详细

导航
相关文章

/

〈 〉