Zabbix4.0历史数据的持久化

Zabbix4.0历史数据的持久化

(一)背景介绍

zabbix是一个大型的分布式的监控系统,监控的范围比较广,是目前比较流行的监控系统,但是由于自身的原因,历史数据不能持久保存,如果数据库的数据大于100G左右查询或其他的速度会非常的慢,会触发很多问题,一般的zabbix历史数据会不超过一个月(按实际获得的数据比例计算),我们一般保存七天。一般为了业务的需要,往往会需要很长的历史数据来进行查看和排查问题,这就需要使zabbix的历史数据进行长久保存(不能存数据库,而可以存ES存储)。

(二)环境

zabbix:zabbix4.0.1(安装部署省略)

ES:5.5.2 (安装部署省略)

(三)具体的配置

3.1、首先修改zabbix_server.conf文件,启用历史数据的配置,具体如下:

$grep '^[a-Z]' /etc/zabbix/zabbix_server.conf

LogFile=/tmp/zabbix_server.log

LogFileSize=0

PidFile=/var/run/zabbix/zabbix_server.pid

SocketDir=/var/run/zabbix

DBHost=localhost

DBName=zabbix

DBUser=zabbix

DBPassword=zabbix

HistoryStorageURL=http://X.X.X.X:9200

HistoryStorageTypes=uint,db1,str,log,text

HistoryStorageDateIndex=1

SNMPTrapperFile=/var/log/snmptrap/snmptrap.log

Timeout=4

AlertScriptsPath=/usr/lib/zabbix/alertscripts

ExternalScripts=/usr

########################下边是具体的解释####################################

### Option: HistoryStorageURL

# History storage HTTP[S] URL.

#

# Mandatory: no

# Default:

# HistoryStorageURL=

#HistoryStorageURL="-uelastic:FatPPdaiCom http://10.114.16.77:9200"

HistoryStorageURL=http://10.114.23.139:9200

### Option: HistoryStorageTypes

# Comma separated list of value types to be sent to the history storage.

#

# Mandatory: no

# Default:

# HistoryStorageTypes=uint,db1,str,log,text

HistoryStorageTypes=uint,db1,str,log,text

### Option: HistoryStorageDateIndex

# Enable preprocessing of history values in history storage to store values in different indices based on date.

# 0 - disable

# 1 - enable

#

# Mandatory: no

# Default:

HistoryStorageDateIndex=1

备注:下边是ES所支持存储的数据类型

3.2、修改zabbix前端的配置文件,添加global $DB,$HISTORY;

$vim /etc/zabbix/web/zabbix.conf.php

// Zabbix GUI configuration file.

global $DB,$HISTORY;

#global $DB;

$DB['TYPE'] = 'MYSQL';

$DB['SERVER'] = 'localhost';

$DB['PORT'] = '0';

$DB['DATABASE'] = 'zabbix';

$DB['USER'] = 'zabbix';

$DB['PASSWORD'] = 'zabbix';

// Schema name. Used for IBM DB2 and PostgreSQL.

$DB['SCHEMA'] = '';

$ZBX_SERVER = '10.114.23.230';

$ZBX_SERVER_PORT = '10051';

$ZBX_SERVER_NAME = 'zabbix';

$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;

// Elasticsearch url (can be string if same url is used for all types).

$HISTORY['url'] = 'http://X.X.X.X:9200';

#$HISTORY['url'] = '-uelastic:FatPPdaiCom http://X.X.X.X:9200';

// Value types stored in Elasticsearch.

$HISTORY['types'] = ['uint', 'text', 'log', 'str', 'db1'];

//--------------------------------------------------------------------------------------------------------------------------

//如果不同类型使用不同的 ES 集群,可以按如下进行配置

#$HISTORY['url'] = [

# 'uint' => 'http://10.114.16.77:9200 ',

# 'text' => 'http://10.114.16.77:9200 '

#];

#$HISTORY['types'] = ['uint', 'text'];

注意的要点:

1、HISTORY['url']和HISTORY['types']被更新。

2、定义zabbix GUI全局数据源路径文件 global $DB, $HISTORY;(告诉zabbix先从数据库里读取,没有的话取HISTORY读取)

3.3、在ES上依次创建五个索引(不创建也可以,会自动创建的)

$curl -XPUT http://X.X.X.X:9200/unit \

-H 'content-type:application/json' \

-d '{

"settings" : {

"index" : {

"number_of_replicas" : 1,

"number_of_shards" : 5

}

},

"mappings" : {

"values" : {

"properties" : {

"itemid" : {

"type" : "long"

},

"clock" : {

"format" : "epoch_second",

"type" : "date"

},

"value" : {

"type" : "long"

}

}

}

}

}'

$curl -XPUT http://X.X.X.X:9200/db1 \

-H 'content-type:application/json' \

-d '{

"settings" : {

"index" : {

"number_of_replicas" : 1,

"number_of_shards" : 5

}

},

"mappings" : {

"values" : {

"properties" : {

"itemid" : {

"type" : "long"

},

"clock" : {

"format" : "epoch_second",

"type" : "date"

},

"value" : {

"type" : "double"

}

}

}

}

}'

$curl -XPUT http://X.X.X.X:9200/log \

-H 'content-type:application/json' \

-d '{

"settings" : {

"index" : {

"number_of_replicas" : 1,

"number_of_shards" : 5

}

},

"mappings" : {

"values" : {

"properties" : {

"itemid" : {

"type" : "long"

},

"clock" : {

"format" : "epoch_second",

"type" : "date"

},

"value" : {

"fields" : {

"analyzed" : {

"index" : true,

"type" : "text",

"analyzer" : "standard"

}

},

"index" : false,

"type" : "text"

}

}

}

}

}'

$curl -XPUT http://X.X.X.X:9200/text \

-H 'content-type:application/json' \

-d '{

"settings" : {

"index" : {

"number_of_replicas" : 1,

"number_of_shards" : 5

}

},

"mappings" : {

"values" : {

"properties" : {

"itemid" : {

"type" : "long"

},

"clock" : {

"format" : "epoch_second",

"type" : "date"

},

"value" : {

"fields" : {

"analyzed" : {

"index" : true,

"type" : "text",

"analyzer" : "standard"

}

},

"index" : false,

"type" : "text"

}

}

}

}

}'

$curl -XPUT http://X.X.X.X:9200/str \

-H 'content-type:application/json' \

-d '{

"settings" : {

"index" : {

"number_of_replicas" : 1,

"number_of_shards" : 5

}

},

"mappings" : {

"values" : {

"properties" : {

"itemid" : {

"type" : "long"

},

"clock" : {

"format" : "epoch_second",

"type" : "date"

},

"value" : {

"fields" : {

"analyzed" : {

"index" : true,

"type" : "text",

"analyzer" : "standard"

}

},

"index" : false,

"type" : "text"

}

}

}

}

}'

3.4、在ES上查看是否生成:

$curl http://X.X.X.X:9200/_cat/indices?v

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

green open str ZTkdqhpFR9ijzTfKvtsMrQ 5 1 0 0 2.2kb 1.1kb

green open text TjY6OEGySJ2AGc1rpfOEhA 5 1 0 0 2.2kb 1.1kb

green open log qG-R2UhiQhypP9stSSodkw 5 1 0 0 2.2kb 1.1kb

green open db1 av_V5XbiSDyDdZXW0rO6aw 5 1 0 0 2.2kb 1.1kb

green open .kibana JZBiyypQSRuMxRnHxOLP1Q 1 1 1 0 8kb 4kb

green open unit 2bxxvaMTTFKqTq0EDX5EjA 5 1 0 0 2.2kb 1.1kb

3.5、在kibana上创建相应的索引并查看数据。

就可以看到历史数据会存储到ES中去

(四)、查看是否通过db访问还是es,修改该文件(/etc/zabbix/web/zabbix.conf.php)的全局变量就可以知道访问的是 mysql 还是ES了。

具体可以参考官方文档( https://www.zabbix.com/documentation/4.0/manual/appendix/install/elastic_search_setup )

🌈 相关推荐