Etcd Confd实现配置文件变化自动更新prometheus服务

一、部署步骤

1、confd部署

# wget https://github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-amd64
# mv confd-0.16.0-linux-amd64 /usr/local/confd
# chmod  x /usr/local/confd
# cd /usr/local/confd
# ./confd --help

2、confd配置
confd通过读取后端存储的配置信息来动态更新对应的配置文件，对应的后端存储可以是etcd，redis等，其中etcd的v3版本对应的存储后端为etcdv3
（1）创建confdir
confdir底下包含两个目录:
conf.d:confd的配置文件，主要包含配置的生成逻辑，例如模板源，后端存储对应的keys，命令执行等。
templates:配置模板Template，即基于不同组件的配置，修改为符合 Golang text templates的模板文件。

# mkdir -p /etc/confd/{conf.d,templates}

（2）Template Resources
模板源配置文件是TOML格式的文件，主要包含配置的生成逻辑，例如模板源，后端存储对应的keys，命令执行等。默认目录在/etc/confd/conf.d。
参数说明：
必要参数
dest (string) - The target file.
keys (array of strings) - An array of keys.
src (string) - The relative path of a configuration template.

可选参数
gid (int) - The gid that should own the file. Defaults to the effective gid.
mode (string) - The permission mode of the file.
uid (int) - The uid that should own the file. Defaults to the effective uid.
reload_cmd (string) - The command to reload config.
check_cmd (string) - The command to check config. Use {{.src}} to reference the rendered source template.
prefix (string) - The string to prefix to keys.

例如：/etc/confd/conf.d/prometheus.conf.toml

# vim /etc/confd/conf.d/prometheus.conf.toml
[template]
prefix = "/prometheus"
src = "prometheus.yml.tmpl"
dest = "/opt/prometheus/prometheus.yml"
mode = "0755"
keys = [
"/job",
]
reload_cmd = "curl -XPOST 'http://10.1.43.1:9090/-/reload'"

（3）Template
Template定义了单一应用配置的模板，默认存储在/etc/confd/templates目录下，模板文件符合Go的text/template格式。

模板文件常用函数有base，get，gets，lsdir，json等。具体可参考https://github.com/kelseyhightower/confd/blob/master/docs/templates.md。

例如：/etc/confd/templates/prometheus.yml.tmpl

# vim /etc/confd/templates/prometheus.yml.tmpl
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

  external_labels:
    monitor: 'monitor1'

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
       - 10.1.43.1:9093
       - 10.1.43.2:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
    - "rules/*.yml"
    - "rules/HLY/*.yml"
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['10.1.43.1:9090','10.1.43.2:9090']
  {{range $job_name := gets "/job/*"}}
  {{$jobJson := json $job_name.Value}}
  - job_name: '{{$jobJson.name}}'
    scheme: '{{$jobJson.scheme}}'
    metrics_path: '{{$jobJson.metrics}}'
    static_configs:
  {{$target := printf "%s/*" $job_name.Key}}{{range $ins_name := gets $target}}
  {{$insJson := json $ins_name.Value}}
    - targets: ['{{$insJson.instance}}']
      labels:
        name: '{{$insJson.name}}'
        ip: '{{$insJson.ip}}'
  {{end}}
  {{end}}

3、创建后端存储的配置数据
以etcdv3存储为例，在etcd中创建以下prometheus数据。

先创建job
# etcdctl --endpoints="10.1.43.1:12379" put /prometheus/job/test '{"scheme":"http","metrics":"/metrics","name":"test"}'
OK
再创建instance
# etcdctl --endpoints="10.1.43.1:12379" put /prometheus/job/test/test1 '{"name":"test1","instance":"1.1.1.1:9093","ip":"1.1.1.1"}'
OK

4、启动confd服务

# cd /usr/local/confd
# ./confd -watch -backend etcdv3 -node http://10.1.43.1:12379 -node http://10.1.43.2:12379 -node http://10.1.43.3:12379                         #后端etcd集群启动

5、验证
查看生成的配置文件，已自动同步并重新加载prometheus服务

  - job_name: 'test'
    scheme: 'http'
    metrics_path: '/metrics'
    static_configs:


    - targets: ['1.1.1.1:9093']
      labels:
        name: 'test1'
        ip: '1.1.1.1'

查看prometheus页面

已创建刚才put到etcd的监控目标！

注意：启动confd的时候只指定一个etcd node节点，就算etcd挂了，confd服务起来，配置文件也不会重置

来源：https://www./content-4-909601.html