Prometheus入门

一、简单介绍

Prometheus组件
Prometheus:监控服务,从被监控机上拉取监控信息,并通过配置的规则文件触发报警
explorter: 采集器
    根据不同监控内容有不同采集器
    node-explorter
    jmx-explorter
    mysql-explorter
    elasticseach-explorter
    ...
altermanager:报警器
    用来配置报警发送方式

prometheus及相关exporter组件下载地址:https://prometheus.io/download/

部署
用ansible-prometheus安装 (推荐)
单独安装

安装prometheus

创建目录 mkdir /opt/prometheus, 把prometheus相关的组件都放到这个目录下

下载最新版本prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.10.0/prometheus-2.10.0.linux-amd64.tar.gz

1
2
tar -zxf prometheus-2.10.0.linux-amd64.tar.gz
mv prometheus-2.10.0.linux-amd64 /opt/prometheus/prometheus

下载最新版本alertmanager
https://github.com/prometheus/alertmanager/releases/download/v0.17.0/alertmanager-0.17.0.linux-amd64.tar.gz

1
2
tar -zxf alertmanager-0.17.0.linux-amd64.tar.gz
mv alertmanager-0.17.0.linux-amd64 /opt/prometheus/alertmanager

优雅的启动prometheus

编写启动脚本:
vim /opt/prometheus/prometheus.sh

1
2
#!/bin/bash
/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus/prometheus.yml &>> /var/log/prometheus.log

#进入system目录,编写systemd服务脚本
cd /usr/lib/systemd/system
vim prometheus.service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
User=root
Group=root
Type=simple
ExecStart=/opt/prometheus/prometheus/prometheus.sh # 启动脚本

[Install]
WantedBy=multi-user.target

启动prometheus

1
2
3
4
systemctl daemon-reload # 重新加载daemon
systemctl start prometheus.service #启动
systemctl status prometheus.service #查看启动状态
systemctl enable prometheus.service #添加开机启动

监控主机

node_exporter安装
下载最新版的node_exporter, node_exporter-0.18.1.linux-amd64.tar.gz

1
2
tar -zxf node_exporter-0.18.1.linux-amd64.tar.gz
cp node_exporter-0.18.1.linux-amd64/node_exporter /usr/local/bin/

把node_exporter做成服务
创建node_exporter.service文件

1
2
3
4
5
6
7
8
9
10
11
12
# vim /usr/lib/systemd/system/node_exporter.service
[Unit]
Description=Prometheus node_exporter
[Service]
User=nobody
ExecStart=/usr/local/bin/node_exporter --log.level=error
ExecStop=/usr/bin/killall node_exporter
MemoryLimit=300M
CPUQuota=100%

[Install]
WantedBy=default.target

使用MemoryLimit限制内存使用大小
使用CPUQuata限制cpu使用核数

1
2
3
systemctl daemon-reload
systemctl enable node_exporter.service
systemctl start node_exporter

安装grafana

1
2
3
4
5
6
wget https://dl.grafana.com/oss/release/grafana-6.2.4-1.x86_64.rpm
yum localinstall grafana-6.2.4-1.x86_64.rpm

systemctl daemon-reload
systemctl start grafana-server
systemctl status grafana-server

配置数据源

监控mysql

在被监控的mysql主机上操作
1 下载mysqld_exporter https://github.com/prometheus/mysqld_exporter/releases

2 进入数据库创建用户并授权

1
2
mysql> GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'prometheus'@'localhost' identified by 'prometheus';
mysql> GRANT SELECT ON performance_schema.* TO 'prometheus'@'localhost';

3 创建.my.cnf文件

1
2
3
4
$ cat ~/.my.cnf
[client]
user=prometheus
password=prometheus

默认情况下,运行./mysqld_exporter时,会去当前用户目录下查找.my.cnf, 可以通过参数–config.my-cnf=”yourpath/.my.cnf”, 参数–web.listen-address=:9104指定端口

4 创建成服务
使用环境变量启动

1
2
3
4
5
6
7
8
9
10
11
12
$ vim /usr/lib/systemd/system/mysqld_exporter.service
[Unit]
Description=mysqld_exporter
After=network.target
[Service]
Type=simple
User=prometheus
Environment=DATA_SOURCE_NAME=prometheus:prometheus@tcp(127.0.0.1:3306)/?loc=Local
ExecStart=/opt/prometheus/mysqld_exporter/mysqld_exporter
Restart=on-failure
[Install]
WantedBy=multi-user.target

使用配置文件启动
在mysqld_exporter目录下创建mysqld_exporter.sh启动脚本

1
2
#!/bin/bash
/opt/prometheus/mysqld_exporter/mysqld_exporter --config.my-cnf="/opt/prometheus/mysqld_exporter/.my.cnf" --web.listen-address=:9104

创建mysqld_exporter.service文件

1
2
3
4
5
6
7
8
9
10
11
12
$ vim /usr/lib/systemd/system/mysqld_exporter.service
[Unit]
Description=mysqld_exporter
After=network.target
[Service]
Type=simple
User=prometheus
#指定启动脚本路径
ExecStart=/opt/prometheus/mysqld_exporter/mysqld_exporter.sh
Restart=on-failure
[Install]
WantedBy=multi-user.target

上面两种方式任选一个
systemctl start mysqld_exporter
启动服务

监控redis

下载redis_exporter, https://github.com/oliver006/redis_exporter/releases

或者

1
2
3
4
$ go get github.com/oliver006/redis_exporter
$ cd $GOPATH/src/github.com/oliver006/redis_exporter
$ go build
$ ./redis_exporter <flags>

0%