Promethues 监控Tomcat 主要用的模块

• Promethus 负载抓取/存储指标信息、并提供查询功能
• grafana 数据可视化
• JMX exporter 提供JMX中JVM相关的metrics

1、利用JMX exporter，在JAVA进程内启动一个小型的Http server

2、配置Prometheus抓取那个Http server提供的metrics。

3、配置Grafana连接Prometheus，配置Dashboard。

一、下载一个tomcat,下载jmx_exporter

• 1、获取jmx_exporter有2种方法，自己编译获取jar包，获取现成的jar包

wget  https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar

或者编译
git clone https://github.com/prometheus/jmx_exporter
cd jmx_exporter
mvn package

• 2、安装tomcat

wget http://mirrors.tuna.tsinghua.edu.cn/Apache/tomcat/tomcat-8/v8.5.51/bin/apache-tomcat-8.5.51.tar.gz
tar zxvf apache-tomcat-8.5.51.tar.gz
mv apache-tomcat-8.5.51 /usr/localt/tomcat_test

• 3、配置文件下载

wget https://github.com/prometheus/jmxexporter/blob/master/exampleconfigs/tomcat.yml

tomcat.yml的内容如下

---   
lowercaseoutputLabelNames: true
lowercaseOutputName: true
rules:
- pattern: 'Catalina<type=GlobalRequestProcessor, name="(w+-w+)-(d+)"><>(w+):'
  name: tomcat_$3_total
  labels:
    port: "$2"
    protocol: "$1"
  help: Tomcat global $3
  type: COUNTER
- pattern: 'Catalina<j2eeType=Servlet, WebModule=//([-a-zA-Z0-9+&@#/%?=~_|!:.,;]*[-a-zA-Z0-9+&@#/%=~_|]), name=([-a-zA-Z0-9+/$%~_-|!.]*), J2EEApplication=none, J2EEServer=none><>(requestCount|maxTime|processingTime|errorCount):'
  name: tomcat_servlet_$3_total
  labels:
    module: "$1"
    servlet: "$2"
  help: Tomcat servlet $3 total
  type: COUNTER
- pattern: 'Catalina<type=ThreadPool, name="(w+-w+)-(d+)"><>(currentThreadCount|currentThreadsBusy|keepAliveCount|pollerThreadCount|connectionCount):'
  name: tomcat_threadpool_$3
  labels:
    port: "$2"
    protocol: "$1"
  help: Tomcat threadpool $3
  type: GAUGE
- pattern: 'Catalina<type=Manager, host=([-a-zA-Z0-9+&@#/%?=~_|!:.,;]*[-a-zA-Z0-9+&@#/%=~_|]), context=([-a-zA-Z0-9+/$%~_-|!.]*)><>(processingTime|sessionCounter|rejectedSessions|expiredSessions):'
  name: tomcat_session_$3_total
  labels:
    context: "$2"
    host: "$1"
  help: Tomcat session $3 total
  type: COUNTER

• 4、收集数据

4-1、Tomcat收集数据

mkdir /usr/local/tomcat_test/jmx/
cp jmx_prometheus_javaagent-0.12.0.jar  /usr/local/tomcat_test/jmx/
cp tomcat.yml  /usr/local/tomcat_test/jmx/

修改配置文件vim /usr/local/tomcat_test/bin/catalina.sh 
JAVA_OPTS="-javaagent:/usr/local/tomcat_test/jmx/jmx_prometheus_javaagent-0.12.0.jar=39081:/usr/local/tomcat_test/jmx/tomcat.yml"

4-2、JAR包运行（案例，关键文件jmxprometheusjavaagent-0.3.0.jar和tomcat.yml）

java -javaagent:./jmx_prometheus_javaagent-0.3.0.jar=9151:tomcat.yaml -jar yourJar.jar

• 5、 测试是否收集到数据 curl -s http://localhost:39081/ | more

[root@Prometheus promethus]# curl -s http://localhost:39081/ | more 
# HELP jmx_exporter_build_info A metric with a constant '1' value labeled with the version of the JMX exporter.
# TYPE jmx_exporter_build_info gauge
jmx_exporter_build_info{version="0.12.0",name="jmx_prometheus_javaagent",} 1.0
# HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.
# TYPE jmx_config_reload_success_total counter
jmx_config_reload_success_total 0.0
# HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.
# TYPE jmx_config_reload_failure_total counter
jmx_config_reload_failure_total 0.0
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 3.99
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.582290765496E9
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 74.0

二、配置promethus 采集数据

• 1、文件引用 (本文采用) /usr/local/prometheus/prometheus.yml

- job_name: 'tomcat'
    file_sd_configs:
    - files: ['/usr/local/prometheus/conf/tomcat.yml']
      refresh_interval: 180s

/usr/local/prometheus/conf/tomcat.yml/tomcat.yml

- targets:
  - 114.67.116.119:39081
  labels:
    idc: test_idc
    service: tomcat_test

• 2、直接配置

scrape_configs:
  - job_name: 'java'
    static_configs:
    - targets: ['114.67.116.119:39081']

• 3、重载配置文件

kill -hup `ps -ef |grep prometheus|grep -v grep|awk '{print $2}'`
或者
kill -9 PID
/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml

• 4、查看promethus界面，tomcat的监控是否发现

三、grafana 数据可视化

导入模板 8563

名称对应 promethus.yml的job名称

展示tomcat监控大屏