tomcat日志收集方式
|
1 2 3 4 5 6 7 |
1.将日志改成Json格式 在企业中，想要将java日志改成json格式，并没有那么容易。 因为将日志改成Json格式，查看起来会很难受，有些开发人员不希望将日志格式改成Json的，所以，在改日志格式之前需要跟开发人员进行沟通，那么将tomcat日志格式改成Json格式也有两种方式。 1）开发自己更改，通过程序代码，或者log4j 2）运维修改tomcat的server配置文件 2.通过logstash的mutiline模块实现多行匹配 |
安装tomcat
1）安装java环境
2）安装tomcat
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
1.上传代码包 [root@web01 ~]# rz [root@web01 ~]# ll -rw-r--r-- 1 root root 11026056 2020-12-04 18:04 apache-tomcat-9.0.30.tar.gz 2.解压tomcat包 [root@web01 ~]# tar xf apache-tomcat-9.0.30.tar.gz 3.将安装包移动并改名 [root@web01 ~]# mv apache-tomcat-9.0.30 /usr/local/tomcat-9.0.30 4.做软连接 [root@web01 ~]# ln -s /usr/local/tomcat-9.0.30 /usr/local/tomcat |
3）配置站点
|
1 2 3 4 5 6 7 8 9 |
1.写一个测试页面到站点目录下的index.html文件中 [root@web01 ~]# echo 'TEST elk' > /usr/local/tomcat/webapps/ROOT/index.html 2.启动tomcat [root@web01 ~]# /usr/local/tomcat/bin/startup.sh 3.检测tomcat端口是否启动 [root@web01 ~]# netstat -lntup|grep 8080 tcp 0 0 :::8080 :::* LISTEN 12569/java |
4）访问测试
|
1 |
http://10.0.0.7:8080/ |
配置logstash收集tomcat日志
1）配置
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[root@web01 ~]# vim /etc/logstash/conf.d/tomcat_log_es.conf input { file { path => "/usr/local/tomcat/logs/localhost_access_log.*.txt" start_position => "end" type => "tomcat_log" } } output { elasticsearch { hosts => ["10.0.0.71:9200"] index => "tomcat_log_%{+YYYY-MM-dd}" } } |
2）启动
|
1 |
[root@web01 ~]# logstash -f /etc/logstash/conf.d/tomcat_log_es.conf |
方式一: 收集tomcat的json格式日志
1）修改tomcat日志格式
|
1 2 3 4 5 6 7 8 9 |
[root@web01 ~]# vim /usr/local/tomcat/conf/server.xml <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="tomcat_access_json" suffix=".log" pattern="{"clientip":"%h","ClientUser":"%l","authenticated":"%u","AccessTime":"%t","method":"%r","status":"%s","SendBytes":"%b","Query?string":"%q","partner":"%{Referer}i","AgentVersion":"%{User-Agent}i"}"/> <!-- Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" / --> |
2）重启Tomcat
|
1 2 |
[root@web01 ~]# /usr/local/tomcat/bin/shutdown.sh [root@web01 ~]# /usr/local/tomcat/bin/startup.sh |
3）查看新的日志
|
1 2 |
[root@web01 ~]# tail -f /usr/local/tomcat/logs/tomcat_access_json.2020-12-07.log {"clientip":"10.0.0.1","ClientUser":"-","authenticated":"-","AccessTime":"[07/Dec/2020:22:51:25 +0800]","method":"GET / HTTP/1.1","status":"200","SendBytes":"9","Query?string":"","partner":"-","AgentVersion":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36"} |
4）配置logstash收集新的日志
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@web01 ~]# vim /etc/logstash/conf.d/tomcat_log_es.conf input { file { path => "/usr/local/tomcat/logs/tomcat_access_json.*.log" start_position => "end" type => "tomcat_log" } } output { elasticsearch { hosts => ["10.0.0.71:9200"] index => "tomcat_json_log_%{+YYYY-MM-dd}" } } |
5）启动服务
|
1 |
[root@web01 ~]# logstash -f /etc/logstash/conf.d/tomcat_log_es.conf |
方式二：使用multiline插件收集java日志
|
1 2 3 4 |
使用codec的multiline插件实现多行匹配，这是一个可以将多行进行合并的插件，而且可以使用what指定将匹配到的行与前面的行合并还是和后面的行合并 帮助文档：https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html 因为目前tomcat日志中没有exception，所以，我们把Logstash部署在ES上，收集一下ES的java日志。 |
1）测试多行匹配
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@web01 ~]# vim /etc/logstash/conf.d/stdin_stdout.conf input { stdin { codec => multiline { pattern => "^\[" negate => "true" what => "previous" } } } output { stdout {} } |
2）启动
|
1 |
[root@web01 ~]# logstash -f /etc/logstash/conf.d/stdin_stdout.conf |
3）测试
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
#测试输入一堆内容，并没有打印，只有当输入一个以 [ 开头的时候才会结束并输出 jhvc jkhv jhv jc [ { "message" => "jhvc\njkhv\njhv\njc", "@timestamp" => 2020-12-07T15:15:49.182Z, "@version" => "1", "tags" => [ [0] "multiline" ], "host" => "web01" } |
4）收集java日志写入ES
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[root@web01 ~]# cat /etc/logstash/conf.d/java_es.conf input { file { path => "/usr/local/tomcat/logs/tomcat_access_json.2020-12-07.log" start_position => "end" codec => multiline { pattern => "^\[" negate => "true" what => "previous" } } } output { elasticsearch { hosts => ['10.0.0.71:9200'] index => "tomcat_json_log_%{+YYYY-MM-DD}" } } |
5）启动
|
1 2 |
[root@web01 ~]# logstash -f /etc/logstash/conf.d/java_es.conf & |
6）测试收集日志
|
1 |
[root@web01 ~]# cat tomcat.log >> /usr/local/tomcat/logs/tomcat_access_json.2020-12-07.log |