前言 我想使用 RHEL/CentOS 系统自带的日志工具Rsyslog, 根据程序日志类别动态存储到一个以程序名命名的文件。
日志中心化给我们带来了好多优势， 那么我们如何收集 Nginx 日志到日志服务器呢？ Nginx 自 1.7.1 版本之后开始支持 syslog.
Originally posted on Centralized logging for fun and profit
Setting up a centralized log server using syslog isn't as hard as many may believe. Whether it's logs from Apache, nginx, email services, or even from your own Python applications having a central log server gives you many benefits:
Benefits to a centralized logs
- Reduces disk space usage and disk I/O on core servers that should be busy doing something else. This is especially true if you want to log all queries to your database. Doing this on the same disk as your actual database creates a write for every read and an extra write for every write.
- Removes logs from the server in the event of an intrusion or system failure. By having the logs elsewhere you at least have a chance of finding something useful about what happened.
- All of your logs are in one place, duh! This makes things like grepping through say Apache error logs across multiple webservers easier than bouncing around between boxes. Any log processing and log rotation can also be centralized which may delay your sysadmin from finally snapping and killing everyone.
前言 本文主要简述的如何使用 Rsyslog 实现日志的中心化，并利用 ELK Stack 实现图形搜索与数据统计。
ELK(Elasticsearch + Logstash + Kibana) 是一套开源的日志管理方案
Logstash: The server component of Logstash that processes incoming logs
Elasticsearch: Stores all of the logs
Kibana 4: Web interface for searching and visualizing logs, which will be proxied through Nginx
Logstash Forwarder: Installed on servers that will send their logs to Logstash, Logstash Forwarder serves as a log forwarding agent that utilizes the lumberjack networking protocol to communicate with Logstash
JDK - http://www.oracle.com/technetwork/java/javase/downloads/index.html
Elasticsearch - https://www.elastic.co/downloads/elasticsearch
Logstash - https://www.elastic.co/downloads/logstash
Kibana - https://www.elastic.co/downloads/kibana
redis - http://redis.io/download