CentOS centralised logging, syslogd, rsyslog, syslog-ng, logstash sender?

Posted by benbradley on Server Fault See other posts from Server Fault or by benbradley
Published on 2012-11-28T16:14:20Z Indexed on 2012/11/28 17:06 UTC
Read the original article Hit count: 342

Filed under:

centos

|

logging

|

syslog

|

rsyslog

|

logstash

I'm trying to figure out the best way to setup a central place to store and interrogate server logs. syslog, Apache, MySQL etc.

I've found a few different options but I'm not sure what would be best. I'm looking for something that is easy to install and keep updated on many virtual machines. I can add it to a VM template going forward but I'd also like it to be easy to install to keep the VM complexity down.

The options I've found so far are:

syslogd
syslog-ng
rsyslog
syslogd/syslog-ng/rsyslog to logstash/ElasticSearch
logstash agent in each log "client" to send to Redis/logstash/ElasticSearch

And all sorts of permutations of the above.

What's the most resilient and light from the log "client" perspective?
I'd like to avoid the situation where log "clients" hang because they are unable to send their logs to the logging server. Also I would still like to keep local logging and the rotation/retention provided by logrotate in place.

Any ideas/suggestions or reasons for or against any of the above?
Or suggestions of a different structure entirely?

Cheers, B

© Server Fault or respective owner

Related posts about centos

Problems with repositories on CentOS 3.9

as seen on Super User - Search for 'Super User'
Hello, I have CentOS 3.9 for i386. When I try to instal some thing with yum, i.e: yum install firefox or yum install firefox* or yum list firefox and so on, I get: +++++++++++++++++++ yum info firefox Gathering header information file(s) from server(s) Server: CentOS-3 - Addons Server: CentOS-3 -… >>> More
Problems with repositories on CentOS 3.9

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I have CentOS 3.9 for i386. When I try to instal some thing with yum, i.e: yum install firefox or yum install firefox* or yum list firefox and so on, I get: +++++++++++++++++++ yum info firefox Gathering header information file(s) from server(s) Server: CentOS-3 - Addons Server: CentOS-3 -… >>> More
centos yum problems

as seen on Server Fault - Search for 'Server Fault'
I am really new to using linux and have just formatted my centos 5.2 vps and am trying to install links by using the command yum install links. But the following error gets displayed: [root@inverses ~]# yum install links Loading "fastestmirror" plugin Loading mirror speeds from cached hostfile *… >>> More
Networking issues with Linux server (CentOS 5.3)

as seen on Server Fault - Search for 'Server Fault'
I have a Linux server hosting our bug tracking software (CentOS 5.2 Kernel 2.6.18-128.4.1.el5) that I have having some strange network problems with. The machine is configured with two NICS, one for the public interface and the other for our server back end network. The problem is that after doing… >>> More
Networking issues with Linux server (CentOS 5.3)

as seen on Server Fault - Search for 'Server Fault'
I have a Linux server hosting our bug tracking software (CentOS 5.2 Kernel 2.6.18-128.4.1.el5) that I have having some strange network problems with. The machine is configured with two NICS, one for the public interface and the other for our server back end network. The problem is that after doing… >>> More

Related posts about logging

log file is not getting created using JDK logging with Commons-logging

as seen on Stack Overflow - Search for 'Stack Overflow'
When I run the TestJcLLoggingService class log messages are coming to Console but no log file is created, please help me if you know the answer. two source files are pasted below. TestJcLLoggingService.java package com.amadeus.psp.pasd.logging; import org.apache.commons.logging.Log; import org… >>> More
Python logging before you run logging.basicConfig?

as seen on Stack Overflow - Search for 'Stack Overflow'
It appears that if you invoke logging.info() BEFORE you run logging.basicConfig, the logging.basicConfig call doesn't have any effect. In fact, no logging occurs. Where is this behavior documented? I don't really understand. >>> More
Python: combine logging and wx so that logging stream is redirectet to stdout/stderr frame

as seen on Stack Overflow - Search for 'Stack Overflow'
Here's the thing: I'm trying to combine the logging module with wx.App()'s redirect feature. My intention is to log to a file AND to stderr. But I want stderr/stdout redirected to a separate frame as is the feature of wx.App. My test code: import logging import wx class MyFrame(wx.Frame): … >>> More
Redirect logging output using custom logging handler

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, I'm using a module in my python app that writes a lot a of messages using the logging module. Initially I was using this in a console application and it was pretty easy to get the logging output to display on the console using a console handler. Now I've developed a GUI version of my app… >>> More
Java Logger API

as seen on Oracle Blogs - Search for 'Oracle Blogs'
This is a more like a tip rather than technical write up and serves as a quick intro for newbies. The logger API helps to diagnose application level or JDK level issues at runtime. There are 7 levels which decide the detailing in logging (SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST)… >>> More