Enterprise Log Management: An Overview (Part 2) – – FOSE
In part 1 of my guest post series for the FOSE Insights Blog, I talked about the importance of enterprise log management and the questions one should ask before implementing a log management solution. In this post, I’ll be covering the different types of log management solutions to help you determine the best one for your organization.
When implementing a log management solution, there are various types of solutions, architectures and definitions. Here are a few you should be familiar with:
- Sinkhole – Traditional single “syslog” server that “receives” remote logs from one or more sources
- Hierarchy – Multi-Tiered sinkhole’s divided by department, network (vlan), or other logical fashion such as accounting, marketing and engineering to collect log data.
- Aggregator – Usually located at the top of a hierarchy, where major functionality such as alerting, reporting, searching and correlations occur.
- Distributed – Independent log repositories, may be searchable/accessible from a central location.
- Store and Forward – logs are written to a local disk or network disk to be spooled and sent later.
- Streaming – Real-time distribution of log data to a remote logging server as they are being generated.
- Agent Based – Operating systems that do not support remote logging often require assistance from software to send log data. Even those operating systems that are capable of sending log data may use agents to send out specific data in a secure manner to a logging server.
- Agent Less – Systems do not send log data directly to the log server, the logger itself obtains the data via secure file copy (store & grab) or WMI (Windows Management Interface)
- Combo – Most mature log management infrastructures use part or all the above in some way or fashion.
There are a number of useful tools in both the open-source and commercial space that can assist in the creation of a log management solution or the upgrade of an existing solution. In the open-source area, the following solutions are:
- Syslog-ng – Unix based tool. Swiss army knife of log management. Can read any file and “tail” it to the network. Commercial versions available (sinkhole/ forwarding agent)
- Rsyslog – Like syslog-ng , with enhanced filtering, encryption, buffering
- OSSEC – Host Based or Server Based SIM/IDS (Aggregator/Agent)
- –SEC.pl – Simple Event Correlator (Aggregator)
- –PHP-Syslog, MySQL – PHP interface to logs in a database
- –Lasso – Agent-less collection agent for Windows (WMI based)
While in the commercial realm, there are many formidable solutions available, but are just a here are a few notable ones:
Splunk – In the “Pro” column, Splunk provides relevant search information very fast due to its use of data indexing. It has a wide support for various operating systems such as Windows, Mac OS X and Linux-based systems and it is extremely easy to use. One can also have use the software at no cost up to 500MB of log data. In the “Con” column, Splunk seem to have quick development cycles that requires numerous software updates and the advanced features, such as “app” development has a bit of a steep learning curve.
Read the rest at FOSE’s Blog Site