Log management is the collection of self-generated data from IT hardware devices and software applications. The collection of this data can contain useful information about business processes such as the number of errors on a website or even a security issue that displays the number of failed attempts to access a perimeter router.
Many organizations conduct log management practices to meet regulatory compliance in their respected industries. For instance, the Graham-Leach-Bliley (GLB) Act provides compliance guidelines to those organizations in the financial industry, while the Healthcare Insurance Portability and Accountability Act (HIPAA) has a direct affect on the health care industry. In addition, the Sarbanes-Oxley (SOX) Act offers protection guidelines to publicly traded companies’ financial systems and the Federal Information Security Management Act (FISMA) aids in the protection of the federal government’s information systems.
However, the use of log management practices serves more than just a vehicle to satisfy compliance requirements, it can also be used to assist in the fault management process to aid in troubleshooting at the tier 1 level. In addition to fault management, performance analysis is another useful area for log data to obtain information about server CPU, memory, disk and even network throughput. Lastly, log data can conduct trend analysis to view peak usage stats such as the “top”10s as well as aid in the detection of security incidents and violations.
The management of log data may represent an organizations’ needs to report and alert on events occurring on a day-to-day operational basis or simply for the storing of the data for forensic purposes in case of an adverse security incident in the future.
Before implementing or upgrading a log management solution, here are some questions that should be asked.
- What is the goal? Is it for the automation of service tickets, data analysis, incident response or all of the above?
- Determine what the event sources are, such as Firewalls, Hosts, Applications, and Network Devices.
- How are the logs going to get from the source devices to its destinations?
- Will the logs operate in a store & forward or streaming for distribution?
- What are the volume requirements? Will you need to support a 100MB/GB/TB per day or per hour or per week?
- What types of events require monitoring? For example, login/logout, port up/ port down, HTTP errors, SMTP failures, Anonymous FTP
- Do you need log for non-repudiation, compartmentalization or with encryption? If so, at what level?
- What are your storage requirements? How long will you need to store your log data?
As you work through the requirements, it may lead to new or additional requirements that may suggest partitioning the architecture into more manageable pieces such as a distributed approach to log management.
In part 2, I’ll discuss log management solutions to consider to help you determine the best one for your organization.
