Splunk 3.3.4 in Software

Good security practices and the pressures of regulatory compliance now demand that log data from all sorts of network devices is collected and made available for scrutiny. This can result in a veritable avalanche of data that can be difficult to search through without an indexing facility.

The brilliantly named Splunk could come to the rescue as it's capable of gathering data from a wide range of sources, storing it in its centralised database and indexing it for fast searches. Sources include data from device logs and configurations, alerts, SNMP traps and so on and the best part is that for a 500MB daily log store limit you don't pay a penny. We installed Splunk on a Windows Server 2003 R2 system and although it only took a few minutes we'd recommend taking time out to read the copious documentation on the Splunk web site and make use of the video tutorials as it does present a steep learning curve.

Splunk can utilise any log data as long as it's in human readable format so sources such as syslog are no problem. It listens on IP address and port combinations and any text log data sent to it will be imported into the database and automatically indexed. It can monitor files and folders used to store text based log data on remote systems and will automatically index these into its database. Audit daemons that create log files as binaries will need to be converted to text format before Splunk can use them but utilities for this are usually provided by the vendor and Splunk also has a scheduler tool so it can run these automatically at regular intervals. Splunk can also index Windows Event Logs, the registry and WMI data.

Larger businesses will no doubt be storing more than 500MB per day and will want the Enterprise version of Splunk. Prices depend on the amount of daily data and the main difference is that this version can send and receive data - the free version can only receive log data. This allows Splunk Enterprise to support a distributed environment where you may have multiple systems gathering log data and passing it to a central database. Other features of Enterprise are the ability to detect changes to files on remote systems and use different user accounts for access control.

The Splunk web console opens with a smart home page that can be easily cusomised to suit. The default is to show log sources, the source type, a list of monitored hosts and detected errors but it can be easily changed to show graphs and tables created from custom reports - rest assured, the possibilities are endless. Log data sources need to be defined to Splunk and range from host systems and ports to folder locations or FIFO queues. Splunk also offers a crawl option where it can scan systems, volumes or folders and provide a list of all the files it finds which can be refined with exclusions and further searches.

We had plenty of syslog sources in the lab so decided to see how well Splunk handled these. Sources are defined as inputs as we created one for listening on port 514 for all LAN addresses so Splunk would pick up any syslog source that was pointing to it. First we configured an HP ProCurve 2848 Gigabit switch using its CLI logging command with the IP address of the Splunk syslog server. And that's all we had to do as the switch's management IP address appeared automatically in Splunk's list of monitored hosts.

We could then view its syslog data either by selecting the source entry for UDP traffic on port 514, the syslog source type or the switch's host entry. We also had a Radware DP102 IPS appliance standing guard in front of our firewall and we set this up to send syslog data to the Splunk server. As with the HP switch the moment it started sending log data it appeared automatically in the host list ready for selection.