The Secret Language of Splunk

Imagine this… first day of your first corporate job. Within an hour of arriving you are sitting in a meeting room listening to a group of people discussing something very important in great detail and you don’t have a clue what they are on about. There are ‘pdis’,’inits’, ‘poes’, ‘cob’ and that’s before you look at systems ‘db2’, ‘sap’, ‘jde’, ‘pptx’ and ‘cfgs’. Its like you are on holidays but not the good kind. The kind where you are in a strange country where you don’t know the language so you pick Bambi hearts served rare off the menu because you see the word for venison and think you have just ordered a juicy steak.

You walk out of the meeting thinking you have made an enormous mistake and you are completely out of your depth. The thing is every company you will work in has their own secret language it might be a system or a corporate standard.

Splunk is no different.
pivots? indexes? search heads? forwarders? buckets? data models? cluster? and slaves?

Pretty sure your Oxford dictionary won’t help here. So the smart play is to head over to the Splunk Website and start searching their product dictionary. Only one problem when Splunk says

A hierarchically structured, search-time mapping of semantic knowledge about one or more datasets that encode the domain knowledge necessary to generate specialized searches of those datasets. Splunk Enterprise uses these specialized searches to generate reports and charts for pivot users.Splunk Splexicon

this is their definition for the term Data Model. English it may be but meaningful it certainly isn’t to the uninitiated.
So stick with me over the following weeks as I put together a Splunk Glossary to describe many of these terms in plain English so that even the non-techos will understand… Whilst I love Splunk I just think that a few English translations will go a long way.
If your a diehard techo knock yourself out with the Splunk Splexicon guide

Splunk Glossary

Splunk stores indexed data in “Buckets” these buckets are organised by the age of the data.
Buckets move through stages as they age. The buckets start as “Hot” and move to “Frozen”;
HOT – most recent data
WARM – hot bucket move to warm when they reach the limit set in the ‘maxDataSize’ or they are older than the ‘maxHotSpanSecs’
COLD – move to cold once the ‘maxWarmDBCount ‘is reached
FROZEN – once the ‘maxTotalDataSizeMB’ is reached if a coldToFrozenScript’ is setup otherwise the bucket is deleted
THAWED – Frozen buckets can be thawed manually
FISHBUCKET – see below
A bunch of pre-defined search queries that power users create so that users can run their own reports (pivots) without understanding the search language.
Dataset constraints filter out events that aren’t relevant to the dataset. A constraint is really just a simple search that doesn’t have additional pipes and search commands
This is the way Splunk keeps track of what files it has read already. Used by engineers for troubleshooting issues.
The file that Splunk creates to make your data search friendly.

Any information that is automatically created without human intervention. This data can be from a wide range of sources, including websites, servers, application, networks, mobile devices, and so on.
(Taken from Mastering Splunk by James Miller)
A chart or a table that users can create in the Splunk. Users can add predefined searches to get the right data without having to understand the search language. There are a bunch of chart types and the user can pick or choose the one that fits their data best.
SPL is the search language used by Splunk to search your logs. It is made up of search rules, commands and functions that allows you to filter, modify, manipulate and display the data visually in table, charts and graphs.

Share this Post

Leave a Reply

Your email address will not be published. Required fields are marked *