The Secret Language of Splunk

Imagine this… first day of your first corporate job. Within an hour of arriving you are sitting in a meeting room listening to a group of people discussing something very important in great detail and you don’t have a clue what they are on about. There are ‘pdis’,’inits’, ‘poes’, ‘cob’ and that’s before you look at systems ‘db2’, ‘sap’, ‘jde’, ‘pptx’ and ‘cfgs’. Its like you are on holidays but not the good kind. The kind where you are in a strange country where you don’t know the language so you pick Bambi hearts served rare off the menu because you see the word for venison and think you have just ordered a juicy steak.

You walk out of the meeting thinking you have made an enormous mistake and you are completely out of your depth. The thing is every company you will work in has their own secret language it might be a system or a corporate standard.

Splunk is no different.
pivots? indexes? search heads? forwarders? buckets? data models? cluster? and slaves?

Pretty sure your Oxford dictionary won’t help here. So the smart play is to head over to the Splunk Website and start searching their product dictionary. Only one problem when Splunk says

A hierarchically structured, search-time mapping of semantic knowledge about one or more datasets that encode the domain knowledge necessary to generate specialized searches of those datasets. Splunk Enterprise uses these specialized searches to generate reports and charts for pivot users.Splunk Splexicon

this is their definition for the term Data Model. English it may be but meaningful it certainly isn’t to the uninitiated.
So stick with me over the following weeks as I put together a Splunk Glossary to describe many of these terms in plain English so that even the non-techos will understand… Whilst I love Splunk I just think that a few English translations will go a long way.
If your a diehard techo knock yourself out with the Splunk Splexicon guide

Splunk Glossary

Splunk stores indexed data in “Buckets” these buckets are organised by the age of the data.
Buckets move through stages as they age. The buckets start as “Hot” and move to “Frozen”;
HOT – most recent data
WARM – hot bucket move to warm when they reach the limit set in the ‘maxDataSize’ or they are older than the ‘maxHotSpanSecs’
COLD – move to cold once the ‘maxWarmDBCount ‘is reached
FROZEN – once the ‘maxTotalDataSizeMB’ is reached if a coldToFrozenScript’ is setup otherwise the bucket is deleted
THAWED – Frozen buckets can be thawed manually
FISHBUCKET – see below
A bunch of pre-defined search queries that power users create so that users can run their own reports (pivots) without understanding the search language.
Dataset constraints filter out events that aren’t relevant to the dataset. A constraint is really just a simple search that doesn’t have additional pipes and search commands

Share this Post

16 Comments on “The Secret Language of Splunk”

  1. Hey There. I found your blog using msn. This is a very well written article. I will make sure to bookmark it and come back to read more of your useful info. Thanks for the post. I’ll definitely comeback.

  2. Howdy would you mind letting me know which web host you’re working with? I’ve loaded your blog in 3 completely different web browsers and I must say this blog loads a lot faster then most. Can you recommend a good web hosting provider at a reasonable price? Thanks a lot, I appreciate it!

  3. Aw, this was an incredibly good post. Spending
    some time and actual effort to create a very good article… but what can I say…
    I procrastinate a lot and don’t seem to get nearly anything done.

  4. Oh my goodness! Awesome article dude! Thanks, However
    I am having difficulties with your RSS. I don’t know the
    reason why I cannot join it. Is there anyone else
    getting similar RSS issues? Anyone that knows the solution can you kindly
    respond? Thanks!!

  5. Thank you for any other great post. The place else could
    anybody get that kind of information in such a perfect approach of writing?

    I have a presentation subsequent week, and I am at the look for such information.

  6. Greetings! This is my first visit to your blog!
    We are a collection of volunteers and starting a new project in a community in the same niche.
    Your blog provided us beneficial information to work on. You have done a wonderful job!

  7. You really make it seem so easy with your presentation but I find this topic to be actually something that
    I think I would never understand. It seems too complex
    and very broad for me. I am looking forward for your
    next post, I will try to get the hang of it!

  8. Oh my gosh! It seems so familiar to my first working day. Why I haven’t found this before.
    I’m trying to optimize the indexes.conf now. I thought that the triggers for migration from hot to warm bucket – ‘maxDataSize’/’maxHotBuckets’/’maxHotIdleSecs’. And the ‘maxHotSpanSecs’ is the timespan between the earliest and latest events in the bucket. Isn’t it?

  9. What’s up, after reading this remarkable post i am also glad to share my experience here with colleagues.

Leave a Reply

Your email address will not be published. Required fields are marked *