{"id":3278,"date":"2018-03-07T19:27:32","date_gmt":"2018-03-08T00:27:32","guid":{"rendered":"https:\/\/antipaucity.com\/?p=3278"},"modified":"2018-03-07T19:27:32","modified_gmt":"2018-03-08T00:27:32","slug":"splunk-oddity-17681-stats-vs-table","status":"publish","type":"post","link":"https:\/\/merikebi.warrenmyers.com\/?p=3278","title":{"rendered":"splunk oddity #17681 – stats vs table"},"content":{"rendered":"

It’s fairly common to want to table<\/a> the data you’ve found in a search in {{Splunk}} – heck, if you’re not prettying the data up somewhy<\/em>, why are you bothering with the tool?<\/p>\n

But I digress.<\/p>\n

There are two (at least) ways of making a table – you can use the |table <field(s)><\/code> syntax, or you can use |stats <some function> <field(s)><\/code> approach.<\/p>\n

Interestingly, in my testing in both test and production environments, using the |stats...<\/code> approach is consistently<\/em> 10-15% faster<\/strong> than the |table...<\/code> option.<\/p>\n

Why? I don’t know. He’s on third. And I don’t give a darn!<\/p>\n

This is another case of technical intricacies mattering<\/a> … but I don’t know what is going on under the hood that makes the apparently-more-complex option run faster than the apparently-simler option.<\/p>\n

Maybe someday someone in Splunk engineering will be able to enlighten me to that.<\/p>\n

This reminds me a bit of an optimization I was able to help a friend with upwards of 12 years ago – they had queries running in {{MySQL}} that were taking forever<\/em> to complete (and by “forever”, I mean they were running sometimes 4-5 times a long as the interval between running them (they ran every 5 mintues, but could take 20+ minutes to finish!)).<\/p>\n

What I found, at least back in the dark days of MySQL 3.x was that using IN(...)<\/code> was loads faster than using OR<\/code> statements.<\/p>\n

So a query that had a clause WHERE name IN(\"bob\",\"sarah\",\"mike\",\"terry\",\"sue\") would run anywhere from 20-90% quicker than the logically-equivalent <\/code>WHERE name=\"bob\" OR name=\"sarah\" OR name=\"mike\" OR name=\"terry\" OR name=\"sue\"<\/code> (given a large enough dataset overwhich it was running … on small [enough] tables (say up to a couplefew thousand records), the OR<\/code> version would run equally, or occasionally faster).<\/p>\n

In their<\/em> case, by switching to the IN(...)<\/code> form, queries went from taking 20+ minutes to finishing in ~20 seconds<\/code>! <\/p>\n

Bonus tidbit:<\/h4>\n

It is well-known in Splunkland that using dedup<\/code> is an “expensive” operation. Want a clever way around it (that is much<\/em> faster)? Instead of doing something like index=myndx | fields ip host | dedup host<\/code>, run index=myndx | fields ip host | stats count by host | fields -count<\/code>. The |stats .. |fields -count<\/code> trick seems to run anywhere from 15-30% faster than dedup<\/code>.<\/p>\n","protected":false},"excerpt":{"rendered":"

It’s fairly common to want to table the data you’ve found in a search in {{Splunk}} – heck, if you’re not prettying the data up somewhy, why are you bothering with the tool? But I digress. There are two (at least) ways of making a table – you can use the |table <field(s)> syntax, or …
Continue reading splunk oddity #17681 – stats vs table<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[19,41],"tags":[535,705],"keyring_services":[],"class_list":["post-3278","post","type-post","status-publish","format-standard","hentry","category-hmmm","category-technical","tag-optimization","tag-splunk"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts\/3278","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3278"}],"version-history":[{"count":0,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts\/3278\/revisions"}],"wp:attachment":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3278"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3278"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3278"},{"taxonomy":"keyring_services","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fkeyring_services&post=3278"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}