Based on the original version from https://github.com/camrunr/hec_perf_report/blob/master/hec_perf_report.xml index=_introspection (`indexerhosts`) OR (`heavyforwarderhosts`) `splunkadmins_hec_metrics_source` http_event_collector_token | bucket _time span=$dd_span$ | stats sum(data.num_of_events) as Events sum(data.total_bytes_received) as Bytes by _time data.token_name $timepicker.earliest$ $timepicker.latest$ 1 $refreshinterval$ index=_introspection (`indexerhosts`) OR (`heavyforwarderhosts`) `splunkadmins_hec_metrics_source` http_event_collector_token | bucket _time span=$dd_span$ | stats sum(data.num_of_events) as Events sum(data.total_bytes_received) as Bytes by _time host | eval host=replace(host,"\..*","") $timepicker.earliest$ $timepicker.latest$ 1 $refreshinterval$
-4h@m now 1 minute 5 minutes 30 minutes 1 hour 1 day 1min 15 300
Events/sec by host timechart limit=$hostcount$ span=$dd_span$ per_second(Events) as Events/sec by host Bytes/sec by host timechart limit=$hostcount$ span=$dd_span$ per_second(Bytes) as Bytes/sec by host Events/sec by input/group timechart span=$dd_span$ per_second(Events) as Events/sec by data.token_name Bytes/sec by input/group timechart span=$dd_span$ per_second(Bytes) as Bytes/sec by data.token_name HEC Batching Efficiency $refreshinterval$ index=_introspection (`indexerhosts`) OR (`heavyforwarderhosts`) `splunkadmins_hec_metrics_source` http_event_collector_token | eval EpR='data.num_of_events'/'data.num_of_requests' | bucket _time span=5m | stats sum(data.num_of_events) as events avg(EpR) as events_per_POST sum(data.num_of_requests) as reqs sum(data.total_bytes_received) as Bytes by _time data.token_name | eval reqs_per_sec=reqs/300, bytes_per_post=Bytes/reqs | rename data.token_name as token_name | stats sum(eval(Bytes/1024/1024)) as MBytes sum(events) as Events p50(events_per_POST) as events_per_post p50(bytes_per_post) as bytes_per_post p90(reqs_per_sec) as posts_per_sec by token_name | eval MBytes = round(MBytes, 2), events_per_post=round(events_per_post,2), bytes_per_post=round(bytes_per_post,2), posts_per_sec=round(posts_per_sec,2) | sort - posts_per_sec $timepicker.earliest$ $timepicker.latest$ [#DC4E41,#DC4E41,#F8BE34,#53A051] 0,5,10 [#53A051,#F8BE34,#DC4E41] 10,50
If useACK is in use num_of_requests_waiting_ack is high then this can be an issue (HEC tokens with useACK will stop allowing data through) $refreshinterval$ index=_introspection (`indexerhosts`) OR (`heavyforwarderhosts`) data.series=http_event_collector data.num_of_requests_waiting_ack=* sourcetype=http_event_collector_metrics | timechart minspan=2m max(data.num_of_requests_waiting_ack) AS num_of_requests_waiting_ack $timepicker.earliest$ $timepicker.latest$