admingit
/
SH-Deployer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add-app_health_check

Browse Source
master
admingit 1 year ago
parent bd8f768606
commit 36646ab7a0
32 changed files with 4036 additions and 0 deletions
Show Stats Download Patch File Download Diff File

42
apps/splunk_health_overview/README
Unescape View File

@ -0,0 +1,42 @@
The Splunk Health Overview app helps you easily manage your Splunk environment and provides a high level overview of key metrics.
It features several helpful views:
Health Status
License Overview
Search Activity
Resource Usage
Scheduler Activity
User Activity
Available Indexes
Indexes and Fields
The Splunk Monitoring App Focuses on the following areas:
Health Status – status of your instances, forwarders, data feeds, and internal messages.
License Overview – shows the status of your license usage, license trending, and index and indexer capacity - takes advantage of the Internal Events data model.
Resource Usage - uses the Introspection Usage data model for a look into Splunk and system resource usage.
Search Activity – illustrates what users are searching in your environment and allows you to easily identify poorly written searches.
Scheduler Activity – information regarding job schedules.
User Activity – information regarding user activity.
Available Indexes – shows what indexes and source types are in your environment with a custom contextual drill down capability that launches a new search with the index/sourcetype in question defined. This has proven very successful as users no longer have to run expensive exploratory searches to determine what indexes and sourcetypes are available to their role.
Indexes and Fields - Contains detail regarding field extractions for sourcetypes and the fields that are extracted.
Getting Started:
Once the app is installed a restart is required. After the restart you will need to run two of the scheduled saved searches avail_indexes and all_servers. These two lookups will gather information regarding your environment and are crucial to the apps functionality. If the all_servers lookup does not generate accurate information in your environment you will need to manually edit this.
Other Gotchas:
There is an Internal Events and Introspection accelerated data model will will need to complete acceleration.
You can customized thresholds based on your license capacity. By default, the filler gauge is set to 10 GB.
Splunk instances running on Mac OSX do not generate Introspection data.
Each instance that wants to be monitored will need to be a search peer so Splunk can collect the role information from that host. If it is not setup as a peer Splunk can still collect information regarding that host granted it is sending data to a host that is a Splunk search peer but will not show information generated from REST.
Currently only supported on Splunk 6.1+. To get this app functioning with 6.0.x you will need remove the panel tags from the xml views and remove any panel embedded time range picker as well as any multiselect inputs.
Special thanks to Sanford Owings and Vlad Skoryk.

46
apps/splunk_health_overview/appserver/static/search_health.css
Unescape View File

@ -0,0 +1,46 @@
/* Custom Icons */
td.icon {
text-align: center;
}
td.icon i {
font-size: 25px;
text-shadow: 1px 1px #aaa;
}
td.icon .severe {
color: red;
}
td.icon .elevated {
color: orangered;
}
td.icon .low {
color: #006400;
}
/* Row Coloring */
#highlight tr td {
background-color: #c1ffc3 !important;
}
#highlight tr.range-elevated td {
background-color: #ffc57a !important;
}
#highlight tr.range-severe td {
background-color: #d59392 !important;
}
#highlight .table td {
border-top: 1px solid #fff;
}
#highlight td.range-severe, td.range-elevated {
font-weight: bold;
}
.icon-inline i {
font-size: 18px;
margin-left: 5px;
}
.icon-inline i.icon-alert-circle {
color: #ef392c;
}
.icon-inline i.icon-alert {
color: #ff9c1a;
}
.icon-inline i.icon-check {
color: #5fff5e;
}

39
apps/splunk_health_overview/appserver/static/search_health.js
Unescape View File

@ -0,0 +1,39 @@
require([
'underscore',
'jquery',
'splunkjs/mvc',
'splunkjs/mvc/tableview',
'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
// Translations from rangemap results to CSS class
var ICONS = {
severe: 'alert-circle',
elevated: 'alert',
low: 'check-circle'
};
var RangeMapIconRenderer = TableView.BaseCellRenderer.extend({
canRender: function(cell) {
// Only use the cell renderer for the range field
return cell.field === 'Status';
},
render: function($td, cell) {
var icon = 'question';
// Fetch the icon for the value
if (ICONS.hasOwnProperty(cell.value)) {
icon = ICONS[cell.value];
}
// Create the icon element and add it to the table cell
$td.addClass('icon').html(_.template('<i class="icon-<%-icon%> <%- range %>" title="<%- range %>"></i>', {
icon: icon,
range: cell.value
}));
}
});
mvc.Components.get('table_status').getVisualization(function(tableView){
// Register custom cell renderer
tableView.table.addCellRenderer(new RangeMapIconRenderer());
// Force the table to re-render
tableView.table.render();
});
});

46
apps/splunk_health_overview/appserver/static/server_health.css
Unescape View File

@ -0,0 +1,46 @@
/* Custom Icons */
td.icon {
text-align: center;
}
td.icon i {
font-size: 25px;
text-shadow: 1px 1px #aaa;
}
td.icon .severe {
color: red;
}
td.icon .elevated {
color: orangered;
}
td.icon .low {
color: #006400;
}
/* Row Coloring */
#highlight tr td {
background-color: #c1ffc3 !important;
}
#highlight tr.range-elevated td {
background-color: #ffc57a !important;
}
#highlight tr.range-severe td {
background-color: #d59392 !important;
}
#highlight .table td {
border-top: 1px solid #fff;
}
#highlight td.range-severe, td.range-elevated {
font-weight: bold;
}
.icon-inline i {
font-size: 18px;
margin-left: 5px;
}
.icon-inline i.icon-alert-circle {
color: #ef392c;
}
.icon-inline i.icon-alert {
color: #ff9c1a;
}
.icon-inline i.icon-check {
color: #5fff5e;
}

52
apps/splunk_health_overview/appserver/static/server_health.js
Unescape View File

@ -0,0 +1,52 @@
require([
'underscore',
'jquery',
'splunkjs/mvc',
'splunkjs/mvc/tableview',
'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
// Translations from rangemap results to CSS class
var ICONS = {
severe: 'alert-circle',
elevated: 'alert',
low: 'check-circle'
};
var RangeMapIconRenderer = TableView.BaseCellRenderer.extend({
canRender: function(cell) {
// Only use the cell renderer for the range field
return cell.field === 'Status';
},
render: function($td, cell) {
var icon = 'question';
// Fetch the icon for the value
if (ICONS.hasOwnProperty(cell.value)) {
icon = ICONS[cell.value];
}
// Create the icon element and add it to the table cell
$td.addClass('icon').html(_.template('<i class="icon-<%-icon%> <%- range %>" title="<%- range %>"></i>', {
icon: icon,
range: cell.value
}));
}
});
mvc.Components.get('table_status').getVisualization(function(tableView){
// Register custom cell renderer
tableView.table.addCellRenderer(new RangeMapIconRenderer());
// Force the table to re-render
tableView.table.render();
});
mvc.Components.get('real_status').getVisualization(function(tableView){
// Register custom cell renderer
tableView.table.addCellRenderer(new RangeMapIconRenderer());
// Force the table to re-render
tableView.table.render();
});
mvc.Components.get('forwarder_status').getVisualization(function(tableView){
// Register custom cell renderer
tableView.table.addCellRenderer(new RangeMapIconRenderer());
// Force the table to re-render
tableView.table.render();
});
});

24
apps/splunk_health_overview/appserver/static/table_cell_highlighting.css
Unescape View File

@ -0,0 +1,24 @@
/* Cell Highlighting */
/*
#highlight td {
background-color: #c1ffc3 !important;
}
*/
#highlight td.range-low {
background-color: #00CC00;
}
#highlight td.range-elevated {
background-color: #FF9933 !important;
font-weight: bold;
}
#highlight td.range-severe {
background-color: #FF0000 !important;
font-weight: bold;
}

51
apps/splunk_health_overview/appserver/static/table_cell_highlighting.js
Unescape View File

@ -0,0 +1,51 @@
require([
'underscore',
'jquery',
'splunkjs/mvc',
'splunkjs/mvc/tableview',
'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
// Row Coloring Example with custom, client-side range interpretation
var CustomRangeRenderer = TableView.BaseCellRenderer.extend({
canRender: function(cell) {
// Enable this custom cell renderer for both the active_hist_searches and the active_realtime_searches field
return _(['Percent']).contains(cell.field);
},
render: function($td, cell) {
// Add a class to the cell based on the returned value
var value = parseFloat(cell.value);
// Apply interpretation for number of historical searches
if (cell.field === 'Percent') {
if (value > 85) {
$td.addClass('range-cell').addClass('range-severe');
}
else if (value > 60) {
$td.addClass('range-cell').addClass('range-elevated');
}
else if (value >= 0) {
$td.addClass('range-cell').addClass('range-low');
}
}
// Update the cell content
$td.text(value.toFixed(2)).addClass('numeric');
}
});
mvc.Components.get('highlight').getVisualization(function(tableView) {
// Add custom cell renderer
tableView.table.addCellRenderer(new CustomRangeRenderer());
// tableView.on('rendered', function() {
// Apply class of the cells to the parent row in order to color the whole row
// tableView.$el.find('td.range-cell').each(function() {
// $(this).addClass(this.className);
// });
//});
// Force the table to re-render
tableView.table.render();
});
});

21
apps/splunk_health_overview/appserver/static/table_data_bar.css
Unescape View File

@ -0,0 +1,21 @@
td.data-bar-cell {
padding: 4px 8px;
}
td.data-bar-cell .data-bar-wrapper .data-bar {
height: 16px;
min-width: 1px;
background-color: #5479AF;
font-weight: bold;
}
.data-bar-over { color: #FFFFFF; }
.data-bar-under { color: #000000; }
.data-bar-wrapper {
border-style: solid;
border-width: 1px;
}

30
apps/splunk_health_overview/appserver/static/table_data_bar.js
Unescape View File

@ -0,0 +1,30 @@
require([
'jquery',
'underscore',
'splunkjs/mvc',
'views/shared/results_table/renderers/BaseCellRenderer',
'splunkjs/mvc/simplexml/ready!'
], function($, _, mvc, BaseCellRenderer) {
var DataBarCellRenderer = BaseCellRenderer.extend({
canRender: function(cell) {
return (cell.field === 'UsedPct');
},
render: function($td, cell) {
var pColor="data-bar-under"
if(cell.value > 15){ pColor="data-bar-over" }
$td.addClass('data-bar-cell').html(_.template('<div class="data-bar-wrapper"><div class="data-bar <%- pColor %>" style="width:<%- percent %>%">&nbsp;<%- ppp %>%</div></div>', {
percent: Math.min(Math.max(parseFloat(cell.value), 0), 100),
ppp: parseFloat(cell.value).toFixed(2),
pColor: pColor
}));
}
});
mvc.Components.get('df').getVisualization(function(tableView) {
tableView.table.addCellRenderer(new DataBarCellRenderer());
tableView.table.render();
});
});

59
apps/splunk_health_overview/appserver/static/table_decorations_shrunk.css
Unescape View File

@ -0,0 +1,59 @@
/* Custom Icons */
td.icon {
text-align: center;
}
td.icon i {
font-size: 15px;
text-shadow: 1px 1px #aaa;
}
td.icon .severe {
color: red;
}
td.icon .elevated {
color: orangered;
}
td.icon .low {
color: #006400;
}
/* Row Coloring */
#highlight tr td {
background-color: #c1ffc3 !important;
}
#highlight tr.range-elevated td {
background-color: #ffc57a !important;
}
#highlight tr.range-severe td {
background-color: #d59392 !important;
}
#highlight .table td {
border-top: 1px solid #fff;
}
#highlight td.range-severe, td.range-elevated {
font-weight: bold;
}
.icon-inline i {
font-size: 10px;
margin-left: 5px;
}
.icon-inline i.icon-alert-circle {
color: #ef392c;
}
.icon-inline i.icon-alert {
color: #ff9c1a;
}
.icon-inline i.icon-check {
color: #5fff5e;
}

56
apps/splunk_health_overview/appserver/static/table_icons_mv.js
Unescape View File

@ -0,0 +1,56 @@
require([
'underscore',
'jquery',
'splunkjs/mvc',
'splunkjs/mvc/tableview',
'splunkjs/mvc/simplexml/ready!'
], function(_, $, mvc, TableView) {
// Translations from rangemap results to CSS class
var ICONS = {
severe: 'alert-circle',
elevated: 'alert',
low: 'check-circle'
};
var RangeMapIconRenderer = TableView.BaseCellRenderer.extend({
canRender: function(cell) {
// Only use the cell renderer for the range field
return cell.field === 'Status';
},
render: function($td, cell) {
var icon = 'question';
//debugger;
if (!(cell.value instanceof Array)) {
console.log("data is array, lets fix it");
cell.value = cell.value.split("##");
}
console.log("dataset is: ", cell.value)
for (var v in cell.value) {
if (cell.value.hasOwnProperty(v)) {
val = cell.value[v];
// Fetch the icon for the value
if (ICONS.hasOwnProperty(val)) {
icon = ICONS[val];
}
var needsBreak = (v == cell.value-1) ? "":"<br />";
// Create the icon element and add it to the table cell
$td.addClass('icon').append(_.template('<i class="icon-<%-icon%> <%- range %>" title="<%- range %>"></i>', {
icon: icon,
range: val
}) + needsBreak);
}
}
}
});
mvc.Components.get('feed_status').getVisualization(function(tableView){
// Register custom cell renderer
tableView.table.addCellRenderer(new RangeMapIconRenderer());
// Force the table to re-render
tableView.table.render();
});
});

16
apps/splunk_health_overview/default/app.conf
Unescape View File

@ -0,0 +1,16 @@
# Splunk app configuration file
[package]
id = splunk_health_overview
[ui]
is_visible = 1
label = Splunk Health Overview
[launcher]
author = Aaron Kornhauser ak@splunk.com
description = This app is a cumulative overview of the health of your Splunk environment.
version = 2.2.1
[install]
build = 1

222
apps/splunk_health_overview/default/data/models/Internal_Events.json
Unescape View File

@ -0,0 +1,222 @@
{
"modelName": "Internal_Events",
"displayName": "Internal Events",
"description": "Model to encapsulate some operational data for executive summary dashboards.",
"objectSummary": {
"Event-Based": 1,
"Transaction-Based": 0,
"Search-Based": 0
},
"objects": [
{
"objectName": "Internal_Log_Events",
"displayName": "Internal Log Events",
"parentName": "BaseEvent",
"fields": [
{
"fieldName": "b",
"owner": "Internal_Log_Events",
"type": "number",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "b",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "series",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "series",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "ev",
"owner": "Internal_Log_Events",
"type": "number",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "ev",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "group",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "group",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "pool",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "pool",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "type",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "type",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "kb",
"owner": "Internal_Log_Events",
"type": "number",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "kb",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "h",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "h",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "idx",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "idx",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "st",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "st",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "splunk_server",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "splunk_server",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "i",
"owner": "Internal_Log_Events",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "i",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "_time",
"owner": "BaseEvent",
"type": "timestamp",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "_time",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "host",
"owner": "BaseEvent",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "host",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "source",
"owner": "BaseEvent",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "source",
"comment": "",
"fieldSearch": ""
},
{
"fieldName": "sourcetype",
"owner": "BaseEvent",
"type": "string",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "sourcetype",
"comment": "",
"fieldSearch": ""
}
],
"calculations": [],
"constraints": [
{
"search": "index=_internal source=*license_usage.log* type=Usage",
"owner": "Internal_Log_Events"
}
],
"lineage": "Internal_Log_Events"
}
],
"objectNameList": [
"Internal_Log_Events"
]
}

1150
apps/splunk_health_overview/default/data/models/Introspection_Usage.json
View File

File diff suppressed because it is too large Load Diff

17
apps/splunk_health_overview/default/data/ui/nav/default.xml
Unescape View File

@ -0,0 +1,17 @@
<nav search_view="search" color="#666699">
<view name="health_status" default='true'/>
<view name="license_usage" />
<view name="resource_usage" />
<collection label="Searches" >
<view name="search_activity" />
<view name="search_detail" />
<view name="scheduler_activity" />
<view name="saved_search_detail" />
</collection>
<view name="user_activity" />
<view name="indexes" />
<view name="indexes_and_fields" />
<view name="deployment_status" />
<view name="search"/>
<view name="pivot"/>
</nav>

42
apps/splunk_health_overview/default/data/ui/views/deployment_status.xml
Unescape View File

@ -0,0 +1,42 @@
<form script="table_icons_mv.js" stylesheet="table_decorations_shrunk.css">
<label>Deployment Status</label>
<fieldset submitButton="true">
<input type="dropdown" token="index" searchWhenChanged="true">
<label>Index</label>
<choice value="*">All</choice>
<fieldForLabel>Index</fieldForLabel>
<fieldForValue>Index</fieldForValue>
<default>*</default>
<populatingSearch earliest="$earliest$" latest="$latest$" fieldForLabel="Index" fieldForValue="Index">|inputlookup avail_indexes.csv | dedup Index | sort + Index</populatingSearch>
</input>
<input type="multiselect" token="sourcetype" searchWhenChanged="true">
<label>Sourcetype</label>
<choice value="*">All</choice>
<fieldForLabel>sourcetype</fieldForLabel>
<fieldForValue>sourcetype</fieldForValue>
<default>*</default>
<valuePrefix>sourcetype=</valuePrefix>
<delimiter> OR </delimiter>
<populatingSearch earliest="-7d@h" latest="now" fieldForLabel="sourcetype" fieldForValue="sourcetype">| tstats values(sourcetype) AS st where index=$index$ by sourcetype | fields - st</populatingSearch>
</input>
<input type="text" token="host" searchWhenChanged="true">
<label>Host</label>
<default>*</default>
</input>
</fieldset>
<row>
<panel>
<table id="feed_status">
<title>Host and Sourcetype Detail - Last 7 Days</title>
<searchString>| tstats latest(_time) AS lastTime WHERE index=$index$ sourcetype!=stash $sourcetype$ host=*$host$* BY host sourcetype index | eval current=now() | eval age_min=round((current-lastTime)/60,2) | rangemap field=age_min default=severe low=0-30 elevated=30-60 | stats max(current) AS "Current Time" values(sourcetype) AS Sourcetypes list(lastTime) AS "Latest Event" list(age_min) AS age_min list(range) AS Status by host index | convert ctime(*Time) ctime("Latest Event") | eventstats avg(age_min) AS avg_age by host | eval avg_age=round(avg_age,2) | sort - avg_age | rename age_min AS "Communicated Minutes Ago" avg_age AS "Avg Communicated Minutes Ago"</searchString>
<earliestTime>-7d@h</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
</form>

243
apps/splunk_health_overview/default/data/ui/views/health_status.xml
Unescape View File

@ -0,0 +1,243 @@
<form stylesheet="server_health.css" script="server_health.js">
<label>Health Status</label>
<fieldset submitButton="false">
<input type="time" token="time" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
</fieldset>
<row>
<panel>
<html>
<div>
<p/>This dashboard is intended to show the overall status and health of your Splunk deployment.<p/>The dashboard provides server availability via REST calls, utilization for CPU and memory, as well as Splunk internal messages.
<p/>To drilldown onto a particular host in question, click on the server name and additional panels will be populated. Also, the timechart has a pan and zoom feature which allows you to specify a timerange on the chart to filter the panel below.</div>
</html>
</panel>
</row>
<row>
<panel>
<table>
<title>Historical Unreachable Splunk Server Instances</title>
<searchString>index=_internal sourcetype=splunkd component=DistributedPeerManager "Down"| rex "named\s(?&lt;Host&gt;\S+)\:" | lookup all_servers.csv host AS Host OUTPUT role |stats sparkline count AS Count values(role) AS Roles earliest(_time) AS "First Occurrence" latest(_time) AS "Last Occurrence" by Host message | convert ctime(*Occurrence) | rename message as Message sparkline AS Sparkline | table Host Roles Message Sparkline Count "First Occurrence" "Last Occurrence"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="count">10</option>
<option name="drilldown">cell</option>
</table>
</panel>
</row>
<row>
<panel>
<table id="table_status">
<title>Current Splunk Server Status</title>
<searchString>|inputlookup all_servers.csv | eval splunk_server=host | join type=left splunk_server [|rest /services/server/info] | join type=left splunk_server [| rest /services/server/status/resource-usage/hostwide | eval mb_mem_used=round(mem_used,2) | fields splunk_server cpu_system_pct cpu_user_pct mb_mem_used] | fillnull value="Non-Reporting" | eval Status=if(updated="Non-Reporting",1,0) | rangemap field=Status low=0-0 severe=1-1 |rename splunk_server AS Server | stats list(role) AS Role values(range) AS Status values(cpu_system_pct) AS "CPU System Utilized" values(cpu_user_pct) AS "CPU User Utilized" values(os_name) AS OS values(numberOfCores) AS "Cores" values(physicalMemoryMB) AS "Avail Mem (MB)" values(mb_mem_used) AS "Mem Utilized (MB)" values(guid) AS guid values(version) AS Version by Server | table Status Server Role OS Cores "Avail Mem (MB)" "CPU System Utilized" "CPU User Utilized" "Mem Utilized (MB)" Version guid | sort - Status "CPU System Utilized" "CPU User Utilized"| fillnull value="Non-Reporting" "Mem Utilized (MB)" "CPU System Utilized" "CPU User Utilized"</searchString>
<earliest>@d</earliest>
<latest>now</latest>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<drilldown>
<set token="showChart1">true</set>
<set token="host">$row.Server$</set>
</drilldown>
<option name="count">10</option>
<option name="drilldown">cell</option>
</table>
</panel>
<panel>
<table id="forwarder_status">
<title>Forwarder Status</title>
<searchString>index=_internal source="*splunkd.log" NOT [| inputlookup all_servers.csv | fields host] | eval current=now() | eval age=round((current-_time)/60,2) | stats latest(current) AS "Current Time" latest(_time) AS "Latest Event" min(age) AS Age by host | convert ctime("Latest Event") ctime("Current Time") | sort -Age | rangemap default=severe field=Age low=0-10 | rename host AS Forwarder Age AS "Communicated Minutes Ago" range AS Status | table Status Forwarder "Current Time" "Latest Event" "Communicated Minutes Ago" | sort - Status " Communicated Minutes Ago"</searchString>
<earliestTime>-4h@m</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<drilldown>
<set token="showChart1">true</set>
<set token="host">$row.Forwarder$</set>
</drilldown>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart depends="$showChart1$">
<title>Average CPU Utilization by $host$</title>
<searchString>| tstats avg(Introspection.data.cpu_system_pct) AS cpu_system_pct avg(Introspection.data.cpu_user_pct) AS cpu_user_pct FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage host=$host$ GROUPBY _time | timechart avg(cpu_system_pct) AS "% CPU System" avg(cpu_user_pct) AS "% CPU User"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">bottom</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart depends="$showChart1$">
<title>System Load Average by $host$</title>
<searchString>| tstats max(Introspection.data.normalized_load_avg_1min) AS load FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage host=$host$ GROUPBY _time | timechart max(load) AS "Load Average"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">bottom</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart depends="$showChart1$">
<title>Memory Usage by $host$</title>
<searchString>| tstats avg(Introspection.data.mem_used) AS mem_used avg(Introspection.data.swap_used) AS swap_used FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage host=$host$ GROUPBY _time | timechart avg(mem_used) AS "MB Average" avg(swap_used) AS "MB Swap"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">bottom</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">true</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.overlayFields">"MB Swap"</option>
</chart>
</panel>
</row>
<row>
<panel>
<table id="real_status">
<title>Data Feed Status</title>
<searchString>| metadata type=sourcetypes index=* |search [inputlookup avail_indexes.csv | fields sourcetype ] | sort - totalCount |eval description=description + " (" + sourcetype + ")" | eval delta=now()-recentTime | rangemap default=severe field=delta low=0-1800 | convert ctime(recentTime) AS "Last Indexed" | table range, sourcetype, "Last Indexed", totalCount | eval totalCount=tostring(totalCount, "commas") | rename totalCount AS Events, range AS Status | sort - Status</searchString>
<earliest>-30d@d</earliest>
<latest>now</latest>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">none</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>Splunk Internal Warning and Error Messages</title>
<searchString>index=_internal source="*splunkd.log" log_level!=INFO | timechart limit=15 count by component</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<selection>
<set token="selection.earliest">$start$</set>
<set token="selection.latest">$end$</set>
</selection>
</chart>
</panel>
</row>
<row>
<panel>
<input type="dropdown" token="grouping" searchWhenChanged="true">
<label>Grouping</label>
<choice value="host, component, log_level, message">Host</choice>
<choice value="component, log_level, message">Message</choice>
<default>component, log_level, message</default>
</input>
<table>
<title>Splunk Internal Message Detail</title>
<searchString>index=_internal source="*splunkd.log" log_level!=INFO | stats last(_time) AS "First Occurrence" first(_time) AS "Latest Occurrence" count by $grouping$ | rename host AS Server component AS Component log_level AS "Log Level" message AS Message | sort - count | convert ctime("Latest Occurrence") ctime("First Occurrence")</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">15</option>
</table>
</panel>
</row>
</form>

112
apps/splunk_health_overview/default/data/ui/views/indexes.xml
Unescape View File

@ -0,0 +1,112 @@
<form>
<label>Available Indexes</label>
<description></description>
<row>
<panel>
<table>
<title>Indexes Available to Search</title>
<searchString>|inputlookup avail_indexes.csv | join type=left sourcetype [|metadata type=sourcetypes index=* | convert ctime(*Time)] | eval totalCount = tostring(totalCount, "commas") | stats list(sourcetype) AS Sourcetype values(retention) AS "Retention Periond (Days)" list(lastTime) AS "Latest Event" list(totalCount) AS "Total Count" by Index</searchString>
<earliestTime>0</earliestTime>
<latestTime></latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">100</option>
<drilldown target="search">
<link field="Index">/app/search/search/?q=search index%3D$row.Index$</link>
<link field="Sourcetype">/app/search/search/?q=search index%3D$row.Index$ sourcetype%3D$click.value2$</link>
<link field="Description">/app/search/search/?q=search index%3D$row.Index$</link>
<link field="Retention Periond (Days)">/app/search/search/?q=search index%3D$row.Index$</link>
<link field="Latest Event">/app/search/search/?q=search index%3D$row.Index$</link>
<link field="Total Events">/app/search/search/?q=search index%3D$row.Index$</link>
</drilldown>
</table>
</panel>
</row>
<row>
<panel>
<input type="time" token="time">
<label></label>
<default>
<earliestTime>-30d@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<input type="dropdown" token="index_name">
<label>Index</label>
<populatingSearch fieldForLabel="Index" fieldForValue="Index">|inputlookup avail_indexes.csv | dedup Index | sort + Index</populatingSearch>
<fieldForLabel>Index</fieldForLabel>
<fieldForValue>Index</fieldForValue>
<selectFirstChoice>true</selectFirstChoice>
</input>
<chart>
<title>Event Count Distribution by Index/Sourcetype</title>
<searchString>| tstats count WHERE index=$index_name$ GROUPBY sourcetype, _time span=1d | timechart span=1d sum(count) by sourcetype</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.chart">column</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">none</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">Count</option>
</chart>
</panel>
<panel>
<input type="dropdown" token="index" searchWhenChanged="true">
<label>Index</label>
<populatingSearch earliest="$earliest$" latest="$latest$" fieldForLabel="Index" fieldForValue="Index">|inputlookup avail_indexes.csv | dedup Index | sort + Index</populatingSearch>
<choice value="*">All</choice>
<default>*</default>
</input>
<table>
<title>Host Statistics in Index: $index$</title>
<searchString>|metadata type=hosts index=$index$ | convert ctime(*Time) | eval totalCount = tostring(totalCount, "commas") | rename firstTime AS "Earliest Event" lastTime AS "Latest Event" totalCount AS "Event Count" | fields host "Earliest Event" "Latest Event" "Event Count" | sort - "Latest Event"</searchString>
<earliestTime>-30d@d</earliestTime>
<latestTime>now</latestTime>
<option name="list.drilldown">full</option>
<option name="list.wrap">1</option>
<option name="maxLines">5</option>
<option name="raw.drilldown">full</option>
<option name="rowNumbers">false</option>
<option name="table.drilldown">all</option>
<option name="table.wrap">1</option>
<option name="type">list</option>
<option name="wrap">true</option>
<option name="dataOverlayMode">none</option>
<drilldown target="search">
<link field="host">/app/search/search/?q=search index%3D$index$%20host=$row.host$</link>
<link field="Earliest Event">/app/search/search/?q=search index%3D$index$%20host=$row.host$</link>
<link field="Total Events">/app/search/search/?q=search index%3D$index$%20host=$row.host$</link>
<link field="Latest Event">/app/search/search/?q=search index%3D$index$%20host=$row.host$</link>
</drilldown>
<option name="drilldown">cell</option>
<option name="count">25</option>
</table>
</panel>
</row>
</form>

33
apps/splunk_health_overview/default/data/ui/views/indexes_and_fields.xml
Unescape View File

@ -0,0 +1,33 @@
<form>
<label>Indexes and Fields</label>
<description/>
<fieldset submitButton="false">
<input type="dropdown" token="index">
<label>Available Indexes</label>
<populatingSearch fieldForLabel="index" fieldForValue="index">| tstats values(sourcetype) AS st where index=* by index | fields - st</populatingSearch>
</input>
<input type="dropdown" token="sourcetype">
<label>Available Sourcetypes</label>
<populatingSearch fieldForLabel="sourcetype" fieldForValue="sourcetype">| tstats values(sourcetype) AS st where index=$index$ by sourcetype | fields - st</populatingSearch>
</input>
</fieldset>
<row>
<panel>
<table>
<title>Available Fields and Values for Index: $index$ and Sourcetype: $sourcetype$</title>
<searchString>index=$index$ sourcetype=$sourcetype$ | fieldsummary maxvals=1 | rex field=values "value\"\:\"(?&lt;Sample_Value&gt;.+)\"\," | eval index="$index$" | eval sourcetype="$sourcetype$" | where Sample_Value!="" |fields field Sample_Value index sourcetype | stats list(field) AS "Field Name" list(Sample_Value) AS "Sample Value" by index sourcetype</searchString>
<earliestTime>-24h</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<drilldown target="search">
<link field="index">/app/search/search/?q=search index%3D$row.index$ earliest=-15m</link>
<link field="sourcetype">/app/search/search/?q=search index%3D$row.index$ sourcetype%3D$sourcetype$ earliest=-15m</link>
<link field="Field Name">/app/search/search/?q=search index%3D$row.index$ sourcetype%3D$sourcetype$ earliest=-15m</link>
<link field="Sample Value">/app/search/search/?q=search index%3D$row.index$ sourcetype%3D$sourcetype$ earliest=-15m</link>
</drilldown>
</table>
</panel>
</row>
</form>

278
apps/splunk_health_overview/default/data/ui/views/license_usage.xml
Unescape View File

@ -0,0 +1,278 @@
<form script="table_cell_highlighting.js,table_data_bar.js" stylesheet="table_cell_highlighting.css,table_data_bar.css">
<label>License Overview</label>
<description/>
<fieldset autoRun="true" submitButton="false">
<input type="dropdown" token="type" searchWhenChanged="true">
<label>Split-by</label>
<default>Internal_Log_Events.idx</default>
<choice value="Internal_Log_Events.idx">Index</choice>
<choice value="Internal_Log_Events.st">Sourcetype</choice>
<choice value="Internal_Log_Events.h">Host</choice>
</input>
</fieldset>
<row>
<panel>
<html>
<div id="usage_help">
<p/>This dashboard is intended to show the overall status of license utilization across the different Splunk servers and data types.<p/>The dashboard provides a high level overview of overall license used today, by index, and trending over time by index/environment option. To gain deeper insights into Splunk's license and data volume consumption reference either<b>FireBrigade</b>or<b>S.o.S Splunk on Splunk</b>.</div>
</html>
</panel>
</row>
<row>
<panel>
<chart>
<title>Todays Overall License Usage (GB)</title>
<searchString>| tstats summariesonly=false sum(Internal_Log_Events.b) AS bytes from datamodel="Internal_Events" GROUPBY Internal_Log_Events.idx | eval gb=round(bytes/1024/1024/1024,2) | stats sum(gb) AS "Total GB"</searchString>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">fillerGauge</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.chart.rangeValues">["0","2","6","10"]</option>
<option name="charting.gaugeColors">[0x84E900,0xFFE800,0xBF3030]</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<table>
<title>Total GB Ingested Today</title>
<searchString>| tstats summariesonly=false sum(Internal_Log_Events.b) AS bytes from datamodel="Internal_Events" WHERE [inputlookup all_servers.csv | search role=indexer | rename guid AS "Internal_Log_Events.i"| fields Internal_Log_Events.i] GROUPBY $type$ | eval gb=round(bytes/1024/1024/1024,2) | stats sum(gb) AS "Total GB" by $type$ | rename Internal_Log_Events.idx AS Index Internal_Log_Events.st AS "Source Type" Internal_Log_Events.h AS Host | sort - "Total GB"</searchString>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
<option name="list.drilldown">full</option>
<option name="list.wrap">1</option>
<option name="maxLines">5</option>
<option name="raw.drilldown">full</option>
<option name="rowNumbers">false</option>
<option name="table.drilldown">1</option>
<option name="table.wrap">1</option>
<option name="type">list</option>
<option name="wrap">true</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
<panel>
<chart>
<title>Week on Week Usage - Current Day is Partial</title>
<searchString>| tstats summariesonly=true sum(Internal_Log_Events.b) AS bytes from datamodel="Internal_Events" WHERE [inputlookup all_servers.csv | search role=indexer | rename guid AS "Internal_Log_Events.i"| fields Internal_Log_Events.i] GROUPBY _time | eval marker=case( _time &lt; relative_time(now(), "-1w@w"), "Two Weeks Ago", _time &lt; relative_time(now(), "@w"), "Last Week", true(), "This Week") | eval _time=case( _time &lt; relative_time(now(), "-1w@w"), _time+2*7*86400, _time &lt; relative_time(now(), "@w"), _time+7*86400, true(), _time) | eval gb=bytes / 1024 / 1024 / 1024 | chart sum(gb) AS gbytes OVER _time BY marker | fieldformat _time=strftime(_time, "%A")</searchString>
<earliestTime>-2w@w</earliestTime>
<latestTime>now</latestTime>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">bottom</option>
<option name="charting.axisTitleY.text">GB Indexed</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
</row>
<row>
<panel>
<input type="time" searchWhenChanged="true">
<label/>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<chart>
<title>License Usage (GB)</title>
<searchString>| tstats summariesonly=false sum(Internal_Log_Events.b) AS bytes from datamodel="Internal_Events" WHERE [inputlookup all_servers.csv | search role=indexer | rename guid AS "Internal_Log_Events.i"| fields Internal_Log_Events.i] GROUPBY _time $type$ span=auto | eval gb=bytes/1024/1024/1024 | timechart sum(gb) by $type$</searchString>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart>
<title>Todays Overall Ingestion by Index/Indexer (GB)</title>
<searchString>|tstats summariesonly=false sum(Internal_Log_Events.b) AS bytes from datamodel="Internal_Events" WHERE [inputlookup all_servers.csv | search role=indexer | rename guid AS "Internal_Log_Events.i"| fields Internal_Log_Events.i] GROUPBY _time $type$ Internal_Log_Events.i span=auto | eval gb=round(bytes/1024/1024/1024,2) | lookup all_servers.csv guid AS "Internal_Log_Events.i" OUTPUTNEW host | chart sum(gb) over host by $type$</searchString>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">-90</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">GB</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">log</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.axisTitleY2.text">Splunk Indexing Limit</option>
<option name="charting.axisY2.maximumNumber">250</option>
</chart>
</panel>
</row>
<row>
<panel>
<input type="time" token="field1" searchWhenChanged="true">
<label/>
<default>
<earliestTime>-7d@h</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<chart>
<title>Data Distribution by Indexer</title>
<searchString>| tstats prestats=t count WHERE index=* [|inputlookup all_servers.csv | search role=*indexer* | rename host AS splunk_server | fields splunk_server ] BY splunk_server, _time span=1d | timechart limit=100 span=1d count by splunk_server</searchString>
<earliestTime>$field1.earliest$</earliestTime>
<latestTime>$field1.latest$</latestTime>
<earliest>$field1.earliest$</earliest>
<latest>$field1.latest$</latest>
<earliest>$field1.earliest$</earliest>
<latest>$field1.latest$</latest>
<option name="charting.chart">column</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">Events</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
<panel>
<chart>
<title>Event Count Ingested by Indexing Delay(s) - Today</title>
<searchString>| tstats count max(_time) as t max(_indextime) as it where index=* by _time | eval delay=it-t | timechart sum(count) avg(delay) as delay(s)</searchString>
<earliest>@d</earliest>
<latest>now</latest>
<option name="charting.chart">column</option>
<option name="charting.axisY2.enabled">true</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.overlayFields">delay(s)</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">Events</option>
</chart>
</panel>
</row>
<row>
<panel>
<table id="highlight">
<title>Matrix Overview - Index Capacity/Usage</title>
<searchString>| rest /services/data/indexes search="totalEventCount!=0" | eval coldPath.maxDataSizeMB=if('coldPath.maxDataSizeMB' = 0, null(), 'coldPath.maxDataSizeMB') | eval homePath.maxDataSizeMB=if('homePath.maxDataSizeMB' = 0, null(), 'homePath.maxDataSizeMB') | eval roof=min((coalesce('homePath.maxDataSizeMB', 4294967295) + coalesce('coldPath.maxDataSizeMB', 4294967295)), maxTotalDataSizeMB) | eval span=tostring(currentDBSizeMB) + " / " + tostring(roof) + " MB" | eval Percent=tostring(round(currentDBSizeMB * 100 / roof)) + "%"| search [inputlookup all_servers.csv | search role=indexer | rename host AS splunk_server | fields splunk_server] | stats first(span) AS "Capacity vs Limit" by splunk_server title minTime maxTime Percent | rename splunk_server AS Indexer title AS Index minTime AS "Oldest Event" maxTime AS "Newest Event" | table Indexer Index "Capacity vs Limit" "Oldest Event" "Newest Event" Percent | sort - Percent</searchString>
<earliestTime>0</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="drilldown">none</option>
<option name="count">100</option>
<option name="dataOverlayMode">none</option>
</table>
</panel>
<panel>
<table id="df">
<title>Disk Usage</title>
<searchString>| tstats max(Introspection.Disk_Objects.Capacity) AS Capacity max(Introspection.Disk_Objects.Avail) AS Avail max(Introspection.Disk_Objects.Used) AS Used max(Introspection.Disk_Objects.UsedPct) AS UsedPct FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Disk_Objects [inputlookup all_servers.csv | fields host] GROUPBY _time host Introspection.data.mount_point | eventstats max(_time) AS latest by host | where _time=latest | lookup all_servers.csv host OUTPUT role | rename Introspection.data.mount_point AS Mount role AS Role |table host Role Mount Capacity Avail Used UsedPct | sort - UsedPct</searchString>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">none</option>
<option name="count">100</option>
</table>
</panel>
</row>
</form>

210
apps/splunk_health_overview/default/data/ui/views/resource_usage.xml
Unescape View File

@ -0,0 +1,210 @@
<form>
<label>Resource Usage</label>
<fieldset submitButton="false">
<input type="time" token="time2" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<input type="multiselect" token="hosts" searchWhenChanged="true">
<label>Host</label>
<choice value="*">All</choice>
<fieldForLabel>host</fieldForLabel>
<fieldForValue>host</fieldForValue>
<default>*</default>
<valuePrefix>host=</valuePrefix>
<delimiter> OR </delimiter>
<populatingSearch fieldForLabel="host" fieldForValue="host">|inputlookup all_servers.csv | dedup host | fields host | sort + host</populatingSearch>
</input>
</fieldset>
<row>
<panel>
<table>
<title>Current Resource Usage</title>
<searchString>|inputlookup all_servers.csv | eval splunk_server=host | join type=left splunk_server [|rest /services/server/info] | join type=left splunk_server [| rest /services/server/status/resource-usage/hostwide | eval pct_cpu_used=(100-cpu_idle_pct)."%" | eval mb_mem_used=round(mem_used,2) | fields splunk_server cpu_system_pct cpu_user_pct mb_mem_used] | fillnull value="Non-Reporting" |rename splunk_server AS Server | stats list(role) AS Role values(cpu_system_pct) AS "CPU System Utilized" values(cpu_user_pct) AS "CPU User Utilized" values(os_name) AS OS values(numberOfCores) AS "Cores" values(physicalMemoryMB) AS "Avail Mem (MB)" values(mb_mem_used) AS "Mem Utilized (MB)" by Server | table Server Role OS Cores "Avail Mem (MB)" "CPU System Utilized" "CPU User Utilized" "Mem Utilized (MB)" | sort - "CPU System Utilized" "CPU User Utilized" | fillnull value="Non-Reporting" "Mem Utilized (MB)" "CPU System Utilized" "CPU User Utilized"</searchString>
<earliest>@d</earliest>
<latest>now</latest>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="drilldown">cell</option>
<option name="dataOverlayMode">none</option>
<option name="count">10</option>
</table>
</panel>
<panel>
<table>
<title>Current Disk Usage by Host/Mount</title>
<searchString>| tstats max(Introspection.Disk_Objects.Capacity) AS Capacity max(Introspection.Disk_Objects.Avail) AS Avail max(Introspection.Disk_Objects.Used) AS Used max(Introspection.Disk_Objects.UsedPct) AS UsedPct FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Disk_Objects $hosts$ GROUPBY _time host Introspection.data.mount_point | eventstats max(_time) AS latest by host | where _time=latest | lookup all_servers.csv host OUTPUT role | rename Introspection.data.mount_point AS Mount role AS Role |table host Role Mount Capacity Avail Used UsedPct | sort - UsedPct</searchString>
<earliest>@d</earliest>
<latest>now</latest>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="count">10</option>
<drilldown>
<set token="host">$row.host$</set>
</drilldown>
<option name="drilldown">cell</option>
<option name="dataOverlayMode">none</option>
</table>
</panel>
</row>
<row>
<panel>
<table>
<title>Today's Resource Usage by PID/Host</title>
<searchString>index=_introspection component=PerProcess $hosts$| stats earliest(_time) AS "Earliest Occurrence" latest(_time) AS "Latest Occurrence" max(data.fd_used) AS "File Descriptors Used" avg(data.mem_used) AS "Mem Used" avg(data.pct_memory) AS "Avg % Mem" avg(data.pct_cpu) AS "Avg % CPU" by data.pid data.ppid data.process host | eval "Runtime Min"=('Latest Occurrence'-'Earliest Occurrence')/60 | convert ctime(*Occurrence) | rename data.pid AS PID data.ppid AS "Parent PID" data.process AS Process | table "Earliest Occurrence" "Latest Occurrence" host PID "Parent PID" Process "File Descriptors Used" "Mem Used" "Avg % Mem" "Avg % CPU" "Runtime Min" | sort - "Avg % CPU" "Avg % Mem"</searchString>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="drilldown">cell</option>
<option name="dataOverlayMode">none</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>System Load Average by Host</title>
<searchString>| tstats max(Introspection.data.normalized_load_avg_1min) AS load FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage $hosts$ GROUPBY _time host | timechart max(load) AS "Load Average" by host</searchString>
<earliestTime>$time2.earliest$</earliestTime>
<latestTime>$time2.latest$</latestTime>
<option name="charting.chart">line</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">Load Average</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
<panel>
<chart>
<title>% CPU by Host</title>
<searchString>| tstats avg(Introspection.data.cpu_user_pct) AS pct_cpu_user FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage $hosts$ GROUPBY _time host | timechart avg(pct_cpu_user) AS "% CPU User Used" by host</searchString>
<earliestTime>$time2.earliest$</earliestTime>
<latestTime>$time2.latest$</latestTime>
<option name="charting.chart">line</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">% CPU Used</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
<panel>
<chart>
<title>Memory Usage MB by Host</title>
<searchString>| tstats avg(Introspection.data.mem_used) AS mem_used avg(Introspection.data.swap_used) AS swap_used FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Hostwide_Resource_Usage $hosts$ GROUPBY _time host | timechart avg(mem_used) AS "MB Average" by host</searchString>
<earliestTime>$time2.earliest$</earliestTime>
<latestTime>$time2.latest$</latestTime>
<option name="charting.chart">line</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">MB Used</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
</row>
<row>
<panel>
<input type="time" token="time" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>-30d@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<input type="dropdown" token="host" searchWhenChanged="true">
<label>Host</label>
<selectFirstChoice>true</selectFirstChoice>
<populatingSearch fieldForLabel="host" fieldForValue="host">|inputlookup all_servers.csv | dedup host | fields host | sort + host</populatingSearch>
<fieldForLabel>host</fieldForLabel>
<fieldForValue>host</fieldForValue>
</input>
<chart>
<title>Disk Usage % by Host/Mount Over Time - $host$</title>
<searchString>| tstats max(Introspection.Disk_Objects.UsedPct) AS UsedPct FROM datamodel=Introspection_Usage WHERE nodename=Introspection.Disk_Objects host=$host$ GROUPBY _time host Introspection.data.mount_point | rename Introspection.data.mount_point AS Mount | timechart max(UsedPct) As UsedPct by Mount</searchString>
<earliest>$time.earliest$</earliest>
<latest>$time.latest$</latest>
<option name="charting.chart">line</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">% Usage</option>
</chart>
</panel>
</row>
</form>

168
apps/splunk_health_overview/default/data/ui/views/saved_search_detail.xml
Unescape View File

@ -0,0 +1,168 @@
<form>
<label>Saved Search Detail</label>
<description/>
<fieldset submitButton="false">
<input type="text" token="search" searchWhenChanged="true">
<label>Search Name</label>
<default>*</default>
</input>
<input type="dropdown" token="app" searchWhenChanged="true">
<label>Application Context</label>
<choice value="*">Any</choice>
<populatingSearch fieldForLabel="label" fieldForValue="title">| rest /servicesNS/-/-/apps/local | search visible=1 | dedup title | fields title label | sort label</populatingSearch>
<default>*</default>
</input>
<input type="dropdown" token="user" searchWhenChanged="true">
<label>User</label>
<choice value="*">Any</choice>
<choice value="nobody">nobody</choice>
<populatingSearch earliest="$earliest$" latest="$latest$" fieldForLabel="name" fieldForValue="title">|rest /services/authentication/users splunk_server=local
|fields title realname | eval name=if(realname="",title,realname) | sort + name</populatingSearch>
<default>*</default>
</input>
<input type="dropdown" token="scheduled" searchWhenChanged="true">
<label>Scheduled</label>
<choice value="*">All</choice>
<choice value="Yes">Yes</choice>
<choice value="No">No</choice>
<default>*</default>
</input>
<input type="dropdown" token="state">
<label>State</label>
<choice value="*">All</choice>
<choice value="Enabled">Enabled</choice>
<choice value="Disabled">Disabled</choice>
<default>*</default>
</input>
</fieldset>
<row>
<panel>
<single>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time | stats count</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="drilldown">none</option>
<option name="beforeLabel">Total Saved Searches:</option>
<option name="linkView">search</option>
</single>
</panel>
<panel>
<single>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time | search state=Enabled is_scheduled=Yes| stats count</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="drilldown">none</option>
<option name="beforeLabel">Enabled and Scheduled:</option>
<option name="linkView">search</option>
</single>
</panel>
<panel>
<single>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time | search state=Enabled is_scheduled=No | stats count</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="drilldown">none</option>
<option name="beforeLabel">Not Scheduled but Enabled:</option>
<option name="linkView">search</option>
</single>
</panel>
<panel>
<single>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time | search state=Disabled | stats count</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="drilldown">none</option>
<option name="beforeLabel">Disabled Searches:</option>
<option name="linkView">search</option>
</single>
</panel>
</row>
<row>
<panel>
<chart>
<title>Search Status by Application</title>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time eai:acl.app | rename eai:acl.app as app | eval status="State: ".state." / Scheduled: ".is_scheduled | chart count over app by status</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">bar</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisEnd</option>
<option name="charting.legend.placement">bottom</option>
<option name="charting.axisTitleX.text">App Context</option>
</chart>
</panel>
<panel>
<chart>
<title>Saved Searches by User</title>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | eval state=if(disabled=0,"Enabled","Disabled") | eval is_scheduled=if(is_scheduled=1,"Yes","No") | rename title AS savedsearch_name | fields cron_schedule state is_scheduled savedsearch_name search dispatch.earliest_time dispatch.latest_time eai:acl.app eai:acl.owner | rename eai:acl.owner AS title | join title type=outer [|rest /services/authentication/users splunk_server=local |fields title realname | eval user=if(realname="",title,realname)] | eval user=case(title="nobody","No Owner Specified - this is OK",isnotnull(title) AND isnull(user),"Investigate - User Possibly No Longer Exists",1=1, user) | chart count by user</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">pie</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
</row>
<row>
<panel>
<table>
<title>Saved Search Execution Details</title>
<searchString>| inputlookup savedsearch_runtimes.csv | sort last_time | convert ctime(*_time) | rename first_time AS "First Run Time" last_time AS "Latest Run Time" | join savedsearch_name type=outer [| rest /servicesNS/-/-/saved/searches | eval State=if(disabled=0,"Enabled","Disabled") | eval Scheduled=if(is_scheduled=1,"Yes","No") | fields title eai:acl.app eai:acl.owner cron_schedule dispatch.earliest_time dispatch.latest_time search State Scheduled | rename title AS savedsearch_name eai:acl.app AS App eai:acl.owner AS User cron_schedule AS "Cron Schedule" dispatch.earliest_time AS "Dispatch Earliest Time" dispatch.latest_time AS "Dispatch Latest Time"] | rename savedsearch_name AS Search | table Search App User "Avg Runtime" "Max Runtime" Scheduled State "First Run Time" "Latest Run Time" "Dispatch Earliest Time" "Dispatch Latest Time" | search Search=$search$* App=$app$ User=$user$ Scheduled=$scheduled$ State=$state$</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">20</option>
</table>
</panel>
</row>
<row>
<panel>
<table>
<title>Inactive Searches To Be Investigated</title>
<searchString>| rest /servicesNS/-/-/saved/searches splunk_server=local | search NOT [|inputlookup savedsearch_runtimes.csv | rename savedsearch_name AS title | fields title] | eval
State=if(disabled=0,"Enabled","Disabled") | eval
Scheduled=if(is_scheduled=1,"Yes","No") | fields title eai:acl.app eai:acl.owner cron_schedule dispatch.earliest_time dispatch.latest_time search State Scheduled | rename title AS
savedsearch_name eai:acl.app AS App eai:acl.owner AS User cron_schedule AS "Cron Schedule" dispatch.earliest_time AS "Dispatch Earliest Time" dispatch.latest_time AS "Dispatch Latest Time"</searchString>
<earliestTime>0</earliestTime>
<latestTime/>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
</form>

189
apps/splunk_health_overview/default/data/ui/views/scheduler_activity.xml
Unescape View File

@ -0,0 +1,189 @@
<form>
<label>Scheduler Activity</label>
<description></description>
<fieldset submitButton="false">
<input type="dropdown" token="host" searchWhenChanged="true">
<label>Select a Search Head</label>
<selectFirstChoice>true</selectFirstChoice>
<populatingSearch fieldForLabel="host" fieldForValue="host">|inputlookup all_servers.csv | search role=*search* | dedup host | fields host | sort + host</populatingSearch>
<prefix>host=</prefix>
<fieldForLabel>host</fieldForLabel>
<fieldForValue>host</fieldForValue>
</input>
<input type="time" token="time" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
</fieldset>
<row>
<panel>
<html>
<div id="usage_help">
<p/>This dashboard is intended to illustrate scheduled search activity and usage across your different Splunk Search Head(s).
<p/>Use the input to choose the Search Head you would like to include results from and the timerange picker to select the dates.
</div>
</html>
</panel>
</row>
<row>
<panel>
<table>
<title>Scheduler Activity</title>
<searchString>index=_internal $host$ source=*scheduler.log app="*" savedsearch_name="*" | stats min(run_time) as "Min runtime (seconds)", median(run_time) as median_runtime, max(run_time) as max_runtime, count(eval(status!="continued")) AS total_exec, count(eval(status=="success")) as "Successful executions", count(eval(status=="skipped")) AS "Skipped executions" by app, savedsearch_name, user host | stats first(*) as * by savedsearch_name | eval interval_usage_ratio=round((median_runtime/schedule_period),2) | search total_exec&gt;0 | rename app AS App median_runtime AS "Median runtime (seconds)" max_runtime AS "Max runtime (seconds)" savedsearch_name AS "Scheduled search name" user AS User total_exec AS "Total executions" | table host App "Scheduled search name" User "Min runtime (seconds)" "Median runtime (seconds)" "Max runtime (seconds)" "Total executions" "Successful executions" "Skipped executions" | sort - "Median runtime (seconds)" "Total executions"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>Scheduler Execution</title>
<searchString>index=_internal $host$ source=*metrics.log group=searchscheduler | timechart partial=false sum(dispatched) AS Started, sum(skipped) AS Skipped | appendcols [search index=_internal $host$ sourcetype=scheduler status=continued | eval savedsearch_id_scheduled_time=savedsearch_id."-".scheduled_time | timechart dc(savedsearch_id_scheduled_time) AS Deferred]</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<table>
<title>Skipped Scheduled Searches</title>
<searchString>index=_internal $host$ source=*scheduler.log savedsplunker status=skipped | stats count by host savedsearch_name, app, user | sort - SkipCount | rename savedsearch_name AS "Scheduled search name" count AS "Skip count" host AS Server</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>Execution Count by Saved Search</title>
<searchString>index=_internal $host$ source=*scheduler.log status="*" AND status!="continued" | timechart count by savedsearch_name</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
</chart>
</panel>
<panel>
<table>
<title>Most Active Searches - Top 10</title>
<searchString>| rest splunk_server=* /servicesNS/-/-/admin/savedsearch/ earliest_time=-0s@s latest_time=+2d@d search="is_scheduled=1" search="disabled=0" | table splunk_server eai:acl.app eai:acl.owner cron_schedule title scheduled_times | mvexpand scheduled_times | rename scheduled_times as _time eai:acl.app as app eai:acl.owner as user title as search | stats count(search) as count by search cron_schedule app | sort 10 -count</searchString>
<earliestTime>0</earliestTime>
<latestTime></latestTime>
<option name="wrap">undefined</option>
<option name="rowNumbers">undefined</option>
<option name="drilldown">row</option>
</table>
</panel>
</row>
<row>
<panel>
<event>
<title>Scheduler Errors</title>
<searchString>index=_internal $host$ source=*scheduler.log error OR warn</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">0</option>
<option name="dataOverlayMode">none</option>
<option name="list.drilldown">full</option>
<option name="list.wrap">1</option>
<option name="maxLines">5</option>
<option name="raw.drilldown">full</option>
<option name="table.drilldown">all</option>
<option name="table.wrap">1</option>
<option name="type">list</option>
<fields>["host","source","sourcetype"]</fields>
</event>
</panel>
</row>
</form>

308
apps/splunk_health_overview/default/data/ui/views/search_activity.xml
Unescape View File

@ -0,0 +1,308 @@
<form stylesheet="search_health.css" script="search_health.js">
<label>Search Activity</label>
<description></description>
<fieldset submitButton="false" autoRun="false">
<input type="dropdown" token="host" searchWhenChanged="true">
<label>Select a Search Head</label>
<selectFirstChoice>true</selectFirstChoice>
<populatingSearch fieldForLabel="host" fieldForValue="host">|inputlookup all_servers.csv | search role=*search* | dedup host| sort + host</populatingSearch>
<prefix>host=</prefix>
<fieldForLabel>host</fieldForLabel>
<fieldForValue>host</fieldForValue>
</input>
<input type="time" token="time" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
</fieldset>
<row>
<panel>
<html>
<div id="usage_help">
<p/>This dashboard is intended to illustrate search activity and usage on the Splunk Search Head(s) in your environment.
<p/>Additionally, you can select a range on the CPU utilization graph to narrow down a time range to filter the other panels in the dashboard to identify correlations.
<p/>The Search Window Statistics table below illustrates different searches ran for a selected period of time. I.e a user selected a search to run for the last 15 Minutes or 30 Days and the panel displays how many searches were executed or cumulative/average execution time during that time frame by index or user.
</div>
</html>
</panel>
</row>
<row>
<panel>
<chart>
<title>Peak Historical Search Concurrency by Host - Adhoc and Scheduled</title>
<searchString>index=_internal $host$ source=*metrics.log group=search_concurrency "system total" NOT user=* | timechart max(active_hist_searches) by host</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">Count</option>
<selection>
<set token="selection.earliest">$start$</set>
<set token="selection.latest">$end$</set>
</selection>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart>
<title>Peak Real-Time Search Concurrency by Host - Adhoc and Scheduled</title>
<searchString>index=_internal $host$ source=*metrics.log group=search_concurrency "system total" NOT user=* | timechart max(active_realtime_searches) by host</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">Count</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<selection>
<set token="selection.earliest">$start$</set>
<set token="selection.latest">$end$</set>
</selection>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
</row>
<row>
<panel>
<chart>
<title>Avg % CPU Utilization by Process</title>
<searchString>| tstats max(Introspection.data.pct_cpu) AS pct_cpu FROM datamodel=Introspection_Usage WHERE nodename=Introspection.PerProcess_Resource_Usage $host$ GROUPBY _time Introspection.data.process | rename Introspection.data.process AS process | timechart avg(pct_cpu) by process</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">none</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<selection>
<set token="selection.earliest">$start$</set>
<set token="selection.latest">$end$</set>
</selection>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">% CPU</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart>
<title>Avg % Memory Utilization by Process</title>
<searchString>| tstats max(Introspection.data.pct_memory) AS pct_memory FROM datamodel=Introspection_Usage WHERE nodename=Introspection.PerProcess_Resource_Usage $host$ GROUPBY _time Introspection.data.process | rename Introspection.data.process AS process | timechart avg(pct_memory) by process</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">MB</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<selection>
<set token="selection.earliest">$start$</set>
<set token="selection.latest">$end$</set>
</selection>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
</row>
<row>
<panel>
<chart>
<title>Search Duration Histogram by Search Head</title>
<searchString>index=_audit $host$ action=search user!=splunk-system-user search_id=* (info=granted OR info=completed) | eval run_time_min=round(total_run_time/60,2) | stats earliest(_time) AS "Start Time" latest(_time) AS "End Time" count values(search) AS Search values(user) AS User max(run_time_min) AS run_time_min by search_id host | convert ctime(*Time) | where count&gt;1 | fields - count | rename search_id AS SID | bucket run_time_min span=5 | chart count over run_time_min by host | rename run_time_min AS "Run Time (Minutes)"</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">bar</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleY.text">Search Execution Count</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
</chart>
</panel>
<panel>
<chart>
<title>Search Count and Average Run Time</title>
<searchString>index=_audit $host$ action=search user!=splunk-system-user search_id=* info=completed | eval run_time_min=round(total_run_time/60,2) | timechart span=1h count AS "Search Count" avg(run_time_min) AS "Run Time Min"</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="charting.chart">column</option>
<option name="charting.axisY2.enabled">true</option>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart.bubbleMaximumSize">50</option>
<option name="charting.chart.bubbleMinimumSize">10</option>
<option name="charting.chart.bubbleSizeBy">area</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">default</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.chart.overlayFields">"Run Time Min"</option>
<option name="charting.axisTitleX.text">Time</option>
</chart>
</panel>
</row>
<row>
<panel>
<table id="table_status">
<title>Searches and Run Time</title>
<searchString>index=_audit $host$ action=search user!=splunk-system-user search_id=* (info=granted OR info=completed) |rex field=apiStartTime "'(?&lt;start_time&gt;[^']+)'" |rex field=apiEndTime "'(?&lt;end_time&gt;[^']+)'" | eval search_id = trim(if(isnull(search_id), id, search_id), "'") | eval run_time_min=round(total_run_time/60,2) |eval range=if(start_time=="ZERO_TIME","All Time", tostring(strptime(end_time, "%a %b %d %H:%M:%S %Y") - strptime(start_time, "%a %b %d %H:%M:%S %Y"),"duration")) | stats earliest(_time) AS "Start Time" latest(_time) AS "End Time" values(start_time) AS "Search Earliest" values(end_time) AS "Search Latest" count values(range) AS range values(search) AS Search values(user) AS User max(run_time_min) AS "Run Time (Min)" by search_id | convert ctime(*Time) | where count&gt;1 | rename search_id AS SID range AS "Search Range" | table "Start Time" "End Time" SID Search User "Run Time (Min)" "Search Range" "Search Earliest" "Search Latest"| sort - "Run Time (Min)"</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<drilldown>
<link>
<![CDATA[
/app/splunk_health_overview/search_detail?form.sid=$row.SID$]]>
</link>
</drilldown>
<option name="count">10</option>
<option name="drilldown">cell</option>
</table>
</panel>
</row>
<row>
<panel>
<input type="dropdown" token="split" searchWhenChanged="true">
<label>Split By:</label>
<choice value="Index">Index</choice>
<choice value="user">User</choice>
<default>Index</default>
</input>
<input type="radio" token="value" searchWhenChanged="true">
<label>Values:</label>
<choice value="sum(run_time_min)">Sum of Duration (Minutes)</choice>
<choice value="avg(run_time_min)">Average Duration (Minutes)</choice>
<choice value="count">Count of Searches</choice>
<default>count</default>
</input>
<table>
<title>Search Window Statistics by $split$</title>
<searchString>index=_audit $host$ action=search search=* |rex field=apiStartTime "'(?&lt;start_time&gt;[^']+)'" |rex field=_raw "[^\_]index=\"?(?&lt;Index&gt;[\_a-zA-Z\-\:]{2,})\"?" |eval Index=lower(Index) |search Index!="|history" Index!="|metadata" Index!="|dbinspect" |eval ctime = _time | join search_id [search index=_audit action=search info=completed total_run_time=* | eval run_time_min=total_run_time/60 | fields run_time_min search_id user] |eval range=case( start_time=="ZERO_TIME","All Time", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 300, "5 Minutes", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 900 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 300, "15 Minutes", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 3600 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 900, "1 Hour", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 14400 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 3600, "4 Hours", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 86400 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 14400, "1 Day", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 610000 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 86400, "7 Days", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 2592000 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 604800, "30 Days", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &lt;= 3888000 AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 2592000, "45 Days", start_time!="ZERO_TIME" AND ctime - strptime(start_time, "%a %b %d %H:%M:%S %Y") &gt; 3888000, "45 Days +", start_time==start_time, "Other") | eval Index=if(Index="","No Index Used in Search",Index) |eval user=if(user="","N/A",user) |chart limit=0 $value$ over $split$ by range | fields $split$ "5 Minutes" "15 Minutes" "1 Hour" "4 Hours" "1 Day" "7 Days" "30 Days" "45 Days" "45 Days +" "All Time" | fillnull $split$ "5 Minutes" "15 Minutes" "1 Hour" "4 Hours" "1 Day" "7 Days" "30 Days" "45 Days" "45 Days +" "All Time" | table $split$ "5 Minutes" "15 Minutes" "1 Hour" "4 Hours" "1 Day" "7 Days" "30 Days" "45 Days" "45 Days +" "All Time" | addtotals col=true | fillnull value=Total</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="list.drilldown">full</option>
<option name="list.wrap">1</option>
<option name="maxLines">5</option>
<option name="raw.drilldown">full</option>
<option name="rowNumbers">false</option>
<option name="table.drilldown">all</option>
<option name="table.wrap">1</option>
<option name="type">list</option>
<option name="wrap">true</option>
<option name="dataOverlayMode">heatmap</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
<panel>
<table>
<title>Usage by User</title>
<searchString>index=_audit $host$ action=search (id=* OR search_id=*) | eval search_id = if(isnull(search_id), id, search_id) | replace '*' with * in search_id | rex "search='(?&lt;search&gt;.*?)', autojoin" | search search_id!=scheduler_* | convert num(total_run_time) | rex "\,\s+user=(?&lt;user&gt;\S+)," | eval user = if(user="n/a", null(), user) | stats min(_time) as _time first(user) as user max(total_run_time) as total_run_time first(search) as search by search_id host | search search=search* search!=*_internal* search!=*_audit* | stats median(total_run_time) as "Median search time" perc95(total_run_time) as "95th Percentile search time" sum(total_run_time) as "Total search time" count as "Search count" max(_time) as "Last use" by user host | fieldformat "Last use" = strftime('Last use', "%F %T.%Q %:z") | rename host AS "Search Head" | sort - "Median search time"</searchString>
<earliestTime>$selection.earliest$</earliestTime>
<latestTime>$selection.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="drilldown">cell</option>
<option name="dataOverlayMode">none</option>
<option name="count">10</option>
</table>
</panel>
</row>
</form>

147
apps/splunk_health_overview/default/data/ui/views/search_detail.xml
Unescape View File

@ -0,0 +1,147 @@
<form>
<label>Search Job Detail</label>
<fieldset>
<input type="text" token="sid"/>
</fieldset>
<searchTemplate>| rest /services/search/jobs/ | search sid=$sid$</searchTemplate>
<row>
<single>
<title>Total Jobs Found</title>
<option name="link.visible">false</option>
<searchPostProcess>stats count</searchPostProcess>
</single>
<single>
<title>Adhoc Jobs Found</title>
<option name="link.visible">false</option>
<searchPostProcess>stats count(eval(if(isSavedSearch=0, title, null()))) AS count</searchPostProcess>
</single>
<single>
<title>Scheduled Jobs Found</title>
<option name="link.visible">false</option>
<searchPostProcess>stats count(eval(if(isSavedSearch=1,isSavedSearch,null()))) AS count</searchPostProcess>
</single>
</row>
<row grouping="3,2,2">
<single>
<title>Time Spent</title>
<option name="link.visible">false</option>
<option name="field">startup_time</option>
<option name="underLabel">Startup</option>
<searchPostProcess>eval startup_time=coalesce('performance.dispatch.createProviderQueue.duration_secs', "None (no remotes)")</searchPostProcess>
</single>
<single>
<option name="link.visible">false</option>
<option name="field">local_wait</option>
<option name="underLabel">Waiting for Indexers</option>
<searchPostProcess>eval local_wait=coalesce('performance.dispatch.fetch.duration_secs', 0)</searchPostProcess>
</single>
<single>
<!-- <title>Total Time</title> -->
<option name="link.visible">false</option>
<option name="field">runDuration</option>
<option name="underLabel">Total</option>
</single>
<single>
<title>Search Space Consumption (bytes)</title>
<option name="link.visible">false</option>
<option name="field">diskUsage</option>
<option name="underLabel">on disk</option>
</single>
<single>
<!-- <title>Size on Disk</title> -->
<option name="link.visible">false</option>
<option name="field">bytes</option>
<option name="underLabel">from indexers</option>
<searchPostProcess>stats sum(performance.dispatch.stream.remote.*.output_count) AS * by sid | transpose | search column!="sid" | stats sum(row 1) AS bytes</searchPostProcess>
</single>
<single>
<title>Scanned Events</title>
<option name="link.visible">false</option>
<option name="field">scanCount</option>
<option name="underLabel">Count</option>
</single>
<single>
<!-- <title>Scan Rate</title> -->
<option name="link.visible">false</option>
<option name="field">rate</option>
<option name="underLabel">rate</option>
<searchPostProcess>eval rate=if(scanCount=0, "N/A", round(scanCount / runDuration, 2) . " E/s")</searchPostProcess>
</single>
</row>
<row>
<single>
<title>Search String</title>
<option name="field">search</option>
<searchPostProcess>eval search=coalesce('request.search', eventSearch)</searchPostProcess>
</single>
</row>
<row>
<chart>
<title>Performance Histogram</title>
<option name="link.visible">false</option>
<option name="charting.chart">column</option>
<option name="charting.chart.stackMode">stacked100</option>
<searchPostProcess>search dispatchState=DONE
| stats max(performance.command.search.index.*.invocations) AS * by sid
| fillnull value=0 usec_1_8, usec_8_64, usec_64_512, usec_512_4096, usec_4096_32768, usec_32768_262144, usec_262144_inf
| eval total=usec_1_8 + usec_8_64 + usec_64_512 + usec_512_4096 + usec_4096_32768 + usec_32768_262144 + usec_262144_inf
| search total> 0
| table sid, usec_262144_inf, usec_32768_262144, usec_4096_32768, usec_512_4096, usec_64_512, usec_8_64, usec_1_8
</searchPostProcess>
</chart>
<chart>
<title>Time spent in commands</title>
<option name="link.visible">false</option>
<option name="charting.chart">pie</option>
<searchPostProcess>stats max(performance.command.*.duration_secs) AS *, max(performance.command.search.*.duration_secs) AS * | transpose | rename column AS command, "row 1" as time</searchPostProcess>
</chart>
</row>
<row>
<chart>
<title>Bytes Returned From Indexers</title>
<option name="link.visible">false</option>
<option name="charting.chart">column</option>
<searchPostProcess>search dispatchState=DONE
| stats max(performance.dispatch.stream.remote.*.output_count) AS * by sid
| table sid, *
</searchPostProcess>
</chart>
<chart>
<title>Ratio of Bytes Returned vs. Time</title>
<option name="link.visible">false</option>
<option name="charting.chart">column</option>
<option name="charting.axisTitleX.text">Hostname</option>
<option name="charting.axisTitleY.text">Bytes / Sec</option>
<option name="charting.legend.placement">none</option>
<searchPostProcess><![CDATA[search dispatchState=DONE
| fields sid, performance.dispatch.stream.remote.* | transpose
| rex field=column "performance\.dispatch\.stream\.remote\.(?<hostname>.+?)\.(?<metric>[^\.]+)$"
| eval sid=if(column="sid", 'row 1', null())
| filldown sid
| eval {metric}='row 1'
| stats max(duration_secs) AS duration_secs
max(output_count) AS output_count
by sid, hostname
| eval bytes_per_sec=round(output_count / duration_secs, 2)
| chart max(bytes_per_sec) over hostname by sid]]></searchPostProcess>
</chart>
</row>
<row>
<table>
<title>Job Details</title>
<searchPostProcess>transpose</searchPostProcess>
</table>
</row>
</form>

169
apps/splunk_health_overview/default/data/ui/views/user_activity.xml
Unescape View File

@ -0,0 +1,169 @@
<form>
<label>User Activity</label>
<description></description>
<fieldset submitButton="false">
<input type="dropdown" token="host" searchWhenChanged="true">
<label>Search Head</label>
<selectFirstChoice>true</selectFirstChoice>
<populatingSearch fieldForLabel="host" fieldForValue="host">|inputlookup all_servers.csv | search role=*search* | dedup host| fields host | sort + host</populatingSearch>
<fieldForLabel>host</fieldForLabel>
<fieldForValue>host</fieldForValue>
</input>
<input type="time" token="time" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@d</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
</fieldset>
<row>
<panel>
<html>
<div id="usage_help">
<p/>This dashboard is intended to illustrate user activity and usage across your Splunk Search Head(s).
<p/>Use the input to choose the Search Heads you would like to include results from and the timerange picker to select the dates.
</div>
</html>
</panel>
</row>
<row>
<panel>
<table>
<title>Users Currently Logged In</title>
<searchString>| rest splunk_server=* /services/authentication/httpauth-tokens | search userName!=splunk-system-user |stats max(updated) AS "Last Activity" values(splunk_server) AS Host by userName | rename userName AS User</searchString>
<earliest>$time.earliest$</earliest>
<latest>$time.latest$</latest>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>Unique User Activity Per Search Head</title>
<searchString>index=_internal host=$host$ sourcetype=splunkd_access "/services/search/jobs" | kv access-extractions | search uri=/services/search/jobs/* user!="-"| timechart dc(user) by host</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.text">Count</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">line</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">bottom</option>
</chart>
</panel>
<panel>
<chart>
<title>Search Activity Run-Time by Host</title>
<searchString>index=_internal host=$host$ sourcetype=splunkd_access "/services/search/jobs" | kv access-extractions | search uri=/services/search/jobs/* user!="-"| rex "(?&lt;run_time&gt;\d+)ms" | timechart eval(sum(run_time)/1000) by host</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option>
<option name="charting.legend.placement">right</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisTitleY.text">Seconds</option>
</chart>
</panel>
<panel>
<input type="time" token="field3" searchWhenChanged="true">
<label></label>
<default>
<earliestTime>@mon</earliestTime>
<latestTime>now</latestTime>
</default>
</input>
<table>
<title>Total Unique Users Active in Splunk</title>
<searchString>index=_internal sourcetype=splunkd_access "/services/search/jobs" user!=splunk-system-user | kv access-extractions |search uri=/services/search/jobs/* user!="-"| stats dc(user) AS "Unique Users" by host | rename host AS "Search Head" | addcoltotals | sort - "Unique Users" | fillnull value=Total</searchString>
<earliestTime>$field3.earliest$</earliestTime>
<latestTime>$field3.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
<row>
<panel>
<chart>
<title>View Hits Over Time</title>
<searchString>index=_internal host=$host$ source=*web_access.log* /app/ | rex "GET /[^/]+/app/(?&lt;app_view&gt;[^/ ?]+/[^/ ?]+) " | timechart limit=15 usenull=f count by app_view</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option>
<option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option>
<option name="charting.axisTitleX.visibility">visible</option>
<option name="charting.axisTitleY.visibility">visible</option>
<option name="charting.axisTitleY2.visibility">visible</option>
<option name="charting.axisX.scale">linear</option>
<option name="charting.axisTitleX.text">Time</option>
<option name="charting.axisY.scale">linear</option>
<option name="charting.axisY2.enabled">false</option>
<option name="charting.axisY2.scale">inherit</option>
<option name="charting.chart">column</option>
<option name="charting.chart.nullValueMode">gaps</option>
<option name="charting.chart.sliceCollapsingThreshold">0.01</option>
<option name="charting.chart.stackMode">stacked</option>
<option name="charting.chart.style">shiny</option>
<option name="charting.drilldown">all</option>
<option name="charting.layout.splitSeries">0</option>
<option name="charting.legend.labelStyle.overflowMode">ellipsisStart</option>
<option name="charting.legend.placement">right</option>
</chart>
</panel>
<panel>
<table>
<title>Top Visited Views</title>
<searchString>index=_internal host=$host$ source=*web_access.log* /app/ | rex "GET /[^/]+/app/(?&lt;app_view&gt;[^/ ?]+/[^/ ?]+) " | rename app_view AS "App View" | top limit=20 host "App View"</searchString>
<earliestTime>$time.earliest$</earliestTime>
<latestTime>$time.latest$</latestTime>
<option name="wrap">true</option>
<option name="rowNumbers">false</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="count">10</option>
</table>
</panel>
</row>
</form>

7
apps/splunk_health_overview/default/datamodels.conf
Unescape View File

@ -0,0 +1,7 @@
[Internal_Events]
acceleration = 1
acceleration.earliest_time = -1mon
[Introspection_Usage]
acceleration = 1
acceleration.earliest_time = -1mon

50
apps/splunk_health_overview/default/savedsearches.conf
Unescape View File

@ -0,0 +1,50 @@
[avail_indexes]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = 0 0 * * 6
dispatch.earliest_time = -30d
dispatch.latest_time = now
disabled = false
enableSched = 1
search = | tstats values(sourcetype) AS sourcetype where index=* by index | mvexpand sourcetype | rename index AS title | join type=left title [| rest /services/data/indexes | eval retention=frozenTimePeriodInSecs/60/60/24 | stats min(retention) AS retention by title] | rename title AS Index | table Index sourcetype retention | sort + Index | outputlookup avail_indexes.csv
run_on_startup = true
[server_lookup]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = 0 0 * * 6
disabled = false
dispatch.earliest_time = 0
dispatch.latest_time =
enableSched = 1
search = | rest splunk_server=* /services/server/info | mvexpand server_roles | search server_roles!=search_peer | rename server_roles AS role splunk_server AS host | table host guid role version | outputlookup all_servers.csv
run_on_startup = true
[server_lookup_v6_1]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = 0 0 * * 6
disabled = true
dispatch.earliest_time = -30d@d
dispatch.latest_time = now
enableSched = 1
search = index=_internal sourcetype=splunkd component=ServerRoles role!=license_slave* role!=search_peer* |rex field=role "(?<role>\S+)\." |dedup host role | join host [|rest splunk_server=* /services/server/info | rename serverName AS host | fields host guid version] | rename server_role AS role | table host role guid version | outputlookup all_servers.csv
[savedsearch_state_lookup]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = 0 * * * *
disabled = 0
dispatch.earliest_time = -1h@h
dispatch.latest_time = now
enableSched = 1
search = index=_internal sourcetype=scheduler OR (sourcetype=splunk_web_service "loading saved search") OR (sourcetype=splunkd_access method=POST /saved/searches) | rex "\/saved\/searches\/(?<savedsearch_name>[^/]+) HTTP?" | rex "saved\ssearch\s\"\/\w+\/(?<user>\w+)\/(?<app>\w+)\/\w+\/\w+\/(?<savedsearch_name>.+)\"\s" | eval savedsearch_name=urldecode(savedsearch_name) | search savedsearch_name!=_ACCELERATE* | stats avg(run_time) AS "Avg Runtime" max(run_time) AS "Max Runtime" min(_time) as first_time,max(_time) as last_time by savedsearch_name | inputlookup append=T savedsearch_runtimes.csv | stats max("Max Runtime") AS "Max Runtime" max("Avg Runtime") AS "Avg Runtime" min(first_time) as first_time, max(last_time) as last_time by savedsearch_name | outputlookup savedsearch_runtimes.csv

2
apps/splunk_health_overview/lookups/README
Unescape View File

@ -0,0 +1,2 @@
all_servers.csv is generated by the all_servers saved search.
all_indexes.csv is generated by th eall_indexes saved search.

6
apps/splunk_health_overview/metadata/default.meta
Unescape View File

@ -0,0 +1,6 @@
[]
access = read : [ admin ], write : [ admin, power ]
export = none
[lookups]
export = system

48
apps/splunk_health_overview/metadata/local.meta
Unescape View File

@ -0,0 +1,48 @@
[app/install/install_source_checksum]
version = 6.2.0
modtime = 1417642279.990818000
[views/search_activity]
version = 6.2.3
modtime = 1435021968.343178000
[savedsearches/avail_indexes]
version = 6.2.0
modtime = 1418588425.683031000
[views/search_detail]
owner = admin
version = 6.2.0
modtime = 1418588689.234899000
[views/user_activity]
version = 6.2.0
modtime = 1418694100.118998000
[views/health_status]
version = 6.2.0
modtime = 1418691739.189140000
[views/resource_usage]
version = 6.2.0
modtime = 1418693713.624970000
[views/scheduler_activity]
version = 6.2.0
modtime = 1418692953.351024000
[views/license_usage]
version = 6.1.0
modtime = 1433265651.802525000
[views/deployment_status]
version = 6.1.0
modtime = 1433265339.806811000
[datamodels/Internal_Events]
version = 6.1.0
modtime = 1433265569.286090000
[models/Internal_Events]
version = 6.1.0
modtime = 1433265547.276039000

153
apps/splunk_health_overview/splunkbase.manifest
Unescape View File

@ -0,0 +1,153 @@
{
"version": "1.0",
"date": "2022-11-14T18:58:45.384302319Z",
"hashAlgorithm": "SHA-256",
"app": {
"id": 1919,
"version": "2.2.1",
"files": [
{
"path": "metadata/default.meta",
"hash": "92003e4bf19e1a44a315eff7b3cc8038699c0fc03dd6ecdcff6525bf2612a05a"
},
{
"path": "metadata/local.meta",
"hash": "ab95b50b3a4104e4a9d4dcb45f6dbef06ffff161194434af56f52b3be7b62883"
},
{
"path": "appserver/static/table_data_bar.css",
"hash": "797f3bc17c47d933506c15c8022c3b3b9f138899b80197f86e9d259ad8a1567d"
},
{
"path": "appserver/static/search_health.js",
"hash": "72340cd9c73db79320e934c77cbe4a1cf1954b6ce91ac948b21e55b4fd63b1da"
},
{
"path": "appserver/static/search_health.css",
"hash": "751367415f66ee04b72d81d61f9c176e110b7cc589442fdc276b8071f897540e"
},
{
"path": "appserver/static/table_cell_highlighting.css",
"hash": "04a4504e1af20a1931194597108c2cf4b2887c16d69cdbaabd428c1a3ec0c92e"
},
{
"path": "appserver/static/table_cell_highlighting.js",
"hash": "bef0340f3c68bc27c28748198fc8332f36a210ee8d00d0bc003842bb0daed9bd"
},
{
"path": "appserver/static/table_data_bar.js",
"hash": "8206bd0016380df81c7389cbf8d69352c35484faaa2ea6384213e1e59e37ff38"
},
{
"path": "appserver/static/server_health.css",
"hash": "751367415f66ee04b72d81d61f9c176e110b7cc589442fdc276b8071f897540e"
},
{
"path": "appserver/static/table_decorations_shrunk.css",
"hash": "afccebe892f133a1e5ccfab1096c921af6273f15b59d6ecf5c7fa4b1dcce0cc7"
},
{
"path": "appserver/static/table_icons_mv.js",
"hash": "6615774fcf6131949caf8241437ebf7d0bfd241296af14080b6b9db28cdb2290"
},
{
"path": "appserver/static/server_health.js",
"hash": "328341b4fc45b7e35c3718114a064661e80c16f653f1d798cf3226c953416526"
},
{
"path": "README",
"hash": "cfa218576b3e77df9b92197427043b71b88f9b0f32028dbbb71cc3e81635979b"
},
{
"path": "default/datamodels.conf",
"hash": "5eed66c04cfdf7c58adda24eac50f1d9f6f2e8674a0d36c8d33e7cad0d869080"
},
{
"path": "default/data/models/Introspection_Usage.json",
"hash": "0ec363990f04209b6c9b8c571cb9edcdf5d7c25d34f5faf069ea5b21902917ec"
},
{
"path": "default/data/models/Internal_Events.json",
"hash": "347d411013a9314c566c044d51bcdd7dde885dab96f6dcea094bcbd5fd877a19"
},
{
"path": "default/data/ui/views/resource_usage.xml",
"hash": "1f80ff6824dfda9650ab27593e363d10fb38cfb65e45600f638e8203ae7e6a1a"
},
{
"path": "default/data/ui/views/indexes.xml",
"hash": "7c6011b3b45a4444c15d32fcc1a3290a65e8a2ba0b3713a13b8e8f7b57d508d9"
},
{
"path": "default/data/ui/views/search_activity.xml",
"hash": "0adf9d703d01a82d223a71031cae914a878b2e0993f118f8bedccc5e31872e9d"
},
{
"path": "default/data/ui/views/health_status.xml",
"hash": "641c6f632ba605258fa26f47e6bafbe3d6dd1c46f27b7624c94909bae164feed"
},
{
"path": "default/data/ui/views/scheduler_activity.xml",
"hash": "f320e946f5344a134889f615af50f8576ecdd6ce9e58043fb5985adee65de7ac"
},
{
"path": "default/data/ui/views/saved_search_detail.xml",
"hash": "8b45f92b143cf485c58a67e634f212ccccc34a91116c063054ae904a5ed845da"
},
{
"path": "default/data/ui/views/license_usage.xml",
"hash": "1d8f30d203586663bdff5cefa13923e048c42ae5a1b1abaf976dad981e614caa"
},
{
"path": "default/data/ui/views/user_activity.xml",
"hash": "d32d397034c1f949988c77e558b275782cc8237e80af9667f4364bde4d611b8c"
},
{
"path": "default/data/ui/views/indexes_and_fields.xml",
"hash": "c7862ae1f3d722dc724f58173c45c897c4d343373e23a04198bc542396927bc2"
},
{
"path": "default/data/ui/views/deployment_status.xml",
"hash": "faefc4c337afd2b122c220228a8ee64b1a3169643c2a9fa5f8170fed27750595"
},
{
"path": "default/data/ui/views/search_detail.xml",
"hash": "50ae8fa17a012a37abc9a41d92525ac349168993fb76f1fa51ffeb8234dc8bcd"
},
{
"path": "default/data/ui/nav/default.xml",
"hash": "12d5f01948c75731728c67618607dc88cc3e162892b125abb4ad1df39d527d82"
},
{
"path": "default/savedsearches.conf",
"hash": "ed284d921baa9d0c4b3d58eba5cffe26fabfb5002b41fda8a930f41eeb7ff433"
},
{
"path": "default/app.conf",
"hash": "c047a1ec0d2c707b4e42a1cd43169510337c687a68cd76cd212f39e82e6d6a48"
},
{
"path": "lookups/README",
"hash": "8df77968580e2571ece8e120c027a3c0a3633f18ce514fe79cf6e4d2ee830bac"
}
]
},
"products": [
{
"platform": "splunk",
"product": "enterprise",
"versions": [],
"architectures": [
"x86_64"
],
"operatingSystems": [
"windows",
"linux",
"macos",
"freebsd",
"solaris",
"aix"
]
}
]
}
Write Preview
Loading…
Cancel
Save