SplunkPy

Use the SplunkPy integration to fetch events (logs) from within Cortex XSOAR, push events from Cortex XSOAR to SplunkPy, and fetch SplunkPy ES notable events as Cortex XSOAR incidents.

This integration was integrated and tested with Splunk v6.5.

Use Cases


  • Query Splunk for events.
  • Create a new event in Splunk.
  • Get results of a search that was executed in Splunk.

Configure SplunkPy on Cortex XSOAR

  1. Navigate to Settings > Integrations > Servers & Services.
  2. Search for SplunkPy.
  3. Click Add instance to create and configure a new integration instance.
ParameterDescriptionRequired
hostThe host name to the server, including the scheme (x.x.x.x).True
authenticationThe username used for authentication.True
portThe port affiliated with the server.True
fetchQueryThe notable events ES query to be fetched.False
fetch_limitThe limit of incidents to fetch. The maximum is 200 (It is recommended to fetch less than 50).False
isFetchThe incidents fetched.False
incidentTypeThe incident type.False
proxyRuns the integration instance using the proxy server (HTTP or HTTPS) that you defined in the server configuration.False
timezoneThe timezone of the Splunk server (in minutes). For example, GMT is gmt +3, set +180 (set this only if it is different than the Cortex XSOAR server). This is relevant only for fetching notable events.False
parseNotableEventsRawParses the raw part of notable events.False
replaceKeysReplace with Underscore in Incident FieldsFalse
extractFieldsThe CSV fields that will be parsed out of _raw notable events.False
useSplunkTimeUses the Splunk clock time for the fetch.False
unsecureWhen selected, certificates are not checked. (not secure)False
earliest_fetch_time_fieldnameThe earliest time to fetch (the name of the Splunk field whose value defines the query's earliest time to fetch).False
latest_fetch_time_fieldnameThe latest time to fetch (the name of the Splunk field whose value defines the query's latest time to fetch).False
appThe context of the application's namespace.False
hec_tokenThe HEC token (HTTP Event Collector).False
hec_urlThe HEC URL. For example, https://localhost:8088.False
fetch_timeThe first timestamp to fetch in \<number>\<time unit> format. For example, "12 hours", "7 days", "3 months", "1 year".False
use_requests_handlerUse Python requests handlerFalse

The (!) Earliest time to fetch and Latest time to fetch are search parameters options. The search uses All Time as the default time range when you run a search from the CLI. Time ranges can be specified using one of the CLI search parameters, such as earliest_time, index_earliest, or latest_time.

  1. Click Test to validate the URLs, token, and connection.

Configure Splunk to Produce Alerts for SplunkPy

It is recommended that Splunk is configured to produce basic alerts that the SplunkPy integration can ingest, by creating a summary index in which alerts are stored. The SplunkPy integration can then query that index for incident ingestion. It is not recommended to use the Cortex XSOAR application with Splunk for routine event consumption because this method is not able to be monitored and is not scalable.

  1. Create a summary index in Splunk. For more information, click here.
  2. Build a query to return relevant alerts. image
  3. Identify the fields list from the Splunk query and save it to a local file. image
  4. Define a search macro to capture the fields list that you saved locally. For more information, click here. Use the following naming convention: (demistofields{type}). imageimage
  5. Define a scheduled search, the results of which are stored in the summary index. For more information about scheduling searches, click here. image
  6. In the Summary indexing section, select the summary index, and enter the {key:value} pair for Cortex XSOAR classification. image
  7. Configure the incident type in Cortex XSOAR by navigating to Settings > Advanced > Incident Types.image
  8. Navigate to Settings > Integrations > Classification & Mapping, and drag the value to the appropriate incident type. image
  9. Click the Edit mapping link to map the Splunk fields to Cortex XSOAR. image
  10. (Optional) Create custom fields.
  11. Build a playbook and assign it as the default for this incident type.

Commands

You can execute these commands from the Cortex XSOAR CLI, as part of an automation, or in a playbook. After you successfully execute a command, a DBot message appears in the War Room with the command details.

Get results


Returns the results of a previous Splunk search. This command can be used in conjunction with the splunk-job-create command.

Base Command

splunk-results

Input
Argument NameDescriptionRequired
sidThe ID of the search for which to return results.Required
Context Output

There is no context output for this command.

Command Example

!splunk-results sid="1566221331.1186"

Search for events


Searches Splunk for events.

Base Command

splunk-search

Input
Argument NameDescriptionRequired
queryThe Splunk search language string to execute. For example, "index=* | head 3".Required
earliest_timeSpecifies the earliest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. The default is 1 week ago, in the format "-7d". You can also specify time in the format: 2014-06-19T12:00:00.000-07:00".Optional
latest_timeSpecifies the latest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. For example: "2014-06-19T12:00:00.000-07:00" or "-3d" (for time 3 days before now).Optional
event_limitThe maximum number of events to return. The default is 100. If "0" is selected, all results are returned.Optional
appThe string that contains the application namespace in which to restrict searches.Optional
batch_limitThe maximum number of returned results to process at a time. For example, if 100 results are returned, and you specify a batch_limit of 10, the results will be processed 10 at a time over 10 iterations. This does not affect the search or the context and outputs returned. In some cases, specifying a batch_size enhances search performance. If you think that the search execution is suboptimal, it is recommended to try several batch_size values to determine which works best for your search. The default is 25,000.Optional
update_contextDetermines whether the results will be entered into the context.Optional
Context Output
PathTypeDescription
Splunk.ResultUnknownThe results of the Splunk search. The results are a JSON array, in which each item is a Splunk event.
Command Example

!splunk-search query="* | head 3" earliest_time="-1000d"

Human Readable Output

Splunk Search results for query: * | head 3

_bkt_cd_indextime_kv_raw_serial_si_sourcetype_timehostindexlinecountsourcesourcetypesplunk_server
main~445~66D21DF4-F4FD-4886-A986-82E72ADCBFE9445:89777415854629061InsertedAt="2020-03-29 06:21:43"; EventID="837005"; EventType="Application control"; Action="None"; ComputerName="ACME-code-007"; ComputerDomain="DOMAIN"; ComputerIPAddress="127.0.0.1"; EventTime="2020-03-29 06:21:43"; EventTypeID="5"; Name="LogMeIn"; EventName="LogMeIn"; UserName=""; ActionID="6"; ScanTypeID="200"; ScanType="Unknown"; SubTypeID="23"; SubType="Remote management tool"; GroupName="";\u003cbr\u003e2ip-172-31-44-193, mainsophos:appcontrol2020-03-28T23:21:43.000-07:00127.0.0.1main2eventgensophos:appcontrolip-172-31-44-193

Create event


Creates a new event in Splunk.

Base Command

splunk-submit-event

Input
Argument NameDescriptionRequired
indexThe Splunk index to which to push the data. Run the splunk-get-indexes command to get all of the indexes.Required
dataThe new event data to push. Can be, any string.Required
sourcetypeThe event source type.Required
hostThe event host. Can be, "Local" or "120.0.0.1".Required
Context Output

There is no context output for this command.

Command Example

!splunk-submit-event index="main" data="test" sourcetype="demisto-ci" host="localhost"

Human Readable Output

image

Print all index names


Prints all Splunk index names.

Base Command

splunk-get-indexes

Input

There are no input arguments for this command.

Context Output

There is no context output for this command.

Command Example

!splunk-get-indexes extend-context="indexes="

Human Readable Output

image

Update notable events


Update an existing notable event in Splunk ES

Base Command

splunk-notable-event-edit

Input
Argument NameDescriptionRequired
eventIDsThe comma-separated list of event IDs of notable events.Required
ownerThe Splunk user to assign to the notable event.Optional
commentThe comment to add to the notable event.Required
urgencyThe urgency of the notable event.Optional
statusThe notable event status. Can be 0 - 5, where 0 - Unassigned, 1 - Assigned, 2 - In Progress, 3 - Pending, 4 - Resolved, 5 - Closed.Optional
Context Output

There is no context output for this command.

Command Example

!splunk-notable-event-edit eventIDs=66D21DF4-F4FD-4886-A986-82E72ADCBFE9@@notable@@a045b8acc3ec93c2c74a2b18c2caabf4 comment="Demisto"

Human Readable Output

image

Create a new job


Creates a new search job in Splunk.

Base Command

splunk-job-create

Input
Argument NameDescriptionRequired
queryThe Splunk search language string to execute. For example, "index=* | head 3".Required
appThe string that contains the application namespace in which to restrict searches.Optional
Context Output
PathTypeDescription
Splunk.JobUnknownThe SID of the created job.
Command Example

!splunk-job-create query="index=* | head 3"

Context Example
{
"Splunk.Job": "1566221733.1628"
}
Human Readable Output

image

Parse an event


Parses the raw part of the event.

Base Command

splunk-parse-raw

Input
Argument NameDescriptionRequired
rawThe raw data of the Splunk event (string).Optional
Context Output
PathTypeDescription
Splunk.Raw.ParsedunknownThe raw event data (parsed).
Command Example

!splunk-parse-raw

Submit an event


Sends events to an HTTP event collector using the Splunk platform JSON event protocol.

Base Command

splunk-submit-event-hec

Input
Argument NameDescriptionRequired
eventThe event payload key-value. An example string: "event": "Access log test message.".Required
fieldsThe fields for indexing that do not occur in the event payload itself. This accepts multiple comma separated fields.Optional
indexThe index name.Optional
hostThe hostname.Optional
source_typeThe user-defined event source type.Optional
sourceThe user-defined event source.Optional
timeThe epoch-formatted time.Optional
Context Output

There is no context output for this command.

Command Example

!splunk-submit-event-hec event="something happened" fields="severity: INFO, category: test, test1" source_type=access source="/var/log/access.log"

Human Readable Output

The event was sent successfully to Splunk.

Get job status


Returns the status of a job.

Base Command

splunk-job-status

Input
Argument NameDescriptionRequired
sidThe ID of the job for which to get the status.Required
Context Output
PathTypeDescription
Splunk.JobStatus.CIDUnknownThe ID of the job.
Splunk.JobStatus.StatusUnknownThe status of the job.
Command Example

!splunk-job-status sid=1234.5667

Context Example
Splank.JobStatus = {
'SID': 1234.5667,
'Status': DONE
}
Human Readable Output

image

Aditional Information

To get the HEC Token

  1. Go to the Splunk UI.
  2. Under "Settings" -> "Data" -> "Data inputs". Screen Shot 2020-01-20 at 10 22 50
  3. Then click on "HTTP Event Collector".
  4. Click on "New Token".
  5. Add all the relevant details until done.

For the HTTP Port number: Click on Global settings (in the http event collector page) Screen Shot 2020-01-20 at 10 27 25

The default port is 8088.

Troubleshooting

In case you encounter HTTP errors (e.g. IncompleteRead), we suggest using Python requests handler.