Search on Regex from Lookup
Reference:
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Format
Objective
Take a list of Regular Expressions (regex) from a lookup table and try to match against each regex.
This approach uses the format command in Splunk to craft multiple match expressions which can then be used in an eval statement.
Example
Say we have the following lookup table called regex_queries.csv with 2 columns:
| regex | remarks |
|---|---|
<regex1> |
Remark #1 |
<regex2> |
Remark #2 |
We can search through the raw web proxy logs to see if any log matches any of the regex expressions in the lookup table:
index=webproxy
| where [
| inputlookup regex_queries.csv
| rename regex AS query
| format "" "match(_raw," "" ")" "OR" ""
]
The output of the subsearch in the above query will be:
match(_raw,"<regex1>") OR match(_raw,"<regex2>")
Hence the effective final query looks like:
If we also need the value of the remarks column from the regex_queries.csv lookup table we can use an eval command instead of a where command:
index=webproxy
| eval [
| inputlookup regex_queries.csv
| fields regex, remarks
| format "ioc_match=case(" "match(_raw," ")," "" "," ")"
| rex mode=sed field=search "s/remarks\=//g"
| rex mode=sed field=search "s/regex\=//g"
]
The output of the subsearch in the above query will be:
ioc_match=case(match(_raw,"<regex1>"),"Remark #1",match(_raw,"<regex2>"),"Remark #2")
Hence the effective final query looks like: