TopicsDetection rules
Detection rules
The rules below are starting points for your own. Events reach the SIEM one to seven minutes after they happen, and after a sink outage, later by as long as the outage lasted; the queue keeps them for seven days by default. Each rule that reports events therefore selects the ones that arrived since its previous run (ingestion time in Sentinel, index time in Splunk) from a look-back that covers late events with a day to spare: three days in Sentinel and eight days in Splunk. Rules that count failed logins within an hour count by the time the logins happened, so a burst that arrives late still adds up. The rules for missing events (no heartbeat in both SIEMs, lost events in Sentinel) use the look-back each one names.
For every rule
- Run it every 15 minutes unless it says otherwise. Consecutive runs select slices that meet without
overlapping, as Microsoft’s
guidance on ingestion delay does, so a run
that starts late leaves events between its slice and the one before. To close that gap, let the slices
overlap by a few minutes, for example
ago(20m)and_index_earliest=-20m@mon a 15-minute schedule. - Throttle its alerts per tenant or network, or per event
idfor a rule that reports single events: a rule fires on every run while its condition holds, and overlapping slices or a batch sent again repeat an event. In Splunk, suppress the alerts by those fields; in Sentinel, group them into one incident by those fields as custom details. - Deduplicate on the event
idwherever the rule counts events, since delivery is at least once.
Keep the tenants you expect in a list, so that a tenant that never sent anything also raises the heartbeat alert.
Microsoft Sentinel
The rules read the parser function PxmLogExporter from
siem/sentinel/PxmLogExporter.kql. The expected tenants are the watchlist
PxmTenants, with the tenant (the tenant of the events) as its search key. In each analytics rule, set
Run query every to 15 minutes and Lookup data from the last to 3 days, unless the rule says otherwise.
Azure can store a late event with its arrival time as TimeGenerated; EventTime keeps the time it happened.
No heartbeat for 30 minutes, with Lookup data from the last set to 2 hours:
let seen = PxmLogExporter
| where TimeGenerated > ago(2h) and EventName == "exporter.heartbeat"
| summarize LastHeartbeat = max(TimeGenerated) by Tenant;
_GetWatchlist('PxmTenants')
| project Tenant = tostring(SearchKey)
| join kind=leftouter seen on Tenant
| where isnull(LastHeartbeat) or LastHeartbeat < ago(30m)
| project Tenant, LastHeartbeat
The exporter’s health: a sink failing or skipping events, a capture plugin missing:
PxmLogExporter
| where IngestionTime > ago(15m)
| where EventName in ("exporter.sink.error", "exporter.dropped")
or (EventName == "exporter.heartbeat"
and (tostring(Data.sinks.http) == "failing"
or tostring(Data.sinks.stream) == "failing"
or tostring(Data.plugins_loaded) has "false"))
Changes that can silence the exporter: the exporter, a sink or Include platform support logins switched off; a change of the events it captures, of where a sink delivers or whether through the outgoing proxy, or of how the SIEM files the events; an option stored under another spelling; the purge of the exporter’s data:
let switchedOn = dynamic(["1", "true", "on", "yes"]);
let flags = dynamic(["Enabled", "HttpEnabled", "StreamEnabled", "IncludeSupportLogins"]);
let routing = dynamic(["EventGroups", "RecordClasses", "Environment", "UseOutgoingProxy",
"HttpPreset", "HttpUrl", "HttpHeaders", "HttpBodyFormat", "SentinelDcrImmutableId",
"SentinelStreamName", "SplunkHecToken", "SplunkIndex", "SplunkSourcetype",
"EventHubsHubName", "StreamHost", "StreamPort", "StreamFraming", "SyslogFacility",
"SyslogAppName"]);
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "exporter.config.changed"
| mv-expand Option = iff(array_length(Data.options) > 0, Data.options, dynamic([""])) to typeof(string)
| extend New = tolower(trim(@"[ \t\n\r\v\x00]+", tostring(Data.changes[Option]['new'])))
| where (Option in (flags) and New !in (switchedOn))
or Option in (routing) or tostring(Data.reason) == "other-spelling"
| summarize Options = make_set(Option)
by EventId, EventName, TimeGenerated, Tenant, ActorUserName
| union (PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "exporter.purged"
| project EventId, EventName, TimeGenerated, Tenant, ActorUserName)
A sink that is switched off, or sends or files its events elsewhere, does not deliver the event about that
change where the rules look, so if the SIEM reads only that sink, the heartbeat rule catches it. A new row
under an accented spelling of the module’s name arrives as config.option_changed, which this rule does not
read.
Many failed logins from one network (SrcIpNetwork: an IPv4 /32 or an IPv6 /64), counted over the hour up to
each failure that arrived since the previous run:
let recent = PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "auth.login.failure"
and isnotempty(SrcIpNetwork)
| summarize by SrcIpNetwork, At = bin(EventTime, 1m) + 1m;
PxmLogExporter
| where EventName == "auth.login.failure"
and EventTime >= toscalar(recent | summarize min(At)) - 1h
| summarize arg_max(TimeGenerated, *) by EventId
| join kind=inner recent on SrcIpNetwork
| where EventTime between ((At - 1h) .. At)
| summarize Failures = sum(EventCount), Users = make_set(PresentedUserName, 20),
Addresses = make_set(SrcIpAddr, 20) by SrcIpNetwork, At
| summarize arg_max(Failures, *) by SrcIpNetwork
| where Failures >= 20
A login after failed logins in the hour before it, counting the user’s failures and the capped failures
(name_cap) from the login’s network:
let failures = PxmLogExporter
| where EventName == "auth.login.failure"
| summarize arg_max(TimeGenerated, *) by EventId
| project FailedAt = EventTime, Failed = EventCount,
FailedUser = tolower(PresentedUserName), FailedFrom = SrcIpNetwork,
Capped = isnotnull(Data.name_cap);
let logins = PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "auth.login.success"
| summarize arg_max(TimeGenerated, *) by EventId
| project LoginAt = EventTime, User = tolower(ActorUserName), SrcIpAddr, SrcIpNetwork;
union
(logins | join kind=inner (failures | where not(Capped))
on $left.User == $right.FailedUser),
(logins | join kind=inner (failures | where Capped and isnotempty(FailedFrom))
on $left.SrcIpNetwork == $right.FailedFrom)
| where FailedAt between ((LoginAt - 1h) .. LoginAt)
| summarize Failures = sum(Failed), LastFailure = max(FailedAt)
by User, LoginAt, SrcIpAddr
| where Failures >= 5
An administrator role assigned (name your administrator roles):
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "user.role_assigned"
| where tostring(Data.role_name) in ("Administrators")
A credential created, changed or deleted:
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "credential.changed"
Events of the HTTPS sink lost on the way, with Run query every set to 1 day and Lookup data from the
last to 4 days; for the TLS stream sink, read stream instead of http:
let beats = PxmLogExporter
| where EventName == "exporter.heartbeat"
| project Tenant, Beat = EventTime, Arrived = IngestionTime,
Sent = tolong(Data.sent.http), LastSeq = tolong(Data.last_seq.http)
| partition hint.strategy=native by Tenant
(sort by Beat asc | extend FromSeq = prev(LastSeq), SentBefore = prev(Sent))
| where isnotnull(FromSeq) and LastSeq > FromSeq and Arrived > ago(1d);
beats
| join kind=leftouter (PxmLogExporter | where isnotnull(Seq) | distinct Tenant, Seq)
on Tenant
| summarize Received = countif(Seq > FromSeq and Seq <= LastSeq)
by Tenant, Beat, Expected = Sent - SentBefore
| where Received < Expected
Splunk
The searches read the sourcetype pxm:logexporter from siem/splunk/props.conf; add
the index you use. The expected tenants are the lookup pxm_tenants.csv, with a column tenant. Schedule
each search every 15 minutes with the time range earliest=-8d, unless it says otherwise, and widen the range
to match when someone raises Keep undelivered events (hours) on the exporter’s Limits page, for example
earliest=-15d for 336 hours, again with a day to spare. _index_earliest=-15m@m _index_latest=@m takes the
events indexed since the previous run. The timestamp settings in props.conf apply to the TLS stream and to
file inputs; the HTTP Event Collector takes the time each event carries.
No heartbeat for 30 minutes:
| inputlookup pxm_tenants.csv
| join type=left tenant
[ search sourcetype=pxm:logexporter event="exporter.heartbeat" earliest=-2h
| stats max(_time) AS last_heartbeat BY tenant ]
| where isnull(last_heartbeat) OR last_heartbeat < relative_time(now(), "-30m")
The exporter’s health:
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
(event="exporter.sink.error" OR event="exporter.dropped"
OR (event="exporter.heartbeat" (data.sinks.http="failing" OR data.sinks.stream="failing"
OR data.plugins_loaded.event="false" OR data.plugins_loaded.authentication="false"
OR data.plugins_loaded.rights="false" OR data.plugins_loaded.options="false")))
Changes that can silence the exporter, as described for Sentinel, including what the rule cannot see:
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
(event="exporter.purged" OR (event="exporter.config.changed"
((data.options{}="Enabled"
NOT data.changes.Enabled.new IN ("1", "true", "on", "yes"))
OR (data.options{}="HttpEnabled"
NOT data.changes.HttpEnabled.new IN ("1", "true", "on", "yes"))
OR (data.options{}="StreamEnabled"
NOT data.changes.StreamEnabled.new IN ("1", "true", "on", "yes"))
OR (data.options{}="IncludeSupportLogins"
NOT data.changes.IncludeSupportLogins.new IN ("1", "true", "on", "yes"))
OR data.options{} IN ("EventGroups", "RecordClasses", "Environment", "UseOutgoingProxy",
"HttpPreset", "HttpUrl", "HttpHeaders", "HttpBodyFormat", "SentinelDcrImmutableId",
"SentinelStreamName", "SplunkHecToken", "SplunkIndex", "SplunkSourcetype",
"EventHubsHubName", "StreamHost", "StreamPort", "StreamFraming",
"SyslogFacility", "SyslogAppName")
OR data.reason="other-spelling")))
Many failed logins from one network (src_network: an IPv4 /32 or an IPv6 /64), counted over the hour up to
each of its failures indexed since the previous run:
sourcetype=pxm:logexporter event="auth.login.failure"
[ search sourcetype=pxm:logexporter event="auth.login.failure"
_index_earliest=-15m@m _index_latest=@m
| dedup src_network | fields src_network ]
| dedup id
| sort 0 _time
| streamstats time_window=1h sum(data.count) AS failures,
values(data.presented_user_name) AS users BY src_network
| where failures >= 20
AND _indextime >= relative_time(now(), "-15m@m")
AND _indextime < relative_time(now(), "@m")
| stats max(failures) AS failures, values(users) AS users BY src_network
A login after failed logins in the hour before it, counting the user’s failures and the capped failures
(name_cap) from the login’s network; each login is reported once it is indexed:
sourcetype=pxm:logexporter (event="auth.login.failure" OR event="auth.login.success")
| dedup id
| eval who=lower(if(event="auth.login.failure", 'data.presented_user_name', user)),
named=if(event="auth.login.failure" AND isnull('data.name_cap'), 'data.count', 0),
capped=if(isnotnull('data.name_cap'), 'data.count', 0)
| sort 0 _time
| streamstats time_window=1h sum(named) AS named_failures BY who
| streamstats time_window=1h sum(capped) AS capped_failures BY src_network
| eval failures=coalesce(named_failures, 0) + coalesce(capped_failures, 0)
| where event="auth.login.success" AND failures >= 5
AND _indextime >= relative_time(now(), "-15m@m")
AND _indextime < relative_time(now(), "@m")
| table _time who src failures
An administrator role assigned, and a credential created, changed or deleted:
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
event="user.role_assigned" data.role_name="Administrators"
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
event="credential.changed"
Centric PXM™ is a trademark of Centric Software, Inc.