TopicsDetection rules

Detection rules

The rules below are starting points for your own. Events reach the SIEM one to seven minutes after they happen, and after a sink outage, later by as long as the outage lasted; the queue keeps them for seven days by default. Each rule that reports events therefore selects the ones that arrived since its previous run (ingestion time in Sentinel, index time in Splunk) from a look-back that covers late events with a day to spare: three days in Sentinel and eight days in Splunk. Rules that count failed logins within an hour count by the time the logins happened, so a burst that arrives late still adds up. The rules for missing events (no heartbeat in both SIEMs, lost events in Sentinel) use the look-back each one names.

For every rule

  • Run it every 15 minutes unless it says otherwise. Consecutive runs select slices that meet without overlapping, as Microsoft’s guidance on ingestion delay does, so a run that starts late leaves events between its slice and the one before. To close that gap, let the slices overlap by a few minutes, for example ago(20m) and _index_earliest=-20m@m on a 15-minute schedule.
  • Throttle its alerts per tenant or network, or per event id for a rule that reports single events: a rule fires on every run while its condition holds, and overlapping slices or a batch sent again repeat an event. In Splunk, suppress the alerts by those fields; in Sentinel, group them into one incident by those fields as custom details.
  • Deduplicate on the event id wherever the rule counts events, since delivery is at least once.

Keep the tenants you expect in a list, so that a tenant that never sent anything also raises the heartbeat alert.

Microsoft Sentinel

The rules read the parser function PxmLogExporter from siem/sentinel/PxmLogExporter.kql. The expected tenants are the watchlist PxmTenants, with the tenant (the tenant of the events) as its search key. In each analytics rule, set Run query every to 15 minutes and Lookup data from the last to 3 days, unless the rule says otherwise. Azure can store a late event with its arrival time as TimeGenerated; EventTime keeps the time it happened.

No heartbeat for 30 minutes, with Lookup data from the last set to 2 hours:

KQL
let seen = PxmLogExporter
    | where TimeGenerated > ago(2h) and EventName == "exporter.heartbeat"
    | summarize LastHeartbeat = max(TimeGenerated) by Tenant;
_GetWatchlist('PxmTenants')
| project Tenant = tostring(SearchKey)
| join kind=leftouter seen on Tenant
| where isnull(LastHeartbeat) or LastHeartbeat < ago(30m)
| project Tenant, LastHeartbeat

The exporter’s health: a sink failing or skipping events, a capture plugin missing:

KQL
PxmLogExporter
| where IngestionTime > ago(15m)
| where EventName in ("exporter.sink.error", "exporter.dropped")
    or (EventName == "exporter.heartbeat"
        and (tostring(Data.sinks.http) == "failing"
            or tostring(Data.sinks.stream) == "failing"
            or tostring(Data.plugins_loaded) has "false"))

Changes that can silence the exporter: the exporter, a sink or Include platform support logins switched off; a change of the events it captures, of where a sink delivers or whether through the outgoing proxy, or of how the SIEM files the events; an option stored under another spelling; the purge of the exporter’s data:

KQL
let switchedOn = dynamic(["1", "true", "on", "yes"]);
let flags = dynamic(["Enabled", "HttpEnabled", "StreamEnabled", "IncludeSupportLogins"]);
let routing = dynamic(["EventGroups", "RecordClasses", "Environment", "UseOutgoingProxy",
    "HttpPreset", "HttpUrl", "HttpHeaders", "HttpBodyFormat", "SentinelDcrImmutableId",
    "SentinelStreamName", "SplunkHecToken", "SplunkIndex", "SplunkSourcetype",
    "EventHubsHubName", "StreamHost", "StreamPort", "StreamFraming", "SyslogFacility",
    "SyslogAppName"]);
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "exporter.config.changed"
| mv-expand Option = iff(array_length(Data.options) > 0, Data.options, dynamic([""])) to typeof(string)
| extend New = tolower(trim(@"[ \t\n\r\v\x00]+", tostring(Data.changes[Option]['new'])))
| where (Option in (flags) and New !in (switchedOn))
    or Option in (routing) or tostring(Data.reason) == "other-spelling"
| summarize Options = make_set(Option)
    by EventId, EventName, TimeGenerated, Tenant, ActorUserName
| union (PxmLogExporter
    | where IngestionTime > ago(15m) and EventName == "exporter.purged"
    | project EventId, EventName, TimeGenerated, Tenant, ActorUserName)

A sink that is switched off, or sends or files its events elsewhere, does not deliver the event about that change where the rules look, so if the SIEM reads only that sink, the heartbeat rule catches it. A new row under an accented spelling of the module’s name arrives as config.option_changed, which this rule does not read.

Many failed logins from one network (SrcIpNetwork: an IPv4 /32 or an IPv6 /64), counted over the hour up to each failure that arrived since the previous run:

KQL
let recent = PxmLogExporter
    | where IngestionTime > ago(15m) and EventName == "auth.login.failure"
        and isnotempty(SrcIpNetwork)
    | summarize by SrcIpNetwork, At = bin(EventTime, 1m) + 1m;
PxmLogExporter
| where EventName == "auth.login.failure"
    and EventTime >= toscalar(recent | summarize min(At)) - 1h
| summarize arg_max(TimeGenerated, *) by EventId
| join kind=inner recent on SrcIpNetwork
| where EventTime between ((At - 1h) .. At)
| summarize Failures = sum(EventCount), Users = make_set(PresentedUserName, 20),
    Addresses = make_set(SrcIpAddr, 20) by SrcIpNetwork, At
| summarize arg_max(Failures, *) by SrcIpNetwork
| where Failures >= 20

A login after failed logins in the hour before it, counting the user’s failures and the capped failures (name_cap) from the login’s network:

KQL
let failures = PxmLogExporter
    | where EventName == "auth.login.failure"
    | summarize arg_max(TimeGenerated, *) by EventId
    | project FailedAt = EventTime, Failed = EventCount,
        FailedUser = tolower(PresentedUserName), FailedFrom = SrcIpNetwork,
        Capped = isnotnull(Data.name_cap);
let logins = PxmLogExporter
    | where IngestionTime > ago(15m) and EventName == "auth.login.success"
    | summarize arg_max(TimeGenerated, *) by EventId
    | project LoginAt = EventTime, User = tolower(ActorUserName), SrcIpAddr, SrcIpNetwork;
union
    (logins | join kind=inner (failures | where not(Capped))
        on $left.User == $right.FailedUser),
    (logins | join kind=inner (failures | where Capped and isnotempty(FailedFrom))
        on $left.SrcIpNetwork == $right.FailedFrom)
| where FailedAt between ((LoginAt - 1h) .. LoginAt)
| summarize Failures = sum(Failed), LastFailure = max(FailedAt)
    by User, LoginAt, SrcIpAddr
| where Failures >= 5

An administrator role assigned (name your administrator roles):

KQL
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "user.role_assigned"
| where tostring(Data.role_name) in ("Administrators")

A credential created, changed or deleted:

KQL
PxmLogExporter
| where IngestionTime > ago(15m) and EventName == "credential.changed"

Events of the HTTPS sink lost on the way, with Run query every set to 1 day and Lookup data from the last to 4 days; for the TLS stream sink, read stream instead of http:

KQL
let beats = PxmLogExporter
    | where EventName == "exporter.heartbeat"
    | project Tenant, Beat = EventTime, Arrived = IngestionTime,
        Sent = tolong(Data.sent.http), LastSeq = tolong(Data.last_seq.http)
    | partition hint.strategy=native by Tenant
        (sort by Beat asc | extend FromSeq = prev(LastSeq), SentBefore = prev(Sent))
    | where isnotnull(FromSeq) and LastSeq > FromSeq and Arrived > ago(1d);
beats
| join kind=leftouter (PxmLogExporter | where isnotnull(Seq) | distinct Tenant, Seq)
    on Tenant
| summarize Received = countif(Seq > FromSeq and Seq <= LastSeq)
    by Tenant, Beat, Expected = Sent - SentBefore
| where Received < Expected

Splunk

The searches read the sourcetype pxm:logexporter from siem/splunk/props.conf; add the index you use. The expected tenants are the lookup pxm_tenants.csv, with a column tenant. Schedule each search every 15 minutes with the time range earliest=-8d, unless it says otherwise, and widen the range to match when someone raises Keep undelivered events (hours) on the exporter’s Limits page, for example earliest=-15d for 336 hours, again with a day to spare. _index_earliest=-15m@m _index_latest=@m takes the events indexed since the previous run. The timestamp settings in props.conf apply to the TLS stream and to file inputs; the HTTP Event Collector takes the time each event carries.

No heartbeat for 30 minutes:

SPL
| inputlookup pxm_tenants.csv
| join type=left tenant
    [ search sourcetype=pxm:logexporter event="exporter.heartbeat" earliest=-2h
      | stats max(_time) AS last_heartbeat BY tenant ]
| where isnull(last_heartbeat) OR last_heartbeat < relative_time(now(), "-30m")

The exporter’s health:

SPL
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
  (event="exporter.sink.error" OR event="exporter.dropped"
  OR (event="exporter.heartbeat" (data.sinks.http="failing" OR data.sinks.stream="failing"
    OR data.plugins_loaded.event="false" OR data.plugins_loaded.authentication="false"
    OR data.plugins_loaded.rights="false" OR data.plugins_loaded.options="false")))

Changes that can silence the exporter, as described for Sentinel, including what the rule cannot see:

SPL
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
  (event="exporter.purged" OR (event="exporter.config.changed"
    ((data.options{}="Enabled"
        NOT data.changes.Enabled.new IN ("1", "true", "on", "yes"))
      OR (data.options{}="HttpEnabled"
        NOT data.changes.HttpEnabled.new IN ("1", "true", "on", "yes"))
      OR (data.options{}="StreamEnabled"
        NOT data.changes.StreamEnabled.new IN ("1", "true", "on", "yes"))
      OR (data.options{}="IncludeSupportLogins"
        NOT data.changes.IncludeSupportLogins.new IN ("1", "true", "on", "yes"))
      OR data.options{} IN ("EventGroups", "RecordClasses", "Environment", "UseOutgoingProxy",
        "HttpPreset", "HttpUrl", "HttpHeaders", "HttpBodyFormat", "SentinelDcrImmutableId",
        "SentinelStreamName", "SplunkHecToken", "SplunkIndex", "SplunkSourcetype",
        "EventHubsHubName", "StreamHost", "StreamPort", "StreamFraming",
        "SyslogFacility", "SyslogAppName")
      OR data.reason="other-spelling")))

Many failed logins from one network (src_network: an IPv4 /32 or an IPv6 /64), counted over the hour up to each of its failures indexed since the previous run:

SPL
sourcetype=pxm:logexporter event="auth.login.failure"
    [ search sourcetype=pxm:logexporter event="auth.login.failure"
        _index_earliest=-15m@m _index_latest=@m
      | dedup src_network | fields src_network ]
| dedup id
| sort 0 _time
| streamstats time_window=1h sum(data.count) AS failures,
    values(data.presented_user_name) AS users BY src_network
| where failures >= 20
    AND _indextime >= relative_time(now(), "-15m@m")
    AND _indextime < relative_time(now(), "@m")
| stats max(failures) AS failures, values(users) AS users BY src_network

A login after failed logins in the hour before it, counting the user’s failures and the capped failures (name_cap) from the login’s network; each login is reported once it is indexed:

SPL
sourcetype=pxm:logexporter (event="auth.login.failure" OR event="auth.login.success")
| dedup id
| eval who=lower(if(event="auth.login.failure", 'data.presented_user_name', user)),
    named=if(event="auth.login.failure" AND isnull('data.name_cap'), 'data.count', 0),
    capped=if(isnotnull('data.name_cap'), 'data.count', 0)
| sort 0 _time
| streamstats time_window=1h sum(named) AS named_failures BY who
| streamstats time_window=1h sum(capped) AS capped_failures BY src_network
| eval failures=coalesce(named_failures, 0) + coalesce(capped_failures, 0)
| where event="auth.login.success" AND failures >= 5
    AND _indextime >= relative_time(now(), "-15m@m")
    AND _indextime < relative_time(now(), "@m")
| table _time who src failures

An administrator role assigned, and a credential created, changed or deleted:

SPL
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
  event="user.role_assigned" data.role_name="Administrators"
SPL
sourcetype=pxm:logexporter _index_earliest=-15m@m _index_latest=@m
  event="credential.changed"

Centric PXM™ is a trademark of Centric Software, Inc.