CoursesAdvanced Linux securityHunting & detections

Threat hunting on Linux

Hypothesis-driven, proactive searching.

Advanced12 min · lesson 14 of 17

Detections are automated and reactive — they fire when a known-bad behaviour occurs. Threat hunting is the proactive complement: a human forms a hypothesis about how an attacker might be operating undetected, then goes looking through the data to prove or disprove it. Hunting exists because no detection set is complete; the techniques you have not automated, or the novel ones, are found by someone deliberately searching rather than waiting for an alert. It is how you find the intrusion your rules missed and, often, how you build the next rule.

A hunt starts with a specific, testable hypothesis grounded in attacker behaviour — not "let me look around" but "if an attacker established persistence via a systemd timer, there would be a recently-created timer unit pointing at an unusual path; let me query every host for that." You use the tools you built (osquery across the fleet, the off-host event store, the audit trail) to gather the relevant data, analyze it for the anomaly, and either find something (now an incident) or confirm the hypothesis is clean (now, ideally, a new automated detection so you never hunt it manually again).

The hunt loop
1hypothesis
how might they hide?
2gather
query fleet + event store
3analyze
anomaly vs baseline
4find or automate
incident, or new detection
A good hunt ends by turning its manual query into an automated detection — so the same technique is caught by machine next time.

Anomaly against a baseline

Most host hunting comes down to "what is unusual here?" — which requires knowing what usual looks like. A process making an outbound connection it never made before, a user logging in at an odd hour from a new location, a host running a binary no other host in its role runs, a parent-child process relationship that should not exist. The baselines you built for privesc and persistence detection double as hunting ground: deviations from normal are the leads. This is where knowing your own environment is a defender’s edge the attacker does not have.

Hunting is where offense knowledge pays off

Effective hunting draws directly on the first half of this course. You hunt for the escalation paths, the persistence locations, the stealth techniques, and the credential theft you studied — because you cannot hypothesize about attacker behaviour you do not understand. The privesc audit, the persistence enumeration, the rootkit discrepancy checks are all hunts you can run on demand and on a schedule. Offense-informed defence is the whole thesis: the more deeply you understand how the attack works, the better you hunt, detect, and respond.

Every hunt should leave you better off
A hunt that finds nothing is not wasted — but only if it ends in one of two outcomes: a new automated detection so the technique is caught by machine next time, or an improved baseline that makes the next hunt sharper. A hunt that finds nothing and changes nothing is just time spent. Close the loop: turn the manual query into a scheduled detection, document what "normal" looked like, and let each hunt compound your coverage rather than repeating it.