CoursesAdvanced Linux securityDetection engineering

osquery for fleet visibility

System state as SQL you can query.

Advanced12 min · lesson 10 of 17

osquery is a tool that exposes the state of a Linux host as a set of SQL tables — running processes, listening ports, installed packages, users, cron jobs, kernel modules, SUID binaries, and dozens more — so you can ask questions about a machine, or a whole fleet, in plain SQL. Instead of scripting an enumeration and parsing text, you write SELECT * FROM listening_ports and get structured answers, the same way across every host. It turns the ad-hoc audit commands from earlier lessons into repeatable, fleet-wide queries.

terminal
$ osqueryi # interactive shell
osquery> SELECT pid, name, path, uid FROM processes WHERE uid = 0; # root processes
osquery> SELECT * FROM listening_ports WHERE address = '0.0.0.0'; # exposed listeners
osquery> SELECT path, permissions FROM suid_bin; # SUID binaries, fleet-wide
osquery> SELECT * FROM crontab; # scheduled tasks

Scheduled queries and change detection

The real power is the scheduled query and its "diff" mode: osquery runs a query on an interval and reports what changed since last time — new listening ports, new SUID binaries, added users, new kernel modules. This is exactly the baseline-and-diff detection the privesc and persistence lessons called for, delivered as a product feature across the whole fleet. A scheduled query for new SUID binaries turns "attacker planted a backdoor" into a diff event in your logs, automatically, everywhere.

osquery.conf
{
"schedule": {
"new_suid": {
"query": "SELECT * FROM suid_bin;",
"interval": 3600, "snapshot": false // report ADDED/REMOVED rows only
},
"new_listeners": {
"query": "SELECT DISTINCT address, port, name FROM listening_ports lp JOIN processes p ON lp.pid=p.pid;",
"interval": 600
}
}
}

Fleet-wide answers during an incident

When a threat is disclosed — a malicious package, an indicator of compromise, a persistence technique — osquery answers "are we affected, and where?" across thousands of hosts in one query, the fleet-scale version of the SBOM zero-day drill. "Which hosts have this file, this process, this kernel module, this user?" becomes a single distributed query instead of a per-host scramble. That ability to interrogate the whole fleet on demand is what makes osquery a hunting and incident-response tool, not just a monitoring one.

osquery reads through the same kernel a rootkit controls
osquery is powerful and it is not magic: it gathers most of its data through the same kernel and userland a sophisticated rootkit can subvert, so a kernel-rootkitted host can hide from osquery too. Use it as your primary broad visibility — it catches the overwhelming majority of real activity and misconfiguration — but for the deepest threats, corroborate with independent sources (network sensors, off-host telemetry, memory forensics). No single agent on a compromised host is the whole truth.