Skip to main content
For the complete documentation index, see llms.txt. Markdown versions are available by appending .md to page URLs.
Lookout is a local-first security observability system for private networks. It discovers the environment, collects security evidence from hosts, network systems, and services, evaluates which detections the available evidence can support, and produces evidence-backed Alerts and Incidents. It is designed for environments where security data cannot be treated as a generic cloud log stream. A deployment may contain Linux servers, Tailscale nodes, databases, SaaS services, mobile devices, appliances, and systems that provide only partial telemetry. Lookout represents all of them in one security model and shows where detection coverage is complete, degraded, or unavailable.

​
What Lookout does

A typical deployment performs the following work:
  1. Discovers assets, services, identities, network paths, and available telemetry sources.
  2. Installs and enrolls collectors on supported Linux endpoints using identity-pinned administrative access.
  3. Collects Linux journald and audit evidence, inventory, service state, listening sockets, identity metadata, and security-control health.
  4. Reads supported Tailscale inventory, configuration-audit, and network-flow data through read-only access.
  5. Accepts Zeek, syslog, OpenTelemetry, and service-specific records through normalizers.
  6. Builds a typed security graph and calculates detection coverage from actual capabilities.
  7. Evaluates deterministic rules and bounded behavioral baselines.
  8. Correlates findings into Alerts and Incidents with evidence, affected systems, confidence, and review state.
  9. Presents local state through the API and optionally synchronizes a raw-log-free summary to the SaaS console.
The result is not only a list of detected events. It is a current view of the environment, the evidence available for each system, the detections that can run, the gaps that limit them, and the findings produced by the evidence. The local deployment remains authoritative. SaaS synchronization is outbound-only, disabled by default, and limited to an explicit projection. Raw events and evidence remain local unless raw-event export is separately enabled.

​
System architecture

The implementation is a Node.js 20 application. The repository includes the local server, collectors, storage, detection engine, onboarding protocol, CLI, installer, API, and browser prototype.

​
Example workflow

Assume a new service starts listening on a Linux host. The recurring survey detects the listener and records its bind scope and service identity. The graph links the service to its host, network zone, owner, and available telemetry. The detection planner checks whether the required service-state, network, identity, and sensor-health capabilities are present. If the change matches an enabled rule, Lookout stores the finding, attaches the supporting evidence, applies Alert policy, and correlates it with related findings. If a required capability is missing, the detection plan reports the gap instead of reporting full coverage.

​
Security graph

The graph ontology includes:
  • Assets, services, software, identities, and credential metadata
  • Network zones, paths, exposures, and cloud resources
  • Telemetry sources and security controls
Relationships include runs, stores, administers, authenticates_to, reachable_from, observed_by, and protected_by. Adapters submit validated facts. Each fact contains provenance, observation time, confidence, and source ownership. Stable identifiers and deterministic sorting make graph materialization reproducible. Conflicting claims are resolved by confidence, observation time, and fact ID. Lower-priority claims remain available as provenance. The graph stores credential metadata and secret-manager references, never credential values.

​
Adapters and collection

The current production profile includes:
  • A read-only Tailscale adapter
  • A Linux endpoint collector
  • Passive network formats
  • Generic service ingestion
  • Explicit-declaration survey input
Supported source normalizers include:
  • Zeek
  • RFC 5424 syslog
  • OpenTelemetry logs
  • Tailscale network-flow and configuration-audit logs
  • Linux journald and audit records
The Linux survey runs repeatedly and records listening services and bind scope, systemd state, installed software, local identities and privilege indicators, SSH authorization metadata, mounted and conventionally sensitive resources, and observable host controls. It does not read credential contents. The first successful survey establishes drift state. Later changes, including new listeners, accounts, persistent services, privilege grants, credential metadata changes, stopped controls, and sensitive resources, become normalized events. The Tailscale collector uses read-only credentials, durable cursors, bounded polling windows, and inclusive-boundary deduplication. Network-flow collection depends on tailnet plan and logging configuration.

​
Event pipeline

All normalized events use a common envelope containing event time, ingest time, site, sensor, category, actor, source and target entities, action, outcome, and optional session, flow, or process correlation keys. Events are stored in an append-only JSONL journal. EventStore provides:
  • Schema validation
  • Deduplication
  • Monotonic sequence numbers
  • SHA-256 consistency chaining
  • Time, category, and entity queries
  • Retention compaction
Collectors submit signed envelopes using Ed25519 identities. The server validates the collector registration, signature, sequence, freshness, and replay state. An exact retry of the latest accepted envelope is idempotent. Failed collectors retain the signed envelope and retry with bounded backoff. The collector submits high-signal journald and audit events in bounded batches, plus inventory and health observations on a periodic interval. Missing permissions or unavailable sensors are reported as capability gaps.

​
Detection planning

Each behavior declares required entities, relationships, event fields, time windows, ordering requirements, and sensor capabilities. The planner evaluates those requirements against the current graph and collector health. Each analytic is reported as:
  • full: required evidence is available
  • partial: some evidence is available, but coverage is incomplete
  • gap: minimum evidence is unavailable
Lookout does not infer coverage from asset discovery, operating system, vendor, or network product. Unsupported, unreachable, and partially instrumented systems remain visible with explicit gaps.

​
Detection engine

Built-in deterministic rules support three types:
  • Event matching
  • Threshold evaluation over a bounded window
  • Ordered sequence matching over a bounded window
Rules are schema-validated and evaluated against normalized events. Built-in analytics include version metadata, severity, required capabilities, applicability predicates, and alert disposition. Imported Sigma-subset rules are validated, compiled, and persisted through the API. Detection content is cross-referenced against pinned MITRE ATT&CK, NIST CSF, CISA CPG, SigmaHQ, and vendor-library revisions. The references provide traceability and do not replace local capability checks.

​
Behavioral analytics

Behavioral analytics are explainable and bounded. Current models cover:
  • Identity to authentication target
  • Host to executable
  • Process to destination
  • Identity to privilege target
  • Identity to protected resource
  • Identity hour-of-week activity
  • Peer-group executable usage
  • Endpoint to network peer
  • Resource to data destination
  • Endpoint egress volume
Set baselines require repeated observations. Numeric baselines use median and median absolute deviation. Models have warm-up requirements, minimum sample counts, bounded update rates, schema versions, capacity limits, and contamination controls. Active Alert evidence is excluded from learning. Behavioral matches remain findings and correlation evidence. By default, only high- and critical-severity deterministic findings become standalone Alerts. Automatic Incident creation requires multiple distinct correlated findings, a shared entity chain, and strong high-confidence deterministic evidence. Anomaly-only findings do not automatically create Incidents.

​
Storage and security controls

The local deployment stores graph snapshots, events, detection state, cases, baselines, imported rules, collector state, export state, and a separate API audit journal. The API uses bearer credentials stored as SHA-256 hashes and maps principals to least-privilege roles, including console, viewer, ingestor, collector, analyst, rule_admin, and admin. Authenticated API activity is recorded in the audit journal. When enabled, storage encryption uses AES-256-GCM. Encrypted backups support graph, event, audit, detection, collector, and export state. Files are created in owner-only directories with restrictive permissions. Snapshot digests and journal chains detect corruption. Key custody and host hardening remain deployment responsibilities.

​
Onboarding and fleet installation

SaaS onboarding uses a provider-neutral HTTPS setup-session protocol:
  1. The authenticated browser creates a short-lived single-use setup token bound to its trusted tenant and user, then embeds it in the agent prompt.
  2. A lightweight bootstrap reports Connected before downloading the release, without consuming the claim token.
  3. The current agent terminal performs parallel authoritative read-only provider discovery and selects a central Linux VM deterministically.
  4. The installer claims the exact discovered scope and starts provider-native remote deployment without requiring a scope file or first connecting to a VM.
  5. The installer consumes the token with its Ed25519 deployment identity and the approved scope. The service returns the exact normalized scope.
  6. The installer generates a separate temporary restricted SSH key for fallback access only.
  7. The service releases provisioning only after deployment-identity signature verification.
  8. The fleet installer probes, enrolls, surveys, configures, and validates only VMs from the returned scope.
  9. The installer removes the exact temporary SSH authorization from reached hosts, and the browser redirects automatically.
  10. The dashboard prompts the operator to connect alert notifications.
  11. Missing central heartbeats create dashboard and email notifications plus a single-use recovery token. Recovery retains the deployment identity and forces collector re-enrollment.
The setup token is never placed in a URL or process argument. Long-lived deployment credentials are stored through the platform secret mechanism. The installer uses pinned host identities and pre-authorized administrative transport. Discovery alone never authorizes remote deployment, and the installer cannot expand the approved scope. The supported Linux installer provisions private TLS, per-endpoint enrollment credentials, locked service accounts, encrypted local storage, signed collectors, hardened systemd services, heartbeat verification, and rollback metadata. Fleet and host-local modes are available.

​
Console and export boundary

The versioned API exposes graph, console snapshot, detection plan, behaviors, rules, events, collector submissions, Alerts, and Incidents. The SaaS console receives a compact, raw-log-free projection containing topology, coverage, Alert and Incident summaries, and sensor health. Console sync uses a durable HTTPS outbox, bounded batches, stable idempotency keys, and retry handling. Cloud export is a separate opt-in policy. It filters records before queueing, removes non-allowlisted attributes, and uses durable retry. Permanent delivery errors enter a blocked state. Local ingestion and detection continue when export fails. Alert webhooks are independent of raw-event export. They deliver Alert summaries through HTTPS using an allowlisted credential reference, durable outbox, idempotency keys, cooldown handling, and bounded retry.

​
Validation

lookout validate-detection-pipeline validates every built-in deterministic rule using isolated state, production normalizers, capability planning, durable ingestion, detection, and Alert policy. Each malicious scenario includes a benign near-miss. validate-live-linux performs controlled SSH and ephemeral-listener actions against an enrolled collector and verifies a fresh Alert through the installed collector-to-server path. It does not modify accounts, credentials, installed services, routes, security controls, existing logs, backups, MFA, or cloud resources. Pipeline validation does not prove complete production coverage. The detection plan and sensor health remain the source of truth for whether required evidence is available in a deployment.

​
Scope

Lookout is security detection and observability infrastructure. It does not perform automated response and does not replace endpoint, network, identity, or cloud hardening. Platform-specific collectors for macOS, Windows, mobile devices, and appliances are not yet complete. These systems can remain in the security graph with partial evidence and explicit capability gaps.

​
Run locally

Development requires Node.js 20 or later:
Useful commands: