Docs Backend

Logs and log patterns

Collect journal units and log files from every host, search and filter by level, and decide what to store with log patterns that drop noise at the collector.

The host agent reads two kinds of source: systemd journal units, and log files. Nothing is collected until you choose it. The agent finds and offers sources, it does not gather them by itself.

The logs page with level filters and search — Backend → Logs: every collected line, filtered by level and searched in place.

Where the agent looks

systemd units, read from the journal with a cursor.
System logs under /var/log.
PM2 application logs in PM2's log directories, which are never under /var/log.
Deploy directories: a logs/ or log/ folder under /opt, /srv, /var/www and home directories.

The search is not a walk of the filesystem. The roots are listed and checked for existence, and symbolic links are not followed. Rotated and compressed copies are skipped. To see what a machine would offer, run this on it:

node /opt/obscenter-agent/discover.cjs --sources

On Windows the agent reads the event log channels you name for that machine.

Worth switching on

A stock Ubuntu machine offers dozens of files and hundreds of units, and most of it is housekeeping. Each source is therefore classed as app, web, system or unit, with its size and when it was last written. Sources written in the last day that look like your own software go in a Worth switching on box at the top, with one button to collect them all. Security logs such as auth.log are always suggested, because a quiet auth log is the good case. System logs sit in a folded group below. You switch sources on and off on the host's settings page.

Reading logs

Backend → Logs has two tabs, Lines and Patterns & storage. On Lines, filter by level: all, warnings and worse, errors and worse, or critical. Lines with no level are counted separately. You can also search the text. A badge says which store answered the search. One host's page shows its own lines too. When the setting to keep what a request logged is on, the log lines a traced request wrote also appear in its trace.

Log patterns

Most log volume is the same few sentences with different numbers in them. The Patterns & storage tab reduces each line to its pattern: the sentence with numbers, ids, addresses and times replaced by <*>. For each pattern it shows its kind, how many lines it had, how many were not stored, a sample, a proposal and your verdict. The kinds are error, security, access, health, debug, periodic, lifecycle and other.

Choose a mode: Store everything, except what I drop (the default), or Store only what I choose to store.
Noise is proposed for dropping, and errors, security and access lines for storing. Mark each pattern keep or drop.
keep every line at error level or worse is on by default, whatever a pattern says.
SSH and auth lines, and access-log GET lines, are always stored, because the security pages read them.

Dropping happens at the collector. The agent still sends the line, and the collector counts it and keeps one sample per pattern per day. A pattern you drop by mistake can therefore be judged later, and you can see what dropping it saved. Changing verdicts is for administrators.

How long logs are kept

Two limits apply to the whole install, and the tighter one wins. Keep logs for (days) is 7 by default, from 1 to 365. Never more than (MB) is 1024 by default. Both are set on a host's settings page. A whole-install ceiling across all data can also be set. See Limits.

Logs and security

Collected logs feed three security views. auth.log or the SSH journal feeds SSH logins. nginx access logs feed the map of who is scanning your servers. The Windows Security log feeds logons. See SSH logins, Windows logons and scanning.