Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c1a05d6dba | ||
|
|
fccf902c07 | ||
|
|
f5a2811333 | ||
|
|
98d1dbef4c | ||
|
|
252ea3b81c | ||
|
|
b685b200cc | ||
|
|
8557710724 | ||
|
|
1b600416f0 | ||
|
|
e622b26d42 | ||
|
|
5daba5d37c | ||
|
|
dacd10fcce | ||
|
|
8a54674e40 | ||
|
|
d45455adf1 | ||
|
|
ac49e2b5d4 |
No files matched your search
@@ -7,6 +7,46 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.31.1] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
- `get_log_sources()` names as a file's writer only a process that holds it open for writing.
|
||||
A log reader -- CrowdSec, Promtail, rsyslog's `imfile`, a `tail -F` -- was given the file
|
||||
it reads as if it wrote it. A file only readers hold is now listed without a producer
|
||||
(`unit` and `process` None); one only a reader inside a container holds is left out, as its
|
||||
path is that container's view of a mount. The access mode comes from
|
||||
`/proc/<pid>/fdinfo/<fd>`, read in the same command (NetOrk/netork#2092).
|
||||
|
||||
## [4.31.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
|
||||
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
|
||||
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
|
||||
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
|
||||
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
|
||||
public (NetOrk/netork#2053).
|
||||
|
||||
## [4.30.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LogSourcesMixin.get_log_sources()`: every log a Linux host writes. Sources are the units
|
||||
journald holds entries for, the log files running processes hold open (with size, last write
|
||||
and the service or container writing them), and the engine's file of each container's output.
|
||||
A file a container writes is named by the container's own path. `LOG_SOURCES_COMMAND` and
|
||||
`parse_log_sources` are public, and a host whose `find` cannot list open files (busybox)
|
||||
raises `LogSourcesUnavailable`. A driver supplies `_run_log_sources_command(command,
|
||||
privileged=)`, as for listening sockets (NetOrk/netork#2049).
|
||||
|
||||
## [4.29.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `remove_local_account_keys(name, keys, *, clear_password=False)`: with `clear_password`
|
||||
the account's password goes along with the keys, and so do its sudo rule and admin
|
||||
groups, as sudo asks for that password. For a person who deleted their key and whose
|
||||
password went with it. The account stays open; the steps are `keys`, `password` and
|
||||
`sudo`, each run when another failed (NetOrk/netork#2033).
|
||||
|
||||
## [4.28.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
|
||||
@@ -191,6 +191,23 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
`LogSourcesMixin` (`get_log_sources`) is mixed in the same way: every log the host writes, in
|
||||
one round trip. The units journald holds entries for (`journalctl -F _SYSTEMD_UNIT`, services
|
||||
only), the log files running processes hold open (`/proc/<pid>/fd`: `*.log`, `*.log.N` and
|
||||
anything under `/var/log`) with size and last write from `stat -L` on the descriptor, and the
|
||||
engine's own file of a container's output (`/var/lib/docker/containers/<id>/…`) as that
|
||||
container's stream. Each file names the service or container of its first writer from
|
||||
`/proc/<pid>/cgroup`; a file a container writes carries the container's own path. journald's
|
||||
store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) are left out; a
|
||||
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
|
||||
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
|
||||
user's processes show their files, and the reading says so (`attributed: false`).
|
||||
The same round trip reads where the logs go beyond the host (`forwarding`,
|
||||
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
|
||||
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
|
||||
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
|
||||
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
|
||||
@@ -50,6 +50,7 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
|
||||
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
|
||||
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
|
||||
* :class:`~napalm_device_types.log_sources.LogSourcesMixin`
|
||||
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
|
||||
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
@@ -166,6 +167,21 @@ from napalm_device_types.local_accounts import (
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.log_forwarding import (
|
||||
FileInputDict,
|
||||
ForwardRuleDict,
|
||||
LogForwardingDict,
|
||||
parse_rsyslog,
|
||||
)
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourceDict,
|
||||
LogSourcesDict,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.management_controller import ManagementControllerDriver
|
||||
from napalm_device_types.media import MediaDriver
|
||||
@@ -398,6 +414,17 @@ __all__ = [
|
||||
"parse_host_firewall",
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"LOG_SOURCES_COMMAND",
|
||||
"FileInputDict",
|
||||
"ForwardRuleDict",
|
||||
"LogForwardingDict",
|
||||
"parse_log_forwarding",
|
||||
"parse_rsyslog",
|
||||
"LogSourceDict",
|
||||
"LogSourcesDict",
|
||||
"LogSourcesMixin",
|
||||
"LogSourcesUnavailable",
|
||||
"parse_log_sources",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
|
||||
@@ -292,7 +292,8 @@ class LocalAccountsMixin:
|
||||
|
||||
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
|
||||
and not a lock or an expiry, which sshd or PAM apply to a key too.
|
||||
Only ``add_local_account(clear_password=True)`` asks for it.
|
||||
Only ``add_local_account(clear_password=True)`` and
|
||||
``remove_local_account_keys(clear_password=True)`` ask for it.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
@@ -424,15 +425,21 @@ class LocalAccountsMixin:
|
||||
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
|
||||
return steps
|
||||
|
||||
def remove_local_account_keys(self, name: str, keys: Sequence[str]) -> LocalAccountChangeDict:
|
||||
def remove_local_account_keys(
|
||||
self, name: str, keys: Sequence[str], *, clear_password: bool = False
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Take *keys* off *name*'s account, for a person who deleted them.
|
||||
|
||||
Only those keys go, matched by their base64 whatever a line's options
|
||||
or comment; the other keys, the password and sudo stay, and the
|
||||
account stays open -- to get a new key once there is one. A locked
|
||||
account has none left to remove: the lock took them all.
|
||||
or comment; the other keys stay, and the account stays open -- to get
|
||||
a new key once there is one. A locked account has none left to
|
||||
remove: the lock took them all. Without *clear_password* the password
|
||||
and sudo stay too; with it the password goes, and so do the sudo rule
|
||||
and :attr:`ACCOUNT_ADMIN_GROUPS`, as sudo asks for that password -- for
|
||||
a person whose password went with their key.
|
||||
|
||||
Step: ``keys``.
|
||||
Steps: ``keys``, then ``password`` and ``sudo`` (with *clear_password*),
|
||||
each of which runs when another failed.
|
||||
|
||||
:raises ValueError: for a name no host takes, no keys or one that is
|
||||
none, the login user's own account, or a system account.
|
||||
@@ -444,8 +451,11 @@ class LocalAccountsMixin:
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
parts = [run_fix_steps([("keys", lambda: self._accounts_drop_keys(name, wanted))])]
|
||||
return self._accounts_result(name, parts)
|
||||
steps: list[FixStep] = [("keys", lambda: self._accounts_drop_keys(name, wanted))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("sudo", lambda: self._accounts_set_sudo(name, admin=False)))
|
||||
return self._accounts_result(name, [run_fix_steps([step]) for step in steps])
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
|
||||
in, and whether it gets the journal (NetOrk/netork#2053).
|
||||
|
||||
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
|
||||
this says who reads it on. It is read in the same command, so a reading of
|
||||
the logs a host writes also says which of them leave it.
|
||||
|
||||
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
|
||||
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
|
||||
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
|
||||
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
|
||||
*complete* when it takes every message of its ruleset -- ``*.*`` with no
|
||||
condition around it. Anything narrower is partial, and keeps its selector or
|
||||
condition for the reader to see: which facility a unit logs to is not
|
||||
computed (decided in #2053).
|
||||
|
||||
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
|
||||
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
|
||||
own a host runs on its distribution's default, which ``journald.conf`` ships
|
||||
commented out: ``#ForwardToSyslog=yes``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class ForwardRuleDict(TypedDict):
|
||||
"""One rule sending messages to another host."""
|
||||
|
||||
protocol: str # "udp", "tcp" or "relp"
|
||||
target: str # the host, as configured
|
||||
port: int | None # None: the protocol's default
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
|
||||
|
||||
path: str # as configured: may hold a glob
|
||||
ruleset: str | None
|
||||
|
||||
|
||||
class RsyslogDict(TypedDict):
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
imjournal: bool
|
||||
imuxsock: bool
|
||||
|
||||
|
||||
class LogForwardingDict(TypedDict):
|
||||
"""Where a host's logs go beyond it.
|
||||
|
||||
``journal`` is None when it cannot be told whether the journal reaches
|
||||
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
|
||||
"""
|
||||
|
||||
rsyslog: bool # rsyslogd runs
|
||||
journal: bool | None # the journal reaches rsyslog
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
|
||||
|
||||
#: Selectors that take every message.
|
||||
_EVERYTHING = {"*.*", "*.debug"}
|
||||
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
|
||||
_CALL_RE = re.compile(
|
||||
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
|
||||
)
|
||||
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
|
||||
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
|
||||
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
|
||||
_FORWARD_RE = re.compile(
|
||||
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
|
||||
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
|
||||
)
|
||||
_FORWARD_TYPES = {"omfwd", "omrelp"}
|
||||
_TRUE = {"yes", "true", "1", "on"}
|
||||
|
||||
|
||||
def _uncomment(text: str) -> str:
|
||||
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
|
||||
out: list[str] = []
|
||||
quoted = commented = False
|
||||
for char in text:
|
||||
if char == "\n":
|
||||
quoted = commented = False
|
||||
elif commented:
|
||||
continue
|
||||
elif char == '"':
|
||||
quoted = not quoted
|
||||
elif char == "#" and not quoted:
|
||||
commented = True
|
||||
continue
|
||||
out.append(char)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
class _Pieces:
|
||||
"""Cuts a configuration into statements, ``{`` and ``}``: a line runs on
|
||||
while a bracket is open, and a brace outside one stands as a piece of its own."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.pieces: list[str] = []
|
||||
self.current: list[str] = []
|
||||
self.depth = 0
|
||||
self.quoted = False
|
||||
|
||||
def close(self) -> None:
|
||||
piece = "".join(self.current).strip()
|
||||
if piece:
|
||||
self.pieces.append(piece)
|
||||
self.current.clear()
|
||||
|
||||
def bracket(self, char: str) -> None:
|
||||
if char in "([":
|
||||
self.depth += 1
|
||||
elif char in ")]":
|
||||
self.depth = max(self.depth - 1, 0)
|
||||
|
||||
def feed(self, char: str) -> None:
|
||||
if char == '"':
|
||||
self.quoted = not self.quoted
|
||||
elif not self.quoted:
|
||||
self.bracket(char)
|
||||
if self.depth == 0 and char in "{}\n":
|
||||
self.close()
|
||||
if char != "\n":
|
||||
self.pieces.append(char)
|
||||
return
|
||||
self.current.append(char)
|
||||
|
||||
|
||||
def _pieces(text: str) -> list[str]:
|
||||
"""Statements, ``{`` and ``}`` in order (:class:`_Pieces`)."""
|
||||
cutter = _Pieces()
|
||||
for char in text:
|
||||
cutter.feed(char)
|
||||
cutter.close()
|
||||
return cutter.pieces
|
||||
|
||||
|
||||
def _params(args: str) -> dict[str, str]:
|
||||
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
|
||||
|
||||
|
||||
def _port(value: str | None) -> int | None:
|
||||
return int(value) if value and value.isdigit() else None
|
||||
|
||||
|
||||
class _Reader:
|
||||
"""Walks the pieces of a configuration, keeping the blocks it is in."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.parsed: RsyslogDict = {
|
||||
"rules": [],
|
||||
"file_inputs": [],
|
||||
"imjournal": False,
|
||||
"imuxsock": False,
|
||||
}
|
||||
self.stack: list[tuple[str, str | None]] = []
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
return names[-1] if names else self.legacy_ruleset
|
||||
|
||||
def condition(self, once: str | None) -> str | None:
|
||||
conditions = [value for kind, value in self.stack if kind == "if"]
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
{
|
||||
"protocol": protocol,
|
||||
"target": target,
|
||||
"port": port,
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
}
|
||||
)
|
||||
|
||||
def piece(self, piece: str) -> None:
|
||||
if piece == "{":
|
||||
self.stack.append(self.pending or ("block", None))
|
||||
self.pending = None
|
||||
return
|
||||
if piece == "}":
|
||||
if self.stack:
|
||||
self.stack.pop()
|
||||
return
|
||||
if piece == "else":
|
||||
self.pending = ("if", "else")
|
||||
return
|
||||
# A condition whose action follows on its own: the action is under it.
|
||||
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
|
||||
self.pending = None
|
||||
match = _IF_RE.match(piece)
|
||||
if match:
|
||||
if match["rest"]:
|
||||
self.statement(match["rest"], once=match["cond"])
|
||||
else:
|
||||
self.pending = ("if", match["cond"])
|
||||
return
|
||||
self.statement(piece, once=once)
|
||||
|
||||
def statement(self, text: str, once: str | None) -> None:
|
||||
call = _CALL_RE.match(text)
|
||||
if call:
|
||||
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
|
||||
elif text.startswith("$"):
|
||||
self.directive(text)
|
||||
else:
|
||||
self.legacy(text, once)
|
||||
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
self.parsed["file_inputs"].append(
|
||||
{"path": params["file"], "ruleset": params.get("ruleset")}
|
||||
)
|
||||
elif name == "action":
|
||||
self.action(params, before, once)
|
||||
|
||||
def action(self, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
selector = self.last_selector if before == "&" else (before or "*.*")
|
||||
self.last_selector = selector
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
self.parsed["imjournal"] = True
|
||||
elif name == "imuxsock":
|
||||
self.parsed["imuxsock"] = True
|
||||
|
||||
def directive(self, text: str) -> None:
|
||||
name, _, value = text.partition(" ")
|
||||
name, value = name.lower(), value.strip()
|
||||
if name == "$modload":
|
||||
self.module(value)
|
||||
elif name == "$inputfilename" and value:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
if not match:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
forward = _FORWARD_RE.match(match["action"].strip())
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
|
||||
)
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
read for its forwarding rules, its file inputs, and the inputs that take
|
||||
the journal."""
|
||||
reader = _Reader()
|
||||
for piece in _pieces(_uncomment(text)):
|
||||
reader.piece(piece)
|
||||
return reader.parsed
|
||||
|
||||
|
||||
def journal_to_syslog(lines: list[str]) -> bool | None:
|
||||
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
|
||||
the order journald reads them: the last setting wins; with none, the
|
||||
default the distribution ships commented out; None when nothing says."""
|
||||
setting = shipped = None
|
||||
for line in lines:
|
||||
commented = line.lstrip().startswith("#")
|
||||
_, _, value = line.lstrip("# \t").partition("=")
|
||||
if commented:
|
||||
shipped = value.strip().lower() in _TRUE
|
||||
else:
|
||||
setting = value.strip().lower() in _TRUE
|
||||
return setting if setting is not None else shipped
|
||||
|
||||
|
||||
def log_forwarding(
|
||||
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
|
||||
) -> LogForwardingDict:
|
||||
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
|
||||
if not running:
|
||||
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
journal: bool | None
|
||||
if rsyslog["imjournal"]:
|
||||
journal = True
|
||||
elif rsyslog["imuxsock"]:
|
||||
journal = journal_to_syslog(journald_lines)
|
||||
else:
|
||||
journal = False
|
||||
return {
|
||||
"rsyslog": True,
|
||||
"journal": journal,
|
||||
"rules": rsyslog["rules"],
|
||||
"file_inputs": rsyslog["file_inputs"],
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
"""Log sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person, CrowdSec, a syslog
|
||||
forwarder -- it has to know which logs there are. That is read the same way on
|
||||
every Linux host, so the command and its parse live here once, and a driver
|
||||
only carries the command across (NetOrk/netork#2049).
|
||||
|
||||
**Three kinds.** journald names every unit it holds entries for
|
||||
(``journalctl -F _SYSTEMD_UNIT``); only services count, not login sessions.
|
||||
``/proc/<pid>/fd`` shows the files running processes hold open, a log being a
|
||||
``*.log`` / ``*.log.N`` file or anything under ``/var/log``; ``stat -L`` on the
|
||||
descriptor gives its size and last write, also for a file inside a container,
|
||||
whose path is the container's own. The engine's own file of a container's
|
||||
output (``/var/lib/docker/containers/<id>/…``) is that container's stream, not
|
||||
a file of ``dockerd``.
|
||||
|
||||
**Who writes it.** Only a process holding the file open for writing writes
|
||||
it: the access mode in ``/proc/<pid>/fdinfo/<fd>``'s ``flags`` is ``O_WRONLY`` or
|
||||
``O_RDWR``. A reader -- CrowdSec, Promtail, rsyslog's ``imfile``, a ``tail -F``
|
||||
-- holds it too, and is no producer (NetOrk/netork#2092). ``/proc/<pid>/cgroup``
|
||||
says which systemd service or container the first writer runs in, as for
|
||||
listening sockets (:mod:`napalm_device_types.listening`). A file only readers
|
||||
hold is listed without a producer; one only readers in a container hold is left
|
||||
out, its path being the reader's view of a mount. journald's own files, files
|
||||
already deleted and a database's write-ahead log (LevelDB's ``000004.log``) are
|
||||
left out.
|
||||
|
||||
**Where they go** is read in the same round trip
|
||||
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
|
||||
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
|
||||
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
|
||||
reads them.
|
||||
|
||||
**One round trip, root, and without it.** As for listening sockets the script
|
||||
goes as one ``sh -c`` argument. Without root only the login user's processes
|
||||
show their files; the command then runs again unprivileged, and the reading
|
||||
says it is not ``attributed``. A report whose end is missing raises: a list cut
|
||||
short must never read as logs that went away, and so must a host whose
|
||||
``find`` cannot list open files (busybox): it raises
|
||||
:class:`LogSourcesUnavailable`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
|
||||
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class LogSourceDict(TypedDict):
|
||||
"""A log the host writes: a unit's journal, a file a process holds open, or a
|
||||
container's stream as the engine keeps it.
|
||||
|
||||
``path`` is as its writer sees it: a file a container writes is named inside
|
||||
that container. Whether anything reads the log is the consumer's question.
|
||||
"""
|
||||
|
||||
kind: str # "journald", "file" or "container"
|
||||
path: str | None # the file; None for a journal
|
||||
unit: str | None # the journal's unit, or the file writer's service; without ".service"
|
||||
container_id: str | None # the full container ID: the writer's, or the stream's
|
||||
process: str | None # the first process writing the file; None when only readers hold it
|
||||
size: int | None # bytes; None for a journal, or when the file could not be read
|
||||
modified: str | None # the last write, ISO 8601 UTC; None as for size
|
||||
|
||||
|
||||
class LogSourcesDict(TypedDict):
|
||||
"""What ``LogSourcesMixin.get_log_sources`` read.
|
||||
|
||||
``attributed`` is False when the reading ran without root: only the login
|
||||
user's own processes show their files then, so a missing log means "not
|
||||
told", not "not there". ``journald`` is False on a host without systemd's
|
||||
journal, where no unit has one.
|
||||
"""
|
||||
|
||||
attributed: bool
|
||||
journald: bool
|
||||
sources: list[LogSourceDict]
|
||||
forwarding: LogForwardingDict | None # where the logs go beyond the host
|
||||
|
||||
|
||||
_BEGIN = "LOGS_BEGIN"
|
||||
_END = "LOGS_END"
|
||||
|
||||
#: One line, POSIX ``sh`` with GNU ``find`` (``-lname``, ``-printf``), read-only.
|
||||
#: The frame markers are printed in two halves so that a transport which echoes
|
||||
#: the command does not show them early. ``-printf '%p %l'`` puts the descriptor
|
||||
#: before its target, so a target with a space in it stays whole.
|
||||
LOG_SOURCES_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' LOGS_ BEGIN; "
|
||||
"if [ -d /run/systemd/system ] && command -v journalctl >/dev/null 2>&1; then "
|
||||
"j=journalctl; command -v timeout >/dev/null 2>&1 && j='timeout 30 journalctl'; "
|
||||
"echo '[journald]'; $j -F _SYSTEMD_UNIT 2>/dev/null; fi; "
|
||||
"if find / -maxdepth 0 -lname x -printf '' >/dev/null 2>&1; then "
|
||||
"o=$(find /proc/[0-9]*/fd -mindepth 1 -maxdepth 1 -type l "
|
||||
"\\( -lname '*.log' -o -lname '*.log.[0-9]*' -o -lname '/var/log/*' \\) "
|
||||
"-printf '%p %l\\n' 2>/dev/null); "
|
||||
"echo '[open]'; printf '%s\\n' \"$o\"; "
|
||||
"echo '[stat]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 "
|
||||
"| xargs -r stat -L -c '%n %s %Y' 2>/dev/null; "
|
||||
"echo '[flags]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 | sed 's|/fd/|/fdinfo/|' "
|
||||
"| xargs -r grep -H '^flags:' 2>/dev/null; "
|
||||
"pids=$(printf '%s\\n' \"$o\" | cut -d/ -f3 | sort -un); "
|
||||
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
|
||||
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
|
||||
"else echo '[no-find]'; fi; "
|
||||
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
|
||||
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
|
||||
"| sed 's/^/| /'; "
|
||||
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
|
||||
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
|
||||
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
|
||||
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
|
||||
"printf '%s%s\\n' LOGS_ END"
|
||||
)
|
||||
|
||||
#: ``/proc/900/fd/5`` -- the process and its descriptor.
|
||||
_FD_RE = re.compile(r"^/proc/(\d+)/fd/\d+$")
|
||||
#: ``/proc/900/fdinfo/5:flags:<tab>0102001`` -- the descriptor's open flags, octal.
|
||||
_FLAGS_RE = re.compile(r"^/proc/(\d+)/fdinfo/(\d+):flags:\s*([0-7]+)$")
|
||||
#: The access mode bits of the open flags: 0 is O_RDONLY, 1 O_WRONLY, 2 O_RDWR.
|
||||
_ACCESS_MODE = 0o3
|
||||
#: The engine's file of a container's output: json-file and local drivers.
|
||||
_STREAM_RE = re.compile(r"^/var/lib/docker/containers/([0-9a-f]{64})/")
|
||||
#: Files that are no log to read: journald's store, a file already deleted, and
|
||||
#: the write-ahead log LevelDB and RocksDB name ``000004.log``.
|
||||
_NOT_A_SOURCE = re.compile(r"^/var/log/journal/|\.journal~?$| \(deleted\)$|/\d{6}\.log$")
|
||||
_SERVICE_SUFFIX = ".service"
|
||||
_NO_FIND = "no-find"
|
||||
|
||||
|
||||
class LogSourcesUnavailable(NotImplementedError):
|
||||
"""The host's ``find`` cannot list open files (busybox); nothing to read, nothing to retry."""
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact log source report in the output") from None
|
||||
return [line for line in lines[start + 1 : end] if line]
|
||||
|
||||
|
||||
def _stats(lines: list[str]) -> dict[str, tuple[int, str]]:
|
||||
"""Each descriptor's size and last write: ``/proc/900/fd/5 2048 1762250200``."""
|
||||
stats: dict[str, tuple[int, str]] = {}
|
||||
for line in lines:
|
||||
fd, _, rest = line.partition(" ")
|
||||
size, _, mtime = rest.partition(" ")
|
||||
if size.isdigit() and mtime.isdigit():
|
||||
stats[fd] = (int(size), datetime.fromtimestamp(int(mtime), UTC).isoformat())
|
||||
return stats
|
||||
|
||||
|
||||
def _names(lines: list[str]) -> dict[int, str]:
|
||||
names: dict[int, str] = {}
|
||||
for line in lines:
|
||||
pid, _, name = line.partition(" ")
|
||||
if pid.isdigit() and name:
|
||||
names[int(pid)] = name
|
||||
return names
|
||||
|
||||
|
||||
def _journals(units: list[str]) -> list[LogSourceDict]:
|
||||
services = {u[: -len(_SERVICE_SUFFIX)] for u in units if u.endswith(_SERVICE_SUFFIX)}
|
||||
return [
|
||||
{
|
||||
"kind": "journald",
|
||||
"path": None,
|
||||
"unit": unit,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": None,
|
||||
"modified": None,
|
||||
}
|
||||
for unit in sorted(services)
|
||||
]
|
||||
|
||||
|
||||
def _held(lines: list[str]) -> list[tuple[int, str, str]]:
|
||||
"""(pid, descriptor, target) for every log file a process holds, by pid."""
|
||||
held = []
|
||||
for line in lines:
|
||||
fd, _, target = line.partition(" ")
|
||||
match = _FD_RE.match(fd)
|
||||
if match and target.startswith("/") and not _NOT_A_SOURCE.search(target):
|
||||
held.append((int(match.group(1)), fd, target))
|
||||
return sorted(held)
|
||||
|
||||
|
||||
def _writes(lines: list[str]) -> dict[str, bool]:
|
||||
"""Whether each descriptor is open for writing, by its ``/proc/<pid>/fd/<fd>``."""
|
||||
writes: dict[str, bool] = {}
|
||||
for line in lines:
|
||||
match = _FLAGS_RE.match(line)
|
||||
if match:
|
||||
pid, fd, flags = match.groups()
|
||||
writes[f"/proc/{pid}/fd/{fd}"] = int(flags, 8) & _ACCESS_MODE != 0
|
||||
return writes
|
||||
|
||||
|
||||
_Key = tuple[str, str | None, str]
|
||||
|
||||
|
||||
def _holders(lines: list[str], paths: dict[int, str]) -> dict[_Key, list[tuple[int, str]]]:
|
||||
"""The (pid, descriptor) pairs holding each log, by kind, container and path."""
|
||||
holders: dict[_Key, list[tuple[int, str]]] = {}
|
||||
for pid, fd, target in _held(lines):
|
||||
stream = _STREAM_RE.match(target)
|
||||
kind, container = (
|
||||
("container", stream.group(1)) if stream else ("file", _container(paths.get(pid)))
|
||||
)
|
||||
holders.setdefault((kind, container, target), []).append((pid, fd))
|
||||
return holders
|
||||
|
||||
|
||||
def _writer(held: list[tuple[int, str]], writes: dict[str, bool] | None) -> int | None:
|
||||
"""The first process writing; the first holding for a report without flags,
|
||||
as an older command printed it."""
|
||||
if writes is None:
|
||||
return held[0][0]
|
||||
return next((pid for pid, fd in held if writes.get(fd)), None)
|
||||
|
||||
|
||||
def _producer(
|
||||
kind: str, writer: int | None, paths: dict[int, str], names: dict[int, str]
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""The writer's service and name; a stream's service is its container."""
|
||||
if writer is None:
|
||||
return None, None
|
||||
return (_unit(paths.get(writer)) if kind == "file" else None), names.get(writer)
|
||||
|
||||
|
||||
def _files(sections: dict[str, list[str]]) -> list[LogSourceDict]:
|
||||
stats = _stats(sections.get("stat", []))
|
||||
paths = _cgroup_paths(sections.get("cgroups", []))
|
||||
names = _names(sections.get("comm", []))
|
||||
writes = _writes(sections["flags"]) if "flags" in sections else None
|
||||
found: list[LogSourceDict] = []
|
||||
for (kind, container, target), held in _holders(sections.get("open", []), paths).items():
|
||||
writer = _writer(held, writes)
|
||||
if writer is None and kind == "file" and container is not None:
|
||||
continue # a reader's view of a mount, inside its container
|
||||
unit, process = _producer(kind, writer, paths, names)
|
||||
size, modified = next((stats[fd] for _, fd in held if fd in stats), (None, None))
|
||||
found.append(
|
||||
{
|
||||
"kind": kind,
|
||||
"path": target,
|
||||
"unit": unit,
|
||||
"container_id": container,
|
||||
"process": process,
|
||||
"size": size,
|
||||
"modified": modified,
|
||||
}
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
def _order(source: LogSourceDict) -> tuple[str, str, str]:
|
||||
return (source["kind"], source["container_id"] or "", source["path"] or source["unit"] or "")
|
||||
|
||||
|
||||
def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
|
||||
"""Parse what :data:`LOG_SOURCES_COMMAND` printed: whether the host has a
|
||||
journal, and every source, sorted by kind, container and name.
|
||||
|
||||
:raises LogSourcesUnavailable: when the host has no GNU ``find``.
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if _NO_FIND in sections:
|
||||
raise LogSourcesUnavailable("the host's find cannot list open files (no GNU find)")
|
||||
journald = "journald" in sections
|
||||
sources = _journals(sections.get("journald", [])) + _files(sections)
|
||||
return journald, sorted(sources, key=_order)
|
||||
|
||||
|
||||
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
|
||||
journald, sources = parse_log_sources(output)
|
||||
forwarding = parse_log_forwarding(output)
|
||||
return {
|
||||
"attributed": attributed,
|
||||
"journald": journald,
|
||||
"sources": sources,
|
||||
"forwarding": forwarding,
|
||||
}
|
||||
|
||||
|
||||
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
|
||||
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
|
||||
None for a report without the sections, as an older command printed it.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if "rsyslog" not in sections:
|
||||
return None
|
||||
config = "\n".join(
|
||||
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
|
||||
)
|
||||
running = "running" in sections["rsyslog"]
|
||||
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
|
||||
|
||||
|
||||
class LogSourcesMixin:
|
||||
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse
|
||||
are the same everywhere, so they are concrete here, and a driver supplies
|
||||
only :meth:`_run_log_sources_command` -- how a command reaches its host, and
|
||||
how it gains root there. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_log_sources")`` stays truthful.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation."""
|
||||
...
|
||||
|
||||
def get_log_sources(self) -> LogSourcesDict:
|
||||
"""
|
||||
Returns every log the host writes, with the service or container writing it.
|
||||
|
||||
* attributed (bool) - read as root, so every process's files are listed
|
||||
* journald (bool) - the host keeps systemd's journal
|
||||
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
|
||||
* forwarding (dict) - where the logs go beyond the host, see
|
||||
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"attributed": True,
|
||||
"journald": True,
|
||||
"sources": [
|
||||
{"kind": "journald", "path": None, "unit": "nginx",
|
||||
"container_id": None, "process": None, "size": None, "modified": None},
|
||||
{"kind": "file", "path": "/var/log/nginx/access.log", "unit": "nginx",
|
||||
"container_id": None, "process": "nginx", "size": 2048,
|
||||
"modified": "2026-11-04T09:56:40+00:00"},
|
||||
],
|
||||
}
|
||||
|
||||
:raises LogSourcesUnavailable: if the host has no GNU ``find``.
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
|
||||
try:
|
||||
output = self._run_log_sources_command(command, privileged=True)
|
||||
return _reading(output, attributed=True)
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_log_sources_command(command, privileged=False)
|
||||
return _reading(output, attributed=False)
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.28.0"
|
||||
version = "4.31.1"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
|
||||
@@ -981,3 +981,69 @@ def test_a_driver_with_its_own_key_store_drops_keys_there(host: _Host) -> None:
|
||||
removed = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
assert removed["success"], removed["output"]
|
||||
assert driver.store == {"alice": []}
|
||||
|
||||
|
||||
# -- removing keys and the password at once (NetOrk/netork#2033) ------------------------
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_takes_the_password_and_sudo_too(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
"""A person who deleted their key: the password netOrk no longer knows goes too, and
|
||||
sudo, which asks for it. The account stays open, its other keys stay."""
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys", "password", "sudo"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and not account["admin"] and not account["locked"]
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_runs_each_step_when_one_failed(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
steps = {s["step"]: s for s in result["steps"]}
|
||||
assert steps["keys"]["ok"] and not steps["password"]["ok"]
|
||||
# sudo does not wait for the password: without a known password it has to go anyway.
|
||||
assert steps["sudo"]["ok"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == ""
|
||||
|
||||
|
||||
def test_removing_keys_with_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = _NoClear(host).remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_clears_nothing(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"] and "no account" in result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
|
||||
|
||||
def test_clear_password_is_a_keyword(driver: _Driver) -> None:
|
||||
with pytest.raises(TypeError):
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY], True) # type: ignore[misc]
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
|
||||
whether it gets the journal (NetOrk/netork#2053)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
|
||||
|
||||
DEBIAN = """\
|
||||
module(load="imuxsock")
|
||||
module(load="imklog")
|
||||
$FileOwner root
|
||||
$IncludeConfig /etc/rsyslog.d/*.conf
|
||||
*.*;auth,authpriv.none\t\t-/var/log/syslog
|
||||
auth,authpriv.*\t\t\t/var/log/auth.log
|
||||
*.emerg\t\t\t\t:omusrmsg:*
|
||||
"""
|
||||
|
||||
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
|
||||
|
||||
|
||||
def _rules(text: str) -> list[dict]:
|
||||
return parse_rsyslog(text)["rules"]
|
||||
|
||||
|
||||
class TestRules:
|
||||
def test_a_debian_host_forwards_nothing(self):
|
||||
assert _rules(DEBIAN) == []
|
||||
|
||||
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
|
||||
assert _rules(DEBIAN + GRAYLOG) == [
|
||||
{
|
||||
"protocol": "udp",
|
||||
"target": "graylog.example.net",
|
||||
"port": 1514,
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
},
|
||||
]
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
def test_an_ipv6_target_keeps_its_brackets_off(self):
|
||||
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
|
||||
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
|
||||
|
||||
def test_a_narrower_selector_is_partial_and_says_which(self):
|
||||
(rule,) = _rules("authpriv.* @10.0.0.5\n")
|
||||
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
|
||||
|
||||
def test_excluding_a_facility_is_partial_too(self):
|
||||
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
|
||||
assert rule["complete"] is False
|
||||
|
||||
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
|
||||
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
|
||||
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
|
||||
|
||||
def test_relp_is_its_own_protocol(self):
|
||||
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
|
||||
|
||||
def test_an_action_over_several_lines_is_read_whole(self):
|
||||
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
|
||||
"tcp",
|
||||
"10.0.0.7",
|
||||
6514,
|
||||
True,
|
||||
)
|
||||
|
||||
def test_an_action_on_its_own_takes_every_message(self):
|
||||
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
|
||||
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
|
||||
|
||||
def test_an_action_under_a_condition_is_partial_and_names_it(self):
|
||||
(rule,) = _rules(
|
||||
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
|
||||
)
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == "$programname == 'nginx'"
|
||||
|
||||
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
|
||||
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
|
||||
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
|
||||
|
||||
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
|
||||
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
|
||||
rules = _rules(text)
|
||||
assert [(r["target"], r["complete"]) for r in rules] == [
|
||||
("a", False),
|
||||
("b", False),
|
||||
("c", True),
|
||||
]
|
||||
|
||||
def test_a_property_filter_is_a_partial_selector(self):
|
||||
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == ':programname, isequal, "sshd"'
|
||||
|
||||
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
|
||||
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
|
||||
|
||||
def test_comments_are_no_rules(self):
|
||||
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
|
||||
|
||||
def test_a_hash_inside_quotes_is_no_comment(self):
|
||||
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
|
||||
assert rule["target"] == "10.0.0.13"
|
||||
|
||||
def test_a_local_action_is_no_forwarding(self):
|
||||
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
|
||||
|
||||
|
||||
class TestInputs:
|
||||
def test_an_imfile_input_names_its_file_and_ruleset(self):
|
||||
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
|
||||
assert parse_rsyslog(text)["file_inputs"] == [
|
||||
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
|
||||
]
|
||||
|
||||
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
|
||||
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
|
||||
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
|
||||
|
||||
|
||||
class TestModules:
|
||||
def test_imjournal_reads_the_journal(self):
|
||||
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
|
||||
|
||||
def test_legacy_modload_counts(self):
|
||||
modules = parse_rsyslog("$ModLoad imuxsock\n")
|
||||
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
|
||||
|
||||
|
||||
class TestJournalToSyslog:
|
||||
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
|
||||
the distribution prints commented out in its ``journald.conf``."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("lines", "expected"),
|
||||
[
|
||||
(["#ForwardToSyslog=yes"], True),
|
||||
(["#ForwardToSyslog=no"], False),
|
||||
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
|
||||
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
|
||||
([], None),
|
||||
],
|
||||
)
|
||||
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
|
||||
assert journal_to_syslog(lines) is expected
|
||||
@@ -0,0 +1,394 @@
|
||||
"""get_log_sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person in a log window, CrowdSec,
|
||||
a syslog forwarder -- something has to say which logs there are. Reading that
|
||||
is the same on every Linux host: journald names the units it holds entries for,
|
||||
``/proc/<pid>/fd`` shows the log files running processes have open, and
|
||||
``/proc/<pid>/cgroup`` says which systemd service or container a writer runs in.
|
||||
So all of it lives here once, and a driver only carries the command across
|
||||
(NetOrk/netork#2049).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
|
||||
CID = "4f1c" + "0" * 60
|
||||
OTHER = "9a2b" + "1" * 60
|
||||
JOURNALD = """nginx.service
|
||||
ssh.service
|
||||
postgresql@16-main.service
|
||||
session-3.scope
|
||||
init.scope
|
||||
"""
|
||||
OPEN = f"""/proc/610/fd/7 /var/log/syslog
|
||||
/proc/610/fd/8 /var/log/auth.log
|
||||
/proc/620/fd/9 /var/log/journal/0123abcd/system.journal
|
||||
/proc/900/fd/5 /var/log/nginx/access.log
|
||||
/proc/900/fd/6 /var/log/nginx/error.log
|
||||
/proc/901/fd/5 /var/log/nginx/access.log
|
||||
/proc/700/fd/4 /var/log/old.log (deleted)
|
||||
/proc/1100/fd/30 /var/lib/docker/containers/{CID}/{CID}-json.log
|
||||
/proc/1100/fd/31 /var/lib/docker/containers/{OTHER}/local-logs/container.log
|
||||
/proc/1400/fd/3 /usr/src/paperless/data/log/paperless.log
|
||||
/proc/1500/fd/3 /srv/app/log with space.log
|
||||
/proc/1500/fd/4 /srv/app/Local Storage/leveldb/000004.log
|
||||
/proc/1500/fd/5 /var/lib/rocksdb/012345.log
|
||||
/proc/300/fd/11 /var/log/nginx/access.log
|
||||
/proc/300/fd/12 /mnt/docroot/data/log/paperless.log
|
||||
/proc/300/fd/13 socket:[48213]
|
||||
/proc/300/fd/14 pipe:[48214]
|
||||
/proc/1600/fd/6 /var/log/host/syslog
|
||||
"""
|
||||
#: Each descriptor's ``flags`` from ``/proc/<pid>/fdinfo/<fd>``, octal: the access
|
||||
#: mode (``& 3``) is 1 or 2 for a writer -- 0102001 is O_WRONLY|O_APPEND|O_LARGEFILE
|
||||
#: -- and 0 for a reader (0100000, O_RDONLY|O_LARGEFILE). CrowdSec (300) reads
|
||||
#: nginx's log and a log on a mount; a promtail in a container (1600) reads the
|
||||
#: host's syslog as its own /var/log/host/syslog. 1500 opens its log O_RDWR.
|
||||
FLAGS = """/proc/610/fdinfo/7:flags:\t0102001
|
||||
/proc/610/fdinfo/8:flags:\t0102001
|
||||
/proc/620/fdinfo/9:flags:\t02100002
|
||||
/proc/900/fdinfo/5:flags:\t0102001
|
||||
/proc/900/fdinfo/6:flags:\t0102001
|
||||
/proc/901/fdinfo/5:flags:\t0102001
|
||||
/proc/700/fdinfo/4:flags:\t0102001
|
||||
/proc/1100/fdinfo/30:flags:\t0102001
|
||||
/proc/1100/fdinfo/31:flags:\t0102001
|
||||
/proc/1400/fdinfo/3:flags:\t0102001
|
||||
/proc/1500/fdinfo/3:flags:\t0100002
|
||||
/proc/1500/fdinfo/4:flags:\t0100002
|
||||
/proc/1500/fdinfo/5:flags:\t0100002
|
||||
/proc/300/fdinfo/11:flags:\t0100000
|
||||
/proc/300/fdinfo/12:flags:\t0100000
|
||||
/proc/300/fdinfo/13:flags:\t02004002
|
||||
/proc/300/fdinfo/14:flags:\t02004001
|
||||
/proc/1600/fdinfo/6:flags:\t0100000
|
||||
"""
|
||||
STAT = """/proc/610/fd/7 99000 1762250000
|
||||
/proc/610/fd/8 4100 1762250100
|
||||
/proc/900/fd/5 2048 1762250200
|
||||
/proc/900/fd/6 512 1762250300
|
||||
/proc/901/fd/5 2048 1762250200
|
||||
/proc/1100/fd/30 7000 1762250400
|
||||
/proc/1100/fd/31 300 1762250500
|
||||
/proc/1400/fd/3 0 1762250600
|
||||
/proc/300/fd/11 2048 1762250200
|
||||
/proc/300/fd/12 81920 1762250700
|
||||
"""
|
||||
CGROUPS = f"""610 0::/system.slice/rsyslog.service
|
||||
900 0::/system.slice/nginx.service
|
||||
901 0::/system.slice/nginx.service
|
||||
1100 0::/system.slice/docker.service
|
||||
1400 0::/system.slice/docker-{CID}.scope
|
||||
1500 0::/user.slice/user-1000.slice/session-3.scope
|
||||
300 0::/system.slice/crowdsec.service
|
||||
1600 0::/system.slice/docker-{OTHER}.scope
|
||||
"""
|
||||
COMM = """610 rsyslogd
|
||||
900 nginx
|
||||
901 nginx
|
||||
1100 dockerd
|
||||
1400 python3
|
||||
1500 app
|
||||
300 crowdsec
|
||||
1600 promtail
|
||||
"""
|
||||
|
||||
|
||||
def _wire(
|
||||
*,
|
||||
journald: str | None = JOURNALD,
|
||||
open_: str = OPEN,
|
||||
stat: str = STAT,
|
||||
cgroups: str = CGROUPS,
|
||||
comm: str = COMM,
|
||||
flags: str | None = FLAGS,
|
||||
noise: str = "",
|
||||
forwarding: str = "",
|
||||
) -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
parts = [noise, "LOGS_BEGIN\n"]
|
||||
if journald is not None:
|
||||
parts += ["[journald]\n", journald]
|
||||
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
|
||||
if flags is not None:
|
||||
parts += ["[flags]\n", flags]
|
||||
parts += [forwarding, "LOGS_END\n"]
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _sources(output: str | None = None) -> list:
|
||||
return parse_log_sources(_wire() if output is None else output)[1]
|
||||
|
||||
|
||||
def _by_key(sources: list) -> dict:
|
||||
return {(s["kind"], s["container_id"], s["path"] or s["unit"]): s for s in sources}
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_journald_names_each_service_unit_once_without_its_suffix(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire())
|
||||
units = [s["unit"] for s in sources if s["kind"] == "journald"]
|
||||
assert units == ["nginx", "postgresql@16-main", "ssh"]
|
||||
assert journald is True
|
||||
|
||||
def test_a_host_without_journald_says_so(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire(journald=None))
|
||||
assert journald is False
|
||||
assert not [s for s in sources if s["kind"] == "journald"]
|
||||
|
||||
def test_an_open_file_names_its_writer_size_and_last_write(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
access = sources[("file", None, "/var/log/nginx/access.log")]
|
||||
assert access == {
|
||||
"kind": "file",
|
||||
"path": "/var/log/nginx/access.log",
|
||||
"unit": "nginx",
|
||||
"container_id": None,
|
||||
"process": "nginx",
|
||||
"size": 2048,
|
||||
"modified": "2025-11-04T09:56:40+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_two_processes_hold_is_one_source(self) -> None:
|
||||
paths = [s["path"] for s in _sources() if s["kind"] == "file"]
|
||||
assert paths.count("/var/log/nginx/access.log") == 1
|
||||
|
||||
def test_a_file_without_a_suffix_under_var_log_counts(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
assert sources[("file", None, "/var/log/syslog")]["unit"] == "rsyslog"
|
||||
|
||||
def test_a_file_a_container_writes_carries_the_container_and_its_own_path(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
paperless = sources[("file", CID, "/usr/src/paperless/data/log/paperless.log")]
|
||||
assert paperless["unit"] is None
|
||||
assert paperless["process"] == "python3"
|
||||
assert paperless["size"] == 0
|
||||
|
||||
def test_the_engines_own_log_file_is_the_containers_stream(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
json_file = sources[("container", CID, f"/var/lib/docker/containers/{CID}/{CID}-json.log")]
|
||||
assert json_file["process"] == "dockerd"
|
||||
assert json_file["size"] == 7000
|
||||
local = sources[
|
||||
("container", OTHER, f"/var/lib/docker/containers/{OTHER}/local-logs/container.log")
|
||||
]
|
||||
assert local["modified"] == "2025-11-04T10:01:40+00:00"
|
||||
assert not [s for s in _sources() if "docker" in (s["unit"] or "")]
|
||||
|
||||
def test_journal_files_and_deleted_files_are_not_sources(self) -> None:
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if "/var/log/journal/" in p or p.startswith("/var/log/old")]
|
||||
|
||||
def test_a_databases_write_ahead_log_is_not_a_source(self) -> None:
|
||||
# LevelDB and RocksDB name theirs 000004.log; it holds records, not lines.
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if p.endswith(("000004.log", "012345.log"))]
|
||||
|
||||
def test_a_path_with_a_space_survives(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
spaced = sources[("file", None, "/srv/app/log with space.log")]
|
||||
# The writer runs in a login session, not in a service; stat did not answer.
|
||||
assert spaced["unit"] is None
|
||||
assert spaced["process"] == "app"
|
||||
assert spaced["size"] is None
|
||||
assert spaced["modified"] is None
|
||||
|
||||
def test_sources_come_sorted_by_kind_then_container_then_name(self) -> None:
|
||||
sources = _sources()
|
||||
order = [(s["kind"], s["container_id"] or "", s["path"] or s["unit"]) for s in sources]
|
||||
assert order == sorted(order)
|
||||
|
||||
def test_noise_before_the_frame_is_ignored(self) -> None:
|
||||
assert _sources(_wire(noise="Last login: today\r\n\x1b[0m"))
|
||||
|
||||
def test_a_host_without_gnu_find_raises_rather_than_reading_as_no_files(self) -> None:
|
||||
# busybox find has neither -lname nor -printf: no files would read as none.
|
||||
with pytest.raises(LogSourcesUnavailable, match="find"):
|
||||
parse_log_sources("LOGS_BEGIN\n[journald]\nnginx.service\n[no-find]\nLOGS_END\n")
|
||||
|
||||
def test_a_report_cut_short_raises_rather_than_reading_as_fewer_logs(self) -> None:
|
||||
cut = _wire().replace("LOGS_END\n", "")
|
||||
with pytest.raises(ValueError, match="log source report"):
|
||||
parse_log_sources(cut)
|
||||
|
||||
|
||||
class TestWriters:
|
||||
"""A file is attributed to the process writing it, not to one reading it
|
||||
(NetOrk/netork#2092): CrowdSec, Promtail or a ``tail -F`` holding a log is
|
||||
no producer of it."""
|
||||
|
||||
def test_a_reader_with_the_lower_pid_does_not_take_the_writers_file(self) -> None:
|
||||
access = _by_key(_sources())[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("nginx", "nginx")
|
||||
|
||||
def test_a_file_only_readers_hold_is_listed_without_a_producer(self) -> None:
|
||||
paperless = _by_key(_sources())[("file", None, "/mnt/docroot/data/log/paperless.log")]
|
||||
assert paperless == {
|
||||
"kind": "file",
|
||||
"path": "/mnt/docroot/data/log/paperless.log",
|
||||
"unit": None,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": 81920,
|
||||
"modified": "2025-11-04T10:05:00+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_open_for_reading_and_writing_is_written(self) -> None:
|
||||
spaced = _by_key(_sources())[("file", None, "/srv/app/log with space.log")]
|
||||
assert spaced["process"] == "app"
|
||||
|
||||
def test_a_file_only_a_reader_in_a_container_holds_is_left_out(self) -> None:
|
||||
# Its path is the reader's view of a mount: it names nothing on the host,
|
||||
# and the container does not write it.
|
||||
assert not [s for s in _sources() if s["path"] == "/var/log/host/syslog"]
|
||||
|
||||
def test_sockets_and_pipes_are_no_sources(self) -> None:
|
||||
assert not [s for s in _sources() if not (s["path"] or "/").startswith("/")]
|
||||
|
||||
def test_a_deleted_file_stays_out_though_it_is_written(self) -> None:
|
||||
assert not [s for s in _sources() if (s["path"] or "").startswith("/var/log/old")]
|
||||
|
||||
def test_a_holder_whose_flags_did_not_answer_is_no_writer(self) -> None:
|
||||
# The process ended between the listing and the read of its fdinfo.
|
||||
flags = FLAGS.replace("/proc/900/fdinfo/5:flags:\t0102001\n", "")
|
||||
flags = flags.replace("/proc/901/fdinfo/5:flags:\t0102001\n", "")
|
||||
access = _by_key(_sources(_wire(flags=flags)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == (None, None)
|
||||
|
||||
def test_a_report_without_flags_names_the_first_holder_as_before(self) -> None:
|
||||
# An older command printed no [flags]: who writes cannot be told apart.
|
||||
access = _by_key(_sources(_wire(flags=None)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("crowdsec", "crowdsec")
|
||||
|
||||
|
||||
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
|
||||
FORWARDING = """[rsyslog]
|
||||
running
|
||||
[rsyslog-conf]
|
||||
| module(load="imuxsock")
|
||||
| [comment-looking line]
|
||||
|
|
||||
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
|
||||
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
|
||||
[journald-conf]
|
||||
#ForwardToSyslog=yes
|
||||
"""
|
||||
|
||||
|
||||
class TestForwarding:
|
||||
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
|
||||
|
||||
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None
|
||||
assert forwarding["rsyslog"] is True
|
||||
assert forwarding["journal"] is True
|
||||
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
|
||||
("graylog.lan", 1514, True)
|
||||
]
|
||||
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
|
||||
|
||||
def test_a_config_line_in_brackets_is_no_section(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None and forwarding["rules"]
|
||||
|
||||
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
|
||||
stopped = FORWARDING.replace("running\n", "")
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
|
||||
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
|
||||
def test_a_report_without_the_sections_says_nothing(self) -> None:
|
||||
assert parse_log_forwarding(_wire()) is None
|
||||
|
||||
def test_the_reading_carries_it(self) -> None:
|
||||
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
|
||||
assert reading["forwarding"] is not None
|
||||
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
|
||||
|
||||
|
||||
class _Host(LogSourcesMixin):
|
||||
"""Carries the command the way a driver does; *answers* per privilege."""
|
||||
|
||||
def __init__(self, answers: dict[bool, str]) -> None:
|
||||
self.answers = answers
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
self.calls.append((command, privileged))
|
||||
return self.answers[privileged]
|
||||
|
||||
|
||||
class TestMixin:
|
||||
def test_it_reads_as_root_in_one_sh_call(self) -> None:
|
||||
host = _Host({True: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is True
|
||||
command, privileged = host.calls[0]
|
||||
assert privileged is True
|
||||
assert command.startswith("sh -c ")
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_without_root_it_reads_again_and_says_so(self) -> None:
|
||||
host = _Host({True: "sudo: a password is required\n", False: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is False
|
||||
assert [privileged for _, privileged in host.calls] == [True, False]
|
||||
assert reading["sources"]
|
||||
|
||||
def test_a_host_without_gnu_find_is_not_read_again_unprivileged(self) -> None:
|
||||
host = _Host({True: "LOGS_BEGIN\n[no-find]\nLOGS_END\n"})
|
||||
with pytest.raises(LogSourcesUnavailable):
|
||||
host.get_log_sources()
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_a_host_that_answers_neither_way_raises(self) -> None:
|
||||
host = _Host({True: "", False: "garbage"})
|
||||
with pytest.raises(ValueError):
|
||||
host.get_log_sources()
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_it_is_valid_posix_sh(self) -> None:
|
||||
assert subprocess.run(["sh", "-n", "-c", LOG_SOURCES_COMMAND]).returncode == 0
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_here_a_file_held_for_writing_has_a_producer_and_one_held_for_reading_none(
|
||||
self, tmp_path
|
||||
) -> None:
|
||||
written, read = tmp_path / "written.log", tmp_path / "read.log"
|
||||
read.write_text("one line\n")
|
||||
with written.open("a") as w, read.open() as r:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert w.writable() and r.readable()
|
||||
sources = {s["path"]: s for s in _sources(out)}
|
||||
assert sources[str(written)]["process"] is not None
|
||||
assert sources[str(read)]["process"] is None
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_it_runs_here_and_parses(self, tmp_path) -> None:
|
||||
log = tmp_path / "held.log"
|
||||
log.write_text("one line\n")
|
||||
with log.open() as held:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert held.readable()
|
||||
# This test's own process holds the file, so it must be among the sources.
|
||||
mine = [s for s in _sources(out) if s["path"] == str(log)]
|
||||
assert mine and mine[0]["size"] == len("one line\n")
|
||||
# Whether rsyslog runs here or not, the report says where logs go.
|
||||
assert parse_log_forwarding(out) is not None
|
||||
Reference in new issue
Block a user