feat: where a host's logs go beyond it (4.31.0)
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Failing after 2m23s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Failing after 2m23s
driver / publish (pull_request) Skipped
get_log_sources() reads, in the same command, whether rsyslog runs, its forwarding rules (legacy @/@@/:omrelp: and RainerScript omfwd/omrelp, with selector, ruleset and whether they take every message), its imfile inputs, and whether the journal reaches it (imjournal, or imuxsock with journald's ForwardToSyslog; failing a setting, the default the distribution ships commented out). Returned as `forwarding`; parse_log_forwarding and parse_rsyslog are public (NetOrk/netork#2053).
This commit is contained in:
1 parent
1b600416f0
commit
8557710724
8 files changed
+586
-6
No files matched your search
@@ -7,6 +7,16 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.31.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
|
||||
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
|
||||
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
|
||||
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
|
||||
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
|
||||
public (NetOrk/netork#2053).
|
||||
|
||||
## [4.30.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
|
||||
@@ -202,6 +202,11 @@ store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) a
|
||||
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
|
||||
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
|
||||
user's processes show their files, and the reading says so (`attributed: false`).
|
||||
The same round trip reads where the logs go beyond the host (`forwarding`,
|
||||
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
|
||||
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
|
||||
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
|
||||
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
|
||||
@@ -167,12 +167,19 @@ from napalm_device_types.local_accounts import (
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.log_forwarding import (
|
||||
FileInputDict,
|
||||
ForwardRuleDict,
|
||||
LogForwardingDict,
|
||||
parse_rsyslog,
|
||||
)
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourceDict,
|
||||
LogSourcesDict,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
@@ -408,6 +415,11 @@ __all__ = [
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"LOG_SOURCES_COMMAND",
|
||||
"FileInputDict",
|
||||
"ForwardRuleDict",
|
||||
"LogForwardingDict",
|
||||
"parse_log_forwarding",
|
||||
"parse_rsyslog",
|
||||
"LogSourceDict",
|
||||
"LogSourcesDict",
|
||||
"LogSourcesMixin",
|
||||
|
||||
@@ -0,0 +1,304 @@
|
||||
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
|
||||
in, and whether it gets the journal (NetOrk/netork#2053).
|
||||
|
||||
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
|
||||
this says who reads it on. It is read in the same command, so a reading of
|
||||
the logs a host writes also says which of them leave it.
|
||||
|
||||
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
|
||||
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
|
||||
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
|
||||
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
|
||||
*complete* when it takes every message of its ruleset -- ``*.*`` with no
|
||||
condition around it. Anything narrower is partial, and keeps its selector or
|
||||
condition for the reader to see: which facility a unit logs to is not
|
||||
computed (decided in #2053).
|
||||
|
||||
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
|
||||
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
|
||||
own a host runs on its distribution's default, which ``journald.conf`` ships
|
||||
commented out: ``#ForwardToSyslog=yes``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class ForwardRuleDict(TypedDict):
|
||||
"""One rule sending messages to another host."""
|
||||
|
||||
protocol: str # "udp", "tcp" or "relp"
|
||||
target: str # the host, as configured
|
||||
port: int | None # None: the protocol's default
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
|
||||
|
||||
path: str # as configured: may hold a glob
|
||||
ruleset: str | None
|
||||
|
||||
|
||||
class RsyslogDict(TypedDict):
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
imjournal: bool
|
||||
imuxsock: bool
|
||||
|
||||
|
||||
class LogForwardingDict(TypedDict):
|
||||
"""Where a host's logs go beyond it.
|
||||
|
||||
``journal`` is None when it cannot be told whether the journal reaches
|
||||
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
|
||||
"""
|
||||
|
||||
rsyslog: bool # rsyslogd runs
|
||||
journal: bool | None # the journal reaches rsyslog
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
|
||||
|
||||
#: Selectors that take every message.
|
||||
_EVERYTHING = {"*.*", "*.debug"}
|
||||
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
|
||||
_CALL_RE = re.compile(
|
||||
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
|
||||
)
|
||||
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
|
||||
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
|
||||
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
|
||||
_FORWARD_RE = re.compile(
|
||||
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
|
||||
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
|
||||
)
|
||||
_FORWARD_TYPES = {"omfwd", "omrelp"}
|
||||
_TRUE = {"yes", "true", "1", "on"}
|
||||
|
||||
|
||||
def _uncomment(text: str) -> str:
|
||||
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
|
||||
out: list[str] = []
|
||||
quoted = commented = False
|
||||
for char in text:
|
||||
if char == "\n":
|
||||
quoted = commented = False
|
||||
elif commented:
|
||||
continue
|
||||
elif char == '"':
|
||||
quoted = not quoted
|
||||
elif char == "#" and not quoted:
|
||||
commented = True
|
||||
continue
|
||||
out.append(char)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _pieces(text: str) -> list[str]:
|
||||
"""Statements, ``{`` and ``}`` in order: a line runs on while a bracket is
|
||||
open, and a brace outside one stands as a piece of its own."""
|
||||
pieces: list[str] = []
|
||||
current: list[str] = []
|
||||
depth = 0
|
||||
quoted = False
|
||||
|
||||
def close() -> None:
|
||||
piece = "".join(current).strip()
|
||||
if piece:
|
||||
pieces.append(piece)
|
||||
current.clear()
|
||||
|
||||
for char in text:
|
||||
if char == '"':
|
||||
quoted = not quoted
|
||||
elif not quoted and char in "([":
|
||||
depth += 1
|
||||
elif not quoted and char in ")]":
|
||||
depth = max(depth - 1, 0)
|
||||
elif not quoted and depth == 0 and char in "{}":
|
||||
close()
|
||||
pieces.append(char)
|
||||
continue
|
||||
elif char == "\n" and depth == 0:
|
||||
close()
|
||||
continue
|
||||
current.append(char)
|
||||
close()
|
||||
return pieces
|
||||
|
||||
|
||||
def _params(args: str) -> dict[str, str]:
|
||||
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
|
||||
|
||||
|
||||
def _port(value: str | None) -> int | None:
|
||||
return int(value) if value and value.isdigit() else None
|
||||
|
||||
|
||||
class _Reader:
|
||||
"""Walks the pieces of a configuration, keeping the blocks it is in."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.parsed: RsyslogDict = {
|
||||
"rules": [],
|
||||
"file_inputs": [],
|
||||
"imjournal": False,
|
||||
"imuxsock": False,
|
||||
}
|
||||
self.stack: list[tuple[str, str | None]] = []
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
return names[-1] if names else self.legacy_ruleset
|
||||
|
||||
def condition(self, once: str | None) -> str | None:
|
||||
conditions = [value for kind, value in self.stack if kind == "if"]
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
{
|
||||
"protocol": protocol,
|
||||
"target": target,
|
||||
"port": port,
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
}
|
||||
)
|
||||
|
||||
def piece(self, piece: str) -> None:
|
||||
if piece == "{":
|
||||
self.stack.append(self.pending or ("block", None))
|
||||
self.pending = None
|
||||
return
|
||||
if piece == "}":
|
||||
if self.stack:
|
||||
self.stack.pop()
|
||||
return
|
||||
if piece == "else":
|
||||
self.pending = ("if", "else")
|
||||
return
|
||||
# A condition whose action follows on its own: the action is under it.
|
||||
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
|
||||
self.pending = None
|
||||
match = _IF_RE.match(piece)
|
||||
if match:
|
||||
if match["rest"]:
|
||||
self.statement(match["rest"], once=match["cond"])
|
||||
else:
|
||||
self.pending = ("if", match["cond"])
|
||||
return
|
||||
self.statement(piece, once=once)
|
||||
|
||||
def statement(self, text: str, once: str | None) -> None:
|
||||
call = _CALL_RE.match(text)
|
||||
if call:
|
||||
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
|
||||
elif text.startswith("$"):
|
||||
self.directive(text)
|
||||
else:
|
||||
self.legacy(text, once)
|
||||
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
self.parsed["file_inputs"].append(
|
||||
{"path": params["file"], "ruleset": params.get("ruleset")}
|
||||
)
|
||||
elif name == "action":
|
||||
selector = self.last_selector if before == "&" else (before or "*.*")
|
||||
self.last_selector = selector
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
self.parsed["imjournal"] = True
|
||||
elif name == "imuxsock":
|
||||
self.parsed["imuxsock"] = True
|
||||
|
||||
def directive(self, text: str) -> None:
|
||||
name, _, value = text.partition(" ")
|
||||
name, value = name.lower(), value.strip()
|
||||
if name == "$modload":
|
||||
self.module(value)
|
||||
elif name == "$inputfilename" and value:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
if not match:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
forward = _FORWARD_RE.match(match["action"].strip())
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
|
||||
)
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
read for its forwarding rules, its file inputs, and the inputs that take
|
||||
the journal."""
|
||||
reader = _Reader()
|
||||
for piece in _pieces(_uncomment(text)):
|
||||
reader.piece(piece)
|
||||
return reader.parsed
|
||||
|
||||
|
||||
def journal_to_syslog(lines: list[str]) -> bool | None:
|
||||
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
|
||||
the order journald reads them: the last setting wins; with none, the
|
||||
default the distribution ships commented out; None when nothing says."""
|
||||
setting = shipped = None
|
||||
for line in lines:
|
||||
commented = line.lstrip().startswith("#")
|
||||
_, _, value = line.lstrip("# \t").partition("=")
|
||||
if commented:
|
||||
shipped = value.strip().lower() in _TRUE
|
||||
else:
|
||||
setting = value.strip().lower() in _TRUE
|
||||
return setting if setting is not None else shipped
|
||||
|
||||
|
||||
def log_forwarding(
|
||||
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
|
||||
) -> LogForwardingDict:
|
||||
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
|
||||
if not running:
|
||||
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
journal: bool | None
|
||||
if rsyslog["imjournal"]:
|
||||
journal = True
|
||||
elif rsyslog["imuxsock"]:
|
||||
journal = journal_to_syslog(journald_lines)
|
||||
else:
|
||||
journal = False
|
||||
return {
|
||||
"rsyslog": True,
|
||||
"journal": journal,
|
||||
"rules": rsyslog["rules"],
|
||||
"file_inputs": rsyslog["file_inputs"],
|
||||
}
|
||||
@@ -19,6 +19,12 @@ container the first process holding a file runs in, as for listening sockets
|
||||
(:mod:`napalm_device_types.listening`). journald's own files, files already
|
||||
deleted and a database's write-ahead log (LevelDB's ``000004.log``) are left out.
|
||||
|
||||
**Where they go** is read in the same round trip
|
||||
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
|
||||
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
|
||||
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
|
||||
reads them.
|
||||
|
||||
**One round trip, root, and without it.** As for listening sockets the script
|
||||
goes as one ``sh -c`` argument. Without root only the login user's processes
|
||||
show their files; the command then runs again unprivileged, and the reading
|
||||
@@ -36,6 +42,7 @@ from shlex import quote
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
|
||||
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
@@ -68,6 +75,7 @@ class LogSourcesDict(TypedDict):
|
||||
attributed: bool
|
||||
journald: bool
|
||||
sources: list[LogSourceDict]
|
||||
forwarding: LogForwardingDict | None # where the logs go beyond the host
|
||||
|
||||
|
||||
_BEGIN = "LOGS_BEGIN"
|
||||
@@ -94,6 +102,13 @@ LOG_SOURCES_COMMAND = (
|
||||
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
|
||||
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
|
||||
"else echo '[no-find]'; fi; "
|
||||
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
|
||||
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
|
||||
"| sed 's/^/| /'; "
|
||||
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
|
||||
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
|
||||
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
|
||||
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
|
||||
"printf '%s%s\\n' LOGS_ END"
|
||||
)
|
||||
|
||||
@@ -213,6 +228,33 @@ def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
|
||||
return journald, sorted(sources, key=_order)
|
||||
|
||||
|
||||
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
|
||||
journald, sources = parse_log_sources(output)
|
||||
forwarding = parse_log_forwarding(output)
|
||||
return {
|
||||
"attributed": attributed,
|
||||
"journald": journald,
|
||||
"sources": sources,
|
||||
"forwarding": forwarding,
|
||||
}
|
||||
|
||||
|
||||
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
|
||||
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
|
||||
None for a report without the sections, as an older command printed it.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if "rsyslog" not in sections:
|
||||
return None
|
||||
config = "\n".join(
|
||||
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
|
||||
)
|
||||
running = "running" in sections["rsyslog"]
|
||||
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
|
||||
|
||||
|
||||
class LogSourcesMixin:
|
||||
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
|
||||
|
||||
@@ -237,6 +279,8 @@ class LogSourcesMixin:
|
||||
* attributed (bool) - read as root, so every process's files are listed
|
||||
* journald (bool) - the host keeps systemd's journal
|
||||
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
|
||||
* forwarding (dict) - where the logs go beyond the host, see
|
||||
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
|
||||
|
||||
Example::
|
||||
|
||||
@@ -258,10 +302,8 @@ class LogSourcesMixin:
|
||||
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
|
||||
try:
|
||||
output = self._run_log_sources_command(command, privileged=True)
|
||||
journald, sources = parse_log_sources(output)
|
||||
return {"attributed": True, "journald": journald, "sources": sources}
|
||||
return _reading(output, attributed=True)
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_log_sources_command(command, privileged=False)
|
||||
journald, sources = parse_log_sources(output)
|
||||
return {"attributed": False, "journald": journald, "sources": sources}
|
||||
return _reading(output, attributed=False)
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.30.0"
|
||||
version = "4.31.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
|
||||
whether it gets the journal (NetOrk/netork#2053)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
|
||||
|
||||
DEBIAN = """\
|
||||
module(load="imuxsock")
|
||||
module(load="imklog")
|
||||
$FileOwner root
|
||||
$IncludeConfig /etc/rsyslog.d/*.conf
|
||||
*.*;auth,authpriv.none\t\t-/var/log/syslog
|
||||
auth,authpriv.*\t\t\t/var/log/auth.log
|
||||
*.emerg\t\t\t\t:omusrmsg:*
|
||||
"""
|
||||
|
||||
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
|
||||
|
||||
|
||||
def _rules(text: str) -> list[dict]:
|
||||
return parse_rsyslog(text)["rules"]
|
||||
|
||||
|
||||
class TestRules:
|
||||
def test_a_debian_host_forwards_nothing(self):
|
||||
assert _rules(DEBIAN) == []
|
||||
|
||||
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
|
||||
assert _rules(DEBIAN + GRAYLOG) == [
|
||||
{
|
||||
"protocol": "udp",
|
||||
"target": "graylog.example.net",
|
||||
"port": 1514,
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
},
|
||||
]
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
def test_an_ipv6_target_keeps_its_brackets_off(self):
|
||||
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
|
||||
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
|
||||
|
||||
def test_a_narrower_selector_is_partial_and_says_which(self):
|
||||
(rule,) = _rules("authpriv.* @10.0.0.5\n")
|
||||
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
|
||||
|
||||
def test_excluding_a_facility_is_partial_too(self):
|
||||
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
|
||||
assert rule["complete"] is False
|
||||
|
||||
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
|
||||
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
|
||||
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
|
||||
|
||||
def test_relp_is_its_own_protocol(self):
|
||||
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
|
||||
|
||||
def test_an_action_over_several_lines_is_read_whole(self):
|
||||
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
|
||||
"tcp",
|
||||
"10.0.0.7",
|
||||
6514,
|
||||
True,
|
||||
)
|
||||
|
||||
def test_an_action_on_its_own_takes_every_message(self):
|
||||
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
|
||||
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
|
||||
|
||||
def test_an_action_under_a_condition_is_partial_and_names_it(self):
|
||||
(rule,) = _rules(
|
||||
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
|
||||
)
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == "$programname == 'nginx'"
|
||||
|
||||
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
|
||||
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
|
||||
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
|
||||
|
||||
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
|
||||
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
|
||||
rules = _rules(text)
|
||||
assert [(r["target"], r["complete"]) for r in rules] == [
|
||||
("a", False),
|
||||
("b", False),
|
||||
("c", True),
|
||||
]
|
||||
|
||||
def test_a_property_filter_is_a_partial_selector(self):
|
||||
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == ':programname, isequal, "sshd"'
|
||||
|
||||
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
|
||||
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
|
||||
|
||||
def test_comments_are_no_rules(self):
|
||||
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
|
||||
|
||||
def test_a_hash_inside_quotes_is_no_comment(self):
|
||||
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
|
||||
assert rule["target"] == "10.0.0.13"
|
||||
|
||||
def test_a_local_action_is_no_forwarding(self):
|
||||
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
|
||||
|
||||
|
||||
class TestInputs:
|
||||
def test_an_imfile_input_names_its_file_and_ruleset(self):
|
||||
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
|
||||
assert parse_rsyslog(text)["file_inputs"] == [
|
||||
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
|
||||
]
|
||||
|
||||
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
|
||||
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
|
||||
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
|
||||
|
||||
|
||||
class TestModules:
|
||||
def test_imjournal_reads_the_journal(self):
|
||||
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
|
||||
|
||||
def test_legacy_modload_counts(self):
|
||||
modules = parse_rsyslog("$ModLoad imuxsock\n")
|
||||
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
|
||||
|
||||
|
||||
class TestJournalToSyslog:
|
||||
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
|
||||
the distribution prints commented out in its ``journald.conf``."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("lines", "expected"),
|
||||
[
|
||||
(["#ForwardToSyslog=yes"], True),
|
||||
(["#ForwardToSyslog=no"], False),
|
||||
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
|
||||
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
|
||||
([], None),
|
||||
],
|
||||
)
|
||||
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
|
||||
assert journal_to_syslog(lines) is expected
|
||||
@@ -20,6 +20,7 @@ from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
|
||||
@@ -78,13 +79,14 @@ def _wire(
|
||||
cgroups: str = CGROUPS,
|
||||
comm: str = COMM,
|
||||
noise: str = "",
|
||||
forwarding: str = "",
|
||||
) -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
parts = [noise, "LOGS_BEGIN\n"]
|
||||
if journald is not None:
|
||||
parts += ["[journald]\n", journald]
|
||||
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
|
||||
parts.append("LOGS_END\n")
|
||||
parts += [forwarding, "LOGS_END\n"]
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
@@ -184,6 +186,51 @@ class TestParse:
|
||||
parse_log_sources(cut)
|
||||
|
||||
|
||||
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
|
||||
FORWARDING = """[rsyslog]
|
||||
running
|
||||
[rsyslog-conf]
|
||||
| module(load="imuxsock")
|
||||
| [comment-looking line]
|
||||
|
|
||||
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
|
||||
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
|
||||
[journald-conf]
|
||||
#ForwardToSyslog=yes
|
||||
"""
|
||||
|
||||
|
||||
class TestForwarding:
|
||||
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
|
||||
|
||||
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None
|
||||
assert forwarding["rsyslog"] is True
|
||||
assert forwarding["journal"] is True
|
||||
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
|
||||
("graylog.lan", 1514, True)
|
||||
]
|
||||
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
|
||||
|
||||
def test_a_config_line_in_brackets_is_no_section(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None and forwarding["rules"]
|
||||
|
||||
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
|
||||
stopped = FORWARDING.replace("running\n", "")
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
|
||||
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
|
||||
def test_a_report_without_the_sections_says_nothing(self) -> None:
|
||||
assert parse_log_forwarding(_wire()) is None
|
||||
|
||||
def test_the_reading_carries_it(self) -> None:
|
||||
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
|
||||
assert reading["forwarding"] is not None
|
||||
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
|
||||
|
||||
|
||||
class _Host(LogSourcesMixin):
|
||||
"""Carries the command the way a driver does; *answers* per privilege."""
|
||||
|
||||
@@ -241,3 +288,5 @@ class TestCommand:
|
||||
# This test's own process holds the file, so it must be among the sources.
|
||||
mine = [s for s in _sources(out) if s["path"] == str(log)]
|
||||
assert mine and mine[0]["size"] == len("one line\n")
|
||||
# Whether rsyslog runs here or not, the report says where logs go.
|
||||
assert parse_log_forwarding(out) is not None
|
||||
Reference in new issue
Block a user