Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
08a62838f7 | ||
|
|
8516f2a1ee | ||
|
|
c74173523a | ||
|
|
6c7f164004 | ||
|
|
073087da69 | ||
|
|
278a707157 | ||
|
|
56e5e82a59 | ||
|
|
e6f145d4a3 | ||
|
|
66037e878e | ||
|
|
afffd0915c | ||
|
|
92891473be | ||
|
|
658f1c8ab2 | ||
|
|
aacde6be3f | ||
|
|
58c7616401 | ||
|
|
b7e31205d5 | ||
|
|
1d71ff8f91 | ||
|
|
58ed0f0a94 | ||
|
|
c1a05d6dba | ||
|
|
fccf902c07 | ||
|
|
f5a2811333 | ||
|
|
98d1dbef4c | ||
|
|
252ea3b81c | ||
|
|
b685b200cc | ||
|
|
8557710724 |
No files matched your search
@@ -7,6 +7,98 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.35.1] - 2026-11-11
|
||||
|
||||
### Fixed
|
||||
- `apply_dhcp_reservationset()` no longer stops at the first reservation the device refuses.
|
||||
One refused write (on OPNsense, an address in a subnet Kea does not manage) ended the run
|
||||
before the commit: every later reservation went unwritten, and the earlier ones stayed
|
||||
saved but inactive, where the next diff found them configured and never committed them. A
|
||||
refused change now gets a `[failed] <mac> -> <ip>: <reason>` line, the others are written,
|
||||
any non-empty diff commits, and `DhcpApplyError` (a `RuntimeError`, exported) is raised
|
||||
after the commit with how many of how many were refused (NetOrk/netork#2381).
|
||||
|
||||
## [4.35.0] - 2026-11-11
|
||||
|
||||
### Added
|
||||
- `TimeSyncMixin`: a device's time sources and time zone. `get_time_sync()` (declared only)
|
||||
returns `TimeSyncDict` -- the configured NTP servers in the device's order, the zone as an
|
||||
IANA name (None where the device keeps no name), whether the clock is synchronised now,
|
||||
and the client (`chrony`, `systemd-timesyncd`, `ntpd`, `openntpd`; None for the device's
|
||||
own). `configure_time_sync(ntp_servers=None, timezone=None)` is a fix template: None
|
||||
leaves a value alone, the device ends up with exactly the servers given, in order, and a
|
||||
`verify` step reads it back. An unknown zone, a malformed server, an empty list or a value
|
||||
outside the driver's `time_sync_writes` is refused before anything is written.
|
||||
`TIME_SYNC_VALUES` and `check_ntp_servers` are public (NetOrk/netork#2361).
|
||||
- `PosixTimeSyncMixin` implements it over `run_command`: read in one round trip as the login
|
||||
user (`timedatectl`, chrony's configuration and `chronyc -n sources`, `timedatectl
|
||||
show-timesync`, ntpd's and OpenNTPD's configuration); the zone by `timedatectl
|
||||
set-timezone`; chrony's servers in a file of their own (`sources.d`, else a `confdir` or
|
||||
an included file) with every other source commented out, then `chronyc reload sources` or
|
||||
a restart; systemd-timesyncd's in a drop-in that resets `NTP=` and empties `FallbackNTP=`.
|
||||
ntpd and OpenNTPD are read, not written. `parse_time_sync` is public.
|
||||
- `napalm_device_types.posix_tz` (from netOrk): `posix_tz`, `is_iana_zone`, `is_posix_tz` and
|
||||
`UnknownTimezone`, and `parse_posix_tz`, which takes a POSIX rule apart into the UTC
|
||||
offset and the summer-time transitions a switch wants (`PosixTzDict`,
|
||||
`DstTransitionDict`).
|
||||
|
||||
### Changed
|
||||
- Needs `tzdata` (2024.1 or later): the zones' rules come from its TZif files.
|
||||
|
||||
## [4.34.0] - 2026-11-10
|
||||
|
||||
### Added
|
||||
- `RunningBackupsMixin.get_running_backups()`: the backups running on a device right now --
|
||||
a hypervisor's backup of its guests (with the guests it is taking now), a backup server's
|
||||
backup, sync, verify, garbage collection, prune or tape run, a host's own file-level backup.
|
||||
Declared only; a driver mixes it in when it can tell, and a driver without it cannot tell.
|
||||
`RunningBackupDict`, `RunningBackupKind` and `RUNNING_BACKUP_KINDS` are public. An
|
||||
application asks right before it restarts a device (NetOrk/netork#2318).
|
||||
|
||||
## [4.33.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `get_host_status()` tells an interrupted package manager on hosts without dpkg too, and
|
||||
says which one it read in the new `package_manager` field (`dpkg`, `dnf`, `yum`, `apk`,
|
||||
`opkg`, or None). dnf and yum: a package installed in two versions (not the kernels and
|
||||
other packages dnf keeps side by side) as state `duplicate`, or a transaction that never
|
||||
ended (dnf 4's history, yum's `transaction-all` journal). apk: a package apk marked with
|
||||
broken files or a broken script (`broken-files`, `broken-script`), the ones `apk fix` would
|
||||
take. opkg: a package left `unpacked`, `half-configured` or `half-installed`, or flagged for
|
||||
reinstallation. All read without root; while one of them runs the answer is unknown
|
||||
(NetOrk/netork#851).
|
||||
|
||||
## [4.32.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `SshServerAuthMixin.get_ssh_server_auth()`: how an OpenSSH server lets people log in --
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication,
|
||||
`AuthenticationMethods` as lists of methods, `PermitRootLogin` and `UsePAM`, from `sshd -T`
|
||||
as root, and sshd's PAM `auth` stack (each module with its control and arguments, includes
|
||||
followed), from `/etc/pam.d` as the login user. `OpenSshServerMixin` carries it, so every
|
||||
driver that can harden sshd has it. `parse_sshd_auth`, `parse_pam_auth`, `parse_pam_files`,
|
||||
`pam_auth_stack` and `SSHD_EFFECTIVE_COMMAND` are public (NetOrk/netork#662).
|
||||
|
||||
## [4.31.1] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
- `get_log_sources()` names as a file's writer only a process that holds it open for writing.
|
||||
A log reader -- CrowdSec, Promtail, rsyslog's `imfile`, a `tail -F` -- was given the file
|
||||
it reads as if it wrote it. A file only readers hold is now listed without a producer
|
||||
(`unit` and `process` None); one only a reader inside a container holds is left out, as its
|
||||
path is that container's view of a mount. The access mode comes from
|
||||
`/proc/<pid>/fdinfo/<fd>`, read in the same command (NetOrk/netork#2092).
|
||||
|
||||
## [4.31.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
|
||||
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
|
||||
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
|
||||
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
|
||||
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
|
||||
public (NetOrk/netork#2053).
|
||||
|
||||
## [4.30.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
|
||||
@@ -124,7 +124,8 @@ application wants made on a device runs as fixed steps the driver implements, wi
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
|
||||
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
|
||||
@@ -174,9 +175,12 @@ A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_fi
|
||||
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
|
||||
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
|
||||
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether dpkg was
|
||||
left halfway (a journal in `/var/lib/dpkg/updates` or a half-configured package; unknown
|
||||
while a package manager is running), and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether its package
|
||||
manager was left halfway (unknown while one is running): dpkg (a journal in
|
||||
`/var/lib/dpkg/updates` or a half-configured package), dnf or yum (two versions of a package
|
||||
in the rpm database, a transaction that never ended), apk (a package marked with broken files
|
||||
or a broken script) or opkg (a package left `unpacked`), with `package_manager` naming which,
|
||||
and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
each filesystem once, in bytes; how full is too full is the caller's decision). `package_updates` holds the shared apt and dnf parsers: apt's suites become
|
||||
an update's `origin`, a `-security` suite makes it a security update, and dnf's security
|
||||
advisories do the same.
|
||||
@@ -202,6 +206,11 @@ store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) a
|
||||
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
|
||||
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
|
||||
user's processes show their files, and the reading says so (`attributed: false`).
|
||||
The same round trip reads where the logs go beyond the host (`forwarding`,
|
||||
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
|
||||
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
|
||||
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
|
||||
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
@@ -290,6 +299,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
|
||||
itself (an appliance with its own database) must not, or opts out with
|
||||
`harden_ssh_server = None`.
|
||||
|
||||
`SshServerAuthMixin` (`get_ssh_server_auth`) says how an OpenSSH server lets people log in:
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication on or off,
|
||||
`AuthenticationMethods` as lists of methods (empty for `any`), `PermitRootLogin` and
|
||||
`UsePAM` (None where sshd has no PAM), from `sshd -T` as root -- the same fixed command the
|
||||
hardening reads. With PAM on, it also names the modules of sshd's PAM `auth` stack
|
||||
with each control and its arguments, read from `/etc/pam.d` in one listing as the
|
||||
login user and followed through `@include`, `include` and `substack`. Which of them is a
|
||||
second factor is the consumer's call. `Match` blocks are not read. `OpenSshServerMixin`
|
||||
carries it; a driver can also mix it in alone and supply `run_command`.
|
||||
|
||||
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
|
||||
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
|
||||
shell: that line, password and all, in the login user's shell history, and files its partial
|
||||
@@ -327,6 +346,28 @@ application's job as a timer where systemd runs, else in `/etc/cron.d` or the us
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
`TimeSyncMixin` (`get_time_sync`, `configure_time_sync`) is how a device keeps its clock:
|
||||
the NTP servers it is configured with, in its order, its time zone as an IANA name, whether
|
||||
it is synchronised now and which client does it (`TimeSyncDict`). `get_time_sync` is
|
||||
declared only; `configure_time_sync(ntp_servers=None, timezone=None)` is a fix template --
|
||||
`None` leaves a value alone, the device ends up with exactly the servers given, in that
|
||||
order, and a `verify` step reads it back. A zone that is not in the IANA database, a
|
||||
malformed server, an empty list or a value outside the driver's `time_sync_writes` is
|
||||
refused before anything is written. A driver states in `time_sync_writes` which of
|
||||
`ntp_servers` and `timezone` it can write (a BMC that runs on UTC only: `ntp_servers`).
|
||||
`napalm_device_types.posix_tz` turns an IANA name into the POSIX rule libc wants in `TZ`
|
||||
(OpenWrt) and that rule into a UTC offset and summer-time transitions (a switch), from the
|
||||
`tzdata` package. `PosixTimeSyncMixin` implements it over `run_command`: one reading as the
|
||||
login user (`timedatectl show`, the active client: chrony's configuration and `chronyc -n
|
||||
sources`, `timedatectl show-timesync`, `ntp.conf`, OpenNTPD's `ntpd.conf`); the zone by
|
||||
`timedatectl set-timezone`; chrony's servers in a file of their own --
|
||||
`/etc/chrony/sources.d/netork.sources` where chrony reads that directory, else a `confdir`
|
||||
or an included `/etc/chrony.d/netork.conf` -- with every other `server`, `pool`, `peer` and
|
||||
foreign `sourcedir` line commented out behind `#napalm-time-sync: `, then `chronyc reload
|
||||
sources` or a restart; systemd-timesyncd's in `/etc/systemd/timesyncd.conf.d/netork.conf`
|
||||
(`NTP=` reset, `FallbackNTP=` emptied), then a restart. ntpd and OpenNTPD are read, not
|
||||
written.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
|
||||
@@ -151,6 +151,7 @@ the driver.
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `package_managers` | rpm, dnf and yum history, apk's and opkg's databases, `/proc/<pid>/comm` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
|
||||
@@ -56,13 +56,17 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
* :class:`~napalm_device_types.packages.PackageManagementMixin`
|
||||
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
|
||||
* :class:`~napalm_device_types.running_backups.RunningBackupsMixin`
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.time_sync.TimeSyncMixin`
|
||||
* :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
|
||||
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
|
||||
@@ -123,7 +127,7 @@ from napalm_device_types.container_engine import (
|
||||
ContainerEngineMixin,
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
from napalm_device_types.dhcp import DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dns_forward import DnsForwardMixin
|
||||
from napalm_device_types.firewall import FirewallDriver
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
@@ -167,12 +171,19 @@ from napalm_device_types.local_accounts import (
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.log_forwarding import (
|
||||
FileInputDict,
|
||||
ForwardRuleDict,
|
||||
LogForwardingDict,
|
||||
parse_rsyslog,
|
||||
)
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourceDict,
|
||||
LogSourcesDict,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
@@ -213,9 +224,24 @@ from napalm_device_types.package_updates import (
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.phone import PhoneDriver
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.posix_tz import (
|
||||
DstTransitionDict,
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.running_backups import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
RunningBackupDict,
|
||||
RunningBackupKind,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
@@ -264,6 +290,14 @@ from napalm_device_types.ssh_server import (
|
||||
SshHardeningResult,
|
||||
plan_ssh_hardening,
|
||||
)
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
PamAuthEntryDict,
|
||||
SshServerAuthDict,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
from napalm_device_types.storage import StorageDriver
|
||||
from napalm_device_types.sudo_history import (
|
||||
HistoryCleanupDict,
|
||||
@@ -285,6 +319,13 @@ from napalm_device_types.systemd import (
|
||||
parse_systemd_services,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
from napalm_device_types.time_sync import (
|
||||
TIME_SYNC_VALUES,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
check_ntp_servers,
|
||||
)
|
||||
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.virtual_ips import (
|
||||
VIRTUAL_IPS_COMMAND,
|
||||
@@ -363,6 +404,7 @@ __all__ = [
|
||||
"DhcpHaDict",
|
||||
"DhcpHaPeerDict",
|
||||
"DeviceTypeDriver",
|
||||
"DhcpApplyError",
|
||||
"DhcpServerMixin",
|
||||
"FingerprintRule",
|
||||
"rules_max_weight",
|
||||
@@ -408,6 +450,11 @@ __all__ = [
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"LOG_SOURCES_COMMAND",
|
||||
"FileInputDict",
|
||||
"ForwardRuleDict",
|
||||
"LogForwardingDict",
|
||||
"parse_log_forwarding",
|
||||
"parse_rsyslog",
|
||||
"LogSourceDict",
|
||||
"LogSourcesDict",
|
||||
"LogSourcesMixin",
|
||||
@@ -447,6 +494,23 @@ __all__ = [
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
"VmStopRequestMixin",
|
||||
"RUNNING_BACKUP_KINDS",
|
||||
"RunningBackupDict",
|
||||
"RunningBackupKind",
|
||||
"RunningBackupsMixin",
|
||||
"TIME_SYNC_VALUES",
|
||||
"TimeSyncDict",
|
||||
"TimeSyncMixin",
|
||||
"check_ntp_servers",
|
||||
"PosixTimeSyncMixin",
|
||||
"parse_time_sync",
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
"VMProvisionIncomplete",
|
||||
"SERVICE_CLUSTERS_COMMAND",
|
||||
"ServiceClustersMixin",
|
||||
@@ -457,6 +521,12 @@ __all__ = [
|
||||
"is_placeholder_identity",
|
||||
"parse_system_identity",
|
||||
"OpenSshServerMixin",
|
||||
"PamAuthEntryDict",
|
||||
"SshServerAuthDict",
|
||||
"SshServerAuthMixin",
|
||||
"SSHD_EFFECTIVE_COMMAND",
|
||||
"pam_auth_stack",
|
||||
"parse_sshd_auth",
|
||||
"SSH_ALGORITHM_CLASSES",
|
||||
"SshHardeningPlan",
|
||||
"SshHardeningResult",
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
"""How chrony is given exactly the time sources a caller names.
|
||||
|
||||
The sources go to one file of their own (:func:`plan_chrony` picks it):
|
||||
|
||||
1. the sources file in a ``sourcedir`` the main configuration reads (Debian's
|
||||
``/etc/chrony/sources.d``): ``chronyc reload sources`` takes it without a restart;
|
||||
2. else ``netork.conf`` -- the include file's name -- in the first ``confdir``;
|
||||
3. else the include file, where an ``include`` pattern already takes it (SUSE's
|
||||
``/etc/chrony.d/*.conf``);
|
||||
4. else the include file, with an ``include`` line appended to the main configuration
|
||||
(RHEL's ``/etc/chrony.conf``).
|
||||
|
||||
Every other source stops counting: a ``server``, ``pool`` or ``peer`` line in any
|
||||
other file chrony reads is commented out behind :data:`COMMENT_MARK`, and so is a
|
||||
``sourcedir`` line other than the one the sources file is in -- the DHCP client's
|
||||
``/run/chrony-dhcp``, whose files are rewritten on every lease and so are left alone.
|
||||
chronyd reads its main configuration only when it starts: a change there means a
|
||||
restart.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from fnmatch import fnmatchcase
|
||||
|
||||
from napalm_device_types._time_sync_reading import SOURCE_LINE, ConfigFile
|
||||
|
||||
#: What a source line taken out of chrony's configuration starts with.
|
||||
COMMENT_MARK = "#napalm-time-sync: "
|
||||
#: The comment above an ``include`` line appended to the main configuration.
|
||||
INCLUDE_COMMENT = "# napalm-time-sync: the time sources"
|
||||
|
||||
_DIRECTORY_LINE = re.compile(r"\s*(sourcedir|confdir|include)\s+(.+?)\s*$")
|
||||
|
||||
|
||||
@dataclass
|
||||
class ChronyPlan:
|
||||
"""What :func:`plan_chrony` decided."""
|
||||
|
||||
target: str # the file the sources go to
|
||||
content: str
|
||||
comment_out: dict[str, list[int]] = field(default_factory=dict) # path: line numbers
|
||||
include_in: str | None = None # the main configuration an include of target is appended to
|
||||
reload: bool = False # True: ``chronyc reload sources`` takes the change
|
||||
|
||||
|
||||
def source_line(server: str) -> str:
|
||||
"""The line a server is written as: a pool's name as a ``pool``, anything else a ``server``."""
|
||||
directive = "pool" if ".pool." in f".{server.lower()}." else "server"
|
||||
return f"{directive} {server} iburst"
|
||||
|
||||
|
||||
def _directives(main: ConfigFile) -> list[tuple[int, str, list[str]]]:
|
||||
found = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
match = _DIRECTORY_LINE.match(line)
|
||||
if match:
|
||||
found.append((number, match[1], match[2].split()))
|
||||
return found
|
||||
|
||||
|
||||
def _arguments(directives: list[tuple[int, str, list[str]]], kind: str) -> list[str]:
|
||||
return [arg for _, k, args in directives if k == kind for arg in args]
|
||||
|
||||
|
||||
def _target(main: ConfigFile, sources_file: str, include_file: str) -> tuple[str, str | None, bool]:
|
||||
"""The file the sources go to, the main configuration to include it from, and
|
||||
whether it is a sources file."""
|
||||
directives = _directives(main)
|
||||
if posixpath.dirname(sources_file) in _arguments(directives, "sourcedir"):
|
||||
return sources_file, None, True
|
||||
confdirs = _arguments(directives, "confdir")
|
||||
if confdirs:
|
||||
return posixpath.join(confdirs[0], posixpath.basename(include_file)), None, False
|
||||
if any(fnmatchcase(include_file, p) for p in _arguments(directives, "include")):
|
||||
return include_file, None, False
|
||||
return include_file, main.path, False
|
||||
|
||||
|
||||
def _dropped_sourcedirs(main: ConfigFile, kept: str | None) -> set[int]:
|
||||
"""The ``sourcedir`` lines of *main* that do not name the directory *kept*."""
|
||||
return {
|
||||
number
|
||||
for number, kind, args in _directives(main)
|
||||
if kind == "sourcedir" and (kept is None or kept not in args)
|
||||
}
|
||||
|
||||
|
||||
def _lines_to_comment(f: ConfigFile, dropped: set[int]) -> list[int]:
|
||||
return [
|
||||
number
|
||||
for number, line in enumerate(f.lines, start=1)
|
||||
if SOURCE_LINE.match(line) or (f.kind == "main" and number in dropped)
|
||||
]
|
||||
|
||||
|
||||
def plan_chrony(
|
||||
files: list[ConfigFile], servers: list[str], *, sources_file: str, include_file: str
|
||||
) -> ChronyPlan:
|
||||
"""Where *servers* go and what else is commented out.
|
||||
|
||||
:raises RuntimeError: when the reading found no chrony configuration.
|
||||
"""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
raise RuntimeError("chrony runs, but no chrony configuration was found to write to")
|
||||
target, include_in, is_sources = _target(main, sources_file, include_file)
|
||||
dropped = _dropped_sourcedirs(main, posixpath.dirname(target) if is_sources else None)
|
||||
comment_out = _comment_out(files, target, dropped)
|
||||
return ChronyPlan(
|
||||
target=target,
|
||||
content="".join(f"{source_line(s)}\n" for s in servers),
|
||||
comment_out=comment_out,
|
||||
include_in=include_in,
|
||||
reload=is_sources and include_in is None and _only_sources_files(files, comment_out),
|
||||
)
|
||||
|
||||
|
||||
def _only_sources_files(files: list[ConfigFile], changed: dict[str, list[int]]) -> bool:
|
||||
"""Whether every file that changes is one ``chronyc reload sources`` reads again."""
|
||||
return all(f.kind == "sourcedir" for f in files if f.path in changed)
|
||||
|
||||
|
||||
def _comment_out(files: list[ConfigFile], target: str, dropped: set[int]) -> dict[str, list[int]]:
|
||||
"""Per file, the lines that take a source; a dropped ``sourcedir``'s files are not read."""
|
||||
found: dict[str, list[int]] = {}
|
||||
for f in files:
|
||||
if f.path == target or (f.kind == "sourcedir" and f.at in dropped):
|
||||
continue
|
||||
numbers = _lines_to_comment(f, dropped)
|
||||
if numbers:
|
||||
found[f.path] = numbers
|
||||
return found
|
||||
|
||||
|
||||
def comment_out_command(path: str, numbers: list[int]) -> str:
|
||||
"""``sed`` that comments out those lines of *path* that still hold a source or a sourcedir."""
|
||||
edits = " ".join(
|
||||
"-e "
|
||||
+ shlex.quote(
|
||||
f"{n}{{/^[[:space:]]*(server|pool|peer|sourcedir)[[:space:]]/s/^/{COMMENT_MARK}/;}}"
|
||||
)
|
||||
for n in numbers
|
||||
)
|
||||
return f"sed -i -E {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def include_command(main: str, target: str) -> str:
|
||||
"""Append an ``include`` of *target* to *main*, on a line of its own."""
|
||||
lines = " ".join(shlex.quote(line) for line in (INCLUDE_COMMENT, f"include {target}"))
|
||||
return (
|
||||
f'{{ [ -z "$(tail -c 1 -- {shlex.quote(main)})" ] || echo; '
|
||||
f"printf '%s\\n' {lines}; }} >> {shlex.quote(main)}"
|
||||
)
|
||||
@@ -0,0 +1,301 @@
|
||||
"""The reading behind :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`.
|
||||
|
||||
One command, as the login user, prints framed sections (``@@ts <name> ...``):
|
||||
|
||||
* ``timedatectl`` -- ``timedatectl show``: the zone, and whether the kernel clock is
|
||||
synchronised;
|
||||
* ``zone`` -- ``/etc/timezone`` or the target of ``/etc/localtime``, for a host
|
||||
without timedatectl;
|
||||
* ``active`` -- one ``<unit> <daemon> <state>`` line per time sync service;
|
||||
* ``timesync`` -- ``timedatectl show-timesync``: systemd-timesyncd's servers;
|
||||
* ``chronyc`` -- ``chronyc -n sources``: whether chronyd has selected a source;
|
||||
* ``chrony <line> <kind> <path>`` -- chrony's main configuration (line 0, kind
|
||||
``main``), then each file an ``include``, ``confdir`` or ``sourcedir`` on that
|
||||
line of it reads;
|
||||
* ``ntpd <path>``, ``openntpd <path>`` -- their configuration;
|
||||
* ``end`` -- the reading is complete.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from napalm_device_types.time_sync import TimeSyncDict
|
||||
|
||||
_MARK = "@@ts "
|
||||
|
||||
#: ``<unit> <daemon>``: the services that keep a host's clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: tuple[tuple[str, str], ...] = (
|
||||
("chrony", "chrony"),
|
||||
("chronyd", "chrony"),
|
||||
("systemd-timesyncd", "systemd-timesyncd"),
|
||||
("ntpsec", "ntpd"),
|
||||
("ntp", "ntpd"),
|
||||
("ntpd", "ntpd"),
|
||||
("openntpd", "openntpd"),
|
||||
)
|
||||
|
||||
#: A source line of chrony's or ntpd's configuration: the directive and what follows.
|
||||
SOURCE_LINE = re.compile(r"\s*(server|pool|peer)\s+(.*)")
|
||||
_OPENNTPD_SOURCE = re.compile(r"\s*(servers?)\s+(\S+)")
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
#: ntpd's reference clocks are addresses in 127.127.0.0/16, not servers.
|
||||
_REFCLOCK = "127.127."
|
||||
|
||||
# The files a chrony configuration reads, each after a marker with the line that names it.
|
||||
_CHRONY_FOLLOW = """\
|
||||
awk '$1=="include"||$1=="confdir"||$1=="sourcedir"{for(i=2;i<=NF;i++)print NR, $1, $i}' "$c" |
|
||||
while read -r n k p; do
|
||||
case $k in include) set -- $p ;; confdir) set -- "$p"/*.conf ;; *) set -- "$p"/*.sources ;; esac
|
||||
for f in "$@"; do
|
||||
[ -f "$f" ] && { printf '@@ts chrony %s %s %s\\n' "$n" "$k" "$f"; cat -- "$f"; echo; }
|
||||
done
|
||||
done"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReadPaths:
|
||||
"""Where a host keeps what the reading reads."""
|
||||
|
||||
units: tuple[tuple[str, str], ...]
|
||||
chrony_configs: tuple[str, ...]
|
||||
ntpd_configs: tuple[str, ...]
|
||||
openntpd_configs: tuple[str, ...]
|
||||
time_zone_sh: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConfigFile:
|
||||
"""One configuration file as the reading printed it."""
|
||||
|
||||
path: str
|
||||
kind: str # "main", "include", "confdir", "sourcedir"; "ntpd", "openntpd"
|
||||
at: int # the line of the main configuration that names it; 0 for that file itself
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""Everything one reading said."""
|
||||
|
||||
timedatectl: dict[str, str] = field(default_factory=dict)
|
||||
zone_files: list[str] = field(default_factory=list)
|
||||
active: list[tuple[str, str]] = field(default_factory=list) # (unit, daemon), active ones
|
||||
timesync: dict[str, str] = field(default_factory=dict)
|
||||
chronyc: str = ""
|
||||
chrony: list[ConfigFile] = field(default_factory=list)
|
||||
ntpd: ConfigFile | None = None
|
||||
openntpd: ConfigFile | None = None
|
||||
|
||||
@property
|
||||
def daemon(self) -> str | None:
|
||||
return self.active[0][1] if self.active else None
|
||||
|
||||
@property
|
||||
def unit(self) -> str | None:
|
||||
return self.active[0][0] if self.active else None
|
||||
|
||||
|
||||
def _first_file(kind: str, paths: tuple[str, ...]) -> str:
|
||||
candidates = " ".join(shlex.quote(p) for p in paths)
|
||||
return (
|
||||
f'for c in {candidates}; do [ -f "$c" ] && '
|
||||
f'{{ printf \'@@ts {kind} %s\\n\' "$c"; cat -- "$c"; echo; break; }}; done'
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: ReadPaths) -> str:
|
||||
"""The command that prints the reading; ``sh`` runs it, whatever the login shell."""
|
||||
active = "\n".join(
|
||||
f"printf '%s %s %s\\n' {shlex.quote(unit)} {shlex.quote(daemon)} "
|
||||
f'"$(systemctl is-active {shlex.quote(unit + ".service")} 2>/dev/null)"'
|
||||
for unit, daemon in paths.units
|
||||
)
|
||||
chrony = " ".join(shlex.quote(p) for p in paths.chrony_configs)
|
||||
script = "\n".join(
|
||||
[
|
||||
"echo '@@ts timedatectl'",
|
||||
"timedatectl show 2>/dev/null",
|
||||
"echo '@@ts zone'",
|
||||
f"{{ {paths.time_zone_sh}; }}",
|
||||
"echo '@@ts active'",
|
||||
active,
|
||||
"echo '@@ts timesync'",
|
||||
"timedatectl show-timesync 2>/dev/null",
|
||||
"echo '@@ts chronyc'",
|
||||
"chronyc -n sources 2>/dev/null",
|
||||
f'for c in {chrony}; do [ -f "$c" ] || continue',
|
||||
'printf \'@@ts chrony 0 main %s\\n\' "$c"; cat -- "$c"; echo',
|
||||
_CHRONY_FOLLOW,
|
||||
"break; done",
|
||||
_first_file("ntpd", paths.ntpd_configs),
|
||||
_first_file("openntpd", paths.openntpd_configs),
|
||||
"echo '@@ts end'",
|
||||
]
|
||||
)
|
||||
return f"sh -c {shlex.quote(script)}"
|
||||
|
||||
|
||||
def parse_key_values(text: str) -> dict[str, str]:
|
||||
"""``Key=value`` lines (``timedatectl show``, ``show-timesync``)."""
|
||||
values: dict[str, str] = {}
|
||||
for line in text.splitlines():
|
||||
key, sep, value = line.partition("=")
|
||||
if sep and key.strip():
|
||||
values[key.strip()] = value.strip()
|
||||
return values
|
||||
|
||||
|
||||
def _sections(text: str) -> list[tuple[list[str], list[str]]]:
|
||||
sections: list[tuple[list[str], list[str]]] = []
|
||||
for line in text.splitlines():
|
||||
if line.startswith(_MARK):
|
||||
sections.append((line[len(_MARK) :].split(maxsplit=3), []))
|
||||
elif sections:
|
||||
sections[-1][1].append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _active(lines: list[str]) -> list[tuple[str, str]]:
|
||||
rows = [line.split() for line in lines]
|
||||
return [(r[0], r[1]) for r in rows if len(r) == 3 and r[2] == "active"]
|
||||
|
||||
|
||||
def _take(reading: Reading, header: list[str], lines: list[str]) -> None:
|
||||
name, text = header[0], "\n".join(lines)
|
||||
if name == "chrony" and len(header) == 4:
|
||||
reading.chrony.append(ConfigFile(header[3], header[2], int(header[1]), lines))
|
||||
elif name in ("ntpd", "openntpd") and len(header) == 2:
|
||||
setattr(reading, name, ConfigFile(header[1], name, 0, lines))
|
||||
else:
|
||||
simple = {
|
||||
"timedatectl": ("timedatectl", parse_key_values(text)),
|
||||
"zone": ("zone_files", lines),
|
||||
"active": ("active", _active(lines)),
|
||||
"timesync": ("timesync", parse_key_values(text)),
|
||||
"chronyc": ("chronyc", text),
|
||||
}
|
||||
if name in simple:
|
||||
setattr(reading, *simple[name])
|
||||
|
||||
|
||||
def parse_reading(text: str) -> Reading:
|
||||
"""The reading's sections. :raises ValueError: for a reading cut short."""
|
||||
sections = _sections(text)
|
||||
if not sections or sections[-1][0] != ["end"]:
|
||||
raise ValueError("the time sync reading was cut short")
|
||||
reading = Reading()
|
||||
for header, lines in sections:
|
||||
_take(reading, header, lines)
|
||||
return reading
|
||||
|
||||
|
||||
def _source_name(rest: str) -> str | None:
|
||||
"""The server in what follows ``server``/``pool``/``peer``: ntpd puts ``-4`` first."""
|
||||
return next((word for word in rest.split() if not word.startswith("-")), None)
|
||||
|
||||
|
||||
def chrony_servers(files: list[ConfigFile]) -> list[str]:
|
||||
"""chrony's sources in the order it reads them: each file in place of the line naming it."""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
return []
|
||||
found: list[str] = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
found.extend(_sources([line]))
|
||||
for included in (f for f in files if f.at == number and f.kind != "main"):
|
||||
found.extend(_sources(included.lines))
|
||||
return found
|
||||
|
||||
|
||||
def _sources(lines: list[str]) -> list[str]:
|
||||
names = []
|
||||
for line in lines:
|
||||
match = SOURCE_LINE.match(line)
|
||||
if match and (name := _source_name(match[2])):
|
||||
names.append(name)
|
||||
return names
|
||||
|
||||
|
||||
def ntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""ntpd's servers, pools and peers; reference clocks are not servers."""
|
||||
return [name for name in _sources(lines) if not name.startswith(_REFCLOCK)]
|
||||
|
||||
|
||||
def openntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""OpenNTPD's ``server`` and ``servers``; ``constraint`` and ``sensor`` are no sources."""
|
||||
return [m[2] for line in lines if (m := _OPENNTPD_SOURCE.match(line))]
|
||||
|
||||
|
||||
def timesyncd_servers(values: dict[str, str]) -> list[str]:
|
||||
"""systemd-timesyncd's servers in the order it tries them; the fallback only alone."""
|
||||
found: list[str] = []
|
||||
for key in ("RuntimeNTPServers", "SystemNTPServers", "LinkNTPServers"):
|
||||
for server in values.get(key, "").split():
|
||||
if server not in found:
|
||||
found.append(server)
|
||||
return found or values.get("FallbackNTPServers", "").split()
|
||||
|
||||
|
||||
def chrony_synchronized(sources: str) -> bool | None:
|
||||
"""Whether chronyd has selected a source (``^*``); None for no answer from chronyc."""
|
||||
rows = [line for line in sources.splitlines() if line[:1] in ("^", "=", "#")]
|
||||
if not rows:
|
||||
return None
|
||||
return any(line[1:2] == "*" for line in rows)
|
||||
|
||||
|
||||
def _zone_name(value: str) -> str | None:
|
||||
value = value.strip()
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
for prefix in ("posix/", "right/"):
|
||||
value = value.removeprefix(prefix)
|
||||
return value if _ZONE_NAME.fullmatch(value) else None
|
||||
|
||||
|
||||
def _time_zone(reading: Reading) -> str | None:
|
||||
named = reading.timedatectl.get("Timezone", "")
|
||||
if named and named != "n/a":
|
||||
return _zone_name(named)
|
||||
return next((z for line in reading.zone_files if (z := _zone_name(line))), None)
|
||||
|
||||
|
||||
def _servers(reading: Reading) -> list[str]:
|
||||
if reading.daemon == "chrony":
|
||||
return chrony_servers(reading.chrony)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return timesyncd_servers(reading.timesync)
|
||||
if reading.daemon == "ntpd" and reading.ntpd is not None:
|
||||
return ntpd_servers(reading.ntpd.lines)
|
||||
if reading.daemon == "openntpd" and reading.openntpd is not None:
|
||||
return openntpd_servers(reading.openntpd.lines)
|
||||
return []
|
||||
|
||||
|
||||
def _synchronized(reading: Reading) -> bool | None:
|
||||
if reading.daemon == "chrony":
|
||||
chrony = chrony_synchronized(reading.chronyc)
|
||||
if chrony is not None:
|
||||
return chrony
|
||||
return {"yes": True, "no": False}.get(reading.timedatectl.get("NTPSynchronized", ""))
|
||||
|
||||
|
||||
def time_sync_of(reading: Reading) -> TimeSyncDict:
|
||||
"""What a reading says, as ``get_time_sync`` returns it."""
|
||||
return {
|
||||
"ntp_servers": _servers(reading),
|
||||
"timezone": _time_zone(reading),
|
||||
"synchronized": _synchronized(reading),
|
||||
"daemon": reading.daemon,
|
||||
}
|
||||
|
||||
|
||||
def parse_time_sync(text: str) -> TimeSyncDict:
|
||||
"""The reading's output as ``get_time_sync`` returns it.
|
||||
|
||||
:raises ValueError: for a reading cut short.
|
||||
"""
|
||||
return time_sync_of(parse_reading(text))
|
||||
+55
-16
@@ -58,6 +58,19 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
|
||||
_HEX_ONLY = re.compile(r"[^0-9a-f]")
|
||||
|
||||
|
||||
class DhcpApplyError(RuntimeError):
|
||||
"""Some reservations of an applied set were refused by the device.
|
||||
|
||||
Raised after the rest were written and committed: ``failed`` of ``attempted``
|
||||
changes did not go through, and each was named in a ``[failed]`` progress line.
|
||||
"""
|
||||
|
||||
def __init__(self, failed: int, attempted: int) -> None:
|
||||
self.failed = failed
|
||||
self.attempted = attempted
|
||||
super().__init__(f"{failed} of {attempted} DHCP reservation change(s) were refused")
|
||||
|
||||
|
||||
def normalize_mac(mac: str | None) -> str:
|
||||
"""Reduces a MAC address to lowercase colon-separated form.
|
||||
|
||||
@@ -298,29 +311,55 @@ class DhcpServerMixin:
|
||||
committing reloads the DHCP daemon and drops in-flight requests, which
|
||||
is too high a price for a no-op run.
|
||||
|
||||
A change the device refuses (say, an address in a subnet its DHCP server
|
||||
does not manage) does not stop the others: it gets a ``[failed]`` line,
|
||||
the rest are written and committed, and ``DhcpApplyError`` is raised at
|
||||
the end. Any non-empty diff commits, even one whose every write was
|
||||
refused, so that writes an earlier, interrupted run saved take effect.
|
||||
|
||||
:param desired: The complete desired reservation set.
|
||||
:yields: Progress lines, one per applied add/update, plus a final
|
||||
commit line.
|
||||
:yields: Progress lines, one per applied, updated or refused change,
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_reservations(desired)
|
||||
|
||||
for reservation in diff["add"]:
|
||||
self.apply_dhcp_reservation(reservation)
|
||||
yield f"[add] {reservation['mac']} -> {reservation['ip']}"
|
||||
|
||||
for entry in diff["update"]:
|
||||
self.apply_dhcp_reservation(entry["reservation"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield (
|
||||
f"[update] {entry['reservation']['mac']} -> {entry['reservation']['ip']} ({fields})"
|
||||
)
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
return
|
||||
|
||||
added = updated = refused = 0
|
||||
|
||||
for reservation in diff["add"]:
|
||||
line, ok = self._write_reservation(reservation, None, "[add]")
|
||||
added, refused = added + ok, refused + (not ok)
|
||||
yield line
|
||||
|
||||
for entry in diff["update"]:
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
line, ok = self._write_reservation(entry["reservation"], entry["uuid"], "[update]")
|
||||
updated, refused = updated + ok, refused + (not ok)
|
||||
yield f"{line} ({fields})" if ok else line
|
||||
|
||||
# Committed even when every write was refused: a run that stopped before its
|
||||
# commit left saved writes behind, which the device reports as configured, so
|
||||
# no later diff would ever commit them.
|
||||
self.commit_dhcp_reservations()
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
yield f"[commit] applied {added} add(s), {updated} update(s)"
|
||||
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted)
|
||||
|
||||
def _write_reservation(
|
||||
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
|
||||
) -> tuple[str, bool]:
|
||||
"""Write one reservation; its progress line, and whether the device took it."""
|
||||
target = f"{reservation['mac']} -> {reservation['ip']}"
|
||||
try:
|
||||
self.apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
except Exception as exc:
|
||||
return f"[failed] {target}: {exc}", False
|
||||
return f"{tag} {target}", True
|
||||
|
||||
def diff_dhcp_subnets(self, desired: list[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
|
||||
"""
|
||||
|
||||
@@ -43,7 +43,9 @@ exits quietly every day (NetOrk/netork#738). While dpkg, apt, aptitude or
|
||||
unattended-upgrade is running, packages are halfway on purpose -- also between
|
||||
two of apt's dpkg calls -- so the answer is then ``None``, as on a host without
|
||||
dpkg. unattended-upgrade is told from Ubuntu's idle shutdown helper by its
|
||||
command line (:data:`BUSY_CHECK`).
|
||||
command line (:data:`BUSY_CHECK`). On a host without dpkg, dnf or yum, apk and opkg
|
||||
are read the same way (:mod:`napalm_device_types.package_managers`,
|
||||
NetOrk/netork#851), and ``package_manager`` names the one the answer is about.
|
||||
|
||||
**Filesystems** are how full ``/``, ``/boot`` and ``/var`` are, from POSIX
|
||||
``df -Pk``: where an upgrade unpacks a kernel, builds its initrd and keeps the
|
||||
@@ -65,6 +67,11 @@ from napalm_device_types.models import (
|
||||
HostStatusDict,
|
||||
InterruptedPackageDict,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
PackageManagerReading,
|
||||
parse_package_managers,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "HSTAT_BEGIN"
|
||||
@@ -117,6 +124,7 @@ HOST_STATUS_COMMAND = (
|
||||
f"echo '[dpkg-journal]'; ls -1 {_DPKG_JOURNAL} 2>/dev/null | head -n 20; "
|
||||
"echo '[dpkg-audit]'; dpkg-query -W -f='${db:Status-Abbrev} ${Package}\\n' 2>/dev/null "
|
||||
"| awk 'substr($0, 2, 1) ~ /[HUFWt]/ || substr($0, 3, 1) == \"R\"' | head -n 50; fi; "
|
||||
f"{PACKAGE_MANAGERS_COMMAND}"
|
||||
f"echo '[filesystems]'; df -Pk {_FILESYSTEMS}; "
|
||||
f"echo '[timers]'; for u in {_APT_TIMER} {' '.join(_DNF_TIMERS)}; do "
|
||||
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled "$u" 2>/dev/null)"; done; '
|
||||
@@ -287,12 +295,24 @@ def _halfway(line: str) -> InterruptedPackageDict | None:
|
||||
return {"name": name, "state": state} if name and state else None
|
||||
|
||||
|
||||
def _dpkg(sections: dict[str, list[str]]) -> tuple[bool | None, list[InterruptedPackageDict]]:
|
||||
if "dpkg-audit" not in sections or "dpkg-busy" in sections:
|
||||
return None, []
|
||||
def _dpkg(sections: dict[str, list[str]]) -> PackageManagerReading | None:
|
||||
if "dpkg-audit" not in sections:
|
||||
return None
|
||||
if "dpkg-busy" in sections:
|
||||
return PackageManagerReading("dpkg", None, [])
|
||||
packages = [p for p in map(_halfway, sections["dpkg-audit"]) if p is not None]
|
||||
journal = any(_JOURNAL_ENTRY.fullmatch(f) for f in sections.get("dpkg-journal") or [])
|
||||
return bool(packages) or journal, packages
|
||||
return PackageManagerReading("dpkg", bool(packages) or journal, packages)
|
||||
|
||||
|
||||
def _package_manager(sections: dict[str, list[str]]) -> PackageManagerReading:
|
||||
"""The reading that says most: one left halfway, then one that is not, then a busy one."""
|
||||
readings = [r for r in (_dpkg(sections), *parse_package_managers(sections)) if r is not None]
|
||||
for interrupted in (True, False):
|
||||
found = next((r for r in readings if r.interrupted is interrupted), None)
|
||||
if found is not None:
|
||||
return found
|
||||
return readings[0] if readings else PackageManagerReading(None, None, [])
|
||||
|
||||
|
||||
def _filesystems(sections: dict[str, list[str]]) -> list[FilesystemUsageDict] | None:
|
||||
@@ -318,15 +338,16 @@ def parse_host_status(output: str) -> HostStatusDict:
|
||||
"""
|
||||
sections = _sections(output)
|
||||
required, reason = _reboot(sections)
|
||||
interrupted, packages = _dpkg(sections)
|
||||
manager = _package_manager(sections)
|
||||
return {
|
||||
"reboot_required": required,
|
||||
"reboot_reason": reason,
|
||||
"auto_updates": _auto_updates(sections),
|
||||
"auto_updates_last_success": _last_success(sections),
|
||||
"auto_updates_interval": _interval(sections),
|
||||
"package_manager_interrupted": interrupted,
|
||||
"interrupted_packages": packages,
|
||||
"package_manager": manager.manager,
|
||||
"package_manager_interrupted": manager.interrupted,
|
||||
"interrupted_packages": manager.packages,
|
||||
"filesystems": _filesystems(sections),
|
||||
}
|
||||
|
||||
@@ -356,6 +377,7 @@ class HostStatusMixin:
|
||||
* auto_updates (bool or None)
|
||||
* auto_updates_last_success (Unix time or None)
|
||||
* auto_updates_interval (seconds or None)
|
||||
* package_manager (dpkg, dnf, yum, apk, opkg, or None)
|
||||
* package_manager_interrupted (bool or None)
|
||||
* interrupted_packages (list of {name, state})
|
||||
* filesystems (list of {mountpoint, total, used, available} in bytes, or None)
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
|
||||
in, and whether it gets the journal (NetOrk/netork#2053).
|
||||
|
||||
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
|
||||
this says who reads it on. It is read in the same command, so a reading of
|
||||
the logs a host writes also says which of them leave it.
|
||||
|
||||
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
|
||||
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
|
||||
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
|
||||
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
|
||||
*complete* when it takes every message of its ruleset -- ``*.*`` with no
|
||||
condition around it. Anything narrower is partial, and keeps its selector or
|
||||
condition for the reader to see: which facility a unit logs to is not
|
||||
computed (decided in #2053).
|
||||
|
||||
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
|
||||
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
|
||||
own a host runs on its distribution's default, which ``journald.conf`` ships
|
||||
commented out: ``#ForwardToSyslog=yes``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class ForwardRuleDict(TypedDict):
|
||||
"""One rule sending messages to another host."""
|
||||
|
||||
protocol: str # "udp", "tcp" or "relp"
|
||||
target: str # the host, as configured
|
||||
port: int | None # None: the protocol's default
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
|
||||
|
||||
path: str # as configured: may hold a glob
|
||||
ruleset: str | None
|
||||
|
||||
|
||||
class RsyslogDict(TypedDict):
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
imjournal: bool
|
||||
imuxsock: bool
|
||||
|
||||
|
||||
class LogForwardingDict(TypedDict):
|
||||
"""Where a host's logs go beyond it.
|
||||
|
||||
``journal`` is None when it cannot be told whether the journal reaches
|
||||
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
|
||||
"""
|
||||
|
||||
rsyslog: bool # rsyslogd runs
|
||||
journal: bool | None # the journal reaches rsyslog
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
|
||||
|
||||
#: Selectors that take every message.
|
||||
_EVERYTHING = {"*.*", "*.debug"}
|
||||
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
|
||||
_CALL_RE = re.compile(
|
||||
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
|
||||
)
|
||||
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
|
||||
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
|
||||
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
|
||||
_FORWARD_RE = re.compile(
|
||||
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
|
||||
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
|
||||
)
|
||||
_FORWARD_TYPES = {"omfwd", "omrelp"}
|
||||
_TRUE = {"yes", "true", "1", "on"}
|
||||
|
||||
|
||||
def _uncomment(text: str) -> str:
|
||||
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
|
||||
out: list[str] = []
|
||||
quoted = commented = False
|
||||
for char in text:
|
||||
if char == "\n":
|
||||
quoted = commented = False
|
||||
elif commented:
|
||||
continue
|
||||
elif char == '"':
|
||||
quoted = not quoted
|
||||
elif char == "#" and not quoted:
|
||||
commented = True
|
||||
continue
|
||||
out.append(char)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
class _Pieces:
|
||||
"""Cuts a configuration into statements, ``{`` and ``}``: a line runs on
|
||||
while a bracket is open, and a brace outside one stands as a piece of its own."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.pieces: list[str] = []
|
||||
self.current: list[str] = []
|
||||
self.depth = 0
|
||||
self.quoted = False
|
||||
|
||||
def close(self) -> None:
|
||||
piece = "".join(self.current).strip()
|
||||
if piece:
|
||||
self.pieces.append(piece)
|
||||
self.current.clear()
|
||||
|
||||
def bracket(self, char: str) -> None:
|
||||
if char in "([":
|
||||
self.depth += 1
|
||||
elif char in ")]":
|
||||
self.depth = max(self.depth - 1, 0)
|
||||
|
||||
def feed(self, char: str) -> None:
|
||||
if char == '"':
|
||||
self.quoted = not self.quoted
|
||||
elif not self.quoted:
|
||||
self.bracket(char)
|
||||
if self.depth == 0 and char in "{}\n":
|
||||
self.close()
|
||||
if char != "\n":
|
||||
self.pieces.append(char)
|
||||
return
|
||||
self.current.append(char)
|
||||
|
||||
|
||||
def _pieces(text: str) -> list[str]:
|
||||
"""Statements, ``{`` and ``}`` in order (:class:`_Pieces`)."""
|
||||
cutter = _Pieces()
|
||||
for char in text:
|
||||
cutter.feed(char)
|
||||
cutter.close()
|
||||
return cutter.pieces
|
||||
|
||||
|
||||
def _params(args: str) -> dict[str, str]:
|
||||
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
|
||||
|
||||
|
||||
def _port(value: str | None) -> int | None:
|
||||
return int(value) if value and value.isdigit() else None
|
||||
|
||||
|
||||
class _Reader:
|
||||
"""Walks the pieces of a configuration, keeping the blocks it is in."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.parsed: RsyslogDict = {
|
||||
"rules": [],
|
||||
"file_inputs": [],
|
||||
"imjournal": False,
|
||||
"imuxsock": False,
|
||||
}
|
||||
self.stack: list[tuple[str, str | None]] = []
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
return names[-1] if names else self.legacy_ruleset
|
||||
|
||||
def condition(self, once: str | None) -> str | None:
|
||||
conditions = [value for kind, value in self.stack if kind == "if"]
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
{
|
||||
"protocol": protocol,
|
||||
"target": target,
|
||||
"port": port,
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
}
|
||||
)
|
||||
|
||||
def piece(self, piece: str) -> None:
|
||||
if piece == "{":
|
||||
self.stack.append(self.pending or ("block", None))
|
||||
self.pending = None
|
||||
return
|
||||
if piece == "}":
|
||||
if self.stack:
|
||||
self.stack.pop()
|
||||
return
|
||||
if piece == "else":
|
||||
self.pending = ("if", "else")
|
||||
return
|
||||
# A condition whose action follows on its own: the action is under it.
|
||||
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
|
||||
self.pending = None
|
||||
match = _IF_RE.match(piece)
|
||||
if match:
|
||||
if match["rest"]:
|
||||
self.statement(match["rest"], once=match["cond"])
|
||||
else:
|
||||
self.pending = ("if", match["cond"])
|
||||
return
|
||||
self.statement(piece, once=once)
|
||||
|
||||
def statement(self, text: str, once: str | None) -> None:
|
||||
call = _CALL_RE.match(text)
|
||||
if call:
|
||||
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
|
||||
elif text.startswith("$"):
|
||||
self.directive(text)
|
||||
else:
|
||||
self.legacy(text, once)
|
||||
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
self.parsed["file_inputs"].append(
|
||||
{"path": params["file"], "ruleset": params.get("ruleset")}
|
||||
)
|
||||
elif name == "action":
|
||||
self.action(params, before, once)
|
||||
|
||||
def action(self, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
selector = self.last_selector if before == "&" else (before or "*.*")
|
||||
self.last_selector = selector
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
self.parsed["imjournal"] = True
|
||||
elif name == "imuxsock":
|
||||
self.parsed["imuxsock"] = True
|
||||
|
||||
def directive(self, text: str) -> None:
|
||||
name, _, value = text.partition(" ")
|
||||
name, value = name.lower(), value.strip()
|
||||
if name == "$modload":
|
||||
self.module(value)
|
||||
elif name == "$inputfilename" and value:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
if not match:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
forward = _FORWARD_RE.match(match["action"].strip())
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
|
||||
)
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
read for its forwarding rules, its file inputs, and the inputs that take
|
||||
the journal."""
|
||||
reader = _Reader()
|
||||
for piece in _pieces(_uncomment(text)):
|
||||
reader.piece(piece)
|
||||
return reader.parsed
|
||||
|
||||
|
||||
def journal_to_syslog(lines: list[str]) -> bool | None:
|
||||
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
|
||||
the order journald reads them: the last setting wins; with none, the
|
||||
default the distribution ships commented out; None when nothing says."""
|
||||
setting = shipped = None
|
||||
for line in lines:
|
||||
commented = line.lstrip().startswith("#")
|
||||
_, _, value = line.lstrip("# \t").partition("=")
|
||||
if commented:
|
||||
shipped = value.strip().lower() in _TRUE
|
||||
else:
|
||||
setting = value.strip().lower() in _TRUE
|
||||
return setting if setting is not None else shipped
|
||||
|
||||
|
||||
def log_forwarding(
|
||||
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
|
||||
) -> LogForwardingDict:
|
||||
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
|
||||
if not running:
|
||||
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
journal: bool | None
|
||||
if rsyslog["imjournal"]:
|
||||
journal = True
|
||||
elif rsyslog["imuxsock"]:
|
||||
journal = journal_to_syslog(journald_lines)
|
||||
else:
|
||||
journal = False
|
||||
return {
|
||||
"rsyslog": True,
|
||||
"journal": journal,
|
||||
"rules": rsyslog["rules"],
|
||||
"file_inputs": rsyslog["file_inputs"],
|
||||
}
|
||||
@@ -14,10 +14,22 @@ whose path is the container's own. The engine's own file of a container's
|
||||
output (``/var/lib/docker/containers/<id>/…``) is that container's stream, not
|
||||
a file of ``dockerd``.
|
||||
|
||||
**Who writes it.** ``/proc/<pid>/cgroup`` says which systemd service or
|
||||
container the first process holding a file runs in, as for listening sockets
|
||||
(:mod:`napalm_device_types.listening`). journald's own files, files already
|
||||
deleted and a database's write-ahead log (LevelDB's ``000004.log``) are left out.
|
||||
**Who writes it.** Only a process holding the file open for writing writes
|
||||
it: the access mode in ``/proc/<pid>/fdinfo/<fd>``'s ``flags`` is ``O_WRONLY`` or
|
||||
``O_RDWR``. A reader -- CrowdSec, Promtail, rsyslog's ``imfile``, a ``tail -F``
|
||||
-- holds it too, and is no producer (NetOrk/netork#2092). ``/proc/<pid>/cgroup``
|
||||
says which systemd service or container the first writer runs in, as for
|
||||
listening sockets (:mod:`napalm_device_types.listening`). A file only readers
|
||||
hold is listed without a producer; one only readers in a container hold is left
|
||||
out, its path being the reader's view of a mount. journald's own files, files
|
||||
already deleted and a database's write-ahead log (LevelDB's ``000004.log``) are
|
||||
left out.
|
||||
|
||||
**Where they go** is read in the same round trip
|
||||
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
|
||||
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
|
||||
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
|
||||
reads them.
|
||||
|
||||
**One round trip, root, and without it.** As for listening sockets the script
|
||||
goes as one ``sh -c`` argument. Without root only the login user's processes
|
||||
@@ -36,6 +48,7 @@ from shlex import quote
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
|
||||
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
@@ -51,7 +64,7 @@ class LogSourceDict(TypedDict):
|
||||
path: str | None # the file; None for a journal
|
||||
unit: str | None # the journal's unit, or the file writer's service; without ".service"
|
||||
container_id: str | None # the full container ID: the writer's, or the stream's
|
||||
process: str | None # the first process holding the file
|
||||
process: str | None # the first process writing the file; None when only readers hold it
|
||||
size: int | None # bytes; None for a journal, or when the file could not be read
|
||||
modified: str | None # the last write, ISO 8601 UTC; None as for size
|
||||
|
||||
@@ -68,6 +81,7 @@ class LogSourcesDict(TypedDict):
|
||||
attributed: bool
|
||||
journald: bool
|
||||
sources: list[LogSourceDict]
|
||||
forwarding: LogForwardingDict | None # where the logs go beyond the host
|
||||
|
||||
|
||||
_BEGIN = "LOGS_BEGIN"
|
||||
@@ -90,15 +104,28 @@ LOG_SOURCES_COMMAND = (
|
||||
"echo '[open]'; printf '%s\\n' \"$o\"; "
|
||||
"echo '[stat]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 "
|
||||
"| xargs -r stat -L -c '%n %s %Y' 2>/dev/null; "
|
||||
"echo '[flags]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 | sed 's|/fd/|/fdinfo/|' "
|
||||
"| xargs -r grep -H '^flags:' 2>/dev/null; "
|
||||
"pids=$(printf '%s\\n' \"$o\" | cut -d/ -f3 | sort -un); "
|
||||
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
|
||||
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
|
||||
"else echo '[no-find]'; fi; "
|
||||
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
|
||||
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
|
||||
"| sed 's/^/| /'; "
|
||||
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
|
||||
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
|
||||
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
|
||||
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
|
||||
"printf '%s%s\\n' LOGS_ END"
|
||||
)
|
||||
|
||||
#: ``/proc/900/fd/5`` -- the process and its descriptor.
|
||||
_FD_RE = re.compile(r"^/proc/(\d+)/fd/\d+$")
|
||||
#: ``/proc/900/fdinfo/5:flags:<tab>0102001`` -- the descriptor's open flags, octal.
|
||||
_FLAGS_RE = re.compile(r"^/proc/(\d+)/fdinfo/(\d+):flags:\s*([0-7]+)$")
|
||||
#: The access mode bits of the open flags: 0 is O_RDONLY, 1 O_WRONLY, 2 O_RDWR.
|
||||
_ACCESS_MODE = 0o3
|
||||
#: The engine's file of a container's output: json-file and local drivers.
|
||||
_STREAM_RE = re.compile(r"^/var/lib/docker/containers/([0-9a-f]{64})/")
|
||||
#: Files that are no log to read: journald's store, a file already deleted, and
|
||||
@@ -159,39 +186,83 @@ def _journals(units: list[str]) -> list[LogSourceDict]:
|
||||
|
||||
|
||||
def _held(lines: list[str]) -> list[tuple[int, str, str]]:
|
||||
"""(pid, descriptor, target) for every log a process holds, by pid."""
|
||||
"""(pid, descriptor, target) for every log file a process holds, by pid."""
|
||||
held = []
|
||||
for line in lines:
|
||||
fd, _, target = line.partition(" ")
|
||||
match = _FD_RE.match(fd)
|
||||
if match and target and not _NOT_A_SOURCE.search(target):
|
||||
if match and target.startswith("/") and not _NOT_A_SOURCE.search(target):
|
||||
held.append((int(match.group(1)), fd, target))
|
||||
return sorted(held)
|
||||
|
||||
|
||||
def _writes(lines: list[str]) -> dict[str, bool]:
|
||||
"""Whether each descriptor is open for writing, by its ``/proc/<pid>/fd/<fd>``."""
|
||||
writes: dict[str, bool] = {}
|
||||
for line in lines:
|
||||
match = _FLAGS_RE.match(line)
|
||||
if match:
|
||||
pid, fd, flags = match.groups()
|
||||
writes[f"/proc/{pid}/fd/{fd}"] = int(flags, 8) & _ACCESS_MODE != 0
|
||||
return writes
|
||||
|
||||
|
||||
_Key = tuple[str, str | None, str]
|
||||
|
||||
|
||||
def _holders(lines: list[str], paths: dict[int, str]) -> dict[_Key, list[tuple[int, str]]]:
|
||||
"""The (pid, descriptor) pairs holding each log, by kind, container and path."""
|
||||
holders: dict[_Key, list[tuple[int, str]]] = {}
|
||||
for pid, fd, target in _held(lines):
|
||||
stream = _STREAM_RE.match(target)
|
||||
kind, container = (
|
||||
("container", stream.group(1)) if stream else ("file", _container(paths.get(pid)))
|
||||
)
|
||||
holders.setdefault((kind, container, target), []).append((pid, fd))
|
||||
return holders
|
||||
|
||||
|
||||
def _writer(held: list[tuple[int, str]], writes: dict[str, bool] | None) -> int | None:
|
||||
"""The first process writing; the first holding for a report without flags,
|
||||
as an older command printed it."""
|
||||
if writes is None:
|
||||
return held[0][0]
|
||||
return next((pid for pid, fd in held if writes.get(fd)), None)
|
||||
|
||||
|
||||
def _producer(
|
||||
kind: str, writer: int | None, paths: dict[int, str], names: dict[int, str]
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""The writer's service and name; a stream's service is its container."""
|
||||
if writer is None:
|
||||
return None, None
|
||||
return (_unit(paths.get(writer)) if kind == "file" else None), names.get(writer)
|
||||
|
||||
|
||||
def _files(sections: dict[str, list[str]]) -> list[LogSourceDict]:
|
||||
stats = _stats(sections.get("stat", []))
|
||||
paths = _cgroup_paths(sections.get("cgroups", []))
|
||||
names = _names(sections.get("comm", []))
|
||||
found: dict[tuple[str, str | None, str], LogSourceDict] = {}
|
||||
for pid, fd, target in _held(sections.get("open", [])):
|
||||
stream = _STREAM_RE.match(target)
|
||||
cgroup = paths.get(pid)
|
||||
kind, container = ("container", stream.group(1)) if stream else ("file", _container(cgroup))
|
||||
source = found.get((kind, container, target))
|
||||
if source is None:
|
||||
source = found[(kind, container, target)] = {
|
||||
writes = _writes(sections["flags"]) if "flags" in sections else None
|
||||
found: list[LogSourceDict] = []
|
||||
for (kind, container, target), held in _holders(sections.get("open", []), paths).items():
|
||||
writer = _writer(held, writes)
|
||||
if writer is None and kind == "file" and container is not None:
|
||||
continue # a reader's view of a mount, inside its container
|
||||
unit, process = _producer(kind, writer, paths, names)
|
||||
size, modified = next((stats[fd] for _, fd in held if fd in stats), (None, None))
|
||||
found.append(
|
||||
{
|
||||
"kind": kind,
|
||||
"path": target,
|
||||
"unit": None if stream else _unit(cgroup),
|
||||
"unit": unit,
|
||||
"container_id": container,
|
||||
"process": names.get(pid),
|
||||
"size": None,
|
||||
"modified": None,
|
||||
"process": process,
|
||||
"size": size,
|
||||
"modified": modified,
|
||||
}
|
||||
if source["size"] is None and fd in stats:
|
||||
source["size"], source["modified"] = stats[fd]
|
||||
return list(found.values())
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
def _order(source: LogSourceDict) -> tuple[str, str, str]:
|
||||
@@ -213,6 +284,33 @@ def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
|
||||
return journald, sorted(sources, key=_order)
|
||||
|
||||
|
||||
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
|
||||
journald, sources = parse_log_sources(output)
|
||||
forwarding = parse_log_forwarding(output)
|
||||
return {
|
||||
"attributed": attributed,
|
||||
"journald": journald,
|
||||
"sources": sources,
|
||||
"forwarding": forwarding,
|
||||
}
|
||||
|
||||
|
||||
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
|
||||
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
|
||||
None for a report without the sections, as an older command printed it.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if "rsyslog" not in sections:
|
||||
return None
|
||||
config = "\n".join(
|
||||
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
|
||||
)
|
||||
running = "running" in sections["rsyslog"]
|
||||
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
|
||||
|
||||
|
||||
class LogSourcesMixin:
|
||||
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
|
||||
|
||||
@@ -237,6 +335,8 @@ class LogSourcesMixin:
|
||||
* attributed (bool) - read as root, so every process's files are listed
|
||||
* journald (bool) - the host keeps systemd's journal
|
||||
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
|
||||
* forwarding (dict) - where the logs go beyond the host, see
|
||||
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
|
||||
|
||||
Example::
|
||||
|
||||
@@ -258,10 +358,8 @@ class LogSourcesMixin:
|
||||
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
|
||||
try:
|
||||
output = self._run_log_sources_command(command, privileged=True)
|
||||
journald, sources = parse_log_sources(output)
|
||||
return {"attributed": True, "journald": journald, "sources": sources}
|
||||
return _reading(output, attributed=True)
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_log_sources_command(command, privileged=False)
|
||||
journald, sources = parse_log_sources(output)
|
||||
return {"attributed": False, "journald": journald, "sources": sources}
|
||||
return _reading(output, attributed=False)
|
||||
@@ -85,8 +85,8 @@ class InterruptedPackageDict(TypedDict):
|
||||
"""A package an interrupted package manager run left halfway."""
|
||||
|
||||
name: str
|
||||
#: dpkg's word for it: "half-installed", "unpacked", "half-configured",
|
||||
#: "triggers-awaited", "triggers-pending", or "reinstall-required".
|
||||
#: The manager's word for it: dpkg's and opkg's ("unpacked", "half-configured", ...),
|
||||
#: rpm's "duplicate", apk's "broken-files" and "broken-script" (``package_managers``).
|
||||
state: str
|
||||
|
||||
|
||||
@@ -111,9 +111,9 @@ class HostStatusDict(TypedDict):
|
||||
reboot_reason: str | None
|
||||
#: True when the host installs updates on its own (unattended-upgrades, dnf-automatic).
|
||||
auto_updates: bool | None
|
||||
#: True when a package manager run was cut off and has to be finished
|
||||
#: (``dpkg --configure -a``) before anything else installs; None when it
|
||||
#: cannot tell, or a package manager is running right now.
|
||||
#: The package manager read (dpkg, dnf, yum, apk, opkg; None: none) and whether a run
|
||||
#: of it was cut off, left halfway (None: cannot tell, or one is running right now).
|
||||
package_manager: NotRequired[str | None]
|
||||
package_manager_interrupted: NotRequired[bool | None]
|
||||
#: The packages that run left halfway; empty when only its journal says so.
|
||||
interrupted_packages: NotRequired[list[InterruptedPackageDict]]
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
"""Whether dnf or yum, apk or opkg was left halfway: the host status's answer for
|
||||
a host without dpkg (NetOrk/netork#851).
|
||||
|
||||
Read the same way as dpkg's state (:mod:`napalm_device_types.host_status`): in the
|
||||
one host status command, without root, and changing nothing. Each manager is asked
|
||||
only where it exists, and while one of its processes runs -- from
|
||||
``/proc/<pid>/comm``, which every Linux has and BusyBox's ``ps`` cannot narrow to --
|
||||
its packages are halfway on purpose and the answer is unknown.
|
||||
|
||||
**dnf and yum** (rpm). A transaction cut off after rpm installed the new version
|
||||
of a package and before it erased the old one leaves both in the rpm database:
|
||||
what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
|
||||
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
|
||||
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
|
||||
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
|
||||
per imported key. Also interrupted is a transaction that never ended: dnf 4's
|
||||
history (libdnf's ``trans`` table) holds its row with ``state`` 0 until
|
||||
``finish()`` writes 1 (done) or 2 (error), so the newest row at 0 is one cut off;
|
||||
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
|
||||
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
|
||||
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
|
||||
has no ``sqlite3`` program and a ``dnf`` command loads every plugin and may reach
|
||||
for its repositories. dnf 5's history lives elsewhere and is not read; its
|
||||
duplicates are.
|
||||
|
||||
**apk** (Alpine, OpenWrt from 25.12). apk marks an installed package whose files
|
||||
or scripts failed with ``f:`` in ``/lib/apk/db/installed`` -- ``f`` broken files,
|
||||
``s`` a broken script; ``x`` (xattrs) and ``S`` (the checksum kind) are no fault --
|
||||
and those are exactly the packages ``apk fix`` without arguments picks. A run
|
||||
killed outright leaves no such mark; its next run usually repeats what it left.
|
||||
|
||||
**opkg** (OpenWrt before 25.12). ``/usr/lib/opkg/status`` holds
|
||||
``Status: <want> <flags> <state>`` per package. A run cut off, or a postinst that
|
||||
failed, leaves ``unpacked`` (or ``half-configured``, ``half-installed``), and the
|
||||
``reinstreq`` flag asks for a reinstall; ``opkg configure`` finishes every unpacked
|
||||
package.
|
||||
|
||||
What to do about it is not decided here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from napalm_device_types.models import InterruptedPackageDict
|
||||
|
||||
#: dnf's default ``installonlypkgs``: names and what such packages provide.
|
||||
_INSTALLONLY = (
|
||||
"kernel",
|
||||
"kernel-PAE",
|
||||
"installonlypkg(kernel)",
|
||||
"installonlypkg(kernel-module)",
|
||||
"installonlypkg(vm)",
|
||||
"multiversion(kernel)",
|
||||
)
|
||||
#: Where a host adds its own installonly packages, dnf's then yum's.
|
||||
_DNF_CONF = "/etc/dnf/dnf.conf /etc/yum.conf"
|
||||
DNF_HISTORY = "/var/lib/dnf/history.sqlite"
|
||||
YUM_DIR = "/var/lib/yum"
|
||||
APK_INSTALLED = "/lib/apk/db/installed"
|
||||
OPKG_STATUS = "/usr/lib/opkg/status"
|
||||
|
||||
#: manager -> the process names that leave its packages halfway while they run.
|
||||
_RUNNING = {
|
||||
"rpm": frozenset({"rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"}),
|
||||
"apk": frozenset({"apk"}),
|
||||
"opkg": frozenset({"opkg"}),
|
||||
}
|
||||
_UNFINISHED = "0"
|
||||
_YUM_JOURNAL = "transaction-all."
|
||||
_YUM_GIVEN_UP = ".disabled"
|
||||
_APK_FLAGS = (("f", "broken-files"), ("s", "broken-script"))
|
||||
_OPKG_HALFWAY = frozenset({"unpacked", "half-configured", "half-installed"})
|
||||
_OPKG_REINSTALL = "reinstreq"
|
||||
|
||||
#: The newest transaction in dnf's history as ``<id> <state>``; nothing when unreadable.
|
||||
#: One literal, with :data:`DNF_HISTORY` spelt out: a fixed query, nothing put into it.
|
||||
_DNF_HISTORY_PY = (
|
||||
"import sqlite3; "
|
||||
"c = sqlite3.connect('file:/var/lib/dnf/history.sqlite?mode=ro', uri=True); "
|
||||
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
|
||||
"r and print(*r)"
|
||||
)
|
||||
|
||||
_RUNNING_PART = (
|
||||
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
|
||||
+ "|".join(sorted(set().union(*_RUNNING.values())))
|
||||
+ "' | sort -u; "
|
||||
)
|
||||
_RPM_PART = (
|
||||
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
|
||||
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
|
||||
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
|
||||
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
|
||||
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
|
||||
"| grep -v ' ' | sort -u; "
|
||||
"echo '[rpm-dupes]'; rpm -qa --qf '%{NAME} %{ARCH}\\n' 2>/dev/null "
|
||||
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
|
||||
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
|
||||
"| head -n 20; "
|
||||
"echo '[dnf-history]'; for py in /usr/libexec/platform-python python3; do "
|
||||
f'if command -v $py >/dev/null 2>&1; then $py -c "{_DNF_HISTORY_PY}" 2>/dev/null; '
|
||||
"break; fi; done; fi; "
|
||||
)
|
||||
_APK_PART = (
|
||||
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
|
||||
"awk '/^P:/ {p = substr($0, 3)} /^f:/ {f = substr($0, 3)} "
|
||||
'/^$/ {if (f ~ /[fs]/) print f, p; p = ""; f = ""} END {if (f ~ /[fs]/) print f, p}\' '
|
||||
f"{APK_INSTALLED} | head -n 50; fi; "
|
||||
)
|
||||
_OPKG_PART = (
|
||||
f"if command -v opkg >/dev/null 2>&1 && [ -r {OPKG_STATUS} ]; then echo '[opkg-audit]'; "
|
||||
"awk '/^Package:/ {p = $2} /^Status:/ && "
|
||||
"($4 ~ /^(unpacked|half-configured|half-installed)$/ || $3 ~ /reinstreq/) "
|
||||
f"{{print $3, $4, p}}' {OPKG_STATUS} | head -n 50; fi; "
|
||||
)
|
||||
|
||||
#: The part of the host status command that reads them; POSIX ``sh``, BusyBox too.
|
||||
PACKAGE_MANAGERS_COMMAND = _RUNNING_PART + _RPM_PART + _APK_PART + _OPKG_PART
|
||||
|
||||
|
||||
class PackageManagerReading(NamedTuple):
|
||||
"""One package manager's state: halfway or not, None while it is running."""
|
||||
|
||||
manager: str | None
|
||||
interrupted: bool | None
|
||||
packages: list[InterruptedPackageDict]
|
||||
|
||||
|
||||
def _unfinished_transaction(sections: dict[str, list[str]]) -> bool:
|
||||
newest = (sections.get("dnf-history") or [""])[0].split()
|
||||
if len(newest) == 2 and newest[1] == _UNFINISHED:
|
||||
return True
|
||||
return any(
|
||||
name.startswith(_YUM_JOURNAL) and not name.endswith(_YUM_GIVEN_UP)
|
||||
for name in sections.get("yum-journal") or []
|
||||
)
|
||||
|
||||
|
||||
def _rpm(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "rpm-dupes" not in sections:
|
||||
return None
|
||||
manager = (sections.get("rpm-frontend") or ["dnf"])[0]
|
||||
if running & _RUNNING["rpm"]:
|
||||
return PackageManagerReading(manager, None, [])
|
||||
installonly = set(sections.get("rpm-installonly") or [])
|
||||
names = dict.fromkeys(line.split()[0] for line in sections["rpm-dupes"])
|
||||
packages: list[InterruptedPackageDict] = [
|
||||
{"name": name, "state": "duplicate"} for name in names if name not in installonly
|
||||
]
|
||||
return PackageManagerReading(
|
||||
manager, bool(packages) or _unfinished_transaction(sections), packages
|
||||
)
|
||||
|
||||
|
||||
def _apk(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "apk-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["apk"]:
|
||||
return PackageManagerReading("apk", None, [])
|
||||
packages: list[InterruptedPackageDict] = []
|
||||
for line in sections["apk-audit"]:
|
||||
flags, _, name = line.partition(" ")
|
||||
name = name.strip()
|
||||
packages += [
|
||||
{"name": name, "state": state} for flag, state in _APK_FLAGS if name and flag in flags
|
||||
]
|
||||
return PackageManagerReading("apk", bool(packages), packages)
|
||||
|
||||
|
||||
def _opkg_package(line: str) -> InterruptedPackageDict | None:
|
||||
parts = line.split()
|
||||
if len(parts) != 3:
|
||||
return None
|
||||
flags, state, name = parts
|
||||
if _OPKG_REINSTALL in flags.split(","):
|
||||
return {"name": name, "state": "reinstall-required"}
|
||||
return {"name": name, "state": state} if state in _OPKG_HALFWAY else None
|
||||
|
||||
|
||||
def _opkg(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "opkg-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["opkg"]:
|
||||
return PackageManagerReading("opkg", None, [])
|
||||
packages = [p for p in map(_opkg_package, sections["opkg-audit"]) if p is not None]
|
||||
return PackageManagerReading("opkg", bool(packages), packages)
|
||||
|
||||
|
||||
def parse_package_managers(sections: dict[str, list[str]]) -> list[PackageManagerReading]:
|
||||
"""What :data:`PACKAGE_MANAGERS_COMMAND` found, one reading per manager it asked."""
|
||||
running = set(sections.get("pm-running") or [])
|
||||
readings = (read(sections, running) for read in (_rpm, _apk, _opkg))
|
||||
return [reading for reading in readings if reading is not None]
|
||||
@@ -0,0 +1,176 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or the rule's parts.
|
||||
|
||||
An application names a zone the IANA way (``Europe/Berlin``). Not every device
|
||||
can: OpenWrt hands ``system.timezone`` to libc as ``TZ``, which wants the rule
|
||||
itself (``CET-1CEST,M3.5.0,M10.5.0/3``), and a switch wants a UTC offset and the
|
||||
summer-time rule. A driver translates with these helpers.
|
||||
|
||||
The rule is the footer of the zone's TZif file (version 2 and later): the line
|
||||
after the last transition that says how the zone goes on. OpenWrt's LuCI table is
|
||||
built from the same footers. The files come from the ``tzdata`` package, a
|
||||
declared dependency, so every driver reads the same database whatever its host
|
||||
ships.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from functools import lru_cache
|
||||
from importlib.resources import files
|
||||
from typing import TypedDict
|
||||
|
||||
#: An IANA name: letters, digits, ``_+-`` in ``/``-separated parts. No dots, so
|
||||
#: nothing reaches outside the database.
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
|
||||
_ABBR = r"(?:[A-Za-z]{3,}|<[A-Za-z0-9+-]{3,}>)"
|
||||
_OFFSET = r"[+-]?\d{1,2}(?::\d{1,2}){0,2}"
|
||||
_DATE = r"(?:J\d{1,3}|\d{1,3}|M\d{1,2}\.\d\.\d)(?:/[+-]?\d{1,3}(?::\d{1,2}){0,2})?"
|
||||
#: ``std offset [dst [offset] [,start,end]]`` (POSIX.1, 8.3), and a bare ``UTC`` or
|
||||
#: ``GMT``, which OpenWrt itself writes by default.
|
||||
_POSIX_TZ = re.compile(
|
||||
rf"(?:UTC|GMT)|{_ABBR}{_OFFSET}(?:{_ABBR}(?:{_OFFSET})?(?:,{_DATE},{_DATE})?)?"
|
||||
)
|
||||
|
||||
_PARTS = re.compile(
|
||||
rf"(?P<std>{_ABBR})(?P<offset>{_OFFSET})"
|
||||
rf"(?:(?P<dst>{_ABBR})(?P<dst_offset>{_OFFSET})?"
|
||||
r"(?:,(?P<start>[^,]+),(?P<end>[^,]+))?)?"
|
||||
)
|
||||
_TRANSITION = re.compile(r"M(\d{1,2})\.([1-5])\.([0-6])(?:/([+-]?\d{1,3}(?::\d{1,2}){0,2}))?")
|
||||
#: When a rule names no time of day for a transition: 02:00 local time.
|
||||
_DEFAULT_TRANSITION_TIME = 7200
|
||||
|
||||
|
||||
class DstTransitionDict(TypedDict):
|
||||
"""When summer time starts or ends: a weekday of a week of a month, and a time."""
|
||||
|
||||
month: int # 1-12
|
||||
week: int # 1-5; 5 is the month's last such weekday
|
||||
weekday: int # 0 = Sunday ... 6 = Saturday
|
||||
time: int # seconds after local midnight, in the time then in force; may be < 0 or > 24h
|
||||
|
||||
|
||||
class PosixTzDict(TypedDict):
|
||||
"""A POSIX TZ rule in parts (:func:`parse_posix_tz`)."""
|
||||
|
||||
std_name: str # "CET"; a quoted name without its <> ("-03")
|
||||
utc_offset: int # seconds east of UTC in standard time: CET 3600, EST -18000
|
||||
dst_name: str | None # "CEST"; None for a zone without summer time
|
||||
dst_offset: int | None # seconds east of UTC in summer time; None without it
|
||||
dst_start: DstTransitionDict | None # None without summer time or without a rule for it
|
||||
dst_end: DstTransitionDict | None
|
||||
|
||||
|
||||
class UnknownTimezone(ValueError):
|
||||
"""A zone that is not in the zone database (nor, where one is taken, a POSIX TZ rule)."""
|
||||
|
||||
def __init__(self, zone: str) -> None:
|
||||
super().__init__(zone)
|
||||
self.zone = zone
|
||||
|
||||
def __str__(self) -> str:
|
||||
return (
|
||||
f"unknown time zone {self.zone!r}: not a name of the IANA time zone database"
|
||||
" such as Europe/Berlin"
|
||||
)
|
||||
|
||||
|
||||
def is_posix_tz(value: str) -> bool:
|
||||
"""Whether *value* reads as a POSIX TZ string -- the form libc takes in ``TZ``."""
|
||||
return bool(_POSIX_TZ.fullmatch(value))
|
||||
|
||||
|
||||
@lru_cache(maxsize=512)
|
||||
def _footer(zone: str) -> str | None:
|
||||
"""The POSIX rule at the end of *zone*'s TZif file; None for no such zone."""
|
||||
if not _ZONE_NAME.fullmatch(zone):
|
||||
return None
|
||||
tzfile = files("tzdata").joinpath("zoneinfo", *zone.split("/"))
|
||||
if not tzfile.is_file():
|
||||
return None
|
||||
data = tzfile.read_bytes()
|
||||
if not data.startswith(b"TZif") or data[4:5] not in (b"2", b"3", b"4") or data[-1:] != b"\n":
|
||||
return None
|
||||
footer = data[:-1].rsplit(b"\n", 1)[-1].decode("ascii", "replace")
|
||||
return footer if is_posix_tz(footer) else None
|
||||
|
||||
|
||||
def is_iana_zone(zone: str) -> bool:
|
||||
"""Whether *zone* names a zone of the database (``Europe/Berlin``, ``UTC``)."""
|
||||
return _footer(zone) is not None
|
||||
|
||||
|
||||
def posix_tz(zone: str) -> str | None:
|
||||
"""The POSIX TZ string for *zone*: an IANA name's rule, a POSIX string as it is.
|
||||
|
||||
None when *zone* is neither -- a device given it would run on UTC.
|
||||
"""
|
||||
return _footer(zone) or (zone if is_posix_tz(zone) else None)
|
||||
|
||||
|
||||
def _seconds(value: str) -> int:
|
||||
"""``[+-]hh[:mm[:ss]]`` in seconds."""
|
||||
sign = -1 if value.startswith("-") else 1
|
||||
parts = [int(p) for p in value.lstrip("+-").split(":")]
|
||||
hours, minutes, seconds = (parts + [0, 0])[:3]
|
||||
return sign * (hours * 3600 + minutes * 60 + seconds)
|
||||
|
||||
|
||||
def _transition(value: str) -> DstTransitionDict | None:
|
||||
match = _TRANSITION.fullmatch(value)
|
||||
if match is None:
|
||||
return None
|
||||
month, week, weekday, time = match.groups()
|
||||
return {
|
||||
"month": int(month),
|
||||
"week": int(week),
|
||||
"weekday": int(weekday),
|
||||
"time": _DEFAULT_TRANSITION_TIME if time is None else _seconds(time),
|
||||
}
|
||||
|
||||
|
||||
def parse_posix_tz(rule: str) -> PosixTzDict | None:
|
||||
"""*rule* in parts; None for what is not a rule, or one with Julian-day transitions.
|
||||
|
||||
POSIX counts offsets west of UTC (``CET-1``); the parts count them east, as a
|
||||
device's "UTC+01:00" does. Summer time without an offset of its own is one hour
|
||||
ahead of standard time.
|
||||
"""
|
||||
if rule in ("UTC", "GMT"):
|
||||
rule += "0"
|
||||
match = _PARTS.fullmatch(rule)
|
||||
if match is None or not is_posix_tz(rule):
|
||||
return None
|
||||
utc_offset = -_seconds(match["offset"])
|
||||
parts: PosixTzDict = {
|
||||
"std_name": match["std"].strip("<>"),
|
||||
"utc_offset": utc_offset,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
if match["dst"] is None:
|
||||
return parts
|
||||
parts["dst_name"] = match["dst"].strip("<>")
|
||||
parts["dst_offset"] = (
|
||||
utc_offset + 3600 if match["dst_offset"] is None else -_seconds(match["dst_offset"])
|
||||
)
|
||||
if match["start"] is not None:
|
||||
start, end = _transition(match["start"]), _transition(match["end"])
|
||||
if start is None or end is None:
|
||||
return None
|
||||
parts["dst_start"], parts["dst_end"] = start, end
|
||||
return parts
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""The backups running on a device right now.
|
||||
|
||||
A restart aborts every backup that runs on the device it restarts: a
|
||||
hypervisor's backup of its guests, a backup server's backup, sync, verify or
|
||||
garbage collection, a host's own file-level backup (restic, borg and the like).
|
||||
A guest's restart breaks the backup its hypervisor is taking of it. So an
|
||||
application that restarts a device asks first, right before the restart.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a driver mixes this in when it can tell;
|
||||
``hasattr`` tells a caller whether it did. No role base carries it, and a driver
|
||||
without it cannot tell -- which is not the same as "nothing runs".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Literal, TypedDict, get_args
|
||||
|
||||
#: What a running backup does: the backup itself, or one of a backup server's
|
||||
#: jobs (``napalm_device_types.backups.BackupJobKind``).
|
||||
RunningBackupKind = Literal["backup", "sync", "verify", "gc", "prune", "tape"]
|
||||
RUNNING_BACKUP_KINDS: tuple[str, ...] = get_args(RunningBackupKind)
|
||||
|
||||
|
||||
class RunningBackupDict(TypedDict):
|
||||
"""One backup running on the device (``get_running_backups``)."""
|
||||
|
||||
kind: RunningBackupKind
|
||||
job: str | None # the job it runs for, as the platform names it; None: started by hand
|
||||
task: str | None # this run's id on the platform: a task id, a unit, a process id
|
||||
guests: list[str] # the guests (get_vms' ids) it is backing up now; [] for the host's own
|
||||
since: int | None # unix seconds it started; None when the platform does not say
|
||||
|
||||
|
||||
class RunningBackupsMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_running_backups(self) -> list[RunningBackupDict]:
|
||||
"""
|
||||
Returns the backups running on the device now; ``[]`` when none is.
|
||||
|
||||
Every entry runs on this device: restarting the device aborts it.
|
||||
A hypervisor names in ``guests`` the guests a backup is taking right
|
||||
now -- a guest that is locked for its backup counts, with or without
|
||||
a task the platform still shows -- so the application can tell a
|
||||
guest's own restart not to break it.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"kind": "backup",
|
||||
"job": "backup-3f2a9c1e-77d1",
|
||||
"task": "UPID:pve1:0001F2A3:00C4B1D2:6731A0C0:vzdump::root@pam:",
|
||||
"guests": ["101"],
|
||||
"since": 1793660400,
|
||||
},
|
||||
{
|
||||
"kind": "gc",
|
||||
"job": "store1",
|
||||
"task": "UPID:localhost:000004D2:...:garbage_collection:store1:root@pam:",
|
||||
"guests": [],
|
||||
"since": 1793664000,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the device could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
@@ -53,6 +53,8 @@ from typing import (
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
from napalm_device_types.ssh_server_auth import SSHD_EFFECTIVE_COMMAND, SshServerAuthMixin
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
@@ -420,7 +422,7 @@ def _unchanged(
|
||||
return result
|
||||
|
||||
|
||||
class OpenSshServerMixin:
|
||||
class OpenSshServerMixin(SshServerAuthMixin):
|
||||
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH, not by a base class,
|
||||
@@ -430,6 +432,9 @@ class OpenSshServerMixin:
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
|
||||
It carries :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`:
|
||||
a host whose sshd can be hardened can also say how it lets people log in.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
@@ -513,7 +518,7 @@ class OpenSshServerMixin:
|
||||
return parse_sshd_effective(self._sshd_effective_text())
|
||||
|
||||
def _sshd_effective_text(self) -> str:
|
||||
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -T")
|
||||
result = self._ssh_run(SSHD_EFFECTIVE_COMMAND)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
return result.stdout
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
"""How an OpenSSH server lets people log in: the effective settings and sshd's PAM stack.
|
||||
|
||||
Whether a host takes a password, only a key, or a key *and* a second factor is
|
||||
the same question on every host that runs OpenSSH, so the reading lives here,
|
||||
and a driver only mixes in :class:`SshServerAuthMixin` and supplies
|
||||
``run_command``.
|
||||
|
||||
**What sshd uses, not what a file says.** ``sshd -T`` prints the settings after
|
||||
sshd's defaults, ``sshd_config``, every ``Include`` and first-match-wins -- the
|
||||
same reading :class:`~napalm_device_types.ssh_server.OpenSshServerMixin` plans
|
||||
its hardening on, by the same fixed command (:data:`SSHD_EFFECTIVE_COMMAND`). It
|
||||
needs root: without it sshd cannot load its host keys and exits.
|
||||
|
||||
**PAM, as far as it can be read.** With ``UsePAM yes``, keyboard-interactive
|
||||
asks whatever PAM's ``auth`` stack for sshd asks: a password from
|
||||
``pam_unix.so``, a one-time code from an OTP module, or both. The stack is read
|
||||
from ``/etc/pam.d`` in one listing as the login user (the files are
|
||||
world-readable), and followed through ``@include``, ``include`` and
|
||||
``substack``. Each module is reported with its control and arguments as
|
||||
written; which of them is a second factor, and whether ``nullok`` lets a user
|
||||
without one through, is the consumer's call.
|
||||
|
||||
Not read: ``Match`` blocks (``sshd -T`` without ``-C`` prints the global
|
||||
settings), and what a PAM module's own configuration asks for.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: sshd's effective configuration. ``sshd`` lives in ``/usr/sbin``, outside a login
|
||||
#: user's ``PATH``. One fixed command, run as root.
|
||||
SSHD_EFFECTIVE_COMMAND = "PATH=$PATH:/usr/sbin:/sbin sshd -T"
|
||||
|
||||
PAM_DIR = "/etc/pam.d"
|
||||
#: Every file in ``/etc/pam.d``, each after a ``==> path <==`` header (GNU, BSD and
|
||||
#: busybox ``tail`` alike). One fixed command, as the login user.
|
||||
PAM_FILES_COMMAND = f"tail -n +1 -- {PAM_DIR}/*"
|
||||
#: The PAM service sshd runs as.
|
||||
PAM_SERVICE = "sshd"
|
||||
|
||||
_TIMEOUT = 30
|
||||
_HEADER_RE = re.compile(rf"^==> {re.escape(PAM_DIR)}/(?P<name>[^/]+) <==$")
|
||||
#: A service name an include may name: a file in ``/etc/pam.d``, never a path out of it.
|
||||
_SERVICE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
|
||||
_INCLUDE_CONTROLS = ("include", "substack")
|
||||
|
||||
|
||||
class PamAuthEntryDict(TypedDict):
|
||||
"""One line of a PAM ``auth`` stack: ``auth required pam_unix.so nullok``."""
|
||||
|
||||
#: The module's file name, without its directory: ``pam_unix.so``.
|
||||
module: str
|
||||
#: ``required``, ``requisite``, ``sufficient``, ``optional``, or a bracketed
|
||||
#: ``[value=action ...]`` as written.
|
||||
control: str
|
||||
args: list[str]
|
||||
|
||||
|
||||
class SshServerAuthDict(TypedDict):
|
||||
"""How an OpenSSH server lets people log in, as ``get_ssh_server_auth`` read it.
|
||||
|
||||
The global settings ``sshd -T`` prints. A ``Match`` block can change them for
|
||||
some users or addresses; ``sshd -T`` without ``-C`` does not show that.
|
||||
|
||||
``authentication_methods`` is ``AuthenticationMethods``: each inner list is one
|
||||
way in, every method of it required in turn (``publickey,keyboard-interactive``
|
||||
-> ``["publickey", "keyboard-interactive"]``); empty for ``any``, where any one
|
||||
enabled method is enough. A method keeps its submethod
|
||||
(``keyboard-interactive:pam``).
|
||||
|
||||
``use_pam`` is None where sshd has no PAM (OpenBSD). ``pam_auth_stack`` is the
|
||||
``auth`` stack PAM runs for sshd, includes and substacks followed; None when
|
||||
PAM is off or its files gave nothing for sshd.
|
||||
"""
|
||||
|
||||
password_authentication: bool
|
||||
#: ``KbdInteractiveAuthentication``; before OpenSSH 8.7, either it or
|
||||
#: ``ChallengeResponseAuthentication``.
|
||||
kbd_interactive_authentication: bool
|
||||
pubkey_authentication: bool
|
||||
hostbased_authentication: bool
|
||||
#: False also where sshd was built without GSSAPI.
|
||||
gssapi_authentication: bool
|
||||
authentication_methods: list[list[str]]
|
||||
#: ``yes``, ``prohibit-password`` (``without-password``), ``forced-commands-only`` or ``no``.
|
||||
permit_root_login: str
|
||||
use_pam: bool | None
|
||||
pam_auth_stack: list[PamAuthEntryDict] | None
|
||||
|
||||
|
||||
def _flag(settings: Mapping[str, str], keyword: str) -> bool:
|
||||
return settings.get(keyword, "no") == "yes"
|
||||
|
||||
|
||||
def _settings(text: str) -> dict[str, str]:
|
||||
"""``keyword value`` per line, the first one of each keyword."""
|
||||
found: dict[str, str] = {}
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if len(parts) == 2:
|
||||
found.setdefault(parts[0].lower(), parts[1].strip())
|
||||
return found
|
||||
|
||||
|
||||
def _methods(value: str) -> list[list[str]]:
|
||||
if value == "any":
|
||||
return []
|
||||
return [[m for m in alternative.split(",") if m] for alternative in value.split()]
|
||||
|
||||
|
||||
def parse_sshd_auth(text: str) -> SshServerAuthDict:
|
||||
"""The authentication settings from ``sshd -T`` output; ``pam_auth_stack`` left None.
|
||||
|
||||
:raises ValueError: if a setting every OpenSSH prints is missing -- not sshd -T
|
||||
output, or cut short.
|
||||
"""
|
||||
settings = _settings(text)
|
||||
missing = [
|
||||
k
|
||||
for k in ("passwordauthentication", "pubkeyauthentication", "permitrootlogin")
|
||||
if k not in settings
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(f"sshd -T printed no {', '.join(missing)}")
|
||||
use_pam = settings.get("usepam")
|
||||
return {
|
||||
"password_authentication": _flag(settings, "passwordauthentication"),
|
||||
"kbd_interactive_authentication": _flag(settings, "kbdinteractiveauthentication")
|
||||
or _flag(settings, "challengeresponseauthentication"),
|
||||
"pubkey_authentication": _flag(settings, "pubkeyauthentication"),
|
||||
"hostbased_authentication": _flag(settings, "hostbasedauthentication"),
|
||||
"gssapi_authentication": _flag(settings, "gssapiauthentication"),
|
||||
"authentication_methods": _methods(settings.get("authenticationmethods", "any")),
|
||||
"permit_root_login": settings["permitrootlogin"],
|
||||
"use_pam": None if use_pam is None else use_pam == "yes",
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def _pam_tokens(line: str) -> list[str]:
|
||||
"""A PAM line's tokens, a bracketed control (``[success=1 default=ignore]``) as one."""
|
||||
tokens: list[str] = []
|
||||
for part in line.split():
|
||||
if tokens and tokens[-1].startswith("[") and not tokens[-1].endswith("]"):
|
||||
tokens[-1] += " " + part
|
||||
else:
|
||||
tokens.append(part)
|
||||
return tokens
|
||||
|
||||
|
||||
def _pam_entry(line: str) -> PamAuthEntryDict | None:
|
||||
tokens = _pam_tokens(line.split("#", 1)[0])
|
||||
if len(tokens) >= 2 and tokens[0] == "@include":
|
||||
tokens = ["auth", "include", tokens[1]]
|
||||
if len(tokens) < 3 or tokens[0].lstrip("-") != "auth":
|
||||
return None
|
||||
control, target, args = tokens[1], tokens[2], tokens[3:]
|
||||
if control in _INCLUDE_CONTROLS:
|
||||
if not _SERVICE_RE.match(target):
|
||||
return None
|
||||
return {"module": target, "control": control, "args": []}
|
||||
return {"module": target.rsplit("/", 1)[-1], "control": control, "args": args}
|
||||
|
||||
|
||||
def parse_pam_auth(text: str) -> list[PamAuthEntryDict]:
|
||||
"""One PAM file's ``auth`` stack, in order.
|
||||
|
||||
A module by its file name (``pam_unix.so``), with its control and arguments.
|
||||
An include is an entry whose control is ``include`` or ``substack`` and whose
|
||||
module is the service it names: ``@include common-auth`` brings in a whole
|
||||
file, ``auth include`` and ``auth substack`` its ``auth`` lines; both are
|
||||
followed by :func:`pam_auth_stack`. An include that names a path rather than
|
||||
a service is left out.
|
||||
"""
|
||||
return [entry for line in text.splitlines() if (entry := _pam_entry(line.strip()))]
|
||||
|
||||
|
||||
def parse_pam_files(text: str) -> dict[str, str]:
|
||||
""":data:`PAM_FILES_COMMAND`'s output, by file name."""
|
||||
files: dict[str, list[str]] = {}
|
||||
current: list[str] | None = None
|
||||
for line in text.splitlines():
|
||||
header = _HEADER_RE.match(line.strip())
|
||||
if header:
|
||||
current = files.setdefault(header["name"], [])
|
||||
elif current is not None:
|
||||
current.append(line)
|
||||
return {name: "\n".join(lines).strip("\n") + "\n" for name, lines in files.items()}
|
||||
|
||||
|
||||
def pam_auth_stack(
|
||||
files: Mapping[str, str], service: str = PAM_SERVICE
|
||||
) -> list[PamAuthEntryDict] | None:
|
||||
"""*service*'s ``auth`` modules, includes and substacks followed, in stack order.
|
||||
|
||||
A file included twice -- or in a loop -- counts once; an include of a file
|
||||
that is not there adds nothing, as with PAM. None without a file for *service*.
|
||||
"""
|
||||
if service not in files:
|
||||
return None
|
||||
stack: list[PamAuthEntryDict] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def walk(name: str) -> None:
|
||||
if name in seen or name not in files:
|
||||
return
|
||||
seen.add(name)
|
||||
for entry in parse_pam_auth(files[name]):
|
||||
if entry["control"] in _INCLUDE_CONTROLS:
|
||||
walk(entry["module"])
|
||||
else:
|
||||
stack.append(entry)
|
||||
|
||||
walk(service)
|
||||
return stack
|
||||
|
||||
|
||||
class SshServerAuthMixin:
|
||||
"""Adds :meth:`get_ssh_server_auth` to a driver that can run commands on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH --
|
||||
:class:`~napalm_device_types.ssh_server.OpenSshServerMixin` carries it -- so
|
||||
``hasattr(driver, "get_ssh_server_auth")`` stays a truthful answer. Reading,
|
||||
not writing: a platform that generates sshd_config itself can carry it too.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def get_ssh_server_auth(self) -> SshServerAuthDict:
|
||||
"""
|
||||
Returns how sshd lets people log in, from ``sshd -T`` and PAM's sshd stack.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"password_authentication": False,
|
||||
"kbd_interactive_authentication": True,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [["publickey", "keyboard-interactive"]],
|
||||
"permit_root_login": "no",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": [
|
||||
{"module": "pam_google_authenticator.so", "control": "required", "args": []}
|
||||
],
|
||||
}
|
||||
|
||||
:raises RuntimeError: if ``sshd -T`` fails -- no OpenSSH, or no root.
|
||||
"""
|
||||
result = self.run_command(SSHD_EFFECTIVE_COMMAND, privileged=True, timeout=_TIMEOUT)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
auth = parse_sshd_auth(result.stdout)
|
||||
if auth["use_pam"]:
|
||||
auth["pam_auth_stack"] = self._pam_auth_stack()
|
||||
return auth
|
||||
|
||||
def _pam_auth_stack(self) -> list[PamAuthEntryDict] | None:
|
||||
"""sshd's PAM auth stack; None when ``/etc/pam.d`` gave nothing for sshd.
|
||||
|
||||
``tail`` goes on past a file it cannot open and exits 1, so what it did
|
||||
print is used whatever the exit status.
|
||||
"""
|
||||
result = self.run_command(PAM_FILES_COMMAND, privileged=False, timeout=_TIMEOUT)
|
||||
return pam_auth_stack(parse_pam_files(result.stdout))
|
||||
@@ -0,0 +1,176 @@
|
||||
"""A device's time sources and time zone: read them, and set them to what the caller says.
|
||||
|
||||
An application keeps one list of NTP servers and a time zone per site, and aligns a
|
||||
device that differs. :class:`TimeSyncMixin` is the contract for that:
|
||||
|
||||
* ``get_time_sync()`` -- declared only, so ``callable(getattr(driver,
|
||||
"get_time_sync", None))`` is the truthful answer to "can this driver tell";
|
||||
* ``configure_time_sync(ntp_servers=None, timezone=None)`` -- a fix template
|
||||
(:mod:`napalm_device_types.fixes`): ``None`` leaves that value as it is, and the
|
||||
device ends up with exactly the servers given, in their order. A driver
|
||||
implements the steps; one whose device takes the change in a single call
|
||||
overrides the method;
|
||||
* ``time_sync_writes`` -- which of the two values the driver can write. A BMC that
|
||||
runs on UTC only writes ``{"ntp_servers"}``; the application never asks it for a
|
||||
zone.
|
||||
|
||||
A driver that inherits the mixin but cannot run it sets ``get_time_sync`` (and
|
||||
``configure_time_sync``) to ``None``.
|
||||
|
||||
A zone is an IANA name on the wire. A device that wants a POSIX rule or an offset
|
||||
with a summer-time rule gets it from :mod:`napalm_device_types.posix_tz`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar, TypedDict
|
||||
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.posix_tz import UnknownTimezone, is_iana_zone
|
||||
|
||||
#: The values a driver can write; ``time_sync_writes`` is a subset.
|
||||
TIME_SYNC_VALUES: frozenset[str] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
_HOST_NAME = re.compile(
|
||||
r"(?=.{1,253}$)[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?"
|
||||
r"(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*\.?"
|
||||
)
|
||||
|
||||
|
||||
class TimeSyncDict(TypedDict):
|
||||
"""How a device keeps its clock (``get_time_sync``)."""
|
||||
|
||||
ntp_servers: list[str] # configured time sources, in the device's order; [] = none configured
|
||||
timezone: str | None # IANA name ("Europe/Berlin"); None = not readable or no named zone
|
||||
synchronized: bool | None # the clock is synchronised to a source right now; None = unknown
|
||||
daemon: str | None # "chrony", "systemd-timesyncd", "ntpd", "openntpd", ...; None = own client
|
||||
|
||||
|
||||
def _is_server(name: str) -> bool:
|
||||
try:
|
||||
ipaddress.ip_address(name)
|
||||
except ValueError:
|
||||
return bool(_HOST_NAME.fullmatch(name))
|
||||
return True
|
||||
|
||||
|
||||
def check_ntp_servers(servers: Sequence[str]) -> list[str]:
|
||||
"""*servers* as a driver writes them: stripped, in their order, each once.
|
||||
|
||||
:raises ValueError: for no server at all, or one that is not a host name or
|
||||
an IP address -- before anything reaches a configuration file.
|
||||
:raises TypeError: for a single string instead of a list of them.
|
||||
"""
|
||||
if isinstance(servers, str):
|
||||
raise TypeError("ntp_servers is a list of servers, not one string")
|
||||
kept: list[str] = []
|
||||
for server in (str(s).strip() for s in servers):
|
||||
if not server:
|
||||
continue
|
||||
if not _is_server(server):
|
||||
raise ValueError(f"{server!r} is not a host name or address of an NTP server")
|
||||
if server.lower() not in (k.lower() for k in kept):
|
||||
kept.append(server)
|
||||
if not kept:
|
||||
raise ValueError("name at least one NTP server; None leaves the servers as they are")
|
||||
return kept
|
||||
|
||||
|
||||
class TimeSyncMixin:
|
||||
"""Read and set a device's time sources and time zone; see the module."""
|
||||
|
||||
#: Which of the two values this driver can write: a subset of {"ntp_servers", "timezone"}.
|
||||
#: A BMC that only runs on UTC writes {"ntp_servers"}; an application never asks it for a
|
||||
#: zone. Declared only: a driver that names nothing writes nothing.
|
||||
time_sync_writes: ClassVar[frozenset[str]]
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""How the device keeps its clock now.
|
||||
|
||||
``ntp_servers`` are the sources its configuration names, in its order
|
||||
(a pool by its name, not the addresses it resolved to). ``timezone``
|
||||
is an IANA name, or None where the device keeps an offset and a rule
|
||||
instead of a name. ``daemon`` names the client that keeps the clock on
|
||||
a host that has a choice; None for a device's own.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"ntp_servers": ["ntp1.example.org", "10.0.0.1"],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
}
|
||||
|
||||
:raises Exception: when the device could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# The steps a driver implements:
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
"""Leave the device with exactly *servers*, in this order: sources it
|
||||
had configured elsewhere stop counting (commented out, replaced)."""
|
||||
...
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
"""Set the device's zone to *zone*, an IANA name of the database."""
|
||||
...
|
||||
|
||||
def configure_time_sync(
|
||||
self, ntp_servers: Sequence[str] | None = None, timezone: str | None = None
|
||||
) -> FixResultDict:
|
||||
"""Give the device exactly *ntp_servers*, in order, and *timezone*; None leaves one alone.
|
||||
|
||||
The steps: ``ntp_servers``, ``timezone``, then ``verify``, which reads the
|
||||
device again.
|
||||
|
||||
:raises UnknownTimezone: for a *timezone* that is not an IANA name of the
|
||||
database, before anything is written.
|
||||
:raises ValueError: for a value this driver does not write
|
||||
(:attr:`time_sync_writes`), for no server or a malformed one, or for
|
||||
nothing asked -- before anything is written.
|
||||
"""
|
||||
servers = None if ntp_servers is None else check_ntp_servers(ntp_servers)
|
||||
if timezone is not None and not is_iana_zone(timezone):
|
||||
raise UnknownTimezone(timezone)
|
||||
asked = {name for name, v in (("ntp_servers", servers), ("timezone", timezone)) if v}
|
||||
if not asked:
|
||||
raise ValueError("nothing to configure: name ntp_servers, timezone or both")
|
||||
refused = asked - getattr(self, "time_sync_writes", frozenset())
|
||||
if refused:
|
||||
raise ValueError(f"this driver does not write {', '.join(sorted(refused))}")
|
||||
steps: list[FixStep] = []
|
||||
if servers is not None:
|
||||
steps.append(("ntp_servers", lambda: self._write_ntp_servers(servers)))
|
||||
if timezone is not None:
|
||||
steps.append(("timezone", lambda: self._write_time_zone(timezone)))
|
||||
steps.append(("verify", lambda: self._verify_time_sync(servers, timezone)))
|
||||
return run_fix_steps(steps)
|
||||
|
||||
def _verify_time_sync(self, servers: list[str] | None, zone: str | None) -> FixStepOutcomeDict:
|
||||
"""Read the device again: does it now have what was written?
|
||||
|
||||
Servers compare without case. A zone is compared where the device reads
|
||||
one back by name.
|
||||
"""
|
||||
now = self.get_time_sync()
|
||||
wrong: list[str] = []
|
||||
if servers is not None and [s.lower() for s in now["ntp_servers"]] != [
|
||||
s.lower() for s in servers
|
||||
]:
|
||||
wrong.append(f"the device has the servers {', '.join(now['ntp_servers']) or 'none'}")
|
||||
if zone is not None and now["timezone"] is not None and now["timezone"] != zone:
|
||||
wrong.append(f"the device is in the time zone {now['timezone']}")
|
||||
if wrong:
|
||||
return {"ok": False, "output": "; ".join(wrong)}
|
||||
return {"ok": True, "output": "the device reads back what was written"}
|
||||
|
||||
|
||||
__all__ = ["TIME_SYNC_VALUES", "TimeSyncDict", "TimeSyncMixin", "check_ntp_servers"]
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
:class:`PosixTimeSyncMixin` implements
|
||||
:class:`~napalm_device_types.time_sync.TimeSyncMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The defaults
|
||||
are Debian's paths; a driver sets its host's.
|
||||
|
||||
**Reading** is one round trip as the login user
|
||||
(:mod:`napalm_device_types._time_sync_reading`): ``timedatectl show`` for the zone
|
||||
(``/etc/timezone`` or ``/etc/localtime``'s target without it), which time sync
|
||||
service is active, and that client's configuration -- chrony's files as chronyd
|
||||
reads them, ``timedatectl show-timesync`` for systemd-timesyncd, ``ntp.conf``,
|
||||
OpenNTPD's ``ntpd.conf``. Synchronised is chronyd's own word where it runs (a
|
||||
selected source in ``chronyc -n sources``), else the kernel's (``NTPSynchronized``).
|
||||
|
||||
**Writing** runs as root through ``run_command(privileged=True)``; root never reads
|
||||
stdin -- a file is staged by the login user and installed by root.
|
||||
|
||||
* The zone: ``timedatectl set-timezone``.
|
||||
* chrony: the servers go to a file of their own and every other source is commented
|
||||
out (:mod:`napalm_device_types._chrony`), then ``chronyc reload sources``, or a
|
||||
restart of chronyd where its main configuration changed or it cannot reload.
|
||||
* systemd-timesyncd: :attr:`~PosixTimeSyncMixin.TIMESYNCD_DROPIN` resets ``NTP=``,
|
||||
sets the servers and empties ``FallbackNTP=``, then the service restarts. Servers a
|
||||
network link brings (DHCP through systemd-networkd) are not timesyncd's to drop;
|
||||
they stay, and the ``verify`` step names them.
|
||||
* ntpd, OpenNTPD, or no client: the servers are read, not written -- the step fails
|
||||
and says so.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._chrony import (
|
||||
COMMENT_MARK,
|
||||
comment_out_command,
|
||||
include_command,
|
||||
plan_chrony,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import TIME_ZONE_SH
|
||||
from napalm_device_types._time_sync_reading import (
|
||||
TIME_SYNC_UNITS,
|
||||
Reading,
|
||||
ReadPaths,
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
parse_reading,
|
||||
parse_time_sync,
|
||||
reading_command,
|
||||
time_sync_of,
|
||||
timesyncd_servers,
|
||||
)
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
from napalm_device_types.time_sync import TIME_SYNC_VALUES, TimeSyncDict, TimeSyncMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
_WRITERS = "chrony and systemd-timesyncd"
|
||||
|
||||
#: The head of the systemd-timesyncd drop-in.
|
||||
TIMESYNCD_HEADER = (
|
||||
"# The time sources, exactly these: written by napalm-device-types'\n"
|
||||
"# PosixTimeSyncMixin and replaced on every change.\n"
|
||||
)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)
|
||||
|
||||
|
||||
def timesyncd_dropin(servers: Sequence[str]) -> str:
|
||||
"""The drop-in that leaves systemd-timesyncd with exactly *servers*."""
|
||||
return f"{TIMESYNCD_HEADER}[Time]\nNTP=\nNTP={' '.join(servers)}\nFallbackNTP=\n"
|
||||
|
||||
|
||||
class PosixTimeSyncMixin(TimeSyncMixin):
|
||||
"""Time sync of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = TIME_SYNC_VALUES
|
||||
|
||||
#: ``(unit, daemon)``: the services that keep the clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: ClassVar[tuple[tuple[str, str], ...]] = TIME_SYNC_UNITS
|
||||
#: chrony's main configuration: the first that exists.
|
||||
CHRONY_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/chrony/chrony.conf", "/etc/chrony.conf")
|
||||
#: The servers' file where the main configuration has a ``sourcedir`` of its directory.
|
||||
CHRONY_SOURCES_FILE: ClassVar[str] = "/etc/chrony/sources.d/netork.sources"
|
||||
#: The servers' file otherwise (its name also in a ``confdir``); included if need be.
|
||||
CHRONY_INCLUDE_FILE: ClassVar[str] = "/etc/chrony.d/netork.conf"
|
||||
TIMESYNCD_DROPIN: ClassVar[str] = "/etc/systemd/timesyncd.conf.d/netork.conf"
|
||||
NTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/ntpsec/ntp.conf", "/etc/ntp.conf")
|
||||
OPENNTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/openntpd/ntpd.conf", "/etc/ntpd.conf")
|
||||
#: ``sh`` that prints the zone without timedatectl: a name, or a path ending in it.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_read(self) -> Reading:
|
||||
paths = ReadPaths(
|
||||
units=self.TIME_SYNC_UNITS,
|
||||
chrony_configs=self.CHRONY_CONFIGS,
|
||||
ntpd_configs=self.NTPD_CONFIGS,
|
||||
openntpd_configs=self.OPENNTPD_CONFIGS,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
return parse_reading(self.run_command(reading_command(paths), timeout=60).stdout)
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""The host's time sources, zone and client, in one round trip as the login user.
|
||||
|
||||
:raises ValueError: when the reading came back cut short.
|
||||
"""
|
||||
return time_sync_of(self._ts_read())
|
||||
|
||||
# -- writing ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_root(self, command: str) -> CommandResult:
|
||||
return self.run_command(command, privileged=True, timeout=60)
|
||||
|
||||
def _ts_checked(self, command: str) -> str:
|
||||
result = self._ts_root(command)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{command}: {_said(result) or f'exit {result.exit_code}'}")
|
||||
return _said(result)
|
||||
|
||||
def _ts_install(self, path: str, content: str) -> None:
|
||||
"""*content* to *path* as root, staged by the login user: root reads no stdin."""
|
||||
staged = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
tmp = (staged.stdout.strip().splitlines() or [""])[-1]
|
||||
if staged.exit_code != 0 or not tmp.startswith("/"):
|
||||
raise RuntimeError(f"cannot stage {path}: {_said(staged)}")
|
||||
try:
|
||||
self._ts_checked(shlex.join(["install", "-D", "-m", "644", "--", tmp, path]))
|
||||
finally:
|
||||
self.run_command(shlex.join(["rm", "-f", "--", tmp]), timeout=30)
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
result = self._ts_root(shlex.join(["timedatectl", "set-timezone", zone]))
|
||||
if result.exit_code != 0:
|
||||
return {"ok": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"ok": True, "output": f"time zone set to {zone}"}
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
reading = self._ts_read()
|
||||
if reading.daemon == "chrony":
|
||||
return self._ts_chrony(reading, servers)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return self._ts_timesyncd(servers)
|
||||
runs = f"this host runs {reading.daemon}" if reading.daemon else "none runs here"
|
||||
return {"ok": False, "output": f"time sources are written for {_WRITERS}; {runs}"}
|
||||
|
||||
def _ts_timesyncd(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
self._ts_install(self.TIMESYNCD_DROPIN, timesyncd_dropin(servers))
|
||||
self._ts_checked("systemctl restart systemd-timesyncd.service")
|
||||
return {"ok": True, "output": f"wrote {self.TIMESYNCD_DROPIN}; restarted systemd-timesyncd"}
|
||||
|
||||
def _ts_chrony(self, reading: Reading, servers: list[str]) -> FixStepOutcomeDict:
|
||||
plan = plan_chrony(
|
||||
reading.chrony,
|
||||
servers,
|
||||
sources_file=self.CHRONY_SOURCES_FILE,
|
||||
include_file=self.CHRONY_INCLUDE_FILE,
|
||||
)
|
||||
said = [f"wrote {plan.target}"]
|
||||
self._ts_install(plan.target, plan.content)
|
||||
for path, numbers in plan.comment_out.items():
|
||||
self._ts_checked(comment_out_command(path, numbers))
|
||||
said.append(f"commented out {path}:{','.join(map(str, numbers))}")
|
||||
if plan.include_in is not None:
|
||||
self._ts_checked(include_command(plan.include_in, plan.target))
|
||||
said.append(f"included it from {plan.include_in}")
|
||||
if not (plan.reload and self._ts_root("chronyc reload sources").exit_code == 0):
|
||||
self._ts_checked(shlex.join(["systemctl", "restart", f"{reading.unit}.service"]))
|
||||
said.append(f"restarted {reading.unit}")
|
||||
else:
|
||||
said.append("reloaded the sources")
|
||||
return {"ok": True, "output": "; ".join(said)}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"COMMENT_MARK",
|
||||
"PosixTimeSyncMixin",
|
||||
"chrony_synchronized",
|
||||
"parse_key_values",
|
||||
"parse_time_sync",
|
||||
"timesyncd_dropin",
|
||||
"timesyncd_servers",
|
||||
]
|
||||
+2
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.30.0"
|
||||
version = "4.35.1"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -40,6 +40,7 @@ dependencies = [
|
||||
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
|
||||
"requests>=2.25.0",
|
||||
"typing-extensions>=4.0",
|
||||
"tzdata>=2024.1", # posix_tz reads the zones' TZif files from it
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -14,7 +14,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin, FirewallDriver, ResidentialGatewayDriver
|
||||
from napalm_device_types.dhcp import normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, normalize_mac
|
||||
from napalm_device_types.models import DhcpReservationDict
|
||||
|
||||
|
||||
@@ -209,6 +209,95 @@ class TestApplyReservationSet:
|
||||
assert lines == ["[commit] no changes"]
|
||||
|
||||
|
||||
class _RefusingDhcpServer(_FakeDhcpServer):
|
||||
"""A server that refuses the reservations for the IPs in *refused*, as Kea does
|
||||
for one whose subnet it does not manage."""
|
||||
|
||||
def __init__(self, live: list[DhcpReservationDict] | None = None, *, refused: set[str]) -> None:
|
||||
super().__init__(live)
|
||||
self.refused = refused
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if reservation["ip"] in self.refused:
|
||||
raise ValueError(f"No Kea-managed subnet contains {reservation['ip']}")
|
||||
return super().apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
|
||||
|
||||
def _three() -> list[DhcpReservationDict]:
|
||||
return [
|
||||
_reservation(mac="aa:bb:cc:dd:ee:01", ip="10.10.20.50"),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:02", ip="10.10.99.5", subnet=""),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:03", ip="10.10.20.52"),
|
||||
]
|
||||
|
||||
|
||||
class TestApplyWithARefusedReservation:
|
||||
"""One reservation the server refuses must not hold back the others, and what was
|
||||
written must take effect: a server that saved a write and was never committed shows
|
||||
it as configured, so the next apply finds nothing to do and never commits it."""
|
||||
|
||||
def test_the_others_are_still_written(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert [entry["reservation"]["ip"] for entry in driver.applied] == [
|
||||
"10.10.20.50",
|
||||
"10.10.20.52",
|
||||
]
|
||||
|
||||
def test_what_was_written_is_committed(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_a_refused_set_still_commits(self):
|
||||
# An earlier apply that stopped before its commit left saved writes behind; a
|
||||
# run whose every write is refused commits too, so those take effect.
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset([_three()[1]]))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_progress_names_the_refused_one_and_why(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError) as raised:
|
||||
for line in driver.apply_dhcp_reservationset(_three()):
|
||||
lines.append(line)
|
||||
|
||||
failed = [line for line in lines if line.startswith("[failed]")]
|
||||
assert failed == [
|
||||
"[failed] aa:bb:cc:dd:ee:02 -> 10.10.99.5: No Kea-managed subnet contains 10.10.99.5"
|
||||
]
|
||||
assert lines[-1] == "[commit] applied 2 add(s), 0 update(s)"
|
||||
assert "1 of 3" in str(raised.value)
|
||||
|
||||
def test_a_refused_update_is_reported_like_an_add(self):
|
||||
driver = _RefusingDhcpServer(
|
||||
[_reservation(uuid="u1", ip="10.10.20.9")], refused={"10.10.20.50"}
|
||||
)
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 1"):
|
||||
list(driver.apply_dhcp_reservationset([_reservation()]))
|
||||
|
||||
assert driver.applied == []
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_the_error_is_a_runtime_error(self):
|
||||
# Callers that catch RuntimeError for a failed apply keep working.
|
||||
assert issubclass(DhcpApplyError, RuntimeError)
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneReservation:
|
||||
def test_a_live_reservation_without_mac_is_never_matched(self):
|
||||
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
|
||||
|
||||
@@ -13,6 +13,8 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
@@ -25,6 +27,12 @@ from napalm_device_types.host_status import (
|
||||
kernel_reboot_pending,
|
||||
parse_host_status,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
APK_INSTALLED,
|
||||
DNF_HISTORY,
|
||||
OPKG_STATUS,
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
)
|
||||
|
||||
|
||||
def _wire(
|
||||
@@ -570,3 +578,371 @@ class TestFilesystems:
|
||||
|
||||
mountpoints = [fs["mountpoint"] for fs in parse_host_status(out.stdout)["filesystems"]]
|
||||
assert "/" in mountpoints
|
||||
|
||||
|
||||
def _other(*sections: tuple[str, tuple[str, ...]], running: tuple[str, ...] = ()) -> str:
|
||||
"""A report from a host without dpkg: the given sections before ``[timers]``."""
|
||||
lines = ["[pm-running]", *running]
|
||||
for name, body in sections:
|
||||
lines += [f"[{name}]", *body]
|
||||
return _wire().replace("[timers]", "\n".join([*lines, "[timers]"]))
|
||||
|
||||
|
||||
class TestInterruptedRpm:
|
||||
"""dnf and yum hosts: a transaction rpm was cut off in leaves the old and the
|
||||
new version of a package both installed (``dnf check --duplicates``,
|
||||
``package-cleanup --dupes``), and dnf's history records a transaction that
|
||||
never ended. netOrk only reports it: there is no one repair that is safe on
|
||||
every host (NetOrk/netork#851, decided 2026-11-07).
|
||||
|
||||
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
|
||||
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
|
||||
1 or ERROR 2), yum's ``transaction-all.*`` journal that
|
||||
``yum-complete-transaction`` reads -- and still have to be checked against a
|
||||
real Oracle Linux 9 host.
|
||||
"""
|
||||
|
||||
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
|
||||
#: An upgrade of glibc and openssl-libs cut off after the new versions were in.
|
||||
CUT_OFF = ("glibc x86_64", "kernel-uek-core x86_64", "openssl-libs x86_64")
|
||||
|
||||
def _rpm(self, dupes=(), history=("42 1",), journal=(), frontend="dnf", running=()):
|
||||
return _other(
|
||||
("rpm-frontend", (frontend,)),
|
||||
("rpm-installonly", self.INSTALLONLY),
|
||||
("rpm-dupes", dupes),
|
||||
("yum-journal", journal),
|
||||
("dnf-history", history),
|
||||
running=running,
|
||||
)
|
||||
|
||||
def test_a_clean_rpm_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._rpm())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_two_versions_of_a_package_left_by_a_cut_off_transaction(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "glibc", "state": "duplicate"},
|
||||
{"name": "openssl-libs", "state": "duplicate"},
|
||||
]
|
||||
|
||||
def test_kernels_are_installed_side_by_side_on_purpose(self):
|
||||
"""dnf's installonly packages: several versions are the rule, not a fault."""
|
||||
status = parse_host_status(self._rpm(dupes=("kernel-uek-core x86_64",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
def test_a_package_duplicated_in_two_arches_is_named_once(self):
|
||||
status = parse_host_status(self._rpm(dupes=("glibc i686", "glibc x86_64")))
|
||||
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_the_last_transaction_never_finished(self):
|
||||
status = parse_host_status(self._rpm(history=("43 0",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
@pytest.mark.parametrize("history", [("44 1",), ("45 2",), ()])
|
||||
def test_a_finished_or_unreadable_history_says_nothing(self, history):
|
||||
"""A failed transaction (ERROR) did end; an unreadable history is no answer."""
|
||||
assert parse_host_status(self._rpm(history=history))["package_manager_interrupted"] is False
|
||||
|
||||
def test_yum_left_its_transaction_journal(self):
|
||||
journal = ("transaction-all.2026-11-01.10:12.33", "transaction-done.2026-11-01.10:12.33")
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["package_manager"] == "yum"
|
||||
|
||||
def test_a_journal_yum_gave_up_on_is_none(self):
|
||||
"""yum-complete-transaction skips a journal it renamed ``.disabled``."""
|
||||
journal = ("transaction-all.2026-11-01.10:12.33.disabled",)
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
@pytest.mark.parametrize("name", ["rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"])
|
||||
def test_a_running_transaction_makes_it_unknown(self, name):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=(name,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_another_managers_run_does_not(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
|
||||
def test_the_command_asks_rpm_only_where_dnf_or_yum_exists(self):
|
||||
assert "command -v rpm" in HOST_STATUS_COMMAND
|
||||
assert "rpm -qa --qf" in HOST_STATUS_COMMAND
|
||||
assert f"file:{DNF_HISTORY}?mode=ro" in HOST_STATUS_COMMAND
|
||||
|
||||
def test_it_never_repairs(self):
|
||||
for word in ("distro-sync", "--cleandupes", "remove", "yum-complete-transaction", "rpm -e"):
|
||||
assert word not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedApk:
|
||||
"""Alpine and OpenWrt 25.12: apk marks a package whose files or scripts failed
|
||||
in ``/lib/apk/db/installed`` (``f:`` with ``f`` broken files, ``s`` a broken
|
||||
script; ``x`` xattrs and ``S`` the checksum kind are no fault), and ``apk fix``
|
||||
with no arguments is what picks exactly those (apk-tools ``app_fix.c``).
|
||||
netOrk only reports it (NetOrk/netork#851). Shaped after apk-tools'
|
||||
``database.c``; to be checked against a real OpenWrt 25.12 host."""
|
||||
|
||||
def _apk(self, audit=(), running=()):
|
||||
return _other(("apk-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._apk())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "apk"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("flags", "states"),
|
||||
[
|
||||
("s", ["broken-script"]),
|
||||
("f", ["broken-files"]),
|
||||
("fsS", ["broken-files", "broken-script"]),
|
||||
],
|
||||
)
|
||||
def test_a_package_apk_marked_broken(self, flags, states):
|
||||
status = parse_host_status(self._apk(audit=(f"{flags} busybox",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "busybox", "state": state} for state in states
|
||||
]
|
||||
|
||||
def test_a_running_apk_makes_it_unknown(self):
|
||||
status = parse_host_status(self._apk(audit=("s busybox",), running=("apk",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_installed_database(self):
|
||||
assert "/lib/apk/db/installed" in HOST_STATUS_COMMAND
|
||||
assert "apk fix" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedOpkg:
|
||||
"""OpenWrt before 25.12: opkg writes ``Status: <want> <flags> <state>`` per
|
||||
package to ``/usr/lib/opkg/status``; a package whose run was cut off or whose
|
||||
postinst failed stays ``unpacked``, and ``opkg configure`` finishes every
|
||||
unpacked package (opkg-lede ``opkg_cmd.c``, ``pkg.c``). netOrk's update run
|
||||
does that first (NetOrk/netork#851). To be checked against a real router."""
|
||||
|
||||
def _opkg(self, audit=(), running=()):
|
||||
return _other(("opkg-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_status_file_is_not_interrupted(self):
|
||||
status = parse_host_status(self._opkg())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "opkg"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("line", "state"),
|
||||
[
|
||||
("user unpacked luci-app-sqm", "unpacked"),
|
||||
("ok half-configured luci-app-sqm", "half-configured"),
|
||||
("ok half-installed luci-app-sqm", "half-installed"),
|
||||
("reinstreq,user installed luci-app-sqm", "reinstall-required"),
|
||||
],
|
||||
)
|
||||
def test_every_state_an_interrupted_run_leaves(self, line, state):
|
||||
status = parse_host_status(self._opkg(audit=(line,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": state}]
|
||||
|
||||
def test_a_running_opkg_makes_it_unknown(self):
|
||||
status = parse_host_status(self._opkg(audit=("ok unpacked x",), running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_status_file_and_never_configures(self):
|
||||
assert "/usr/lib/opkg/status" in HOST_STATUS_COMMAND
|
||||
assert "opkg configure" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestPackageManager:
|
||||
def test_dpkg_says_so(self):
|
||||
assert parse_host_status(_wire(dpkg=()))["package_manager"] == "dpkg"
|
||||
|
||||
def test_a_busy_dpkg_still_says_which(self):
|
||||
status = parse_host_status(_wire(dpkg=(), busy=True))
|
||||
|
||||
assert (status["package_manager"], status["package_manager_interrupted"]) == (
|
||||
"dpkg",
|
||||
None,
|
||||
)
|
||||
|
||||
def test_no_package_manager_read_is_none(self):
|
||||
status = parse_host_status(_wire())
|
||||
|
||||
assert status["package_manager"] is None
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
|
||||
#: An rpm that answers the two queries the command asks, as on an EL host whose
|
||||
#: glibc upgrade was cut off; and a dnf that only has to exist.
|
||||
_RPM_STUB = """#!/bin/sh
|
||||
case "$*" in
|
||||
*--whatprovides*) printf '%s\\n' kernel-core kernel-core 'no package provides kernel-PAE' ;;
|
||||
-qa*) printf '%s\\n' 'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' \\
|
||||
'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' 'bash x86_64' ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
|
||||
_APK_DB = """C:Q1abc=
|
||||
P:alpine-baselayout
|
||||
V:3.6.8-r1
|
||||
A:x86_64
|
||||
F:etc
|
||||
R:fstab
|
||||
|
||||
C:Q1def=
|
||||
P:busybox
|
||||
V:1.37.0-r12
|
||||
A:x86_64
|
||||
f:s
|
||||
F:bin
|
||||
R:busybox
|
||||
|
||||
C:Q1ghi=
|
||||
P:musl
|
||||
V:1.2.5-r9
|
||||
f:S
|
||||
"""
|
||||
|
||||
#: /usr/lib/opkg/status as opkg writes it: luci-app-sqm's postinst never ran.
|
||||
_OPKG_STATUS = """Package: libc
|
||||
Version: 1.2.4-4
|
||||
Depends: libgcc1
|
||||
Status: install hold installed
|
||||
Architecture: mipsel_24kc
|
||||
Installed-Time: 1717000000
|
||||
|
||||
Package: luci-app-sqm
|
||||
Version: 1.5.3-r1
|
||||
Depends: libc, sqm-scripts
|
||||
Status: install user unpacked
|
||||
Architecture: all
|
||||
Installed-Time: 1730000000
|
||||
"""
|
||||
|
||||
|
||||
def _stubs(tmp_path, *names: str, rpm: bool = False) -> dict:
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
for name in names:
|
||||
stub = bin_dir / name
|
||||
stub.write_text(_RPM_STUB if name == "rpm" else "#!/bin/sh\nexit 0\n")
|
||||
stub.chmod(0o755)
|
||||
return {**os.environ, "PATH": f"{bin_dir}:/usr/bin:/bin"}
|
||||
|
||||
|
||||
def _run(command: str, env: dict, shell: tuple[str, ...]) -> str:
|
||||
report = f"printf '%s%s\\n' HSTAT_ BEGIN; {command} printf '%s%s\\n' HSTAT_ END"
|
||||
program = shutil.which(shell[0]) or shell[0]
|
||||
return subprocess.run(
|
||||
[program, *shell[1:], report], capture_output=True, text=True, env=env, timeout=30
|
||||
).stdout
|
||||
|
||||
|
||||
_SHELLS = [("sh", "-c")] + ([("busybox", "sh", "-c")] if shutil.which("busybox") else [])
|
||||
|
||||
|
||||
def _skip_busybox_rpm(shell: tuple[str, ...]) -> None:
|
||||
"""A BusyBox built with its own ``rpm`` applet runs that before any stand-in."""
|
||||
if shell[0] == "busybox":
|
||||
applets = subprocess.run(["busybox", "--list"], capture_output=True, text=True).stdout
|
||||
if "rpm" in applets.split():
|
||||
pytest.skip("this busybox has an rpm applet of its own")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shell", _SHELLS)
|
||||
class TestThePackageManagerCommandOnARealShell:
|
||||
"""The command part run for real, with stand-in programs and files."""
|
||||
|
||||
def test_rpm_duplicates_without_the_kernel_and_the_keys(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "dnf"
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
history = tmp_path / "history.sqlite"
|
||||
with sqlite3.connect(history) as db:
|
||||
db.execute("CREATE TABLE trans (id INTEGER PRIMARY KEY, state INTEGER NOT NULL)")
|
||||
db.executemany("INSERT INTO trans VALUES (?, ?)", [(41, 1), (42, 0)])
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(DNF_HISTORY, str(history)), env, shell)
|
||||
|
||||
assert "[dnf-history]\n42 0\n" in out
|
||||
|
||||
def test_apk_names_the_package_it_marked(self, tmp_path, shell):
|
||||
installed = tmp_path / "installed"
|
||||
installed.write_text(_APK_DB)
|
||||
env = _stubs(tmp_path, "apk")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(APK_INSTALLED, str(installed)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "apk"
|
||||
assert status["interrupted_packages"] == [{"name": "busybox", "state": "broken-script"}]
|
||||
|
||||
def test_opkg_names_the_unpacked_package(self, tmp_path, shell):
|
||||
status_file = tmp_path / "status"
|
||||
status_file.write_text(_OPKG_STATUS)
|
||||
env = _stubs(tmp_path, "opkg")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(OPKG_STATUS, str(status_file)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "opkg"
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": "unpacked"}]
|
||||
|
||||
def test_a_host_with_none_of_them_reads_none(self, tmp_path, shell):
|
||||
env = _stubs(tmp_path)
|
||||
env["PATH"] = str(tmp_path / "bin")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
assert parse_host_status(out)["package_manager"] is None
|
||||
|
||||
def test_the_running_processes_come_from_proc(self, tmp_path, shell):
|
||||
"""The test runner's own process is there under its own name; no manager is."""
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, _stubs(tmp_path), shell)
|
||||
|
||||
assert "[pm-running]\n" in out
|
||||
|
||||
|
||||
class TestTheWholeCommand:
|
||||
def test_it_runs_on_busybox(self):
|
||||
"""OpenWrt's and Alpine's shell: still one intact report."""
|
||||
if not shutil.which("busybox"):
|
||||
pytest.skip("no busybox here")
|
||||
out = subprocess.run(
|
||||
["busybox", "sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, timeout=30
|
||||
).stdout
|
||||
|
||||
assert parse_host_status(out)["filesystems"]
|
||||
|
||||
def test_the_package_managers_are_part_of_it(self):
|
||||
assert PACKAGE_MANAGERS_COMMAND in HOST_STATUS_COMMAND
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
|
||||
whether it gets the journal (NetOrk/netork#2053)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
|
||||
|
||||
DEBIAN = """\
|
||||
module(load="imuxsock")
|
||||
module(load="imklog")
|
||||
$FileOwner root
|
||||
$IncludeConfig /etc/rsyslog.d/*.conf
|
||||
*.*;auth,authpriv.none\t\t-/var/log/syslog
|
||||
auth,authpriv.*\t\t\t/var/log/auth.log
|
||||
*.emerg\t\t\t\t:omusrmsg:*
|
||||
"""
|
||||
|
||||
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
|
||||
|
||||
|
||||
def _rules(text: str) -> list[dict]:
|
||||
return parse_rsyslog(text)["rules"]
|
||||
|
||||
|
||||
class TestRules:
|
||||
def test_a_debian_host_forwards_nothing(self):
|
||||
assert _rules(DEBIAN) == []
|
||||
|
||||
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
|
||||
assert _rules(DEBIAN + GRAYLOG) == [
|
||||
{
|
||||
"protocol": "udp",
|
||||
"target": "graylog.example.net",
|
||||
"port": 1514,
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
},
|
||||
]
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
def test_an_ipv6_target_keeps_its_brackets_off(self):
|
||||
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
|
||||
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
|
||||
|
||||
def test_a_narrower_selector_is_partial_and_says_which(self):
|
||||
(rule,) = _rules("authpriv.* @10.0.0.5\n")
|
||||
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
|
||||
|
||||
def test_excluding_a_facility_is_partial_too(self):
|
||||
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
|
||||
assert rule["complete"] is False
|
||||
|
||||
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
|
||||
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
|
||||
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
|
||||
|
||||
def test_relp_is_its_own_protocol(self):
|
||||
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
|
||||
|
||||
def test_an_action_over_several_lines_is_read_whole(self):
|
||||
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
|
||||
"tcp",
|
||||
"10.0.0.7",
|
||||
6514,
|
||||
True,
|
||||
)
|
||||
|
||||
def test_an_action_on_its_own_takes_every_message(self):
|
||||
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
|
||||
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
|
||||
|
||||
def test_an_action_under_a_condition_is_partial_and_names_it(self):
|
||||
(rule,) = _rules(
|
||||
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
|
||||
)
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == "$programname == 'nginx'"
|
||||
|
||||
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
|
||||
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
|
||||
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
|
||||
|
||||
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
|
||||
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
|
||||
rules = _rules(text)
|
||||
assert [(r["target"], r["complete"]) for r in rules] == [
|
||||
("a", False),
|
||||
("b", False),
|
||||
("c", True),
|
||||
]
|
||||
|
||||
def test_a_property_filter_is_a_partial_selector(self):
|
||||
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == ':programname, isequal, "sshd"'
|
||||
|
||||
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
|
||||
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
|
||||
|
||||
def test_comments_are_no_rules(self):
|
||||
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
|
||||
|
||||
def test_a_hash_inside_quotes_is_no_comment(self):
|
||||
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
|
||||
assert rule["target"] == "10.0.0.13"
|
||||
|
||||
def test_a_local_action_is_no_forwarding(self):
|
||||
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
|
||||
|
||||
|
||||
class TestInputs:
|
||||
def test_an_imfile_input_names_its_file_and_ruleset(self):
|
||||
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
|
||||
assert parse_rsyslog(text)["file_inputs"] == [
|
||||
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
|
||||
]
|
||||
|
||||
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
|
||||
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
|
||||
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
|
||||
|
||||
|
||||
class TestModules:
|
||||
def test_imjournal_reads_the_journal(self):
|
||||
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
|
||||
|
||||
def test_legacy_modload_counts(self):
|
||||
modules = parse_rsyslog("$ModLoad imuxsock\n")
|
||||
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
|
||||
|
||||
|
||||
class TestJournalToSyslog:
|
||||
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
|
||||
the distribution prints commented out in its ``journald.conf``."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("lines", "expected"),
|
||||
[
|
||||
(["#ForwardToSyslog=yes"], True),
|
||||
(["#ForwardToSyslog=no"], False),
|
||||
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
|
||||
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
|
||||
([], None),
|
||||
],
|
||||
)
|
||||
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
|
||||
assert journal_to_syslog(lines) is expected
|
||||
+152
-1
@@ -20,6 +20,7 @@ from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
|
||||
@@ -44,6 +45,35 @@ OPEN = f"""/proc/610/fd/7 /var/log/syslog
|
||||
/proc/1500/fd/3 /srv/app/log with space.log
|
||||
/proc/1500/fd/4 /srv/app/Local Storage/leveldb/000004.log
|
||||
/proc/1500/fd/5 /var/lib/rocksdb/012345.log
|
||||
/proc/300/fd/11 /var/log/nginx/access.log
|
||||
/proc/300/fd/12 /mnt/docroot/data/log/paperless.log
|
||||
/proc/300/fd/13 socket:[48213]
|
||||
/proc/300/fd/14 pipe:[48214]
|
||||
/proc/1600/fd/6 /var/log/host/syslog
|
||||
"""
|
||||
#: Each descriptor's ``flags`` from ``/proc/<pid>/fdinfo/<fd>``, octal: the access
|
||||
#: mode (``& 3``) is 1 or 2 for a writer -- 0102001 is O_WRONLY|O_APPEND|O_LARGEFILE
|
||||
#: -- and 0 for a reader (0100000, O_RDONLY|O_LARGEFILE). CrowdSec (300) reads
|
||||
#: nginx's log and a log on a mount; a promtail in a container (1600) reads the
|
||||
#: host's syslog as its own /var/log/host/syslog. 1500 opens its log O_RDWR.
|
||||
FLAGS = """/proc/610/fdinfo/7:flags:\t0102001
|
||||
/proc/610/fdinfo/8:flags:\t0102001
|
||||
/proc/620/fdinfo/9:flags:\t02100002
|
||||
/proc/900/fdinfo/5:flags:\t0102001
|
||||
/proc/900/fdinfo/6:flags:\t0102001
|
||||
/proc/901/fdinfo/5:flags:\t0102001
|
||||
/proc/700/fdinfo/4:flags:\t0102001
|
||||
/proc/1100/fdinfo/30:flags:\t0102001
|
||||
/proc/1100/fdinfo/31:flags:\t0102001
|
||||
/proc/1400/fdinfo/3:flags:\t0102001
|
||||
/proc/1500/fdinfo/3:flags:\t0100002
|
||||
/proc/1500/fdinfo/4:flags:\t0100002
|
||||
/proc/1500/fdinfo/5:flags:\t0100002
|
||||
/proc/300/fdinfo/11:flags:\t0100000
|
||||
/proc/300/fdinfo/12:flags:\t0100000
|
||||
/proc/300/fdinfo/13:flags:\t02004002
|
||||
/proc/300/fdinfo/14:flags:\t02004001
|
||||
/proc/1600/fdinfo/6:flags:\t0100000
|
||||
"""
|
||||
STAT = """/proc/610/fd/7 99000 1762250000
|
||||
/proc/610/fd/8 4100 1762250100
|
||||
@@ -53,6 +83,8 @@ STAT = """/proc/610/fd/7 99000 1762250000
|
||||
/proc/1100/fd/30 7000 1762250400
|
||||
/proc/1100/fd/31 300 1762250500
|
||||
/proc/1400/fd/3 0 1762250600
|
||||
/proc/300/fd/11 2048 1762250200
|
||||
/proc/300/fd/12 81920 1762250700
|
||||
"""
|
||||
CGROUPS = f"""610 0::/system.slice/rsyslog.service
|
||||
900 0::/system.slice/nginx.service
|
||||
@@ -60,6 +92,8 @@ CGROUPS = f"""610 0::/system.slice/rsyslog.service
|
||||
1100 0::/system.slice/docker.service
|
||||
1400 0::/system.slice/docker-{CID}.scope
|
||||
1500 0::/user.slice/user-1000.slice/session-3.scope
|
||||
300 0::/system.slice/crowdsec.service
|
||||
1600 0::/system.slice/docker-{OTHER}.scope
|
||||
"""
|
||||
COMM = """610 rsyslogd
|
||||
900 nginx
|
||||
@@ -67,6 +101,8 @@ COMM = """610 rsyslogd
|
||||
1100 dockerd
|
||||
1400 python3
|
||||
1500 app
|
||||
300 crowdsec
|
||||
1600 promtail
|
||||
"""
|
||||
|
||||
|
||||
@@ -77,14 +113,18 @@ def _wire(
|
||||
stat: str = STAT,
|
||||
cgroups: str = CGROUPS,
|
||||
comm: str = COMM,
|
||||
flags: str | None = FLAGS,
|
||||
noise: str = "",
|
||||
forwarding: str = "",
|
||||
) -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
parts = [noise, "LOGS_BEGIN\n"]
|
||||
if journald is not None:
|
||||
parts += ["[journald]\n", journald]
|
||||
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
|
||||
parts.append("LOGS_END\n")
|
||||
if flags is not None:
|
||||
parts += ["[flags]\n", flags]
|
||||
parts += [forwarding, "LOGS_END\n"]
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
@@ -184,6 +224,100 @@ class TestParse:
|
||||
parse_log_sources(cut)
|
||||
|
||||
|
||||
class TestWriters:
|
||||
"""A file is attributed to the process writing it, not to one reading it
|
||||
(NetOrk/netork#2092): CrowdSec, Promtail or a ``tail -F`` holding a log is
|
||||
no producer of it."""
|
||||
|
||||
def test_a_reader_with_the_lower_pid_does_not_take_the_writers_file(self) -> None:
|
||||
access = _by_key(_sources())[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("nginx", "nginx")
|
||||
|
||||
def test_a_file_only_readers_hold_is_listed_without_a_producer(self) -> None:
|
||||
paperless = _by_key(_sources())[("file", None, "/mnt/docroot/data/log/paperless.log")]
|
||||
assert paperless == {
|
||||
"kind": "file",
|
||||
"path": "/mnt/docroot/data/log/paperless.log",
|
||||
"unit": None,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": 81920,
|
||||
"modified": "2025-11-04T10:05:00+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_open_for_reading_and_writing_is_written(self) -> None:
|
||||
spaced = _by_key(_sources())[("file", None, "/srv/app/log with space.log")]
|
||||
assert spaced["process"] == "app"
|
||||
|
||||
def test_a_file_only_a_reader_in_a_container_holds_is_left_out(self) -> None:
|
||||
# Its path is the reader's view of a mount: it names nothing on the host,
|
||||
# and the container does not write it.
|
||||
assert not [s for s in _sources() if s["path"] == "/var/log/host/syslog"]
|
||||
|
||||
def test_sockets_and_pipes_are_no_sources(self) -> None:
|
||||
assert not [s for s in _sources() if not (s["path"] or "/").startswith("/")]
|
||||
|
||||
def test_a_deleted_file_stays_out_though_it_is_written(self) -> None:
|
||||
assert not [s for s in _sources() if (s["path"] or "").startswith("/var/log/old")]
|
||||
|
||||
def test_a_holder_whose_flags_did_not_answer_is_no_writer(self) -> None:
|
||||
# The process ended between the listing and the read of its fdinfo.
|
||||
flags = FLAGS.replace("/proc/900/fdinfo/5:flags:\t0102001\n", "")
|
||||
flags = flags.replace("/proc/901/fdinfo/5:flags:\t0102001\n", "")
|
||||
access = _by_key(_sources(_wire(flags=flags)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == (None, None)
|
||||
|
||||
def test_a_report_without_flags_names_the_first_holder_as_before(self) -> None:
|
||||
# An older command printed no [flags]: who writes cannot be told apart.
|
||||
access = _by_key(_sources(_wire(flags=None)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("crowdsec", "crowdsec")
|
||||
|
||||
|
||||
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
|
||||
FORWARDING = """[rsyslog]
|
||||
running
|
||||
[rsyslog-conf]
|
||||
| module(load="imuxsock")
|
||||
| [comment-looking line]
|
||||
|
|
||||
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
|
||||
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
|
||||
[journald-conf]
|
||||
#ForwardToSyslog=yes
|
||||
"""
|
||||
|
||||
|
||||
class TestForwarding:
|
||||
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
|
||||
|
||||
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None
|
||||
assert forwarding["rsyslog"] is True
|
||||
assert forwarding["journal"] is True
|
||||
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
|
||||
("graylog.lan", 1514, True)
|
||||
]
|
||||
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
|
||||
|
||||
def test_a_config_line_in_brackets_is_no_section(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None and forwarding["rules"]
|
||||
|
||||
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
|
||||
stopped = FORWARDING.replace("running\n", "")
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
|
||||
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
|
||||
def test_a_report_without_the_sections_says_nothing(self) -> None:
|
||||
assert parse_log_forwarding(_wire()) is None
|
||||
|
||||
def test_the_reading_carries_it(self) -> None:
|
||||
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
|
||||
assert reading["forwarding"] is not None
|
||||
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
|
||||
|
||||
|
||||
class _Host(LogSourcesMixin):
|
||||
"""Carries the command the way a driver does; *answers* per privilege."""
|
||||
|
||||
@@ -229,6 +363,21 @@ class TestCommand:
|
||||
def test_it_is_valid_posix_sh(self) -> None:
|
||||
assert subprocess.run(["sh", "-n", "-c", LOG_SOURCES_COMMAND]).returncode == 0
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_here_a_file_held_for_writing_has_a_producer_and_one_held_for_reading_none(
|
||||
self, tmp_path
|
||||
) -> None:
|
||||
written, read = tmp_path / "written.log", tmp_path / "read.log"
|
||||
read.write_text("one line\n")
|
||||
with written.open("a") as w, read.open() as r:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert w.writable() and r.readable()
|
||||
sources = {s["path"]: s for s in _sources(out)}
|
||||
assert sources[str(written)]["process"] is not None
|
||||
assert sources[str(read)]["process"] is None
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_it_runs_here_and_parses(self, tmp_path) -> None:
|
||||
log = tmp_path / "held.log"
|
||||
@@ -241,3 +390,5 @@ class TestCommand:
|
||||
# This test's own process holds the file, so it must be among the sources.
|
||||
mine = [s for s in _sources(out) if s["path"] == str(log)]
|
||||
assert mine and mine[0]["size"] == len("one line\n")
|
||||
# Whether rsyslog runs here or not, the report says where logs go.
|
||||
assert parse_log_forwarding(out) is not None
|
||||
@@ -0,0 +1,161 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or its parts.
|
||||
|
||||
The wire carries IANA names (``Europe/Berlin``). OpenWrt hands ``TZ`` to libc and
|
||||
wants the rule itself (``CET-1CEST,M3.5.0,M10.5.0/3``); a switch wants a UTC
|
||||
offset and the summer-time rule. The rule is the footer of the zone's TZif file
|
||||
from the ``tzdata`` package (moved here from netOrk's ``core/posix_tz.py`` for
|
||||
NetOrk/netork#2361).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib.resources import files
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
|
||||
BERLIN = "CET-1CEST,M3.5.0,M10.5.0/3"
|
||||
|
||||
|
||||
def _zones() -> list[str]:
|
||||
return files("tzdata").joinpath("zones").read_text().split()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("zone", "rule"),
|
||||
[
|
||||
("Europe/Berlin", BERLIN),
|
||||
("America/New_York", "EST5EDT,M3.2.0,M11.1.0"),
|
||||
("America/Sao_Paulo", "<-03>3"),
|
||||
("Australia/Lord_Howe", "<+1030>-10:30<+11>-11,M10.1.0,M4.1.0"),
|
||||
("Asia/Kolkata", "IST-5:30"),
|
||||
("UTC", "UTC0"),
|
||||
],
|
||||
)
|
||||
def test_an_iana_name_becomes_its_posix_rule(zone, rule):
|
||||
assert posix_tz(zone) == rule
|
||||
assert is_iana_zone(zone)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("rule", [BERLIN, "<+03>-3", "EST5EDT,M3.2.0/2,M11.1.0/2", "UTC0"])
|
||||
def test_a_posix_string_is_taken_as_it_is(rule):
|
||||
assert posix_tz(rule) == rule
|
||||
assert not is_iana_zone(rule)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
["Mars/Olympus_Mons", "Europe", "../../etc/passwd", "/etc/localtime", "Europe/../UTC", ""],
|
||||
)
|
||||
def test_neither_a_zone_nor_a_rule_has_none(value):
|
||||
assert posix_tz(value) is None
|
||||
assert not is_iana_zone(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["UTC", "GMT0", BERLIN, "<-03>3", "IST-5:30"])
|
||||
def test_posix_strings(value):
|
||||
assert is_posix_tz(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["Europe/Berlin", "America/Argentina/Buenos_Aires", "", "CET-"])
|
||||
def test_not_posix_strings(value):
|
||||
assert not is_posix_tz(value)
|
||||
|
||||
|
||||
def test_every_zone_of_the_database_has_a_rule_the_check_accepts():
|
||||
# A caller tells an IANA name from a rule with is_posix_tz; it must never
|
||||
# take the database's own rule for a name.
|
||||
zones = _zones()
|
||||
missing = [z for z in zones if not is_posix_tz(posix_tz(z) or "")]
|
||||
assert len(zones) > 500
|
||||
assert missing == []
|
||||
|
||||
|
||||
def test_the_refusal_names_the_zone_in_plain_english():
|
||||
exc = UnknownTimezone("Mars/Olympus_Mons")
|
||||
|
||||
assert isinstance(exc, ValueError)
|
||||
assert exc.zone == "Mars/Olympus_Mons"
|
||||
assert "Mars/Olympus_Mons" in str(exc)
|
||||
assert "Europe/Berlin" in str(exc)
|
||||
|
||||
|
||||
class TestTheRuleInParts:
|
||||
"""What a switch takes: the standard offset and, where there is one, the summer-time rule."""
|
||||
|
||||
def test_berlin(self):
|
||||
assert parse_posix_tz(BERLIN) == PosixTzDict(
|
||||
std_name="CET",
|
||||
utc_offset=3600,
|
||||
dst_name="CEST",
|
||||
dst_offset=7200,
|
||||
dst_start={"month": 3, "week": 5, "weekday": 0, "time": 7200},
|
||||
dst_end={"month": 10, "week": 5, "weekday": 0, "time": 10800},
|
||||
)
|
||||
|
||||
def test_utc_has_no_summer_time(self):
|
||||
assert parse_posix_tz("UTC0") == {
|
||||
"std_name": "UTC",
|
||||
"utc_offset": 0,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
|
||||
@pytest.mark.parametrize("bare", ["UTC", "GMT"])
|
||||
def test_the_bare_names_openwrt_writes(self, bare):
|
||||
parts = parse_posix_tz(bare)
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_name"]) == (bare, 0, None)
|
||||
|
||||
def test_sao_paulo_has_given_up_summer_time(self):
|
||||
parts = parse_posix_tz(posix_tz("America/Sao_Paulo") or "")
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_offset"]) == (
|
||||
"-03",
|
||||
-10800,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_lord_howe_moves_by_half_an_hour(self):
|
||||
parts = parse_posix_tz(posix_tz("Australia/Lord_Howe") or "")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (37800, 39600)
|
||||
assert parts["dst_start"] == {"month": 10, "week": 1, "weekday": 0, "time": 7200}
|
||||
assert parts["dst_end"] == {"month": 4, "week": 1, "weekday": 0, "time": 7200}
|
||||
|
||||
def test_new_york_is_west_of_utc_and_gains_an_hour_by_default(self):
|
||||
parts = parse_posix_tz("EST5EDT,M3.2.0,M11.1.0")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (-18000, -14400)
|
||||
|
||||
def test_a_transition_time_may_lie_outside_the_day(self):
|
||||
# Greenland's rule: a day's -1:00 and 0:00 (RFC 8536 3.3.1 allows -167..167 hours).
|
||||
parts = parse_posix_tz("<-02>2<-01>,M3.5.0/-1,M10.5.0/0")
|
||||
assert parts is not None
|
||||
assert parts["dst_start"] == {"month": 3, "week": 5, "weekday": 0, "time": -3600}
|
||||
assert parts["dst_end"] == {"month": 10, "week": 5, "weekday": 0, "time": 0}
|
||||
|
||||
def test_offsets_with_minutes_and_seconds(self):
|
||||
parts = parse_posix_tz("<+1245>-12:45<+1345>,M9.5.0/2:45,M4.1.0/3:45")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (45900, 49500)
|
||||
assert parts["dst_start"] == {"month": 9, "week": 5, "weekday": 0, "time": 9900}
|
||||
|
||||
@pytest.mark.parametrize("rule", ["Europe/Berlin", "", "CET-", "CET-1CEST,J60,J300"])
|
||||
def test_what_it_cannot_take_apart_is_none(self, rule):
|
||||
# Julian-day rules are POSIX, but no zone of the database uses them.
|
||||
assert parse_posix_tz(rule) is None
|
||||
|
||||
def test_every_zone_of_the_database_comes_apart(self):
|
||||
broken = [z for z in _zones() if parse_posix_tz(posix_tz(z) or "") is None]
|
||||
assert broken == []
|
||||
@@ -0,0 +1,51 @@
|
||||
"""get_running_backups: the backups running on a device right now.
|
||||
|
||||
An application that restarts a device asks first, right before the restart: a
|
||||
restart aborts a backup that runs on the device -- a hypervisor's vzdump, a
|
||||
backup server's backup, sync, verify or garbage collection, a host's own restic
|
||||
or borg run -- and a guest's restart breaks the backup its hypervisor is taking
|
||||
of it (NetOrk/netork#2318). A driver mixes it in when it can tell; no role
|
||||
demands it, and a driver without it cannot tell.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
HypervisorDriver,
|
||||
OSDriver,
|
||||
RunningBackupDict,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (BackupServerDriver, HypervisorDriver, OSDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, RunningBackupsMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
"""hasattr is the caller's probe: only a driver that can tell answers True."""
|
||||
assert not hasattr(RunningBackupsMixin, "get_running_backups")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(RunningBackupsMixin)
|
||||
assert "def get_running_backups(self) -> list[RunningBackupDict]" in source
|
||||
assert "empty list for a failed read" in source
|
||||
|
||||
|
||||
def test_the_kinds_cover_a_backup_servers_jobs_and_the_backup_itself():
|
||||
assert set(RUNNING_BACKUP_KINDS) == {"backup", "sync", "verify", "gc", "prune", "tape"}
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(RunningBackupDict)
|
||||
assert set(hints) == {"kind", "job", "task", "guests", "since"}
|
||||
assert hints["guests"] == list[str]
|
||||
assert hints["since"] == int | None
|
||||
@@ -0,0 +1,353 @@
|
||||
"""How an OpenSSH server lets people in: the effective settings and sshd's PAM auth stack.
|
||||
|
||||
The parsers are pure. The getter runs against a fake host that answers ``sshd -T``
|
||||
and the dump of ``/etc/pam.d`` from a dict, so the root read, the PAM includes
|
||||
and a failed read are covered without a machine.
|
||||
|
||||
The ``sshd -T`` texts are abridged from what OpenSSH 9.2p1 on Debian 12 prints
|
||||
(the keywords and their order as sshd writes them; the lines that say nothing
|
||||
about authentication are left out). OpenSSH before 8.7 also prints
|
||||
``challengeresponseauthentication``; OpenBSD's sshd has no ``usepam``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import OpenSshServerMixin
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
PAM_FILES_COMMAND,
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_pam_auth,
|
||||
parse_pam_files,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
|
||||
#: Debian 12 as installed: password login on, PAM on, no AuthenticationMethods.
|
||||
SSHD_T_DEBIAN = """\
|
||||
port 22
|
||||
addressfamily any
|
||||
listenaddress [::]:22
|
||||
listenaddress 0.0.0.0:22
|
||||
usepam yes
|
||||
logingracetime 120
|
||||
x11displayoffset 10
|
||||
maxauthtries 6
|
||||
maxsessions 10
|
||||
clientaliveinterval 0
|
||||
clientalivecountmax 3
|
||||
permitrootlogin without-password
|
||||
ignorerhosts yes
|
||||
ignoreuserknownhosts no
|
||||
hostbasedauthentication no
|
||||
hostbasedusesnamefrompacketonly no
|
||||
pubkeyauthentication yes
|
||||
kerberosauthentication no
|
||||
kerberosorlocalpasswd yes
|
||||
kerberosticketcleanup yes
|
||||
gssapiauthentication no
|
||||
gssapicleanupcredentials yes
|
||||
gssapikeyexchange no
|
||||
gssapistrictacceptorcheck yes
|
||||
gssapistorecredentialsonrotate no
|
||||
passwordauthentication yes
|
||||
kbdinteractiveauthentication no
|
||||
printmotd no
|
||||
x11forwarding yes
|
||||
authenticationmethods any
|
||||
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256
|
||||
"""
|
||||
|
||||
#: Key only: password and keyboard-interactive off.
|
||||
SSHD_T_KEY_ONLY = SSHD_T_DEBIAN.replace(
|
||||
"passwordauthentication yes", "passwordauthentication no"
|
||||
).replace("permitrootlogin without-password", "permitrootlogin no")
|
||||
|
||||
#: A key and then an OTP through PAM, for everyone.
|
||||
SSHD_T_KEY_AND_OTP = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no", "kbdinteractiveauthentication yes"
|
||||
).replace("authenticationmethods any", "authenticationmethods publickey,keyboard-interactive")
|
||||
|
||||
#: OpenSSH 8.4p1 (Debian 11): the old keyword beside the new one.
|
||||
SSHD_T_OLD = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication yes\nchallengeresponseauthentication no",
|
||||
)
|
||||
|
||||
#: OpenBSD 7.6: no usepam line.
|
||||
SSHD_T_OPENBSD = SSHD_T_KEY_ONLY.replace("usepam yes\n", "")
|
||||
|
||||
#: Debian 12's /etc/pam.d/sshd, abridged: the auth stack comes from common-auth.
|
||||
PAM_SSHD_DEBIAN = """\
|
||||
# PAM configuration for the Secure Shell service
|
||||
|
||||
# Standard Un*x authentication.
|
||||
@include common-auth
|
||||
|
||||
# Disallow non-root logins when /etc/nologin exists.
|
||||
account required pam_nologin.so
|
||||
|
||||
# Standard Un*x authorization.
|
||||
@include common-account
|
||||
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
|
||||
@include common-session
|
||||
@include common-password
|
||||
"""
|
||||
|
||||
PAM_COMMON_AUTH = """\
|
||||
auth\t[success=1 default=ignore]\tpam_unix.so nullok
|
||||
auth\trequisite\t\t\tpam_deny.so
|
||||
auth\trequired\t\t\tpam_permit.so
|
||||
"""
|
||||
|
||||
PAM_COMMON_ACCOUNT = "account [success=1 new_authtok_reqd=done default=ignore] pam_unix.so\n"
|
||||
|
||||
#: The usual way to add a TOTP: a line of its own in sshd's stack.
|
||||
PAM_SSHD_OTP = PAM_SSHD_DEBIAN.replace(
|
||||
"@include common-auth",
|
||||
"#@include common-auth\nauth required pam_google_authenticator.so nullok",
|
||||
)
|
||||
|
||||
#: Red Hat's form: ``auth substack password-auth``, a module by its full path.
|
||||
PAM_SSHD_RHEL = """\
|
||||
#%PAM-1.0
|
||||
auth substack password-auth
|
||||
auth include postlogin
|
||||
-auth optional /usr/lib64/security/pam_reauthorize.so prepare
|
||||
account required pam_sepermit.so
|
||||
"""
|
||||
PAM_PASSWORD_AUTH = "auth required pam_env.so\nauth sufficient pam_unix.so try_first_pass nullok\n"
|
||||
PAM_POSTLOGIN = "session optional pam_umask.so silent\n"
|
||||
|
||||
|
||||
# ── parse_sshd_auth ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_parse_reads_the_settings_that_decide_how_people_log_in():
|
||||
auth = parse_sshd_auth(SSHD_T_DEBIAN)
|
||||
assert auth == {
|
||||
"password_authentication": True,
|
||||
"kbd_interactive_authentication": False,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [],
|
||||
"permit_root_login": "without-password",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def test_parse_splits_authentication_methods_into_lists_of_methods():
|
||||
auth = parse_sshd_auth(
|
||||
SSHD_T_KEY_ONLY.replace(
|
||||
"authenticationmethods any",
|
||||
"authenticationmethods publickey,keyboard-interactive:pam publickey,password",
|
||||
)
|
||||
)
|
||||
assert auth["authentication_methods"] == [
|
||||
["publickey", "keyboard-interactive:pam"],
|
||||
["publickey", "password"],
|
||||
]
|
||||
|
||||
|
||||
def test_parse_counts_the_old_challenge_response_keyword_as_keyboard_interactive():
|
||||
"""Before 8.7 either keyword could switch PAM's prompts on: either one is enough."""
|
||||
assert parse_sshd_auth(SSHD_T_OLD)["kbd_interactive_authentication"] is True
|
||||
old_only = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication no\nchallengeresponseauthentication yes",
|
||||
)
|
||||
assert parse_sshd_auth(old_only)["kbd_interactive_authentication"] is True
|
||||
|
||||
|
||||
def test_parse_says_none_for_usepam_where_sshd_has_no_pam():
|
||||
assert parse_sshd_auth(SSHD_T_OPENBSD)["use_pam"] is None
|
||||
|
||||
|
||||
def test_parse_takes_a_method_sshd_was_built_without_as_off():
|
||||
"""sshd built without GSSAPI prints no gssapiauthentication line: that method cannot be used."""
|
||||
text = SSHD_T_DEBIAN.replace("gssapiauthentication no\n", "")
|
||||
assert parse_sshd_auth(text)["gssapi_authentication"] is False
|
||||
|
||||
|
||||
def test_parse_refuses_output_that_is_not_sshd_t():
|
||||
with pytest.raises(ValueError, match="passwordauthentication"):
|
||||
parse_sshd_auth("sshd: no hostkeys available -- exiting.\n")
|
||||
|
||||
|
||||
# ── PAM ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _pairs(entries) -> list[tuple[str, str]]:
|
||||
return [(e["control"], e["module"]) for e in entries]
|
||||
|
||||
|
||||
def _modules(stack) -> list[str]:
|
||||
return [e["module"] for e in stack]
|
||||
|
||||
|
||||
def test_pam_auth_lists_modules_and_includes_in_stack_order():
|
||||
# Every @include: which of them hold auth lines is for the included file to say.
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_DEBIAN)) == [
|
||||
("include", "common-auth"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_reads_bracketed_controls_paths_and_optional_lines():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_RHEL)) == [
|
||||
("substack", "password-auth"),
|
||||
("include", "postlogin"),
|
||||
("optional", "pam_reauthorize.so"),
|
||||
]
|
||||
assert parse_pam_auth(PAM_COMMON_AUTH) == [
|
||||
{"module": "pam_unix.so", "control": "[success=1 default=ignore]", "args": ["nullok"]},
|
||||
{"module": "pam_deny.so", "control": "requisite", "args": []},
|
||||
{"module": "pam_permit.so", "control": "required", "args": []},
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_keeps_a_modules_arguments():
|
||||
"""``nullok`` lets a user without an OTP secret through: the consumer has to see it."""
|
||||
assert parse_pam_auth(PAM_SSHD_OTP)[0] == {
|
||||
"module": "pam_google_authenticator.so",
|
||||
"control": "required",
|
||||
"args": ["nullok"],
|
||||
}
|
||||
|
||||
|
||||
def test_pam_auth_skips_comments_and_other_types():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_OTP)) == [
|
||||
("required", "pam_google_authenticator.so"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_ignores_an_include_that_is_not_a_service_name():
|
||||
assert parse_pam_auth("auth include ../../etc/shadow\n@include\nauth required\n") == []
|
||||
|
||||
|
||||
def _dump(files: dict[str, str]) -> str:
|
||||
"""What ``tail -n +1`` prints for several files: a header before each."""
|
||||
return "\n".join(f"==> /etc/pam.d/{name} <==\n{text}" for name, text in files.items())
|
||||
|
||||
|
||||
def test_pam_files_splits_the_dump_by_file():
|
||||
files = {"sshd": PAM_SSHD_DEBIAN, "common-auth": PAM_COMMON_AUTH}
|
||||
assert parse_pam_files(_dump(files)) == files
|
||||
|
||||
|
||||
def test_auth_stack_follows_includes_and_substacks_in_stack_order():
|
||||
files = {
|
||||
"sshd": PAM_SSHD_RHEL + "auth include password-auth\n",
|
||||
"password-auth": PAM_PASSWORD_AUTH,
|
||||
"postlogin": PAM_POSTLOGIN,
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_env.so", "pam_unix.so", "pam_reauthorize.so"]
|
||||
|
||||
|
||||
def test_auth_stack_stops_at_an_include_loop_and_skips_a_missing_file():
|
||||
files = {
|
||||
"sshd": "@include a\n@include gone\n",
|
||||
"a": "@include sshd\nauth required pam_unix.so\n",
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_auth_stack_is_none_without_a_file_for_the_service():
|
||||
assert pam_auth_stack({"login": PAM_COMMON_AUTH}) is None
|
||||
|
||||
|
||||
# ── the getter ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class _Host(SshServerAuthMixin):
|
||||
"""Answers ``sshd -T`` (root only) and the dump of /etc/pam.d."""
|
||||
|
||||
def __init__(self, sshd_t: str | None, pam: dict[str, str] | None = None) -> None:
|
||||
self.sshd_t = sshd_t
|
||||
self.pam = pam
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
self.calls.append((command, privileged))
|
||||
if command == SSHD_EFFECTIVE_COMMAND:
|
||||
if self.sshd_t is None or not privileged:
|
||||
return CommandResult("", "sshd: no hostkeys available -- exiting.", 1)
|
||||
return CommandResult(self.sshd_t, "", 0)
|
||||
assert command == PAM_FILES_COMMAND, command
|
||||
if self.pam is None:
|
||||
return CommandResult("", "tail: cannot open '/etc/pam.d/*' for reading", 1)
|
||||
return CommandResult(_dump(self.pam), "", 0)
|
||||
|
||||
|
||||
def test_getter_reads_sshd_t_as_root_and_pam_without():
|
||||
host = _Host(
|
||||
SSHD_T_DEBIAN,
|
||||
{
|
||||
"sshd": PAM_SSHD_DEBIAN,
|
||||
"common-auth": PAM_COMMON_AUTH,
|
||||
"common-account": PAM_COMMON_ACCOUNT,
|
||||
},
|
||||
)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["password_authentication"] is True
|
||||
assert _modules(auth["pam_auth_stack"]) == ["pam_unix.so", "pam_deny.so", "pam_permit.so"]
|
||||
# sshd -T once, as root; PAM's files once, as the login user.
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True), (PAM_FILES_COMMAND, False)]
|
||||
|
||||
|
||||
def test_getter_names_an_otp_module_in_sshds_own_stack():
|
||||
host = _Host(SSHD_T_KEY_AND_OTP, {"sshd": PAM_SSHD_OTP})
|
||||
stack = host.get_ssh_server_auth()["pam_auth_stack"]
|
||||
assert _modules(stack) == ["pam_google_authenticator.so"]
|
||||
|
||||
|
||||
def test_getter_keeps_what_it_read_when_one_pam_file_is_unreadable():
|
||||
"""tail goes on past a file it cannot open and exits 1: the rest is still read."""
|
||||
|
||||
class _Partial(_Host):
|
||||
def run_command(self, command: str, **kwargs) -> CommandResult: # type: ignore[override]
|
||||
result = super().run_command(command, **kwargs)
|
||||
if command == PAM_FILES_COMMAND:
|
||||
return CommandResult(result.stdout, "tail: cannot open 'x'", 1)
|
||||
return result
|
||||
|
||||
host = _Partial(SSHD_T_DEBIAN, {"sshd": "auth required pam_unix.so\n"})
|
||||
assert _modules(host.get_ssh_server_auth()["pam_auth_stack"]) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_getter_leaves_pam_unread_without_usepam():
|
||||
host = _Host(SSHD_T_OPENBSD)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["pam_auth_stack"] is None
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True)]
|
||||
|
||||
|
||||
def test_getter_says_none_when_pam_cannot_be_read():
|
||||
assert _Host(SSHD_T_DEBIAN).get_ssh_server_auth()["pam_auth_stack"] is None
|
||||
|
||||
|
||||
def test_getter_raises_when_sshd_t_fails():
|
||||
with pytest.raises(RuntimeError, match="no hostkeys"):
|
||||
_Host(None).get_ssh_server_auth()
|
||||
|
||||
|
||||
def test_the_hardening_mixin_carries_the_reader():
|
||||
"""Every driver that can harden sshd can also say how it lets people in."""
|
||||
assert issubclass(OpenSshServerMixin, SshServerAuthMixin)
|
||||
@@ -0,0 +1,231 @@
|
||||
"""TimeSyncMixin: a device's time sources and time zone, read and set (NetOrk/netork#2361).
|
||||
|
||||
The application keeps one list of NTP servers and a zone per site, and aligns a
|
||||
device that differs. ``get_time_sync`` is declared only, so ``callable(getattr(
|
||||
driver, "get_time_sync", None))`` is the truthful answer; ``configure_time_sync``
|
||||
is a template of steps like every fix, and an API driver overrides it whole.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import ClassVar, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
TIME_SYNC_VALUES,
|
||||
AccessPointDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
OSDriver,
|
||||
SwitchDriver,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
UnknownTimezone,
|
||||
check_ntp_servers,
|
||||
)
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(TimeSyncDict)
|
||||
assert hints == {
|
||||
"ntp_servers": list[str],
|
||||
"timezone": str | None,
|
||||
"synchronized": bool | None,
|
||||
"daemon": str | None,
|
||||
}
|
||||
|
||||
|
||||
def test_the_two_values_a_driver_may_write():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == TIME_SYNC_VALUES
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (AccessPointDriver, FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, TimeSyncMixin)
|
||||
|
||||
|
||||
def test_the_reading_is_declared_not_implemented():
|
||||
assert not hasattr(TimeSyncMixin, "get_time_sync")
|
||||
source = inspect.getsource(TimeSyncMixin)
|
||||
assert "def get_time_sync(self) -> TimeSyncDict" in source
|
||||
assert "time_sync_writes: ClassVar[frozenset[str]]" in source
|
||||
|
||||
|
||||
def test_the_template_takes_both_values_or_none_of_them():
|
||||
signature = inspect.signature(TimeSyncMixin.configure_time_sync)
|
||||
assert list(signature.parameters) == ["self", "ntp_servers", "timezone"]
|
||||
assert signature.parameters["ntp_servers"].default is None
|
||||
assert signature.parameters["timezone"].default is None
|
||||
|
||||
|
||||
class _Device(TimeSyncMixin):
|
||||
"""A device that keeps its values in memory and reads back what it was given."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
def __init__(self, *, keeps_zone_name: bool = True, fail: str = "") -> None:
|
||||
self.state: TimeSyncDict = {
|
||||
"ntp_servers": ["0.pool.ntp.org"],
|
||||
"timezone": "UTC",
|
||||
"synchronized": True,
|
||||
"daemon": None,
|
||||
}
|
||||
self.keeps_zone_name = keeps_zone_name
|
||||
self.fail = fail
|
||||
self.calls: list[tuple[str, object]] = []
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
state = dict(self.state)
|
||||
if not self.keeps_zone_name:
|
||||
state["timezone"] = None
|
||||
return state # type: ignore[return-value]
|
||||
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
if self.fail == "ntp_servers":
|
||||
return {"ok": False, "output": "refused"}
|
||||
self.state["ntp_servers"] = list(servers)
|
||||
return {"ok": True, "output": "written"}
|
||||
|
||||
def _write_time_zone(self, zone):
|
||||
self.calls.append(("timezone", zone))
|
||||
self.state["timezone"] = zone
|
||||
return {"ok": True, "output": f"zone {zone}"}
|
||||
|
||||
|
||||
class TestConfigureTimeSync:
|
||||
def test_both_values_then_a_check(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(["ntp1.example.org", "10.0.0.1"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
assert device.calls == [
|
||||
("ntp_servers", ["ntp1.example.org", "10.0.0.1"]),
|
||||
("timezone", "Europe/Berlin"),
|
||||
]
|
||||
|
||||
def test_none_leaves_a_value_alone(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert device.calls == [("timezone", "Europe/Berlin")]
|
||||
assert device.state["ntp_servers"] == ["0.pool.ntp.org"]
|
||||
|
||||
def test_servers_alone(self):
|
||||
device = _Device()
|
||||
assert device.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self):
|
||||
device = _Device()
|
||||
with pytest.raises(UnknownTimezone, match="Mars/Olympus_Mons"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_posix_rule_is_not_a_zone_on_the_wire(self):
|
||||
with pytest.raises(UnknownTimezone):
|
||||
_Device().configure_time_sync(timezone="CET-1CEST,M3.5.0,M10.5.0/3")
|
||||
|
||||
def test_a_value_the_driver_cannot_write_is_refused_before_anything_is_written(self):
|
||||
class Bmc(_Device):
|
||||
time_sync_writes = frozenset({"ntp_servers"})
|
||||
|
||||
device = Bmc()
|
||||
with pytest.raises(ValueError, match="timezone"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_driver_that_names_nothing_it_writes_writes_nothing(self):
|
||||
class Silent(TimeSyncMixin):
|
||||
"""Mixes the contract in and never says what it writes."""
|
||||
|
||||
with pytest.raises(ValueError, match="ntp_servers"):
|
||||
Silent().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
def test_nothing_asked_is_a_mistake(self):
|
||||
with pytest.raises(ValueError, match="nothing"):
|
||||
_Device().configure_time_sync()
|
||||
|
||||
def test_a_failed_step_stops_the_rest(self):
|
||||
device = _Device(fail="ntp_servers")
|
||||
result = device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers"]
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_the_check_reads_back_what_was_written(self):
|
||||
class Stubborn(_Device):
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
return {"ok": True, "output": "written"} # and nothing changed
|
||||
|
||||
result = Stubborn().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
verify = result["steps"][-1]
|
||||
assert verify["step"] == "verify"
|
||||
assert "0.pool.ntp.org" in verify["output"]
|
||||
|
||||
def test_the_check_names_the_zone_the_device_kept(self):
|
||||
class Elsewhere(_Device):
|
||||
def _write_time_zone(self, zone):
|
||||
return {"ok": True, "output": "set"} # and the device stayed on UTC
|
||||
|
||||
result = Elsewhere().configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["output"] == "the device is in the time zone UTC"
|
||||
|
||||
def test_servers_compare_without_case(self):
|
||||
class Lower(_Device):
|
||||
def get_time_sync(self):
|
||||
state = super().get_time_sync()
|
||||
state["ntp_servers"] = [s.lower() for s in state["ntp_servers"]]
|
||||
return state
|
||||
|
||||
assert Lower().configure_time_sync(["NTP1.Example.org"])["success"] is True
|
||||
|
||||
def test_a_device_without_a_zone_name_is_not_checked_for_one(self):
|
||||
# A switch keeps an offset and a summer-time rule, not a name: it reads None.
|
||||
result = _Device(keeps_zone_name=False).configure_time_sync(timezone="Europe/Berlin")
|
||||
assert result["success"] is True
|
||||
|
||||
|
||||
class TestCheckNtpServers:
|
||||
def test_in_order_stripped_each_once(self):
|
||||
assert check_ntp_servers([" ntp1.example.org", "10.0.0.1", "NTP1.example.org", "::1"]) == [
|
||||
"ntp1.example.org",
|
||||
"10.0.0.1",
|
||||
"::1",
|
||||
]
|
||||
|
||||
def test_none_at_all_is_refused(self):
|
||||
# A device without a source drifts; leaving the value alone is None, not [].
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([])
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([" "])
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"ntp1.example.org iburst",
|
||||
"-ntp.example.org",
|
||||
"ntp1.example.org\nserver evil",
|
||||
"ntp;reboot",
|
||||
"$(id)",
|
||||
"a" * 254,
|
||||
],
|
||||
)
|
||||
def test_only_a_host_name_or_an_address(self, bad):
|
||||
with pytest.raises(ValueError, match="not a host name or address"):
|
||||
check_ntp_servers([bad])
|
||||
|
||||
def test_a_string_is_not_a_list_of_servers(self):
|
||||
with pytest.raises(TypeError):
|
||||
check_ntp_servers("ntp1.example.org") # type: ignore[arg-type]
|
||||
@@ -0,0 +1,543 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
The reading is run by a real ``sh`` over a directory tree that stands in for the
|
||||
host, with ``timedatectl``, ``systemctl`` and ``chronyc`` replaced by scripts that
|
||||
keep a log -- so the shell itself is tested, not a hand-written copy of its output.
|
||||
``sed`` and ``install`` are the real ones, so the files the writes leave are what a
|
||||
host would get (NetOrk/netork#2361).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixTimeSyncMixin, UnknownTimezone, parse_time_sync
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.time_sync_posix import (
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
timesyncd_servers,
|
||||
)
|
||||
|
||||
# What the tools print, as recorded.
|
||||
|
||||
TIMEDATECTL = """\
|
||||
Timezone=Europe/Berlin
|
||||
LocalRTC=no
|
||||
CanNTP=yes
|
||||
NTP=yes
|
||||
NTPSynchronized=yes
|
||||
TimeUSec=Wed 2026-11-11 14:23:16 CET
|
||||
RTCTimeUSec=Wed 2026-11-11 13:23:16 CET
|
||||
"""
|
||||
|
||||
#: Ubuntu 24.04 without any server configured: only the compiled-in fallback.
|
||||
TIMESYNC_FALLBACK = """\
|
||||
FallbackNTPServers=ntp.ubuntu.com
|
||||
ServerName=ntp.ubuntu.com
|
||||
ServerAddress=91.189.91.157
|
||||
RootDistanceMaxUSec=5s
|
||||
PollIntervalMinUSec=32s
|
||||
PollIntervalMaxUSec=34min 8s
|
||||
PollIntervalUSec=34min 8s
|
||||
NTPMessage={ Leap=0, Version=4, Mode=4, Stratum=2, Precision=-24, RootDelay=42.190ms, RootDispersion=1.190ms, Reference=84A36001, Ignored=no, PacketCount=119, Jitter=7.481ms }
|
||||
Frequency=455471
|
||||
"""
|
||||
|
||||
TIMESYNC_CONFIGURED = """\
|
||||
LinkNTPServers=192.168.1.1
|
||||
SystemNTPServers=ntp1.example.org ntp2.example.org
|
||||
FallbackNTPServers=0.debian.pool.ntp.org 1.debian.pool.ntp.org
|
||||
ServerName=ntp1.example.org
|
||||
"""
|
||||
|
||||
CHRONYC_SYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^- 162.159.200.123 3 6 377 41 +814us[ +814us] +/- 14ms
|
||||
^* 192.168.1.1 2 6 377 40 -45us[ -63us] +/- 10ms
|
||||
"""
|
||||
|
||||
CHRONYC_UNSYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^? 162.159.200.123 0 6 0 - +0ns[ +0ns] +/- 0ns
|
||||
"""
|
||||
|
||||
#: Debian 12's chrony.conf, comments shortened, its directories under the test's root.
|
||||
DEBIAN_CHRONY_CONF = """\
|
||||
# Welcome to the chrony configuration file. See chrony.conf(5) for more
|
||||
# information about usable directives.
|
||||
|
||||
# Include configuration files found in /etc/chrony/conf.d.
|
||||
confdir {root}/etc/chrony/conf.d
|
||||
|
||||
# Use Debian vendor zone.
|
||||
pool 2.debian.pool.ntp.org iburst
|
||||
|
||||
# Use time sources from DHCP.
|
||||
sourcedir {root}/run/chrony-dhcp
|
||||
|
||||
# Use NTP sources found in /etc/chrony/sources.d.
|
||||
sourcedir {root}/etc/chrony/sources.d
|
||||
|
||||
keyfile /etc/chrony/chrony.keys
|
||||
driftfile /var/lib/chrony/chrony.drift
|
||||
rtcsync
|
||||
makestep 1 3
|
||||
leapsectz right/UTC
|
||||
"""
|
||||
|
||||
TIMEDATECTL_TOOL = """#!/bin/sh
|
||||
echo "timedatectl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
show) cat "$FIX/timedatectl" ;;
|
||||
show-timesync) [ -f "$FIX/timesync" ] && cat "$FIX/timesync" ;;
|
||||
set-timezone)
|
||||
[ "$2" = "$REFUSE_ZONE" ] && { echo "Failed to set time zone: Invalid or not installed time zone '$2'" >&2; exit 1; }
|
||||
sed -i "s#^Timezone=.*#Timezone=$2#" "$FIX/timedatectl" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
SYSTEMCTL_TOOL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
is-active)
|
||||
u=${2%.service}
|
||||
case " $ACTIVE " in *" $u "*) echo active ;; *) echo inactive; exit 3 ;; esac ;;
|
||||
restart)
|
||||
[ -n "$FAIL_RESTART" ] && { echo "Job for $2 failed." >&2; exit 1; }
|
||||
if [ "$2" = systemd-timesyncd.service ]; then
|
||||
servers=$(sed -n 's/^NTP=\\(..*\\)$/\\1/p' "$DROPIN")
|
||||
printf 'SystemNTPServers=%s\\n' "$servers" > "$FIX/timesync"
|
||||
fi ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CHRONYC_TOOL = """#!/bin/sh
|
||||
echo "chronyc $*" >> "$LOG"
|
||||
case "$*" in
|
||||
"-n sources") [ -f "$FIX/chronyc" ] && cat "$FIX/chronyc" ;;
|
||||
"reload sources") exit "${RELOAD_EXIT:-0}" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixTimeSyncMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, active: str = "chrony") -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CHRONY_CONFIGS = (str(etc / "chrony/chrony.conf"), str(etc / "chrony.conf")) # type: ignore[misc]
|
||||
self.CHRONY_SOURCES_FILE = str(etc / "chrony/sources.d/netork.sources") # type: ignore[misc]
|
||||
self.CHRONY_INCLUDE_FILE = str(etc / "chrony.d/netork.conf") # type: ignore[misc]
|
||||
self.TIMESYNCD_DROPIN = str(etc / "systemd/timesyncd.conf.d/netork.conf") # type: ignore[misc]
|
||||
self.NTPD_CONFIGS = (str(etc / "ntpsec/ntp.conf"), str(etc / "ntp.conf")) # type: ignore[misc]
|
||||
self.OPENNTPD_CONFIGS = (str(etc / "openntpd/ntpd.conf"),) # type: ignore[misc]
|
||||
self.TIME_ZONE_SH = ( # type: ignore[misc]
|
||||
f"cat {etc}/timezone 2>/dev/null || readlink {etc}/localtime 2>/dev/null"
|
||||
)
|
||||
self.env = {
|
||||
"PATH": f"{root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(root / "log"),
|
||||
"FIX": str(root / "fix"),
|
||||
"ACTIVE": active,
|
||||
"DROPIN": self.TIMESYNCD_DROPIN,
|
||||
}
|
||||
self.commands: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append((command, privileged))
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command],
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
env={**os.environ, **self.env},
|
||||
timeout=30,
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> list[str]:
|
||||
log = self.root / "log"
|
||||
return log.read_text().splitlines() if log.exists() else []
|
||||
|
||||
def privileged(self) -> list[str]:
|
||||
return [c for c, p in self.commands if p]
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
def _write(path: Path, text: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in ("bin", "fix", "etc"):
|
||||
(tmp_path / d).mkdir()
|
||||
(tmp_path / "fix/timedatectl").write_text(TIMEDATECTL)
|
||||
for name, body in (
|
||||
("timedatectl", TIMEDATECTL_TOOL),
|
||||
("systemctl", SYSTEMCTL_TOOL),
|
||||
("chronyc", CHRONYC_TOOL),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def debian(root: Path) -> Path:
|
||||
"""chrony as Debian 12 ships it, with a source in each place it reads."""
|
||||
_write(root / "etc/chrony/chrony.conf", DEBIAN_CHRONY_CONF.format(root=root))
|
||||
_write(root / "etc/chrony/conf.d/local.conf", "server ntp-conf.example.org iburst\n")
|
||||
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
|
||||
_write(root / "etc/chrony/sources.d/other.sources", "server ntp-other.example.org\n")
|
||||
(root / "fix/chronyc").write_text(CHRONYC_SYNCED)
|
||||
return root
|
||||
|
||||
|
||||
def _no_timedatectl(root: Path) -> None:
|
||||
"""A host without systemd: the command is not there (the build host's must not answer)."""
|
||||
_tool(root, "timedatectl", '#!/bin/sh\necho "sh: timedatectl: not found" >&2\nexit 127\n')
|
||||
|
||||
|
||||
def _text(path: Path) -> str:
|
||||
return path.read_text()
|
||||
|
||||
|
||||
class TestParsing:
|
||||
def test_key_values(self):
|
||||
values = parse_key_values(TIMEDATECTL)
|
||||
assert values["Timezone"] == "Europe/Berlin"
|
||||
assert values["NTPSynchronized"] == "yes"
|
||||
|
||||
def test_timesyncd_uses_the_fallback_only_when_nothing_else_is_known(self):
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_FALLBACK)) == ["ntp.ubuntu.com"]
|
||||
# runtime, then system, then link: the order timesyncd tries them in.
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_CONFIGURED)) == [
|
||||
"ntp1.example.org",
|
||||
"ntp2.example.org",
|
||||
"192.168.1.1",
|
||||
]
|
||||
assert timesyncd_servers({}) == []
|
||||
|
||||
def test_chrony_is_synchronised_while_it_has_selected_a_source(self):
|
||||
assert chrony_synchronized(CHRONYC_SYNCED) is True
|
||||
assert chrony_synchronized(CHRONYC_UNSYNCED) is False
|
||||
assert chrony_synchronized("") is None
|
||||
|
||||
def test_chrony_without_a_main_configuration_has_no_sources(self):
|
||||
assert parse_time_sync("@@ts active\nchrony chrony active\n@@ts end\n")["ntp_servers"] == []
|
||||
|
||||
def test_a_reading_cut_short_raises(self):
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
parse_time_sync("@@ts timedatectl\nTimezone=UTC\n")
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_chrony_names_its_sources_in_the_order_it_reads_them(self, debian: Path):
|
||||
host = Host(debian)
|
||||
assert host.get_time_sync() == {
|
||||
"ntp_servers": [
|
||||
"ntp-conf.example.org",
|
||||
"2.debian.pool.ntp.org",
|
||||
"192.168.1.1",
|
||||
"ntp-other.example.org",
|
||||
],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
}
|
||||
|
||||
def test_the_reading_is_one_round_trip_without_root(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.get_time_sync()
|
||||
assert [p for _, p in host.commands] == [False]
|
||||
|
||||
def test_chrony_says_whether_it_is_synchronised_not_the_kernel(self, debian: Path):
|
||||
(debian / "fix/chronyc").write_text(CHRONYC_UNSYNCED)
|
||||
assert Host(debian).get_time_sync()["synchronized"] is False
|
||||
|
||||
def test_without_an_answer_from_chronyc_the_kernel_says(self, debian: Path):
|
||||
(debian / "fix/chronyc").unlink()
|
||||
assert Host(debian).get_time_sync()["synchronized"] is True
|
||||
|
||||
def test_commented_and_other_directives_are_no_sources(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"#pool 2.rhel.pool.ntp.org iburst\n! server old.example.org\n"
|
||||
" server ntp1.example.org iburst\nrefclock PPS /dev/pps0\npeer peer.example.org\n",
|
||||
)
|
||||
reading = Host(root, active="chronyd").get_time_sync()
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "peer.example.org"]
|
||||
assert reading["daemon"] == "chrony"
|
||||
|
||||
def test_an_include_pattern_is_followed(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\nmakestep 1 3\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
assert Host(root).get_time_sync()["ntp_servers"] == ["suse.pool.ntp.org"]
|
||||
|
||||
def test_timesyncd(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
reading = Host(root, active="systemd-timesyncd").get_time_sync()
|
||||
assert reading["daemon"] == "systemd-timesyncd"
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "ntp2.example.org", "192.168.1.1"]
|
||||
assert reading["synchronized"] is True
|
||||
|
||||
def test_ntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/ntpsec/ntp.conf",
|
||||
"driftfile /var/lib/ntpsec/ntp.drift\npool 0.debian.pool.ntp.org iburst\n"
|
||||
"server 127.127.1.0\nserver ntp1.example.org iburst\nrestrict default kod\n",
|
||||
)
|
||||
reading = Host(root, active="ntpsec").get_time_sync()
|
||||
assert reading["daemon"] == "ntpd"
|
||||
assert reading["ntp_servers"] == ["0.debian.pool.ntp.org", "ntp1.example.org"]
|
||||
|
||||
def test_openntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/openntpd/ntpd.conf",
|
||||
'servers pool.ntp.org\nserver ntp1.example.org\nconstraints from "https://www.google.com"\n',
|
||||
)
|
||||
reading = Host(root, active="openntpd").get_time_sync()
|
||||
assert reading["daemon"] == "openntpd"
|
||||
assert reading["ntp_servers"] == ["pool.ntp.org", "ntp1.example.org"]
|
||||
|
||||
def test_no_client_runs(self, root: Path):
|
||||
(root / "fix/timedatectl").write_text(
|
||||
TIMEDATECTL.replace("NTPSynchronized=yes", "NTPSynchronized=no")
|
||||
)
|
||||
assert Host(root, active="").get_time_sync() == {
|
||||
"ntp_servers": [],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": False,
|
||||
"daemon": None,
|
||||
}
|
||||
|
||||
def test_without_timedatectl_the_zone_comes_from_the_files(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/localtime").symlink_to("/usr/share/zoneinfo/America/Sao_Paulo")
|
||||
reading = Host(root, active="").get_time_sync()
|
||||
assert reading["timezone"] == "America/Sao_Paulo"
|
||||
assert reading["synchronized"] is None
|
||||
|
||||
def test_etc_timezone_wins_over_the_link(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/timezone").write_text("Australia/Lord_Howe\n")
|
||||
assert Host(root, active="").get_time_sync()["timezone"] == "Australia/Lord_Howe"
|
||||
|
||||
def test_a_copied_localtime_has_no_name(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
assert Host(root, active="").get_time_sync()["timezone"] is None
|
||||
|
||||
|
||||
class TestChronyServers:
|
||||
def test_exactly_these_servers(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp1.example.org iburst\nserver 10.0.0.1 iburst\npool 2.de.pool.ntp.org iburst\n"
|
||||
)
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
assert "\n#napalm-time-sync: pool 2.debian.pool.ntp.org iburst\n" in conf
|
||||
assert f"\n#napalm-time-sync: sourcedir {debian}/run/chrony-dhcp\n" in conf
|
||||
assert f"\nsourcedir {debian}/etc/chrony/sources.d\n" in conf
|
||||
assert f"\nconfdir {debian}/etc/chrony/conf.d\n" in conf
|
||||
assert _text(debian / "etc/chrony/conf.d/local.conf") == (
|
||||
"#napalm-time-sync: server ntp-conf.example.org iburst\n"
|
||||
)
|
||||
assert _text(debian / "etc/chrony/sources.d/other.sources") == (
|
||||
"#napalm-time-sync: server ntp-other.example.org\n"
|
||||
)
|
||||
# DHCP's own file is left alone: it is not read any more.
|
||||
assert _text(debian / "run/chrony-dhcp/eth0.sources") == "server 192.168.1.1 iburst\n"
|
||||
# The main configuration changed: chronyd reads it only when it starts.
|
||||
assert "systemctl restart chrony.service" in host.log()
|
||||
assert host.get_time_sync()["ntp_servers"] == [
|
||||
"ntp1.example.org",
|
||||
"10.0.0.1",
|
||||
"2.de.pool.ntp.org",
|
||||
]
|
||||
|
||||
def test_a_second_run_only_reloads_the_sources(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
(debian / "log").unlink()
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp2.example.org", "ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/chrony.conf") == conf
|
||||
assert "chronyc reload sources" in host.log()
|
||||
assert not any(line.startswith("systemctl restart") for line in host.log())
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp2.example.org iburst\nserver ntp1.example.org iburst\n"
|
||||
)
|
||||
|
||||
def test_a_chronyd_that_cannot_reload_sources_is_restarted(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
host = Host(debian)
|
||||
host.env["RELOAD_EXIT"] = "1"
|
||||
assert host.configure_time_sync(["ntp2.example.org"])["success"] is True
|
||||
log = host.log()
|
||||
reload = log.index("chronyc reload sources")
|
||||
assert log[reload + 1] == "systemctl restart chrony.service"
|
||||
|
||||
def test_writes_are_root_s_and_root_reads_no_stdin(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.configure_time_sync(["ntp1.example.org"])
|
||||
staged = [c for c, p in host.commands if not p and "mktemp" in c]
|
||||
assert len(staged) == 1
|
||||
assert host.privileged()
|
||||
assert all("mktemp" not in c for c in host.privileged())
|
||||
|
||||
def test_rhel_without_a_directory_gets_an_include(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"pool 2.rhel.pool.ntp.org iburst\ndriftfile /var/lib/chrony/drift\n",
|
||||
)
|
||||
host = Host(root, active="chronyd")
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
include = root / "etc/chrony.d/netork.conf"
|
||||
assert _text(include) == "server ntp1.example.org iburst\n"
|
||||
assert _text(root / "etc/chrony.conf") == (
|
||||
"#napalm-time-sync: pool 2.rhel.pool.ntp.org iburst\n"
|
||||
"driftfile /var/lib/chrony/drift\n"
|
||||
f"# napalm-time-sync: the time sources\ninclude {include}\n"
|
||||
)
|
||||
assert "systemctl restart chronyd.service" in host.log()
|
||||
|
||||
def test_an_include_that_already_takes_the_file_is_used(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony.conf") == f"include {root}/etc/chrony.d/*.conf\n"
|
||||
assert (
|
||||
_text(root / "etc/chrony.d/pool.conf")
|
||||
== "#napalm-time-sync: pool suse.pool.ntp.org iburst\n"
|
||||
)
|
||||
assert _text(root / "etc/chrony.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
|
||||
def test_a_confdir_takes_the_file_where_sources_d_is_not_read(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony/chrony.conf",
|
||||
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n",
|
||||
)
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony/conf.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
assert not (root / "etc/chrony/sources.d/netork.sources").exists()
|
||||
|
||||
def test_a_failed_restart_fails_the_step(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.env["FAIL_RESTART"] = "1"
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Job for chrony.service failed." in result["steps"][0]["output"]
|
||||
|
||||
def test_a_file_that_cannot_be_staged_is_not_installed(self, debian: Path):
|
||||
_tool(debian, "mktemp", "#!/bin/sh\necho 'mktemp: failed' >&2\nexit 1\n")
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "cannot stage" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
def test_no_configuration_is_a_failed_step(self, root: Path):
|
||||
result = Host(root).configure_time_sync(["ntp1.example.org"])
|
||||
assert result["success"] is False
|
||||
assert "chrony" in result["steps"][0]["output"]
|
||||
|
||||
|
||||
class TestTimesyncdServers:
|
||||
def test_a_drop_in_with_exactly_these_servers(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_FALLBACK)
|
||||
host = Host(root, active="systemd-timesyncd")
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(root / "etc/systemd/timesyncd.conf.d/netork.conf").endswith(
|
||||
"[Time]\nNTP=\nNTP=ntp1.example.org 10.0.0.1\nFallbackNTP=\n"
|
||||
)
|
||||
assert "systemctl restart systemd-timesyncd.service" in host.log()
|
||||
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org", "10.0.0.1"]
|
||||
|
||||
def test_servers_from_dhcp_stay_and_the_check_says_so(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
tool = (root / "bin/systemctl").read_text()
|
||||
(root / "bin/systemctl").write_text(
|
||||
tool.replace(
|
||||
"printf 'SystemNTPServers", "printf 'LinkNTPServers=192.168.1.1\\nSystemNTPServers"
|
||||
)
|
||||
)
|
||||
result = Host(root, active="systemd-timesyncd").configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "verify"
|
||||
assert "192.168.1.1" in result["steps"][-1]["output"]
|
||||
|
||||
|
||||
class TestOtherClients:
|
||||
@pytest.mark.parametrize("active", ["ntpsec", "openntpd", ""])
|
||||
def test_only_chrony_and_timesyncd_are_written(self, root: Path, active: str):
|
||||
host = Host(root, active=active)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "chrony and systemd-timesyncd" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
|
||||
class TestTimeZone:
|
||||
def test_timedatectl_sets_it(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
result = host.configure_time_sync(timezone="America/Sao_Paulo")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert host.privileged() == ["timedatectl set-timezone America/Sao_Paulo"]
|
||||
assert host.get_time_sync()["timezone"] == "America/Sao_Paulo"
|
||||
|
||||
def test_a_zone_the_host_does_not_know_fails_the_step(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
host.env["REFUSE_ZONE"] = "Europe/Kyiv"
|
||||
result = host.configure_time_sync(timezone="Europe/Kyiv")
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Invalid or not installed time zone" in result["steps"][0]["output"]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self, debian: Path):
|
||||
host = Host(debian)
|
||||
with pytest.raises(UnknownTimezone):
|
||||
host.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert host.commands == []
|
||||
|
||||
def test_both_at_once(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"], "UTC")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["timezone"]) == (["ntp1.example.org"], "UTC")
|
||||
|
||||
|
||||
def test_it_writes_both_values():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == PosixTimeSyncMixin.time_sync_writes
|
||||
Reference in new issue
Block a user