Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b4929990f1 | ||
|
|
3db6c6385c | ||
|
|
c5154d954b | ||
|
|
211d0c02eb | ||
|
|
74d2f42b0e | ||
|
|
c25f39768f | ||
|
|
9ecc6823c8 | ||
|
|
2080ccbfd7 | ||
|
|
ce1fe18405 | ||
|
|
e43926f1d5 | ||
|
|
43e0210ff6 | ||
|
|
241acef6c1 | ||
|
|
4c0ba5b2c0 | ||
|
|
e9da399568 | ||
|
|
8673d1eda1 | ||
|
|
46d7125aa6 | ||
|
|
8e43fe2ed9 | ||
|
|
679584766f | ||
|
|
1b04818982 | ||
|
|
65991d0930 | ||
|
|
9dfb5596bc | ||
|
|
b4cdcfd347 | ||
|
|
5258a43dc9 | ||
|
|
f3cb3601a8 | ||
|
|
a8fedf299a | ||
|
|
c736fb30d0 | ||
|
|
cf555d006d | ||
|
|
4e79cd1fc8 | ||
|
|
304f404567 | ||
|
|
aaa9f0504a | ||
|
|
bc47220318 | ||
|
|
08a62838f7 | ||
|
|
ac38c68feb | ||
|
|
8516f2a1ee | ||
|
|
c74173523a | ||
|
|
6c7f164004 | ||
|
|
073087da69 | ||
|
|
278a707157 | ||
|
|
56e5e82a59 | ||
|
|
e6f145d4a3 | ||
|
|
66037e878e | ||
|
|
afffd0915c | ||
|
|
92891473be | ||
|
|
658f1c8ab2 | ||
|
|
aacde6be3f | ||
|
|
58c7616401 | ||
|
|
b7e31205d5 | ||
|
|
1d71ff8f91 | ||
|
|
58ed0f0a94 | ||
|
|
c1a05d6dba | ||
|
|
fccf902c07 | ||
|
|
f5a2811333 | ||
|
|
98d1dbef4c | ||
|
|
252ea3b81c | ||
|
|
b685b200cc | ||
|
|
8557710724 | ||
|
|
1b600416f0 | ||
|
|
e622b26d42 | ||
|
|
5daba5d37c | ||
|
|
dacd10fcce | ||
|
|
8a54674e40 | ||
|
|
d45455adf1 | ||
|
|
ac49e2b5d4 | ||
|
|
c967635cba | ||
|
|
c470686305 | ||
|
|
1edf458feb | ||
|
|
00452d0d37 |
No files matched your search
+234
@@ -7,6 +7,240 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.39.0] - 2026-11-14
|
||||
|
||||
### Added
|
||||
- `PackageIndexMissing` (a `RuntimeError`): what `get_available_updates()` raises when
|
||||
the host holds no package index to read from -- OpenWrt's, which lives in RAM, after
|
||||
every reboot. `refresh_available_updates()` loads it again, so a caller can tell this
|
||||
from a failed read and refresh instead of reporting an error (NetOrk/netork#2527).
|
||||
|
||||
## [4.38.0] - 2026-11-13
|
||||
|
||||
### Added
|
||||
- `get_system_identity()` also reads the host's system firmware (BIOS or UEFI) from the
|
||||
same SMBIOS tables in the same round trip: `bios_vendor`, `bios_version` and
|
||||
`bios_date` (ISO 8601, from sysfs's `MM/DD/YYYY`; anything else reads as empty). Only
|
||||
the vendors' text defaults (`To be filled by O.E.M.`, `Default string`) are
|
||||
placeholders there -- a version of `1` is a version. Output of the three identity
|
||||
fields alone still parses, with the firmware empty (NetOrk/netork#2518).
|
||||
- `ConfigRevisionsMixin`: a device's own configuration history (NetOrk/netork#2511).
|
||||
`get_config_revisions()` (newest first, the first in place now, as
|
||||
`ConfigRevisionDict`: id, UTC time, description, user, size) and
|
||||
`get_config_revision(revision_id)` are declared. `revert_config_revision(revision_id)`
|
||||
is a template over the driver's `_revert_to_config_revision`: it refuses an id that is
|
||||
not in the history (`UnknownConfigRevision`), and after the revert checks that the
|
||||
newest revision holds the revision's configuration, else `ConfigRevertFailed`; it
|
||||
returns that revision. `config_revision_content(text)` leaves out what a device stamps
|
||||
into every save, so an application can compare revisions with a configuration it has.
|
||||
- `add_service_account(..., replaces=[...])`: netOrk's keys the new one takes the place of
|
||||
go from its account once the new key is in it (step `old keys`), never the new key
|
||||
itself; while the new key could not be added they stay, so netOrk keeps its way in.
|
||||
netOrk calls it through a login with the new key, after trying that login, to move a
|
||||
host from its installation-wide key to its site's (NetOrk/netork#2462).
|
||||
- Each forwarding rule of `LogForwardingDict` names its `template`: the one the rule
|
||||
gives (`@host;RSYSLOG_SyslogProtocol23Format`, `action(… template="…")`), else the
|
||||
default set before it (`$ActionForwardDefaultTemplate`, or
|
||||
`module(load="builtin:omfwd" template="…")` for omfwd actions), else None -- rsyslog's
|
||||
own default, RFC 3164, which sends the host's local time without a time zone. A
|
||||
reader can tell a rule whose timestamps a collector files shifted (NetOrk/netork#493).
|
||||
|
||||
### Fixed
|
||||
- `apply_dhcp_reservationset` writes its changes so an address is released before it is
|
||||
taken: an update that moves a reservation off its address goes before the change that
|
||||
takes that address (along a chain, from its free end), and adds come after every
|
||||
update. A server that keeps addresses unique at each write, such as Kea, no longer
|
||||
refuses the second reservation as a duplicate when an address moves from one to
|
||||
another. A swap -- reservations trading addresses, or a longer rotation -- has no such
|
||||
order: its changes are still tried, and where the server refuses them the `[failed]`
|
||||
line says it was a swap and with which reservations. Everything else is still written
|
||||
and committed, and `DhcpApplyError` follows (#106).
|
||||
- `apply_dhcp_subnetset` no longer stops at the first subnet the device refuses. As
|
||||
`apply_dhcp_reservationset` has since 4.35.1, a refused subnet gets a `[failed]` line,
|
||||
the others are still written, any non-empty diff is committed -- even one whose every
|
||||
write was refused -- and `DhcpApplyError` follows. Before, the subnets written ahead of
|
||||
the refused one stayed saved but inactive, and since the device reports them as
|
||||
configured, no later apply committed them. `DhcpApplyError` takes an optional `kind`
|
||||
(`"reservation"` by default, `"subnet"` here) that its message names (#113).
|
||||
|
||||
## [4.37.0] - 2026-11-12
|
||||
|
||||
### Added
|
||||
- `UpdateMixin.get_installed_advisory_cves()` (declared): the CVEs a host's installed
|
||||
security advisories close, per package. With `DNF_INSTALLED_CVES_COMMAND` (dnf 4,
|
||||
read from the cache) and `parse_dnf_installed_cves()`, the parser every dnf driver
|
||||
shares. netOrk reads it where its knowledge base holds no records of the host's
|
||||
release, as the distribution's word that a backport is in (netOrk #2394).
|
||||
|
||||
## [4.36.2] - 2026-11-12
|
||||
|
||||
### Fixed
|
||||
- `kernel_reboot_pending` compares kernel releases the way rpm does (`rpmvercmp`). A
|
||||
number is newer than letters in the same place, so an Oracle Linux host running
|
||||
`6.12.0-208.112.5.1.el9uek` no longer asks to reboot into the older
|
||||
`6.12.0-208.112.5.el9uek`. A `~` (release candidate) comes before its release (#109).
|
||||
|
||||
## [4.36.1] - 2026-11-12
|
||||
|
||||
### Fixed
|
||||
- The host status no longer reports an Oracle Linux host's UEK kernels as an interrupted
|
||||
dnf transaction. The installonly packages are now also the ones the host's libdnf was
|
||||
built with, read from its Python binding: Oracle adds `kernel-uek` and its parts there,
|
||||
and their packages provide nothing of upstream's defaults (#107).
|
||||
|
||||
## [4.36.0] - 2026-11-11
|
||||
|
||||
### Added
|
||||
- `TimeSyncDict.dhcp_ntp_servers` (not required): the NTP servers a device got by DHCP
|
||||
(option 42) and uses, in its order; `[]` for none, absent or None where the driver cannot
|
||||
tell. `ntp_servers` stays the servers configured on the device and no longer holds
|
||||
DHCP's. A device that gets the site's time giver by DHCP is aligned, so an application
|
||||
has to tell the two apart (NetOrk/netork#2380).
|
||||
- `PosixTimeSyncMixin` reads them: chrony's sources from the DHCP client's `sourcedir`
|
||||
(`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp` by default), systemd-timesyncd's
|
||||
`LinkNTPServers`. Where ntpd or OpenNTPD runs they are unknown (None).
|
||||
`timesyncd_link_servers` is public.
|
||||
|
||||
### Changed
|
||||
- `configure_time_sync` leaves DHCP-provided NTP servers alone. chrony: only `server`,
|
||||
`pool` and `peer` lines are commented out; no `sourcedir` line is any more, and the DHCP
|
||||
client's files are left alone. systemd-timesyncd: a link's servers stay and no longer
|
||||
fail the `verify` step.
|
||||
- A write gives back a DHCP client's `sourcedir` that 4.35.0 commented out
|
||||
(`#napalm-time-sync: sourcedir /run/chrony-dhcp`), so a host aligned by 4.35.0 hears DHCP
|
||||
again; chronyd restarts for it. Another `sourcedir` 4.35.0 took out stays out.
|
||||
- `timesyncd_servers` returns the configured servers (runtime, then system) without a
|
||||
link's; the fallback only while there is no server at all.
|
||||
|
||||
## [4.35.1] - 2026-11-11
|
||||
|
||||
### Fixed
|
||||
- `apply_dhcp_reservationset()` no longer stops at the first reservation the device refuses.
|
||||
One refused write (on OPNsense, an address in a subnet Kea does not manage) ended the run
|
||||
before the commit: every later reservation went unwritten, and the earlier ones stayed
|
||||
saved but inactive, where the next diff found them configured and never committed them. A
|
||||
refused change now gets a `[failed] <mac> -> <ip>: <reason>` line, the others are written,
|
||||
any non-empty diff commits, and `DhcpApplyError` (a `RuntimeError`, exported) is raised
|
||||
after the commit with how many of how many were refused (NetOrk/netork#2381).
|
||||
|
||||
## [4.35.0] - 2026-11-11
|
||||
|
||||
### Added
|
||||
- `TimeSyncMixin`: a device's time sources and time zone. `get_time_sync()` (declared only)
|
||||
returns `TimeSyncDict` -- the configured NTP servers in the device's order, the zone as an
|
||||
IANA name (None where the device keeps no name), whether the clock is synchronised now,
|
||||
and the client (`chrony`, `systemd-timesyncd`, `ntpd`, `openntpd`; None for the device's
|
||||
own). `configure_time_sync(ntp_servers=None, timezone=None)` is a fix template: None
|
||||
leaves a value alone, the device ends up with exactly the servers given, in order, and a
|
||||
`verify` step reads it back. An unknown zone, a malformed server, an empty list or a value
|
||||
outside the driver's `time_sync_writes` is refused before anything is written.
|
||||
`TIME_SYNC_VALUES` and `check_ntp_servers` are public (NetOrk/netork#2361).
|
||||
- `PosixTimeSyncMixin` implements it over `run_command`: read in one round trip as the login
|
||||
user (`timedatectl`, chrony's configuration and `chronyc -n sources`, `timedatectl
|
||||
show-timesync`, ntpd's and OpenNTPD's configuration); the zone by `timedatectl
|
||||
set-timezone`; chrony's servers in a file of their own (`sources.d`, else a `confdir` or
|
||||
an included file) with every other source commented out, then `chronyc reload sources` or
|
||||
a restart; systemd-timesyncd's in a drop-in that resets `NTP=` and empties `FallbackNTP=`.
|
||||
ntpd and OpenNTPD are read, not written. `parse_time_sync` is public.
|
||||
- `napalm_device_types.posix_tz` (from netOrk): `posix_tz`, `is_iana_zone`, `is_posix_tz` and
|
||||
`UnknownTimezone`, and `parse_posix_tz`, which takes a POSIX rule apart into the UTC
|
||||
offset and the summer-time transitions a switch wants (`PosixTzDict`,
|
||||
`DstTransitionDict`).
|
||||
|
||||
### Changed
|
||||
- Needs `tzdata` (2024.1 or later): the zones' rules come from its TZif files.
|
||||
|
||||
## [4.34.0] - 2026-11-10
|
||||
|
||||
### Added
|
||||
- `RunningBackupsMixin.get_running_backups()`: the backups running on a device right now --
|
||||
a hypervisor's backup of its guests (with the guests it is taking now), a backup server's
|
||||
backup, sync, verify, garbage collection, prune or tape run, a host's own file-level backup.
|
||||
Declared only; a driver mixes it in when it can tell, and a driver without it cannot tell.
|
||||
`RunningBackupDict`, `RunningBackupKind` and `RUNNING_BACKUP_KINDS` are public. An
|
||||
application asks right before it restarts a device (NetOrk/netork#2318).
|
||||
|
||||
## [4.33.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `get_host_status()` tells an interrupted package manager on hosts without dpkg too, and
|
||||
says which one it read in the new `package_manager` field (`dpkg`, `dnf`, `yum`, `apk`,
|
||||
`opkg`, or None). dnf and yum: a package installed in two versions (not the kernels and
|
||||
other packages dnf keeps side by side) as state `duplicate`, or a transaction that never
|
||||
ended (dnf 4's history, yum's `transaction-all` journal). apk: a package apk marked with
|
||||
broken files or a broken script (`broken-files`, `broken-script`), the ones `apk fix` would
|
||||
take. opkg: a package left `unpacked`, `half-configured` or `half-installed`, or flagged for
|
||||
reinstallation. All read without root; while one of them runs the answer is unknown
|
||||
(NetOrk/netork#851).
|
||||
|
||||
## [4.32.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `SshServerAuthMixin.get_ssh_server_auth()`: how an OpenSSH server lets people log in --
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication,
|
||||
`AuthenticationMethods` as lists of methods, `PermitRootLogin` and `UsePAM`, from `sshd -T`
|
||||
as root, and sshd's PAM `auth` stack (each module with its control and arguments, includes
|
||||
followed), from `/etc/pam.d` as the login user. `OpenSshServerMixin` carries it, so every
|
||||
driver that can harden sshd has it. `parse_sshd_auth`, `parse_pam_auth`, `parse_pam_files`,
|
||||
`pam_auth_stack` and `SSHD_EFFECTIVE_COMMAND` are public (NetOrk/netork#662).
|
||||
|
||||
## [4.31.1] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
- `get_log_sources()` names as a file's writer only a process that holds it open for writing.
|
||||
A log reader -- CrowdSec, Promtail, rsyslog's `imfile`, a `tail -F` -- was given the file
|
||||
it reads as if it wrote it. A file only readers hold is now listed without a producer
|
||||
(`unit` and `process` None); one only a reader inside a container holds is left out, as its
|
||||
path is that container's view of a mount. The access mode comes from
|
||||
`/proc/<pid>/fdinfo/<fd>`, read in the same command (NetOrk/netork#2092).
|
||||
|
||||
## [4.31.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
|
||||
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
|
||||
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
|
||||
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
|
||||
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
|
||||
public (NetOrk/netork#2053).
|
||||
|
||||
## [4.30.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LogSourcesMixin.get_log_sources()`: every log a Linux host writes. Sources are the units
|
||||
journald holds entries for, the log files running processes hold open (with size, last write
|
||||
and the service or container writing them), and the engine's file of each container's output.
|
||||
A file a container writes is named by the container's own path. `LOG_SOURCES_COMMAND` and
|
||||
`parse_log_sources` are public, and a host whose `find` cannot list open files (busybox)
|
||||
raises `LogSourcesUnavailable`. A driver supplies `_run_log_sources_command(command,
|
||||
privileged=)`, as for listening sockets (NetOrk/netork#2049).
|
||||
|
||||
## [4.29.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `remove_local_account_keys(name, keys, *, clear_password=False)`: with `clear_password`
|
||||
the account's password goes along with the keys, and so do its sudo rule and admin
|
||||
groups, as sudo asks for that password. For a person who deleted their key and whose
|
||||
password went with it. The account stays open; the steps are `keys`, `password` and
|
||||
`sudo`, each run when another failed (NetOrk/netork#2033).
|
||||
|
||||
## [4.28.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LocalAccountsMixin.add_local_account(..., replaces=[...])`: the keys the new one takes
|
||||
the place of go from the account, matched by their base64 whatever a line's options or
|
||||
comment. Keys the person added on the host stay. The step is `old keys`; it runs even
|
||||
when the new key could not be added (NetOrk/netork#2013).
|
||||
- `add_local_account(..., clear_password=True)`: the account's password goes, and it logs
|
||||
in with a key only. Step `password`. It cannot be combined with a `password_hash` or
|
||||
with `admin=True`, as the sudo rule asks for that password. A driver supplies
|
||||
`_account_clear_password`; one that does not fails the step and says so.
|
||||
- `LocalAccountsMixin.remove_local_account_keys(name, keys)`: takes the named keys off an
|
||||
account and leaves it open, its password and sudo as they are -- for a person who
|
||||
deleted a key and has no new one. Step `keys`.
|
||||
- A driver whose host keeps keys elsewhere overrides `_account_drop_keys` next to
|
||||
`_account_add_key` and `_account_remove_keys`.
|
||||
|
||||
## [4.27.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
|
||||
@@ -124,7 +124,8 @@ application wants made on a device runs as fixed steps the driver implements, wi
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
|
||||
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
|
||||
@@ -174,9 +175,12 @@ A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_fi
|
||||
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
|
||||
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
|
||||
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether dpkg was
|
||||
left halfway (a journal in `/var/lib/dpkg/updates` or a half-configured package; unknown
|
||||
while a package manager is running), and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether its package
|
||||
manager was left halfway (unknown while one is running): dpkg (a journal in
|
||||
`/var/lib/dpkg/updates` or a half-configured package), dnf or yum (two versions of a package
|
||||
in the rpm database, a transaction that never ended), apk (a package marked with broken files
|
||||
or a broken script) or opkg (a package left `unpacked`), with `package_manager` naming which,
|
||||
and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
each filesystem once, in bytes; how full is too full is the caller's decision). `package_updates` holds the shared apt and dnf parsers: apt's suites become
|
||||
an update's `origin`, a `-security` suite makes it a security update, and dnf's security
|
||||
advisories do the same.
|
||||
@@ -191,6 +195,23 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
`LogSourcesMixin` (`get_log_sources`) is mixed in the same way: every log the host writes, in
|
||||
one round trip. The units journald holds entries for (`journalctl -F _SYSTEMD_UNIT`, services
|
||||
only), the log files running processes hold open (`/proc/<pid>/fd`: `*.log`, `*.log.N` and
|
||||
anything under `/var/log`) with size and last write from `stat -L` on the descriptor, and the
|
||||
engine's own file of a container's output (`/var/lib/docker/containers/<id>/…`) as that
|
||||
container's stream. Each file names the service or container of its first writer from
|
||||
`/proc/<pid>/cgroup`; a file a container writes carries the container's own path. journald's
|
||||
store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) are left out; a
|
||||
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
|
||||
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
|
||||
user's processes show their files, and the reading says so (`attributed: false`).
|
||||
The same round trip reads where the logs go beyond the host (`forwarding`,
|
||||
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
|
||||
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
|
||||
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
|
||||
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
@@ -265,6 +286,9 @@ O.E.M.`, `Default string`, all-zero or the common `03000200-…-000700080009` UU
|
||||
empty: matching on them would join unrelated machines. `is_placeholder_identity` is the
|
||||
same filter for a BMC driver's side. `product_uuid` and `product_serial` are root's, so a
|
||||
driver supplies `_run_system_identity_command(command, privileged=)`.
|
||||
The same round trip reads the host's system firmware (BIOS or UEFI): `bios_vendor`,
|
||||
`bios_version` and `bios_date` (`YYYY-MM-DD`, from sysfs's `MM/DD/YYYY`). Only the vendors'
|
||||
text defaults read as empty there; a short version such as `1` is a version.
|
||||
|
||||
`OpenSshServerMixin` (`harden_ssh_server`, `get_ssh_server_algorithms`) takes algorithms
|
||||
out of what an OpenSSH server offers -- the ones ssh-audit flags. It reads the lists sshd
|
||||
@@ -278,6 +302,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
|
||||
itself (an appliance with its own database) must not, or opts out with
|
||||
`harden_ssh_server = None`.
|
||||
|
||||
`SshServerAuthMixin` (`get_ssh_server_auth`) says how an OpenSSH server lets people log in:
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication on or off,
|
||||
`AuthenticationMethods` as lists of methods (empty for `any`), `PermitRootLogin` and
|
||||
`UsePAM` (None where sshd has no PAM), from `sshd -T` as root -- the same fixed command the
|
||||
hardening reads. With PAM on, it also names the modules of sshd's PAM `auth` stack
|
||||
with each control and its arguments, read from `/etc/pam.d` in one listing as the
|
||||
login user and followed through `@include`, `include` and `substack`. Which of them is a
|
||||
second factor is the consumer's call. `Match` blocks are not read. `OpenSshServerMixin`
|
||||
carries it; a driver can also mix it in alone and supply `run_command`.
|
||||
|
||||
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
|
||||
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
|
||||
shell: that line, password and all, in the login user's shell history, and files its partial
|
||||
@@ -315,6 +349,51 @@ application's job as a timer where systemd runs, else in `/etc/cron.d` or the us
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
`TimeSyncMixin` (`get_time_sync`, `configure_time_sync`) is how a device keeps its clock:
|
||||
the NTP servers it is configured with, in its order, its time zone as an IANA name, whether
|
||||
it is synchronised now and which client does it (`TimeSyncDict`). The servers it got by
|
||||
DHCP (option 42) and uses are apart, in `dhcp_ntp_servers` (`[]` for none, left out where
|
||||
the driver cannot tell): DHCP counts, a device that gets the site's time giver that way is
|
||||
aligned. `get_time_sync` is declared only; `configure_time_sync(ntp_servers=None,
|
||||
timezone=None)` is a fix template -- `None` leaves a value alone, the device ends up
|
||||
configured with exactly the servers given, in that order, its DHCP-provided servers left
|
||||
alone, and a `verify` step reads it back. A zone that is not in the IANA database, a
|
||||
malformed server, an empty list or a value outside the driver's `time_sync_writes` is
|
||||
refused before anything is written. A driver states in `time_sync_writes` which of
|
||||
`ntp_servers` and `timezone` it can write (a BMC that runs on UTC only: `ntp_servers`).
|
||||
`napalm_device_types.posix_tz` turns an IANA name into the POSIX rule libc wants in `TZ`
|
||||
(OpenWrt) and that rule into a UTC offset and summer-time transitions (a switch), from the
|
||||
`tzdata` package. `PosixTimeSyncMixin` implements it over `run_command`: one reading as the
|
||||
login user (`timedatectl show`, the active client: chrony's configuration and `chronyc -n
|
||||
sources`, `timedatectl show-timesync`, `ntp.conf`, OpenNTPD's `ntpd.conf`); the zone by
|
||||
`timedatectl set-timezone`; chrony's servers in a file of their own --
|
||||
`/etc/chrony/sources.d/netork.sources` where chrony reads that directory, else a `confdir`
|
||||
or an included `/etc/chrony.d/netork.conf` -- with every other `server`, `pool` and `peer`
|
||||
line commented out behind `#napalm-time-sync: `, then `chronyc reload sources` or a
|
||||
restart; systemd-timesyncd's in `/etc/systemd/timesyncd.conf.d/netork.conf` (`NTP=` reset,
|
||||
`FallbackNTP=` emptied), then a restart. What DHCP brings is read apart and kept: chrony's
|
||||
DHCP `sourcedir` (`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp`) is never commented out -- one
|
||||
that 4.35.0 commented out comes back on the next write -- and systemd-timesyncd's
|
||||
`LinkNTPServers` stay. ntpd and OpenNTPD are read, not written; what DHCP gives them is
|
||||
unknown.
|
||||
|
||||
`ConfigRevisionsMixin` (`get_config_revisions`, `get_config_revision`,
|
||||
`revert_config_revision`) is a device's own configuration history: a copy of the
|
||||
configuration for every save, which the device can go back to. An application restores a
|
||||
configuration there by reverting to the revision that holds it -- on OPNsense the only way,
|
||||
since its API takes no uploaded config.xml (NetOrk/netork#2511). Listing (newest first, the
|
||||
first in place now: id, UTC time, description, user, size as `ConfigRevisionDict`) and
|
||||
reading one revision are declared only. `revert_config_revision(revision_id)` is a
|
||||
template: it refuses an id that is not in the history (`UnknownConfigRevision`, a
|
||||
`ValueError`) before anything changes, reads the revision -- before the revert, which may
|
||||
rotate it out -- lets the driver revert (`_revert_to_config_revision`), and checks that the
|
||||
newest revision afterwards holds the revision's configuration (`ConfigRevertFailed`
|
||||
otherwise: a device may say it reverted when it did not). It returns that newest revision.
|
||||
`config_revision_content(text)` is what of a revision is its configuration; a driver whose
|
||||
device stamps every save (who, when) strips the stamp there, and an application that looks
|
||||
for the revision holding a configuration compares through it too. A driver that can list
|
||||
but not revert sets `revert_config_revision = None`.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
@@ -434,7 +513,10 @@ The same split applies to `DhcpServerMixin`: `get_dhcp_reservations`/
|
||||
`apply_dhcp_reservation`/`commit_dhcp_reservations` are abstract (Kea REST on
|
||||
OPNsense, dnsmasq/odhcpd UCI on OpenWrt), while `diff_dhcp_reservations` and
|
||||
`apply_dhcp_reservationset` are concrete — matching by normalised MAC and the
|
||||
apply-then-commit orchestration are identical for every DHCP server.
|
||||
apply-then-commit orchestration are identical for every DHCP server. The apply writes
|
||||
its changes so an address is released before another reservation takes it (an update
|
||||
that frees an address first, adds last); a swap of addresses has no such order and is
|
||||
reported as one where the server refuses it.
|
||||
|
||||
A new driver (FortiGate, pfSense, …) gets `diff_firewall_rules`/
|
||||
`apply_firewall_ruleset` for free the moment it implements the three abstract
|
||||
|
||||
+2
-1
@@ -39,7 +39,7 @@ application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `_dhcp_order`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
@@ -151,6 +151,7 @@ the driver.
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `package_managers` | rpm, dnf and yum history, apk's and opkg's databases, `/proc/<pid>/comm` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
|
||||
@@ -38,6 +38,7 @@ instead of being restated on every role that happens to need it:
|
||||
|
||||
* :class:`~napalm_device_types.channel.CommandChannelMixin`
|
||||
* :class:`~napalm_device_types.config_lifecycle.ConfigLifecycleMixin`
|
||||
* :class:`~napalm_device_types.config_revisions.ConfigRevisionsMixin`
|
||||
* :class:`~napalm_device_types.config_sync.ConfigSyncMixin`
|
||||
* :class:`~napalm_device_types.container_engine.ContainerEngineMixin`
|
||||
* :class:`~napalm_device_types.dhcp.DhcpServerMixin`
|
||||
@@ -50,18 +51,23 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
|
||||
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
|
||||
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
|
||||
* :class:`~napalm_device_types.log_sources.LogSourcesMixin`
|
||||
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
|
||||
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
* :class:`~napalm_device_types.packages.PackageManagementMixin`
|
||||
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
|
||||
* :class:`~napalm_device_types.running_backups.RunningBackupsMixin`
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.time_sync.TimeSyncMixin`
|
||||
* :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
|
||||
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
|
||||
@@ -116,13 +122,19 @@ from napalm_device_types.clusters import (
|
||||
HAResourceDict,
|
||||
)
|
||||
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
|
||||
from napalm_device_types.config_revisions import (
|
||||
ConfigRevertFailed,
|
||||
ConfigRevisionDict,
|
||||
ConfigRevisionsMixin,
|
||||
UnknownConfigRevision,
|
||||
)
|
||||
from napalm_device_types.config_sync import ConfigSyncMixin
|
||||
from napalm_device_types.container_engine import (
|
||||
ContainerEngineConnection,
|
||||
ContainerEngineMixin,
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
from napalm_device_types.dhcp import DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dns_forward import DnsForwardMixin
|
||||
from napalm_device_types.firewall import FirewallDriver
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
@@ -166,6 +178,21 @@ from napalm_device_types.local_accounts import (
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.log_forwarding import (
|
||||
FileInputDict,
|
||||
ForwardRuleDict,
|
||||
LogForwardingDict,
|
||||
parse_rsyslog,
|
||||
)
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourceDict,
|
||||
LogSourcesDict,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.management_controller import ManagementControllerDriver
|
||||
from napalm_device_types.media import MediaDriver
|
||||
@@ -197,16 +224,33 @@ from napalm_device_types.os import OSDriver
|
||||
from napalm_device_types.ospf import OspfMixin
|
||||
from napalm_device_types.package_updates import (
|
||||
APT_UPGRADABLE_COMMAND,
|
||||
DNF_INSTALLED_CVES_COMMAND,
|
||||
DNF_SECURITY_COMMAND,
|
||||
parse_apt_upgradable,
|
||||
parse_dnf_installed_cves,
|
||||
parse_dnf_security,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.phone import PhoneDriver
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.posix_tz import (
|
||||
DstTransitionDict,
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.running_backups import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
RunningBackupDict,
|
||||
RunningBackupKind,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
@@ -255,6 +299,14 @@ from napalm_device_types.ssh_server import (
|
||||
SshHardeningResult,
|
||||
plan_ssh_hardening,
|
||||
)
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
PamAuthEntryDict,
|
||||
SshServerAuthDict,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
from napalm_device_types.storage import StorageDriver
|
||||
from napalm_device_types.sudo_history import (
|
||||
HistoryCleanupDict,
|
||||
@@ -276,7 +328,14 @@ from napalm_device_types.systemd import (
|
||||
parse_systemd_services,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.time_sync import (
|
||||
TIME_SYNC_VALUES,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
check_ntp_servers,
|
||||
)
|
||||
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
|
||||
from napalm_device_types.updates import PackageIndexMissing, UpdateMixin
|
||||
from napalm_device_types.virtual_ips import (
|
||||
VIRTUAL_IPS_COMMAND,
|
||||
VirtualIpsMixin,
|
||||
@@ -349,11 +408,16 @@ __all__ = [
|
||||
"open_stream_on_transport",
|
||||
"run_on_transport",
|
||||
"ConfigLifecycleMixin",
|
||||
"ConfigRevisionsMixin",
|
||||
"ConfigRevisionDict",
|
||||
"ConfigRevertFailed",
|
||||
"UnknownConfigRevision",
|
||||
"ConfigSyncDict",
|
||||
"ConfigSyncMixin",
|
||||
"DhcpHaDict",
|
||||
"DhcpHaPeerDict",
|
||||
"DeviceTypeDriver",
|
||||
"DhcpApplyError",
|
||||
"DhcpServerMixin",
|
||||
"FingerprintRule",
|
||||
"rules_max_weight",
|
||||
@@ -381,8 +445,10 @@ __all__ = [
|
||||
"OSDriver",
|
||||
"PackageManagementMixin",
|
||||
"APT_UPGRADABLE_COMMAND",
|
||||
"DNF_INSTALLED_CVES_COMMAND",
|
||||
"DNF_SECURITY_COMMAND",
|
||||
"parse_apt_upgradable",
|
||||
"parse_dnf_installed_cves",
|
||||
"parse_dnf_security",
|
||||
"parse_host_status",
|
||||
"strip_terminal_codes",
|
||||
@@ -398,6 +464,17 @@ __all__ = [
|
||||
"parse_host_firewall",
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"LOG_SOURCES_COMMAND",
|
||||
"FileInputDict",
|
||||
"ForwardRuleDict",
|
||||
"LogForwardingDict",
|
||||
"parse_log_forwarding",
|
||||
"parse_rsyslog",
|
||||
"LogSourceDict",
|
||||
"LogSourcesDict",
|
||||
"LogSourcesMixin",
|
||||
"LogSourcesUnavailable",
|
||||
"parse_log_sources",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
@@ -428,10 +505,28 @@ __all__ = [
|
||||
"SystemdServicesMixin",
|
||||
"SystemdUnavailable",
|
||||
"parse_systemd_services",
|
||||
"PackageIndexMissing",
|
||||
"UpdateMixin",
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
"VmStopRequestMixin",
|
||||
"RUNNING_BACKUP_KINDS",
|
||||
"RunningBackupDict",
|
||||
"RunningBackupKind",
|
||||
"RunningBackupsMixin",
|
||||
"TIME_SYNC_VALUES",
|
||||
"TimeSyncDict",
|
||||
"TimeSyncMixin",
|
||||
"check_ntp_servers",
|
||||
"PosixTimeSyncMixin",
|
||||
"parse_time_sync",
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
"VMProvisionIncomplete",
|
||||
"SERVICE_CLUSTERS_COMMAND",
|
||||
"ServiceClustersMixin",
|
||||
@@ -442,6 +537,12 @@ __all__ = [
|
||||
"is_placeholder_identity",
|
||||
"parse_system_identity",
|
||||
"OpenSshServerMixin",
|
||||
"PamAuthEntryDict",
|
||||
"SshServerAuthDict",
|
||||
"SshServerAuthMixin",
|
||||
"SSHD_EFFECTIVE_COMMAND",
|
||||
"pam_auth_stack",
|
||||
"parse_sshd_auth",
|
||||
"SSH_ALGORITHM_CLASSES",
|
||||
"SshHardeningPlan",
|
||||
"SshHardeningResult",
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
"""How chrony is given exactly the time sources a caller names.
|
||||
|
||||
The sources go to one file of their own (:func:`plan_chrony` picks it):
|
||||
|
||||
1. the sources file in a ``sourcedir`` the main configuration reads (Debian's
|
||||
``/etc/chrony/sources.d``): ``chronyc reload sources`` takes it without a restart;
|
||||
2. else ``netork.conf`` -- the include file's name -- in the first ``confdir``;
|
||||
3. else the include file, where an ``include`` pattern already takes it (SUSE's
|
||||
``/etc/chrony.d/*.conf``);
|
||||
4. else the include file, with an ``include`` line appended to the main configuration
|
||||
(RHEL's ``/etc/chrony.conf``).
|
||||
|
||||
Every other configured source stops counting: a ``server``, ``pool`` or ``peer``
|
||||
line in any other file chrony reads is commented out behind :data:`COMMENT_MARK`.
|
||||
``sourcedir`` lines stay. The DHCP client's directory (``/run/chrony-dhcp``) and its
|
||||
files are left alone: DHCP option 42 counts (NetOrk/netork#2380), and the client
|
||||
rewrites those files on every lease anyway.
|
||||
|
||||
4.35.0 commented out every ``sourcedir`` other than the sources file's, the DHCP
|
||||
client's among them, which left a host deaf to DHCP. A write gives a DHCP client's
|
||||
``sourcedir`` back where it finds one behind :data:`COMMENT_MARK`; any other one it
|
||||
took out stays out, its sources would be commented out now anyway.
|
||||
|
||||
chronyd reads its main configuration only when it starts: a change there means a
|
||||
restart.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from fnmatch import fnmatchcase
|
||||
|
||||
from napalm_device_types._time_sync_reading import (
|
||||
CHRONY_DHCP_DIRS,
|
||||
SOURCE_LINE,
|
||||
ConfigFile,
|
||||
in_dhcp_dir,
|
||||
)
|
||||
|
||||
#: What a source line taken out of chrony's configuration starts with.
|
||||
COMMENT_MARK = "#napalm-time-sync: "
|
||||
#: The comment above an ``include`` line appended to the main configuration.
|
||||
INCLUDE_COMMENT = "# napalm-time-sync: the time sources"
|
||||
|
||||
_DIRECTORY_LINE = re.compile(r"\s*(sourcedir|confdir|include)\s+(.+?)\s*$")
|
||||
#: A ``sourcedir`` line 4.35.0 commented out.
|
||||
_MARKED_SOURCEDIR = re.compile(rf"{re.escape(COMMENT_MARK)}sourcedir\s+(.+?)\s*$")
|
||||
|
||||
|
||||
@dataclass
|
||||
class ChronyPlan:
|
||||
"""What :func:`plan_chrony` decided."""
|
||||
|
||||
target: str # the file the sources go to
|
||||
content: str
|
||||
comment_out: dict[str, list[int]] = field(default_factory=dict) # path: line numbers
|
||||
restore: dict[str, list[int]] = field(default_factory=dict) # path: marked sourcedir lines
|
||||
include_in: str | None = None # the main configuration an include of target is appended to
|
||||
reload: bool = False # True: ``chronyc reload sources`` takes the change
|
||||
|
||||
|
||||
def source_line(server: str) -> str:
|
||||
"""The line a server is written as: a pool's name as a ``pool``, anything else a ``server``."""
|
||||
directive = "pool" if ".pool." in f".{server.lower()}." else "server"
|
||||
return f"{directive} {server} iburst"
|
||||
|
||||
|
||||
def _directives(main: ConfigFile) -> list[tuple[int, str, list[str]]]:
|
||||
found = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
match = _DIRECTORY_LINE.match(line)
|
||||
if match:
|
||||
found.append((number, match[1], match[2].split()))
|
||||
return found
|
||||
|
||||
|
||||
def _arguments(directives: list[tuple[int, str, list[str]]], kind: str) -> list[str]:
|
||||
return [arg for _, k, args in directives if k == kind for arg in args]
|
||||
|
||||
|
||||
def _target(main: ConfigFile, sources_file: str, include_file: str) -> tuple[str, str | None, bool]:
|
||||
"""The file the sources go to, the main configuration to include it from, and
|
||||
whether it is a sources file."""
|
||||
directives = _directives(main)
|
||||
if posixpath.dirname(sources_file) in _arguments(directives, "sourcedir"):
|
||||
return sources_file, None, True
|
||||
confdirs = _arguments(directives, "confdir")
|
||||
if confdirs:
|
||||
return posixpath.join(confdirs[0], posixpath.basename(include_file)), None, False
|
||||
if any(fnmatchcase(include_file, p) for p in _arguments(directives, "include")):
|
||||
return include_file, None, False
|
||||
return include_file, main.path, False
|
||||
|
||||
|
||||
def _dhcp_sourcedirs_to_restore(main: ConfigFile, dhcp_dirs: tuple[str, ...]) -> list[int]:
|
||||
"""The lines of *main* where 4.35.0 commented out a DHCP client's ``sourcedir``."""
|
||||
dhcp = {posixpath.normpath(d) for d in dhcp_dirs}
|
||||
return [
|
||||
number
|
||||
for number, line in enumerate(main.lines, start=1)
|
||||
if (marked := _MARKED_SOURCEDIR.match(line))
|
||||
and all(posixpath.normpath(arg) in dhcp for arg in marked[1].split())
|
||||
]
|
||||
|
||||
|
||||
def plan_chrony(
|
||||
files: list[ConfigFile],
|
||||
servers: list[str],
|
||||
*,
|
||||
sources_file: str,
|
||||
include_file: str,
|
||||
dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS,
|
||||
) -> ChronyPlan:
|
||||
"""Where *servers* go, what else is commented out, and which DHCP ``sourcedir``
|
||||
(one of *dhcp_dirs*) comes back.
|
||||
|
||||
:raises RuntimeError: when the reading found no chrony configuration.
|
||||
"""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
raise RuntimeError("chrony runs, but no chrony configuration was found to write to")
|
||||
target, include_in, is_sources = _target(main, sources_file, include_file)
|
||||
comment_out = _comment_out(files, target, dhcp_dirs)
|
||||
restore = _dhcp_sourcedirs_to_restore(main, dhcp_dirs)
|
||||
return ChronyPlan(
|
||||
target=target,
|
||||
content="".join(f"{source_line(s)}\n" for s in servers),
|
||||
comment_out=comment_out,
|
||||
restore={main.path: restore} if restore else {},
|
||||
include_in=include_in,
|
||||
reload=(
|
||||
is_sources
|
||||
and include_in is None
|
||||
and not restore
|
||||
and _only_sources_files(files, comment_out)
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _only_sources_files(files: list[ConfigFile], changed: dict[str, list[int]]) -> bool:
|
||||
"""Whether every file that changes is one ``chronyc reload sources`` reads again."""
|
||||
return all(f.kind == "sourcedir" for f in files if f.path in changed)
|
||||
|
||||
|
||||
def _comment_out(
|
||||
files: list[ConfigFile], target: str, dhcp_dirs: tuple[str, ...]
|
||||
) -> dict[str, list[int]]:
|
||||
"""Per file, the lines that take a source; the DHCP client's files are left alone."""
|
||||
found: dict[str, list[int]] = {}
|
||||
for f in files:
|
||||
if f.path == target or in_dhcp_dir(f, dhcp_dirs):
|
||||
continue
|
||||
numbers = [n for n, line in enumerate(f.lines, start=1) if SOURCE_LINE.match(line)]
|
||||
if numbers:
|
||||
found[f.path] = numbers
|
||||
return found
|
||||
|
||||
|
||||
def comment_out_command(path: str, numbers: list[int]) -> str:
|
||||
"""``sed`` that comments out those lines of *path* that still hold a source."""
|
||||
edits = " ".join(
|
||||
"-e "
|
||||
+ shlex.quote(f"{n}{{/^[[:space:]]*(server|pool|peer)[[:space:]]/s/^/{COMMENT_MARK}/;}}")
|
||||
for n in numbers
|
||||
)
|
||||
return f"sed -i -E {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def restore_command(path: str, numbers: list[int]) -> str:
|
||||
"""``sed`` that takes :data:`COMMENT_MARK` off those lines of *path* that are still a
|
||||
``sourcedir`` behind it. The mark holds nothing a regular expression or ``sed``'s
|
||||
``s///`` would read as special."""
|
||||
edits = " ".join(
|
||||
"-e " + shlex.quote(f"{n}{{s/^{COMMENT_MARK}(sourcedir[[:space:]])/\\1/;}}")
|
||||
for n in numbers
|
||||
)
|
||||
return f"sed -i -E {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def include_command(main: str, target: str) -> str:
|
||||
"""Append an ``include`` of *target* to *main*, on a line of its own."""
|
||||
lines = " ".join(shlex.quote(line) for line in (INCLUDE_COMMENT, f"include {target}"))
|
||||
return (
|
||||
f'{{ [ -z "$(tail -c 1 -- {shlex.quote(main)})" ] || echo; '
|
||||
f"printf '%s\\n' {lines}; }} >> {shlex.quote(main)}"
|
||||
)
|
||||
@@ -0,0 +1,116 @@
|
||||
"""The order an applied reservation set is written in (#106).
|
||||
|
||||
A DHCP server such as Kea keeps every address unique at each write, not only at the
|
||||
commit: a reservation that takes an address another one still holds is refused as a
|
||||
duplicate. So the changes are ordered for an address to be released before it is taken:
|
||||
|
||||
* An update that moves a reservation off its address goes before the change that takes
|
||||
that address -- along a chain, from its free end.
|
||||
* Adds come after every update: they release nothing, and an update may free their
|
||||
address.
|
||||
|
||||
A *swap* -- two reservations trading addresses, or a longer rotation -- has no such
|
||||
order. Its changes are still tried, in the order of the desired set: a server that checks
|
||||
only at the commit takes them. A server that refuses them refuses each with its own
|
||||
message, and the progress line says it was a swap. Nothing is moved to a temporary
|
||||
address: the contract knows no free one (pools, leases and reservations nobody manages
|
||||
all hold addresses), and a second write that failed would leave the reservation on it.
|
||||
|
||||
Without I/O: a reconciler's part, see ``docs/STRUCTURE.md``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
from dataclasses import dataclass
|
||||
|
||||
from napalm_device_types.models import DhcpReservationDict, DhcpReservationDiffDict
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReservationChange:
|
||||
"""One write of an applied set, in the order it is made."""
|
||||
|
||||
reservation: DhcpReservationDict
|
||||
#: The live reservation's id for an update, None for an add.
|
||||
uuid: str | None
|
||||
#: The changed fields of an update (``"ip, hostname"``), empty for an add.
|
||||
fields: str = ""
|
||||
#: The other reservations' MACs of the swap this change is part of, if any.
|
||||
swap: tuple[str, ...] = ()
|
||||
|
||||
|
||||
def _address(ip: str | None) -> str:
|
||||
"""*ip* in one spelling, so ``192.0.2.010``-style differences cannot hide a match."""
|
||||
text = (ip or "").strip()
|
||||
try:
|
||||
return str(ipaddress.ip_address(text))
|
||||
except ValueError:
|
||||
return text.lower()
|
||||
|
||||
|
||||
def _waits_on(updates: list[tuple[str, str]]) -> list[int | None]:
|
||||
"""For each ``(old, new)`` address pair, the update whose release it waits for."""
|
||||
released = {old: i for i, (old, new) in enumerate(updates) if old and old != new}
|
||||
return [released.get(new) if new != old else None for old, new in updates]
|
||||
|
||||
|
||||
def _cycles(waits_on: list[int | None]) -> dict[int, list[int]]:
|
||||
"""Each update that is part of a cycle, with the cycle's members in order."""
|
||||
members: dict[int, list[int]] = {}
|
||||
for start in range(len(waits_on)):
|
||||
path: list[int] = []
|
||||
node: int | None = start
|
||||
while node is not None and node not in path and node not in members:
|
||||
path.append(node)
|
||||
node = waits_on[node]
|
||||
if node is not None and node in path:
|
||||
cycle = sorted(path[path.index(node) :])
|
||||
members.update(dict.fromkeys(cycle, cycle))
|
||||
return members
|
||||
|
||||
|
||||
def _update_order(waits_on: list[int | None], cycles: dict[int, list[int]]) -> list[int]:
|
||||
"""Every update after the one it waits on; stable otherwise. Where only cycles are
|
||||
left, the first of them goes as it is, and so on."""
|
||||
order: list[int] = []
|
||||
done: set[int] = set()
|
||||
pending = list(range(len(waits_on)))
|
||||
while pending:
|
||||
ready = [i for i in pending if waits_on[i] is None or waits_on[i] in done]
|
||||
if not ready:
|
||||
ready = [next(i for i in pending if i in cycles)]
|
||||
order += ready
|
||||
done.update(ready)
|
||||
pending = [i for i in pending if i not in done]
|
||||
return order
|
||||
|
||||
|
||||
def order_reservation_changes(
|
||||
diff: DhcpReservationDiffDict, live: list[DhcpReservationDict]
|
||||
) -> list[ReservationChange]:
|
||||
"""The changes of *diff* in the order they are written; see the module.
|
||||
|
||||
:param diff: What ``diff_dhcp_reservations`` found.
|
||||
:param live: The reservations it compared against, for the addresses updates release.
|
||||
"""
|
||||
held = {r.get("uuid"): _address(r.get("ip")) for r in live if r.get("uuid")}
|
||||
updates = diff["update"]
|
||||
waits_on = _waits_on(
|
||||
[(held.get(e["uuid"], ""), _address(e["reservation"].get("ip"))) for e in updates]
|
||||
)
|
||||
cycles = _cycles(waits_on)
|
||||
|
||||
def swap(i: int) -> tuple[str, ...]:
|
||||
return tuple(updates[j]["reservation"]["mac"] for j in cycles.get(i, []) if j != i)
|
||||
|
||||
ordered = [
|
||||
ReservationChange(
|
||||
updates[i]["reservation"],
|
||||
updates[i]["uuid"],
|
||||
", ".join(updates[i]["changed_fields"]),
|
||||
swap(i),
|
||||
)
|
||||
for i in _update_order(waits_on, cycles)
|
||||
]
|
||||
return ordered + [ReservationChange(r, None) for r in diff["add"]]
|
||||
@@ -0,0 +1,369 @@
|
||||
"""The reading behind :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`.
|
||||
|
||||
One command, as the login user, prints framed sections (``@@ts <name> ...``):
|
||||
|
||||
* ``timedatectl`` -- ``timedatectl show``: the zone, and whether the kernel clock is
|
||||
synchronised;
|
||||
* ``zone`` -- ``/etc/timezone`` or the target of ``/etc/localtime``, for a host
|
||||
without timedatectl;
|
||||
* ``active`` -- one ``<unit> <daemon> <state>`` line per time sync service;
|
||||
* ``timesync`` -- ``timedatectl show-timesync``: systemd-timesyncd's servers;
|
||||
* ``chronyc`` -- ``chronyc -n sources``: whether chronyd has selected a source;
|
||||
* ``chrony <line> <kind> <path>`` -- chrony's main configuration (line 0, kind
|
||||
``main``), then each file an ``include``, ``confdir`` or ``sourcedir`` on that
|
||||
line of it reads;
|
||||
* ``ntpd <path>``, ``openntpd <path>`` -- their configuration;
|
||||
* ``end`` -- the reading is complete.
|
||||
|
||||
What DHCP brought is read apart from the configured servers: chrony's files in a
|
||||
DHCP client's ``sourcedir`` (:data:`CHRONY_DHCP_DIRS`), systemd-timesyncd's
|
||||
``LinkNTPServers``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from napalm_device_types.time_sync import TimeSyncDict
|
||||
|
||||
_MARK = "@@ts "
|
||||
|
||||
#: ``<unit> <daemon>``: the services that keep a host's clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: tuple[tuple[str, str], ...] = (
|
||||
("chrony", "chrony"),
|
||||
("chronyd", "chrony"),
|
||||
("systemd-timesyncd", "systemd-timesyncd"),
|
||||
("ntpsec", "ntpd"),
|
||||
("ntp", "ntpd"),
|
||||
("ntpd", "ntpd"),
|
||||
("openntpd", "openntpd"),
|
||||
)
|
||||
|
||||
#: The ``sourcedir`` a DHCP client writes chrony's sources to, one file per interface
|
||||
#: and lease: the NetworkManager dispatcher and dhclient hook of Debian, Ubuntu and
|
||||
#: Red Hat. What chrony reads from there is DHCP's (option 42), not configuration.
|
||||
CHRONY_DHCP_DIRS: tuple[str, ...] = ("/run/chrony-dhcp", "/var/run/chrony-dhcp")
|
||||
|
||||
#: A source line of chrony's or ntpd's configuration: the directive and what follows.
|
||||
SOURCE_LINE = re.compile(r"\s*(server|pool|peer)\s+(.*)")
|
||||
_OPENNTPD_SOURCE = re.compile(r"\s*(servers?)\s+(\S+)")
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
#: ntpd's reference clocks are addresses in 127.127.0.0/16, not servers.
|
||||
_REFCLOCK = "127.127."
|
||||
|
||||
# The files a chrony configuration reads, each after a marker with the line that names it.
|
||||
_CHRONY_FOLLOW = """\
|
||||
awk '$1=="include"||$1=="confdir"||$1=="sourcedir"{for(i=2;i<=NF;i++)print NR, $1, $i}' "$c" |
|
||||
while read -r n k p; do
|
||||
case $k in include) set -- $p ;; confdir) set -- "$p"/*.conf ;; *) set -- "$p"/*.sources ;; esac
|
||||
for f in "$@"; do
|
||||
[ -f "$f" ] && { printf '@@ts chrony %s %s %s\\n' "$n" "$k" "$f"; cat -- "$f"; echo; }
|
||||
done
|
||||
done"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReadPaths:
|
||||
"""Where a host keeps what the reading reads."""
|
||||
|
||||
units: tuple[tuple[str, str], ...]
|
||||
chrony_configs: tuple[str, ...]
|
||||
ntpd_configs: tuple[str, ...]
|
||||
openntpd_configs: tuple[str, ...]
|
||||
time_zone_sh: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConfigFile:
|
||||
"""One configuration file as the reading printed it."""
|
||||
|
||||
path: str
|
||||
kind: str # "main", "include", "confdir", "sourcedir"; "ntpd", "openntpd"
|
||||
at: int # the line of the main configuration that names it; 0 for that file itself
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""Everything one reading said."""
|
||||
|
||||
timedatectl: dict[str, str] = field(default_factory=dict)
|
||||
zone_files: list[str] = field(default_factory=list)
|
||||
active: list[tuple[str, str]] = field(default_factory=list) # (unit, daemon), active ones
|
||||
timesync: dict[str, str] = field(default_factory=dict)
|
||||
chronyc: str = ""
|
||||
chrony: list[ConfigFile] = field(default_factory=list)
|
||||
ntpd: ConfigFile | None = None
|
||||
openntpd: ConfigFile | None = None
|
||||
|
||||
@property
|
||||
def daemon(self) -> str | None:
|
||||
return self.active[0][1] if self.active else None
|
||||
|
||||
@property
|
||||
def unit(self) -> str | None:
|
||||
return self.active[0][0] if self.active else None
|
||||
|
||||
|
||||
def _first_file(kind: str, paths: tuple[str, ...]) -> str:
|
||||
candidates = " ".join(shlex.quote(p) for p in paths)
|
||||
return (
|
||||
f'for c in {candidates}; do [ -f "$c" ] && '
|
||||
f'{{ printf \'@@ts {kind} %s\\n\' "$c"; cat -- "$c"; echo; break; }}; done'
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: ReadPaths) -> str:
|
||||
"""The command that prints the reading; ``sh`` runs it, whatever the login shell."""
|
||||
active = "\n".join(
|
||||
f"printf '%s %s %s\\n' {shlex.quote(unit)} {shlex.quote(daemon)} "
|
||||
f'"$(systemctl is-active {shlex.quote(unit + ".service")} 2>/dev/null)"'
|
||||
for unit, daemon in paths.units
|
||||
)
|
||||
chrony = " ".join(shlex.quote(p) for p in paths.chrony_configs)
|
||||
script = "\n".join(
|
||||
[
|
||||
"echo '@@ts timedatectl'",
|
||||
"timedatectl show 2>/dev/null",
|
||||
"echo '@@ts zone'",
|
||||
f"{{ {paths.time_zone_sh}; }}",
|
||||
"echo '@@ts active'",
|
||||
active,
|
||||
"echo '@@ts timesync'",
|
||||
"timedatectl show-timesync 2>/dev/null",
|
||||
"echo '@@ts chronyc'",
|
||||
"chronyc -n sources 2>/dev/null",
|
||||
f'for c in {chrony}; do [ -f "$c" ] || continue',
|
||||
'printf \'@@ts chrony 0 main %s\\n\' "$c"; cat -- "$c"; echo',
|
||||
_CHRONY_FOLLOW,
|
||||
"break; done",
|
||||
_first_file("ntpd", paths.ntpd_configs),
|
||||
_first_file("openntpd", paths.openntpd_configs),
|
||||
"echo '@@ts end'",
|
||||
]
|
||||
)
|
||||
return f"sh -c {shlex.quote(script)}"
|
||||
|
||||
|
||||
def parse_key_values(text: str) -> dict[str, str]:
|
||||
"""``Key=value`` lines (``timedatectl show``, ``show-timesync``)."""
|
||||
values: dict[str, str] = {}
|
||||
for line in text.splitlines():
|
||||
key, sep, value = line.partition("=")
|
||||
if sep and key.strip():
|
||||
values[key.strip()] = value.strip()
|
||||
return values
|
||||
|
||||
|
||||
def _sections(text: str) -> list[tuple[list[str], list[str]]]:
|
||||
sections: list[tuple[list[str], list[str]]] = []
|
||||
for line in text.splitlines():
|
||||
if line.startswith(_MARK):
|
||||
sections.append((line[len(_MARK) :].split(maxsplit=3), []))
|
||||
elif sections:
|
||||
sections[-1][1].append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _active(lines: list[str]) -> list[tuple[str, str]]:
|
||||
rows = [line.split() for line in lines]
|
||||
return [(r[0], r[1]) for r in rows if len(r) == 3 and r[2] == "active"]
|
||||
|
||||
|
||||
def _take(reading: Reading, header: list[str], lines: list[str]) -> None:
|
||||
name, text = header[0], "\n".join(lines)
|
||||
if name == "chrony" and len(header) == 4:
|
||||
reading.chrony.append(ConfigFile(header[3], header[2], int(header[1]), lines))
|
||||
elif name in ("ntpd", "openntpd") and len(header) == 2:
|
||||
setattr(reading, name, ConfigFile(header[1], name, 0, lines))
|
||||
else:
|
||||
simple = {
|
||||
"timedatectl": ("timedatectl", parse_key_values(text)),
|
||||
"zone": ("zone_files", lines),
|
||||
"active": ("active", _active(lines)),
|
||||
"timesync": ("timesync", parse_key_values(text)),
|
||||
"chronyc": ("chronyc", text),
|
||||
}
|
||||
if name in simple:
|
||||
setattr(reading, *simple[name])
|
||||
|
||||
|
||||
def parse_reading(text: str) -> Reading:
|
||||
"""The reading's sections. :raises ValueError: for a reading cut short."""
|
||||
sections = _sections(text)
|
||||
if not sections or sections[-1][0] != ["end"]:
|
||||
raise ValueError("the time sync reading was cut short")
|
||||
reading = Reading()
|
||||
for header, lines in sections:
|
||||
_take(reading, header, lines)
|
||||
return reading
|
||||
|
||||
|
||||
def _source_name(rest: str) -> str | None:
|
||||
"""The server in what follows ``server``/``pool``/``peer``: ntpd puts ``-4`` first."""
|
||||
return next((word for word in rest.split() if not word.startswith("-")), None)
|
||||
|
||||
|
||||
def in_dhcp_dir(f: ConfigFile, dhcp_dirs: tuple[str, ...]) -> bool:
|
||||
"""Whether *f* is a sources file a DHCP client wrote to one of *dhcp_dirs*."""
|
||||
if f.kind != "sourcedir":
|
||||
return False
|
||||
directory = posixpath.normpath(posixpath.dirname(f.path))
|
||||
return directory in {posixpath.normpath(d) for d in dhcp_dirs}
|
||||
|
||||
|
||||
def _named_on(files: list[ConfigFile], number: int) -> list[ConfigFile]:
|
||||
"""The files the main configuration's line *number* reads."""
|
||||
return [f for f in files if f.at == number and f.kind != "main"]
|
||||
|
||||
|
||||
def _read_order(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[tuple[bool, str]]:
|
||||
"""Every line chrony reads, in its order -- each file in place of the line naming
|
||||
it -- and whether a DHCP client wrote it."""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
return []
|
||||
found: list[tuple[bool, str]] = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
found.append((False, line))
|
||||
for included in _named_on(files, number):
|
||||
dhcp = in_dhcp_dir(included, dhcp_dirs)
|
||||
found.extend((dhcp, text) for text in included.lines)
|
||||
return found
|
||||
|
||||
|
||||
def chrony_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...] = ()) -> list[str]:
|
||||
"""chrony's configured sources in the order it reads them; not those in *dhcp_dirs*."""
|
||||
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if not dhcp])
|
||||
|
||||
|
||||
def chrony_dhcp_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[str]:
|
||||
"""The sources chrony reads from the DHCP client's files in *dhcp_dirs*, in its order.
|
||||
|
||||
A ``sourcedir`` chrony does not read (commented out) prints no files: nothing then.
|
||||
"""
|
||||
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if dhcp])
|
||||
|
||||
|
||||
def _sources(lines: list[str]) -> list[str]:
|
||||
names = []
|
||||
for line in lines:
|
||||
match = SOURCE_LINE.match(line)
|
||||
if match and (name := _source_name(match[2])):
|
||||
names.append(name)
|
||||
return names
|
||||
|
||||
|
||||
def ntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""ntpd's servers, pools and peers; reference clocks are not servers."""
|
||||
return [name for name in _sources(lines) if not name.startswith(_REFCLOCK)]
|
||||
|
||||
|
||||
def openntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""OpenNTPD's ``server`` and ``servers``; ``constraint`` and ``sensor`` are no sources."""
|
||||
return [m[2] for line in lines if (m := _OPENNTPD_SOURCE.match(line))]
|
||||
|
||||
|
||||
def _listed(values: dict[str, str], *keys: str) -> list[str]:
|
||||
found: list[str] = []
|
||||
for key in keys:
|
||||
for server in values.get(key, "").split():
|
||||
if server not in found:
|
||||
found.append(server)
|
||||
return found
|
||||
|
||||
|
||||
def timesyncd_servers(values: dict[str, str]) -> list[str]:
|
||||
"""systemd-timesyncd's configured servers, runtime then system; the fallback only
|
||||
while it has no other server, a link's included. A link's servers are not among
|
||||
them: :func:`timesyncd_link_servers`."""
|
||||
configured = _listed(values, "RuntimeNTPServers", "SystemNTPServers")
|
||||
if configured or timesyncd_link_servers(values):
|
||||
return configured
|
||||
return _listed(values, "FallbackNTPServers")
|
||||
|
||||
|
||||
def timesyncd_link_servers(values: dict[str, str]) -> list[str]:
|
||||
"""The servers network links gave systemd-timesyncd: systemd-networkd's, from a DHCP
|
||||
lease (option 42) or a ``.network`` file's ``NTP=`` -- the reading cannot tell those
|
||||
apart and counts them as DHCP's."""
|
||||
return _listed(values, "LinkNTPServers")
|
||||
|
||||
|
||||
def chrony_synchronized(sources: str) -> bool | None:
|
||||
"""Whether chronyd has selected a source (``^*``); None for no answer from chronyc."""
|
||||
rows = [line for line in sources.splitlines() if line[:1] in ("^", "=", "#")]
|
||||
if not rows:
|
||||
return None
|
||||
return any(line[1:2] == "*" for line in rows)
|
||||
|
||||
|
||||
def _zone_name(value: str) -> str | None:
|
||||
value = value.strip()
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
for prefix in ("posix/", "right/"):
|
||||
value = value.removeprefix(prefix)
|
||||
return value if _ZONE_NAME.fullmatch(value) else None
|
||||
|
||||
|
||||
def _time_zone(reading: Reading) -> str | None:
|
||||
named = reading.timedatectl.get("Timezone", "")
|
||||
if named and named != "n/a":
|
||||
return _zone_name(named)
|
||||
return next((z for line in reading.zone_files if (z := _zone_name(line))), None)
|
||||
|
||||
|
||||
def _servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str]:
|
||||
if reading.daemon == "chrony":
|
||||
return chrony_servers(reading.chrony, dhcp_dirs)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return timesyncd_servers(reading.timesync)
|
||||
if reading.daemon == "ntpd" and reading.ntpd is not None:
|
||||
return ntpd_servers(reading.ntpd.lines)
|
||||
if reading.daemon == "openntpd" and reading.openntpd is not None:
|
||||
return openntpd_servers(reading.openntpd.lines)
|
||||
return []
|
||||
|
||||
|
||||
def _dhcp_servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str] | None:
|
||||
"""What DHCP brought: chrony's and timesyncd's; None where ntpd or OpenNTPD runs,
|
||||
whose DHCP hooks keep a configuration of their own that is not read."""
|
||||
if reading.daemon == "chrony":
|
||||
return chrony_dhcp_servers(reading.chrony, dhcp_dirs)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return timesyncd_link_servers(reading.timesync)
|
||||
if reading.daemon is None:
|
||||
return []
|
||||
return None
|
||||
|
||||
|
||||
def _synchronized(reading: Reading) -> bool | None:
|
||||
if reading.daemon == "chrony":
|
||||
chrony = chrony_synchronized(reading.chronyc)
|
||||
if chrony is not None:
|
||||
return chrony
|
||||
return {"yes": True, "no": False}.get(reading.timedatectl.get("NTPSynchronized", ""))
|
||||
|
||||
|
||||
def time_sync_of(reading: Reading, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
|
||||
"""What a reading says, as ``get_time_sync`` returns it; *dhcp_dirs* are the DHCP
|
||||
client's ``sourcedir`` of chrony."""
|
||||
return {
|
||||
"ntp_servers": _servers(reading, dhcp_dirs),
|
||||
"timezone": _time_zone(reading),
|
||||
"synchronized": _synchronized(reading),
|
||||
"daemon": reading.daemon,
|
||||
"dhcp_ntp_servers": _dhcp_servers(reading, dhcp_dirs),
|
||||
}
|
||||
|
||||
|
||||
def parse_time_sync(text: str, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
|
||||
"""The reading's output as ``get_time_sync`` returns it.
|
||||
|
||||
:raises ValueError: for a reading cut short.
|
||||
"""
|
||||
return time_sync_of(parse_reading(text), dhcp_dirs)
|
||||
@@ -0,0 +1,140 @@
|
||||
"""A device's own configuration history: list it, read a revision, revert to one.
|
||||
|
||||
Some devices keep their earlier configurations themselves: a copy of the
|
||||
configuration for every save, which the device can go back to. An application
|
||||
restores a configuration there by reverting to the revision that holds it -- on a
|
||||
device whose API takes no uploaded configuration (OPNsense), the only way
|
||||
(NetOrk/netork#2511). :class:`ConfigRevisionsMixin` is the contract:
|
||||
|
||||
* ``get_config_revisions()`` and ``get_config_revision(revision_id)`` -- declared
|
||||
only, so ``callable(getattr(driver, "get_config_revisions", None))`` is the
|
||||
truthful answer to "does this device keep a history";
|
||||
* ``revert_config_revision(revision_id)`` -- a template: it refuses an id that is not
|
||||
in the history, reads the revision, lets the driver revert
|
||||
(``_revert_to_config_revision``), and reads the configuration in place afterwards
|
||||
to check that it holds the revision's. A device may answer that it reverted when it
|
||||
did not; the check does not take its word;
|
||||
* ``config_revision_content(text)`` -- what of a revision is its configuration. Every
|
||||
save may stamp a revision anew (OPNsense writes who saved it and when into
|
||||
config.xml), so two revisions with the same configuration differ as text. A driver
|
||||
whose device does that strips the stamp here; by default the whole text counts. An
|
||||
application that looks for the revision holding a configuration compares through
|
||||
it too.
|
||||
|
||||
A driver mixes it in when its device keeps a history; no role base carries it. One
|
||||
that can list the history but not revert to it sets ``revert_config_revision`` to
|
||||
``None``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
|
||||
class ConfigRevisionDict(TypedDict):
|
||||
"""One revision of the device's configuration (``get_config_revisions``)."""
|
||||
|
||||
id: str # what get_config_revision and revert_config_revision take, as the device names it
|
||||
time: str | None # when it was saved, UTC ISO 8601; None when the device does not say
|
||||
description: str | None # what made the change, as the device records it; None: none
|
||||
user: str | None # who saved it, as the device names them; None when it names nobody
|
||||
size: int | None # its size in bytes; None when not given
|
||||
|
||||
|
||||
class UnknownConfigRevision(ValueError):
|
||||
"""The device's configuration history has no revision of that id."""
|
||||
|
||||
def __init__(self, revision_id: object) -> None:
|
||||
super().__init__(f"no configuration revision {revision_id!r} in the device's history")
|
||||
|
||||
|
||||
class ConfigRevertFailed(RuntimeError):
|
||||
"""The device refused the revert, or its configuration afterwards is not the revision's."""
|
||||
|
||||
|
||||
class ConfigRevisionsMixin:
|
||||
"""List, read and revert to the device's configuration revisions; see the module."""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_config_revisions(self) -> list[ConfigRevisionDict]:
|
||||
"""The device's configuration history, newest first.
|
||||
|
||||
The first revision is the configuration in place now.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "config-1794603885.1239.xml",
|
||||
"time": "2026-11-13T21:04:45+00:00",
|
||||
"description": "/api/firewall/filter/apply made changes",
|
||||
"user": "apiuser@192.0.2.10",
|
||||
"size": 58213,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the history could not be read; never an empty
|
||||
list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_config_revision(self, revision_id: str) -> str:
|
||||
"""The configuration of revision *revision_id*, as the device keeps it.
|
||||
|
||||
:raises UnknownConfigRevision: for an id that is not in the history.
|
||||
:raises Exception: when the device could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# The step a driver implements:
|
||||
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
"""Make revision *revision_id* the device's configuration; return once it is
|
||||
in place, as the newest revision of the history.
|
||||
|
||||
:raises UnknownConfigRevision: when the device no longer has it.
|
||||
:raises ConfigRevertFailed: when the device refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
"""What of a revision's *text* is its configuration: the whole text by default.
|
||||
|
||||
A driver whose device stamps every save into the configuration (who, when)
|
||||
returns the text without that stamp, so that two revisions with the same
|
||||
configuration compare equal.
|
||||
"""
|
||||
return text
|
||||
|
||||
def revert_config_revision(self, revision_id: str) -> ConfigRevisionDict:
|
||||
"""Make revision *revision_id* the device's configuration again.
|
||||
|
||||
Returns the revision in place afterwards: the newest of the history, which
|
||||
holds the configuration of *revision_id* (on most devices a new revision the
|
||||
revert saved). What the device does beyond that -- reload its services,
|
||||
restart, nothing -- is the driver's to document.
|
||||
|
||||
:raises UnknownConfigRevision: for an id that is not in the history now;
|
||||
nothing has changed then.
|
||||
:raises ConfigRevertFailed: when the device refuses, or when the configuration
|
||||
in place afterwards is not the revision's.
|
||||
:raises Exception: when the device could not be reached.
|
||||
"""
|
||||
if revision_id not in {r["id"] for r in self.get_config_revisions()}:
|
||||
raise UnknownConfigRevision(revision_id)
|
||||
# Read before the revert: saving it may rotate the revision out of the history.
|
||||
wanted = self.config_revision_content(self.get_config_revision(revision_id))
|
||||
self._revert_to_config_revision(revision_id)
|
||||
revisions = self.get_config_revisions()
|
||||
if not revisions:
|
||||
raise ConfigRevertFailed(
|
||||
f"the device has no configuration after reverting to {revision_id!r}"
|
||||
)
|
||||
now = revisions[0]
|
||||
if self.config_revision_content(self.get_config_revision(now["id"])) != wanted:
|
||||
raise ConfigRevertFailed(
|
||||
f"the configuration in place ({now['id']!r}) is not revision {revision_id!r}"
|
||||
" after the revert"
|
||||
)
|
||||
return now
|
||||
+156
-61
@@ -26,6 +26,7 @@ import re
|
||||
from collections.abc import Iterator
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types._dhcp_order import ReservationChange, order_reservation_changes
|
||||
from napalm_device_types._matching import check_desired, index_live
|
||||
from napalm_device_types.models import (
|
||||
DhcpHaDict,
|
||||
@@ -57,6 +58,27 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
|
||||
|
||||
_HEX_ONLY = re.compile(r"[^0-9a-f]")
|
||||
|
||||
#: Added to a refused change that is part of an address swap (#106).
|
||||
_SWAP_NOTE = (
|
||||
"an address swap with {}: no order frees an address first; "
|
||||
"move one of them to a free address in a run of its own"
|
||||
)
|
||||
|
||||
|
||||
class DhcpApplyError(RuntimeError):
|
||||
"""Some changes of an applied reservation or subnet set were refused by the device.
|
||||
|
||||
Raised after the rest were written and committed: ``failed`` of ``attempted``
|
||||
changes did not go through, and each was named in a ``[failed]`` progress line.
|
||||
``kind`` is what was applied: ``"reservation"`` or ``"subnet"``.
|
||||
"""
|
||||
|
||||
def __init__(self, failed: int, attempted: int, kind: str = "reservation") -> None:
|
||||
self.failed = failed
|
||||
self.attempted = attempted
|
||||
self.kind = kind
|
||||
super().__init__(f"{failed} of {attempted} DHCP {kind} change(s) were refused")
|
||||
|
||||
|
||||
def normalize_mac(mac: str | None) -> str:
|
||||
"""Reduces a MAC address to lowercase colon-separated form.
|
||||
@@ -106,6 +128,43 @@ def _subnet_field_differs(field: str, live: DhcpSubnetDict, desired: DhcpSubnetD
|
||||
return any(live_options.get(option) != value for option, value in desired_options.items())
|
||||
|
||||
|
||||
def _reservation_key(reservation: DhcpReservationDict) -> str:
|
||||
return normalize_mac(reservation.get("mac"))
|
||||
|
||||
|
||||
def _diff_reservations(
|
||||
live: list[DhcpReservationDict], desired: list[DhcpReservationDict]
|
||||
) -> DhcpReservationDiffDict:
|
||||
"""``diff_dhcp_reservations`` against reservations already read."""
|
||||
live_by_mac, shared = index_live(live, _reservation_key)
|
||||
check_desired(desired, _reservation_key, shared, "MAC")
|
||||
|
||||
add: list[DhcpReservationDict] = []
|
||||
update: list[DhcpReservationUpdateDict] = []
|
||||
|
||||
for desired_reservation in desired:
|
||||
current = live_by_mac.get(_reservation_key(desired_reservation))
|
||||
if current is None:
|
||||
add.append(desired_reservation)
|
||||
continue
|
||||
|
||||
changed_fields = [
|
||||
field
|
||||
for field in _DHCP_RESERVATION_COMPARE_FIELDS
|
||||
if current.get(field) != desired_reservation.get(field)
|
||||
]
|
||||
if changed_fields:
|
||||
update.append(
|
||||
{
|
||||
"uuid": current["uuid"],
|
||||
"reservation": desired_reservation,
|
||||
"changed_fields": changed_fields,
|
||||
}
|
||||
)
|
||||
|
||||
return {"add": add, "update": update}
|
||||
|
||||
|
||||
class DhcpServerMixin:
|
||||
"""Mixin providing DHCP reservation and subnet read/diff/apply.
|
||||
|
||||
@@ -258,36 +317,7 @@ class DhcpServerMixin:
|
||||
:raises ValueError: When a desired reservation cannot name a single live one.
|
||||
"""
|
||||
|
||||
def key(reservation: DhcpReservationDict) -> str:
|
||||
return normalize_mac(reservation.get("mac"))
|
||||
|
||||
live_by_mac, shared = index_live(self.get_dhcp_reservations(), key)
|
||||
check_desired(desired, key, shared, "MAC")
|
||||
|
||||
add: list[DhcpReservationDict] = []
|
||||
update: list[DhcpReservationUpdateDict] = []
|
||||
|
||||
for desired_reservation in desired:
|
||||
live = live_by_mac.get(key(desired_reservation))
|
||||
if live is None:
|
||||
add.append(desired_reservation)
|
||||
continue
|
||||
|
||||
changed_fields = [
|
||||
field
|
||||
for field in _DHCP_RESERVATION_COMPARE_FIELDS
|
||||
if live.get(field) != desired_reservation.get(field)
|
||||
]
|
||||
if changed_fields:
|
||||
update.append(
|
||||
{
|
||||
"uuid": live["uuid"],
|
||||
"reservation": desired_reservation,
|
||||
"changed_fields": changed_fields,
|
||||
}
|
||||
)
|
||||
|
||||
return {"add": add, "update": update}
|
||||
return _diff_reservations(self.get_dhcp_reservations(), desired)
|
||||
|
||||
def apply_dhcp_reservationset(self, desired: list[DhcpReservationDict]) -> Iterator[str]:
|
||||
"""
|
||||
@@ -298,29 +328,65 @@ class DhcpServerMixin:
|
||||
committing reloads the DHCP daemon and drops in-flight requests, which
|
||||
is too high a price for a no-op run.
|
||||
|
||||
The changes are written so an address is released before it is taken; a
|
||||
swap of addresses has no such order and is reported as one (``_dhcp_order``).
|
||||
|
||||
A change the device refuses (say, an address in a subnet its DHCP server
|
||||
does not manage) does not stop the others: it gets a ``[failed]`` line,
|
||||
the rest are written and committed, and ``DhcpApplyError`` is raised at
|
||||
the end. Any non-empty diff commits, even one whose every write was
|
||||
refused, so that writes an earlier, interrupted run saved take effect.
|
||||
|
||||
:param desired: The complete desired reservation set.
|
||||
:yields: Progress lines, one per applied add/update, plus a final
|
||||
commit line.
|
||||
:yields: Progress lines, one per applied, updated or refused change,
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_reservations(desired)
|
||||
|
||||
for reservation in diff["add"]:
|
||||
self.apply_dhcp_reservation(reservation)
|
||||
yield f"[add] {reservation['mac']} -> {reservation['ip']}"
|
||||
|
||||
for entry in diff["update"]:
|
||||
self.apply_dhcp_reservation(entry["reservation"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield (
|
||||
f"[update] {entry['reservation']['mac']} -> {entry['reservation']['ip']} ({fields})"
|
||||
)
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
live = self.get_dhcp_reservations()
|
||||
diff = _diff_reservations(live, desired)
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
return
|
||||
|
||||
added = updated = refused = 0
|
||||
|
||||
for change in order_reservation_changes(diff, live):
|
||||
line, ok = self._write_change(change)
|
||||
refused += not ok
|
||||
added += ok and change.uuid is None
|
||||
updated += ok and change.uuid is not None
|
||||
yield line
|
||||
|
||||
# Committed even when every write was refused: a run that stopped before its
|
||||
# commit left saved writes behind, which the device reports as configured, so
|
||||
# no later diff would ever commit them.
|
||||
self.commit_dhcp_reservations()
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
yield f"[commit] applied {added} add(s), {updated} update(s)"
|
||||
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted)
|
||||
|
||||
def _write_change(self, change: ReservationChange) -> tuple[str, bool]:
|
||||
"""Write one change of an ordered set; its progress line, and whether it went in."""
|
||||
tag = "[add]" if change.uuid is None else "[update]"
|
||||
line, ok = self._write_reservation(change.reservation, change.uuid, tag)
|
||||
if ok and change.fields:
|
||||
line = f"{line} ({change.fields})"
|
||||
elif not ok and change.swap:
|
||||
line = f"{line} -- {_SWAP_NOTE.format(', '.join(change.swap))}"
|
||||
return line, ok
|
||||
|
||||
def _write_reservation(
|
||||
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
|
||||
) -> tuple[str, bool]:
|
||||
"""Write one reservation; its progress line, and whether the device took it."""
|
||||
target = f"{reservation['mac']} -> {reservation['ip']}"
|
||||
try:
|
||||
self.apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
except Exception as exc:
|
||||
return f"[failed] {target}: {exc}", False
|
||||
return f"{tag} {target}", True
|
||||
|
||||
def diff_dhcp_subnets(self, desired: list[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
|
||||
"""
|
||||
@@ -391,24 +457,53 @@ class DhcpServerMixin:
|
||||
As with ``apply_dhcp_reservationset``, an empty diff does **not**
|
||||
commit: reconfiguring the DHCP daemon is too expensive for a no-op.
|
||||
|
||||
A subnet the device refuses (say, one that overlaps a subnet it serves)
|
||||
does not stop the others: it gets a ``[failed]`` line, the rest are
|
||||
written and committed, and ``DhcpApplyError`` is raised at the end. Any
|
||||
non-empty diff commits, even one whose every write was refused, so that
|
||||
writes an earlier, interrupted run saved take effect (#113).
|
||||
|
||||
:param desired: The complete desired subnet set.
|
||||
:yields: Progress lines, one per applied add/update, plus a final
|
||||
commit line.
|
||||
:yields: Progress lines, one per applied, updated or refused change,
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_subnets(desired)
|
||||
|
||||
for subnet in diff["add"]:
|
||||
self.apply_dhcp_subnet(subnet)
|
||||
yield f"[add] {subnet['subnet']}"
|
||||
|
||||
for entry in diff["update"]:
|
||||
self.apply_dhcp_subnet(entry["subnet"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield f"[update] {entry['subnet']['subnet']} ({fields})"
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
return
|
||||
|
||||
changes: list[tuple[DhcpSubnetDict, str | None, str]] = [
|
||||
(subnet, None, "") for subnet in diff["add"]
|
||||
]
|
||||
changes += [
|
||||
(e["subnet"], e["uuid"], ", ".join(e["changed_fields"])) for e in diff["update"]
|
||||
]
|
||||
added = updated = refused = 0
|
||||
|
||||
for subnet, uuid, fields in changes:
|
||||
line, ok = self._write_subnet(subnet, uuid, fields)
|
||||
refused += not ok
|
||||
added += ok and uuid is None
|
||||
updated += ok and uuid is not None
|
||||
yield line
|
||||
|
||||
# Committed even when every write was refused; see apply_dhcp_reservationset.
|
||||
self.commit_dhcp_subnets()
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
yield f"[commit] applied {added} add(s), {updated} update(s)"
|
||||
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted, "subnet")
|
||||
|
||||
def _write_subnet(
|
||||
self, subnet: DhcpSubnetDict, uuid: str | None, fields: str
|
||||
) -> tuple[str, bool]:
|
||||
"""Write one subnet; its progress line, and whether the device took it."""
|
||||
try:
|
||||
self.apply_dhcp_subnet(subnet, uuid=uuid)
|
||||
except Exception as exc:
|
||||
return f"[failed] {subnet['subnet']}: {exc}", False
|
||||
if uuid is None:
|
||||
return f"[add] {subnet['subnet']}", True
|
||||
return f"[update] {subnet['subnet']} ({fields})", True
|
||||
@@ -43,7 +43,9 @@ exits quietly every day (NetOrk/netork#738). While dpkg, apt, aptitude or
|
||||
unattended-upgrade is running, packages are halfway on purpose -- also between
|
||||
two of apt's dpkg calls -- so the answer is then ``None``, as on a host without
|
||||
dpkg. unattended-upgrade is told from Ubuntu's idle shutdown helper by its
|
||||
command line (:data:`BUSY_CHECK`).
|
||||
command line (:data:`BUSY_CHECK`). On a host without dpkg, dnf or yum, apk and opkg
|
||||
are read the same way (:mod:`napalm_device_types.package_managers`,
|
||||
NetOrk/netork#851), and ``package_manager`` names the one the answer is about.
|
||||
|
||||
**Filesystems** are how full ``/``, ``/boot`` and ``/var`` are, from POSIX
|
||||
``df -Pk``: where an upgrade unpacks a kernel, builds its initrd and keeps the
|
||||
@@ -58,6 +60,7 @@ too full is not decided here.
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from functools import cmp_to_key
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import (
|
||||
@@ -65,6 +68,11 @@ from napalm_device_types.models import (
|
||||
HostStatusDict,
|
||||
InterruptedPackageDict,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
PackageManagerReading,
|
||||
parse_package_managers,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "HSTAT_BEGIN"
|
||||
@@ -117,6 +125,7 @@ HOST_STATUS_COMMAND = (
|
||||
f"echo '[dpkg-journal]'; ls -1 {_DPKG_JOURNAL} 2>/dev/null | head -n 20; "
|
||||
"echo '[dpkg-audit]'; dpkg-query -W -f='${db:Status-Abbrev} ${Package}\\n' 2>/dev/null "
|
||||
"| awk 'substr($0, 2, 1) ~ /[HUFWt]/ || substr($0, 3, 1) == \"R\"' | head -n 50; fi; "
|
||||
f"{PACKAGE_MANAGERS_COMMAND}"
|
||||
f"echo '[filesystems]'; df -Pk {_FILESYSTEMS}; "
|
||||
f"echo '[timers]'; for u in {_APT_TIMER} {' '.join(_DNF_TIMERS)}; do "
|
||||
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled "$u" 2>/dev/null)"; done; '
|
||||
@@ -163,9 +172,39 @@ def _sections(output: str) -> dict[str, list[str]]:
|
||||
return sections
|
||||
|
||||
|
||||
def _version_key(version: str) -> tuple[object, ...]:
|
||||
"""Natural order: 6.8.0-142 after 6.8.0-87, 7.0.14 after 7.0.2."""
|
||||
return tuple(int(part) if part.isdigit() else part for part in re.split(r"(\d+)", version))
|
||||
#: A version's segments as rpm compares them: runs of digits, runs of letters, and
|
||||
#: ``~``; whatever else stands between them only separates.
|
||||
_VERSION_SEGMENT = re.compile(r"~|\d+|[A-Za-z]+")
|
||||
|
||||
|
||||
def _segment_cmp(x: str, y: str) -> int:
|
||||
"""One segment against another, as :func:`_version_cmp` takes them; 0 when equal."""
|
||||
if "~" in (x, y):
|
||||
return (y == "~") - (x == "~")
|
||||
if x.isdigit() and y.isdigit():
|
||||
return (int(x) > int(y)) - (int(x) < int(y))
|
||||
if x.isdigit() or y.isdigit():
|
||||
return 1 if x.isdigit() else -1
|
||||
return (x > y) - (x < y)
|
||||
|
||||
|
||||
def _version_cmp(a: str, b: str) -> int:
|
||||
"""rpm's ``rpmvercmp``: below 0 when ``a`` is older, above when newer.
|
||||
|
||||
Numbers compare as numbers (6.8.0-142 after 6.8.0-87), letters as text, and a
|
||||
number is newer than letters in its place: ``112.5.1.el9uek`` after
|
||||
``112.5.el9uek`` (#109). A version that has more segments where the other
|
||||
ends is the newer one, unless its next is ``~``, which comes before anything:
|
||||
``6.13~rc5`` before ``6.13``, as dpkg sorts it too.
|
||||
"""
|
||||
left, right = _VERSION_SEGMENT.findall(a), _VERSION_SEGMENT.findall(b)
|
||||
for x, y in zip(left, right, strict=False):
|
||||
if order := _segment_cmp(x, y):
|
||||
return order
|
||||
if len(left) == len(right):
|
||||
return 0
|
||||
longer, sign = (left, 1) if len(left) > len(right) else (right, -1)
|
||||
return -sign if longer[min(len(left), len(right))] == "~" else sign
|
||||
|
||||
|
||||
def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
|
||||
@@ -180,8 +219,9 @@ def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
|
||||
same = [k for k in installed if _kernel_flavour(k)[1] == flavour]
|
||||
if not same:
|
||||
return None
|
||||
newest = max(same, key=lambda k: _version_key(_kernel_flavour(k)[0]))
|
||||
if _version_key(_kernel_flavour(newest)[0]) > _version_key(version):
|
||||
order = cmp_to_key(_version_cmp)
|
||||
newest = max(same, key=lambda k: order(_kernel_flavour(k)[0]))
|
||||
if order(_kernel_flavour(newest)[0]) > order(version):
|
||||
return newest
|
||||
return None
|
||||
|
||||
@@ -287,12 +327,24 @@ def _halfway(line: str) -> InterruptedPackageDict | None:
|
||||
return {"name": name, "state": state} if name and state else None
|
||||
|
||||
|
||||
def _dpkg(sections: dict[str, list[str]]) -> tuple[bool | None, list[InterruptedPackageDict]]:
|
||||
if "dpkg-audit" not in sections or "dpkg-busy" in sections:
|
||||
return None, []
|
||||
def _dpkg(sections: dict[str, list[str]]) -> PackageManagerReading | None:
|
||||
if "dpkg-audit" not in sections:
|
||||
return None
|
||||
if "dpkg-busy" in sections:
|
||||
return PackageManagerReading("dpkg", None, [])
|
||||
packages = [p for p in map(_halfway, sections["dpkg-audit"]) if p is not None]
|
||||
journal = any(_JOURNAL_ENTRY.fullmatch(f) for f in sections.get("dpkg-journal") or [])
|
||||
return bool(packages) or journal, packages
|
||||
return PackageManagerReading("dpkg", bool(packages) or journal, packages)
|
||||
|
||||
|
||||
def _package_manager(sections: dict[str, list[str]]) -> PackageManagerReading:
|
||||
"""The reading that says most: one left halfway, then one that is not, then a busy one."""
|
||||
readings = [r for r in (_dpkg(sections), *parse_package_managers(sections)) if r is not None]
|
||||
for interrupted in (True, False):
|
||||
found = next((r for r in readings if r.interrupted is interrupted), None)
|
||||
if found is not None:
|
||||
return found
|
||||
return readings[0] if readings else PackageManagerReading(None, None, [])
|
||||
|
||||
|
||||
def _filesystems(sections: dict[str, list[str]]) -> list[FilesystemUsageDict] | None:
|
||||
@@ -318,15 +370,16 @@ def parse_host_status(output: str) -> HostStatusDict:
|
||||
"""
|
||||
sections = _sections(output)
|
||||
required, reason = _reboot(sections)
|
||||
interrupted, packages = _dpkg(sections)
|
||||
manager = _package_manager(sections)
|
||||
return {
|
||||
"reboot_required": required,
|
||||
"reboot_reason": reason,
|
||||
"auto_updates": _auto_updates(sections),
|
||||
"auto_updates_last_success": _last_success(sections),
|
||||
"auto_updates_interval": _interval(sections),
|
||||
"package_manager_interrupted": interrupted,
|
||||
"interrupted_packages": packages,
|
||||
"package_manager": manager.manager,
|
||||
"package_manager_interrupted": manager.interrupted,
|
||||
"interrupted_packages": manager.packages,
|
||||
"filesystems": _filesystems(sections),
|
||||
}
|
||||
|
||||
@@ -356,6 +409,7 @@ class HostStatusMixin:
|
||||
* auto_updates (bool or None)
|
||||
* auto_updates_last_success (Unix time or None)
|
||||
* auto_updates_interval (seconds or None)
|
||||
* package_manager (dpkg, dnf, yum, apk, opkg, or None)
|
||||
* package_manager_interrupted (bool or None)
|
||||
* interrupted_packages (list of {name, state})
|
||||
* filesystems (list of {mountpoint, total, used, available} in bytes, or None)
|
||||
|
||||
@@ -5,6 +5,14 @@ with their public key and, where they are to administer the host, may use
|
||||
sudo with their own password. When they leave, the account is locked, not
|
||||
deleted: their key goes, sudo goes, the login is refused, and the home stays.
|
||||
|
||||
**What the person changes follows them** (NetOrk/netork#2013). A key they
|
||||
replace goes from the account as the new one comes in (``replaces``); a key
|
||||
they delete goes without a new one (``remove_local_account_keys``). Either way
|
||||
only the keys named go: one the person added on the host stays, and the
|
||||
account stays open. A password they remove goes from the account too
|
||||
(``clear_password``): it logs in with a key only and has no sudo, as the sudo
|
||||
rule asks for that password.
|
||||
|
||||
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
|
||||
checks, reading an account back, the order of the steps, the key file and the
|
||||
sudo rule. A driver supplies the commands that differ between operating
|
||||
@@ -48,7 +56,7 @@ from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
@@ -179,6 +187,18 @@ def check_public_key(key: str) -> None:
|
||||
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
|
||||
|
||||
|
||||
def _checked_keys(keys: Sequence[str]) -> list[str]:
|
||||
"""*keys*, each one stripped.
|
||||
|
||||
:raises ValueError: when *keys* is one string, or one of them no public key.
|
||||
"""
|
||||
if isinstance(keys, str):
|
||||
raise ValueError("expected a list of public keys, not one string")
|
||||
for key in keys:
|
||||
check_public_key(key)
|
||||
return [key.strip() for key in keys]
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
@@ -225,9 +245,11 @@ class LocalAccountsMixin:
|
||||
|
||||
# -- what a driver supplies ------------------------------------------------------
|
||||
#
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the key
|
||||
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
|
||||
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the two after
|
||||
# them (``_account_create_with_uid``, ``_account_clear_password``) it may, and
|
||||
# the key store (``_account_key_files``, ``_account_read_keys``,
|
||||
# ``_account_add_key``, ``_account_drop_keys``, ``_account_remove_keys``) it may
|
||||
# replace. ``_accounts_*`` are this mixin's own.
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
|
||||
@@ -265,6 +287,16 @@ class LocalAccountsMixin:
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
"""The commands that take *name*'s password away; its key login stays as it is.
|
||||
|
||||
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
|
||||
and not a lock or an expiry, which sshd or PAM apply to a key too.
|
||||
Only ``add_local_account(clear_password=True)`` and
|
||||
``remove_local_account_keys(clear_password=True)`` ask for it.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
# -- reading ---------------------------------------------------------------------
|
||||
|
||||
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
|
||||
@@ -339,6 +371,8 @@ class LocalAccountsMixin:
|
||||
*,
|
||||
password_hash: str | None = None,
|
||||
admin: bool = False,
|
||||
clear_password: bool = False,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Give *name* an account that logs in with *public_key*, or open it again.
|
||||
|
||||
@@ -347,30 +381,82 @@ class LocalAccountsMixin:
|
||||
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
|
||||
the sudo rule, which asks for that password -- so an administrator
|
||||
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
|
||||
are taken away.
|
||||
are taken away. Without *password_hash* the account keeps the password
|
||||
it has, unless *clear_password* takes it away: the account then logs
|
||||
in with a key only, and can have no sudo.
|
||||
|
||||
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
|
||||
the other failed.
|
||||
*replaces* are keys *public_key* takes the place of -- the person's
|
||||
old ones. They go from the account, matched by their base64 whatever
|
||||
a line's options or comment; every other key stays.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, the login
|
||||
user's own account, or a system account.
|
||||
Steps: ``account``, then ``sudo``, ``password`` (with
|
||||
*clear_password*), ``keys`` and ``old keys`` (with *replaces*), each of
|
||||
which runs when another failed: an old key goes even when the new one
|
||||
could not be added.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, a replaced
|
||||
key that is none, *clear_password* with a *password_hash* or with
|
||||
*admin*, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
check_public_key(public_key)
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
new = _key_blob(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != new]
|
||||
self._accounts_check_password(password_hash, admin=admin, clear=clear_password)
|
||||
existing = self._accounts_changeable(name)
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
steps = self._accounts_after_open(name, public_key, admin, clear_password, old)
|
||||
parts += [run_fix_steps([step]) for step in steps]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_after_open(
|
||||
self, name: str, public_key: str, admin: bool, clear_password: bool, old: list[str]
|
||||
) -> list[FixStep]:
|
||||
"""The steps of ``add_local_account`` once the account is there, each on its own."""
|
||||
steps: list[FixStep] = [("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("keys", lambda: self._accounts_add_key(name, public_key)))
|
||||
if old:
|
||||
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
|
||||
return steps
|
||||
|
||||
def remove_local_account_keys(
|
||||
self, name: str, keys: Sequence[str], *, clear_password: bool = False
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Take *keys* off *name*'s account, for a person who deleted them.
|
||||
|
||||
Only those keys go, matched by their base64 whatever a line's options
|
||||
or comment; the other keys stay, and the account stays open -- to get
|
||||
a new key once there is one. A locked account has none left to
|
||||
remove: the lock took them all. Without *clear_password* the password
|
||||
and sudo stay too; with it the password goes, and so do the sudo rule
|
||||
and :attr:`ACCOUNT_ADMIN_GROUPS`, as sudo asks for that password -- for
|
||||
a person whose password went with their key.
|
||||
|
||||
Steps: ``keys``, then ``password`` and ``sudo`` (with *clear_password*),
|
||||
each of which runs when another failed.
|
||||
|
||||
:raises ValueError: for a name no host takes, no keys or one that is
|
||||
none, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
wanted = _checked_keys(keys)
|
||||
if not wanted:
|
||||
raise ValueError("no public key to remove")
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
steps: list[FixStep] = [("keys", lambda: self._accounts_drop_keys(name, wanted))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("sudo", lambda: self._accounts_set_sudo(name, admin=False)))
|
||||
return self._accounts_result(name, [run_fix_steps([step]) for step in steps])
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
|
||||
|
||||
@@ -395,7 +481,12 @@ class LocalAccountsMixin:
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def add_service_account(
|
||||
self, name: str, public_key: str, *, uid: int | None = None
|
||||
self,
|
||||
name: str,
|
||||
public_key: str,
|
||||
*,
|
||||
uid: int | None = None,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
|
||||
root without a password.
|
||||
@@ -409,36 +500,59 @@ class LocalAccountsMixin:
|
||||
once no other account has it; an account that is there keeps its own,
|
||||
as a uid is never changed under files that carry it.
|
||||
|
||||
*replaces* are netOrk's keys *public_key* takes the place of: they go
|
||||
from the account once *public_key* is in it, matched as in
|
||||
:meth:`add_local_account`, and never *public_key* itself. Call it so
|
||||
through a login with *public_key*, tried before: netOrk keeps its way
|
||||
in (NetOrk/netork#2462).
|
||||
|
||||
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
|
||||
``keys``. The new login itself is for the caller to try.
|
||||
``keys``, and ``old keys`` with *replaces* when ``keys`` went. The new
|
||||
login itself is for the caller to try.
|
||||
|
||||
An account that is there and cannot log in (``nologin``, ``false``) is
|
||||
another service's -- a satellite's ``netork``, say -- and is left alone.
|
||||
|
||||
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
|
||||
less, or another service's account.
|
||||
:raises ValueError: for a name or key no host takes, a replaced key that
|
||||
is none, root, a uid of 0 or less, or another service's account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
check_public_key(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != _key_blob(public_key)]
|
||||
login, existing = self._accounts_service_target(name, uid)
|
||||
first = self._accounts_service_account_step(name, login, existing, uid)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += self._accounts_service_keys(name, public_key, old)
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_service_account_step(
|
||||
self, name: str, login: str, existing: list[str] | None, uid: int | None
|
||||
) -> FixResultDict:
|
||||
"""The ``account`` step of netOrk's own account; nothing to do when the driver
|
||||
logs in as *name* already."""
|
||||
if name == login:
|
||||
step: FixStepDict = {
|
||||
"step": "account",
|
||||
"ok": True,
|
||||
"output": f"this driver logs in as {name} already",
|
||||
}
|
||||
first: FixResultDict = _joined([step])
|
||||
else:
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
return _joined([step])
|
||||
return run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
|
||||
def _accounts_service_keys(
|
||||
self, name: str, public_key: str, old: list[str]
|
||||
) -> list[FixResultDict]:
|
||||
"""``sudo`` and ``keys`` of netOrk's own account, then ``old keys`` once the key is in."""
|
||||
parts = [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
if old and parts[-1]["success"]:
|
||||
parts.append(run_fix_steps([("old keys", lambda: self._accounts_drop_keys(name, old))]))
|
||||
return parts
|
||||
|
||||
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
|
||||
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
|
||||
@@ -490,6 +604,20 @@ class LocalAccountsMixin:
|
||||
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
|
||||
return result.stdout.split(":", 1)[0].strip() or None
|
||||
|
||||
def _accounts_check_password(
|
||||
self, password_hash: str | None, *, admin: bool, clear: bool
|
||||
) -> None:
|
||||
""":raises ValueError: for a hash the host does not take, or a password to
|
||||
clear that is to be set, or that sudo would ask for."""
|
||||
if clear and password_hash is not None:
|
||||
raise ValueError(
|
||||
"a password to clear cannot be set too: clear_password or password_hash"
|
||||
)
|
||||
if clear and admin:
|
||||
raise ValueError("an account without a password cannot use sudo: its rule asks for it")
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
|
||||
def _accounts_check_hash(self, password_hash: str) -> None:
|
||||
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
|
||||
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
|
||||
@@ -550,6 +678,16 @@ class LocalAccountsMixin:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": done + (", password set" if password_hash else "")}
|
||||
|
||||
def _accounts_clear_password(self, name: str) -> FixStepOutcomeDict:
|
||||
try:
|
||||
commands = self._account_clear_password(name)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": "this driver cannot remove a password"}
|
||||
failed = self._accounts_run(commands)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"{name} has no password: it logs in with a key only"}
|
||||
|
||||
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
@@ -683,9 +821,23 @@ class LocalAccountsMixin:
|
||||
mine = self._accounts_key_file(name, path)
|
||||
if mine is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
|
||||
failed = self._accounts_write_key_file(name, home, path, [*mine, public_key.strip()])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
|
||||
def _accounts_write_key_file(
|
||||
self, name: str, home: str, path: str, lines: list[str]
|
||||
) -> CommandResult | None:
|
||||
"""*lines* as *name*'s key file *path*, written as *name*; None, or what failed.
|
||||
|
||||
Without lines the file goes: nothing is staged from an empty stdin.
|
||||
"""
|
||||
if not lines:
|
||||
return self._accounts_run_as(name, [["rm", "-f", "--", path]])
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in lines))
|
||||
try:
|
||||
failed = self._accounts_run_as(
|
||||
return self._accounts_run_as(
|
||||
name,
|
||||
[
|
||||
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
|
||||
@@ -694,9 +846,49 @@ class LocalAccountsMixin:
|
||||
)
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
|
||||
def _accounts_drop_from(
|
||||
self, name: str, home: str, path: str, blobs: set[str | None]
|
||||
) -> FixStepOutcomeDict:
|
||||
"""The keys of *blobs* out of *name*'s key file *path*; no output when none was in it."""
|
||||
lines = self._accounts_key_file(name, path)
|
||||
if lines is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
kept = [line for line in lines if _key_blob(line) not in blobs]
|
||||
if len(kept) == len(lines):
|
||||
return {"ok": True, "output": ""}
|
||||
failed = self._accounts_write_key_file(name, home, path, kept)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
return {"ok": True, "output": f"{len(lines) - len(kept)} removed from {path}"}
|
||||
|
||||
def _accounts_drop_keys(self, name: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_drop_keys(name, home, keys)
|
||||
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
"""Remove exactly *keys* from *name*'s key files in *home*, as *name*.
|
||||
|
||||
A key is matched by its base64, whatever a line's options or comment;
|
||||
every other line stays. Afterwards none of *keys* may be left in
|
||||
:meth:`_account_key_files`. A driver whose host keeps keys elsewhere
|
||||
overrides this, with :meth:`_account_add_key`.
|
||||
"""
|
||||
blobs = {_key_blob(key) for key in keys}
|
||||
said: list[str] = []
|
||||
for path in authorized_keys_paths(home):
|
||||
outcome = self._accounts_drop_from(name, home, path, blobs)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
said += [outcome["output"]] if outcome["output"] else []
|
||||
left = self._account_read_keys(name, home)
|
||||
if left is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
if any(_key_blob(line) in blobs for line in left):
|
||||
return {"ok": False, "output": f"{name} still has a key that was to go"}
|
||||
return {"ok": True, "output": "\n".join(said) or f"none of the keys is in {home}/.ssh"}
|
||||
|
||||
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
|
||||
in, and whether it gets the journal (NetOrk/netork#2053).
|
||||
|
||||
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
|
||||
this says who reads it on. It is read in the same command, so a reading of
|
||||
the logs a host writes also says which of them leave it.
|
||||
|
||||
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
|
||||
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
|
||||
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
|
||||
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
|
||||
*complete* when it takes every message of its ruleset -- ``*.*`` with no
|
||||
condition around it. Anything narrower is partial, and keeps its selector or
|
||||
condition for the reader to see: which facility a unit logs to is not
|
||||
computed (decided in #2053).
|
||||
|
||||
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
|
||||
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
|
||||
own a host runs on its distribution's default, which ``journald.conf`` ships
|
||||
commented out: ``#ForwardToSyslog=yes``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class ForwardRuleDict(TypedDict):
|
||||
"""One rule sending messages to another host."""
|
||||
|
||||
protocol: str # "udp", "tcp" or "relp"
|
||||
target: str # the host, as configured
|
||||
port: int | None # None: the protocol's default
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
#: The template its messages are sent in; None: rsyslog's default, RFC 3164,
|
||||
#: which carries the sender's local time without a zone (NetOrk/netork#493).
|
||||
template: str | None
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
|
||||
|
||||
path: str # as configured: may hold a glob
|
||||
ruleset: str | None
|
||||
|
||||
|
||||
class RsyslogDict(TypedDict):
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
imjournal: bool
|
||||
imuxsock: bool
|
||||
|
||||
|
||||
class LogForwardingDict(TypedDict):
|
||||
"""Where a host's logs go beyond it.
|
||||
|
||||
``journal`` is None when it cannot be told whether the journal reaches
|
||||
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
|
||||
"""
|
||||
|
||||
rsyslog: bool # rsyslogd runs
|
||||
journal: bool | None # the journal reaches rsyslog
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
|
||||
|
||||
#: Selectors that take every message.
|
||||
_EVERYTHING = {"*.*", "*.debug"}
|
||||
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
|
||||
_CALL_RE = re.compile(
|
||||
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
|
||||
)
|
||||
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
|
||||
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
|
||||
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
|
||||
_FORWARD_RE = re.compile(
|
||||
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
|
||||
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
|
||||
)
|
||||
_FORWARD_TYPES = {"omfwd", "omrelp"}
|
||||
_TRUE = {"yes", "true", "1", "on"}
|
||||
|
||||
|
||||
def _uncomment(text: str) -> str:
|
||||
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
|
||||
out: list[str] = []
|
||||
quoted = commented = False
|
||||
for char in text:
|
||||
if char == "\n":
|
||||
quoted = commented = False
|
||||
elif commented:
|
||||
continue
|
||||
elif char == '"':
|
||||
quoted = not quoted
|
||||
elif char == "#" and not quoted:
|
||||
commented = True
|
||||
continue
|
||||
out.append(char)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
class _Pieces:
|
||||
"""Cuts a configuration into statements, ``{`` and ``}``: a line runs on
|
||||
while a bracket is open, and a brace outside one stands as a piece of its own."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.pieces: list[str] = []
|
||||
self.current: list[str] = []
|
||||
self.depth = 0
|
||||
self.quoted = False
|
||||
|
||||
def close(self) -> None:
|
||||
piece = "".join(self.current).strip()
|
||||
if piece:
|
||||
self.pieces.append(piece)
|
||||
self.current.clear()
|
||||
|
||||
def bracket(self, char: str) -> None:
|
||||
if char in "([":
|
||||
self.depth += 1
|
||||
elif char in ")]":
|
||||
self.depth = max(self.depth - 1, 0)
|
||||
|
||||
def feed(self, char: str) -> None:
|
||||
if char == '"':
|
||||
self.quoted = not self.quoted
|
||||
elif not self.quoted:
|
||||
self.bracket(char)
|
||||
if self.depth == 0 and char in "{}\n":
|
||||
self.close()
|
||||
if char != "\n":
|
||||
self.pieces.append(char)
|
||||
return
|
||||
self.current.append(char)
|
||||
|
||||
|
||||
def _pieces(text: str) -> list[str]:
|
||||
"""Statements, ``{`` and ``}`` in order (:class:`_Pieces`)."""
|
||||
cutter = _Pieces()
|
||||
for char in text:
|
||||
cutter.feed(char)
|
||||
cutter.close()
|
||||
return cutter.pieces
|
||||
|
||||
|
||||
def _params(args: str) -> dict[str, str]:
|
||||
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
|
||||
|
||||
|
||||
def _port(value: str | None) -> int | None:
|
||||
return int(value) if value and value.isdigit() else None
|
||||
|
||||
|
||||
class _Reader:
|
||||
"""Walks the pieces of a configuration, keeping the blocks it is in."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.parsed: RsyslogDict = {
|
||||
"rules": [],
|
||||
"file_inputs": [],
|
||||
"imjournal": False,
|
||||
"imuxsock": False,
|
||||
}
|
||||
self.stack: list[tuple[str, str | None]] = []
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
#: ``$ActionForwardDefaultTemplate``, for the legacy rules after it.
|
||||
self.legacy_template: str | None = None
|
||||
#: ``module(load="builtin:omfwd" template=…)``, for every omfwd action.
|
||||
self.module_template: str | None = None
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
return names[-1] if names else self.legacy_ruleset
|
||||
|
||||
def condition(self, once: str | None) -> str | None:
|
||||
conditions = [value for kind, value in self.stack if kind == "if"]
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self,
|
||||
protocol: str,
|
||||
target: str,
|
||||
port: int | None,
|
||||
selector: str,
|
||||
once: str | None,
|
||||
template: str | None,
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
{
|
||||
"protocol": protocol,
|
||||
"target": target,
|
||||
"port": port,
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
"template": template,
|
||||
}
|
||||
)
|
||||
|
||||
def piece(self, piece: str) -> None:
|
||||
if piece == "{":
|
||||
self.stack.append(self.pending or ("block", None))
|
||||
self.pending = None
|
||||
return
|
||||
if piece == "}":
|
||||
if self.stack:
|
||||
self.stack.pop()
|
||||
return
|
||||
if piece == "else":
|
||||
self.pending = ("if", "else")
|
||||
return
|
||||
# A condition whose action follows on its own: the action is under it.
|
||||
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
|
||||
self.pending = None
|
||||
match = _IF_RE.match(piece)
|
||||
if match:
|
||||
if match["rest"]:
|
||||
self.statement(match["rest"], once=match["cond"])
|
||||
else:
|
||||
self.pending = ("if", match["cond"])
|
||||
return
|
||||
self.statement(piece, once=once)
|
||||
|
||||
def statement(self, text: str, once: str | None) -> None:
|
||||
call = _CALL_RE.match(text)
|
||||
if call:
|
||||
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
|
||||
elif text.startswith("$"):
|
||||
self.directive(text)
|
||||
else:
|
||||
self.legacy(text, once)
|
||||
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
if params.get("load", "").removeprefix("builtin:") == "omfwd":
|
||||
self.module_template = params.get("template") or self.module_template
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
self.parsed["file_inputs"].append(
|
||||
{"path": params["file"], "ruleset": params.get("ruleset")}
|
||||
)
|
||||
elif name == "action":
|
||||
self.action(params, before, once)
|
||||
|
||||
def action(self, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
selector = self.last_selector if before == "&" else (before or "*.*")
|
||||
self.last_selector = selector
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
template = params.get("template") or (self.module_template if kind == "omfwd" else None)
|
||||
self.forward(
|
||||
protocol, params["target"], _port(params.get("port")), selector, once, template
|
||||
)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
self.parsed["imjournal"] = True
|
||||
elif name == "imuxsock":
|
||||
self.parsed["imuxsock"] = True
|
||||
|
||||
def directive(self, text: str) -> None:
|
||||
name, _, value = text.partition(" ")
|
||||
name, value = name.lower(), value.strip()
|
||||
if name == "$modload":
|
||||
self.module(value)
|
||||
elif name == "$inputfilename" and value:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
elif name == "$actionforwarddefaulttemplate":
|
||||
self.legacy_template = value or None
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
if not match:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
action = match["action"].strip()
|
||||
forward = _FORWARD_RE.match(action)
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol,
|
||||
forward["v6"] or forward["host"],
|
||||
_port(forward["port"]),
|
||||
selector,
|
||||
once,
|
||||
self.legacy_template_of(action[forward.end() :], protocol),
|
||||
)
|
||||
|
||||
def legacy_template_of(self, rest: str, protocol: str) -> str | None:
|
||||
"""The template after a legacy rule's target (``;Name``), else the default
|
||||
set before it; RELP takes no forwarding default."""
|
||||
_, semicolon, named = rest.partition(";")
|
||||
if semicolon and named.strip():
|
||||
return named.strip()
|
||||
return self.legacy_template if protocol != "relp" else None
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
read for its forwarding rules, its file inputs, and the inputs that take
|
||||
the journal."""
|
||||
reader = _Reader()
|
||||
for piece in _pieces(_uncomment(text)):
|
||||
reader.piece(piece)
|
||||
return reader.parsed
|
||||
|
||||
|
||||
def journal_to_syslog(lines: list[str]) -> bool | None:
|
||||
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
|
||||
the order journald reads them: the last setting wins; with none, the
|
||||
default the distribution ships commented out; None when nothing says."""
|
||||
setting = shipped = None
|
||||
for line in lines:
|
||||
commented = line.lstrip().startswith("#")
|
||||
_, _, value = line.lstrip("# \t").partition("=")
|
||||
if commented:
|
||||
shipped = value.strip().lower() in _TRUE
|
||||
else:
|
||||
setting = value.strip().lower() in _TRUE
|
||||
return setting if setting is not None else shipped
|
||||
|
||||
|
||||
def log_forwarding(
|
||||
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
|
||||
) -> LogForwardingDict:
|
||||
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
|
||||
if not running:
|
||||
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
journal: bool | None
|
||||
if rsyslog["imjournal"]:
|
||||
journal = True
|
||||
elif rsyslog["imuxsock"]:
|
||||
journal = journal_to_syslog(journald_lines)
|
||||
else:
|
||||
journal = False
|
||||
return {
|
||||
"rsyslog": True,
|
||||
"journal": journal,
|
||||
"rules": rsyslog["rules"],
|
||||
"file_inputs": rsyslog["file_inputs"],
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
"""Log sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person, CrowdSec, a syslog
|
||||
forwarder -- it has to know which logs there are. That is read the same way on
|
||||
every Linux host, so the command and its parse live here once, and a driver
|
||||
only carries the command across (NetOrk/netork#2049).
|
||||
|
||||
**Three kinds.** journald names every unit it holds entries for
|
||||
(``journalctl -F _SYSTEMD_UNIT``); only services count, not login sessions.
|
||||
``/proc/<pid>/fd`` shows the files running processes hold open, a log being a
|
||||
``*.log`` / ``*.log.N`` file or anything under ``/var/log``; ``stat -L`` on the
|
||||
descriptor gives its size and last write, also for a file inside a container,
|
||||
whose path is the container's own. The engine's own file of a container's
|
||||
output (``/var/lib/docker/containers/<id>/…``) is that container's stream, not
|
||||
a file of ``dockerd``.
|
||||
|
||||
**Who writes it.** Only a process holding the file open for writing writes
|
||||
it: the access mode in ``/proc/<pid>/fdinfo/<fd>``'s ``flags`` is ``O_WRONLY`` or
|
||||
``O_RDWR``. A reader -- CrowdSec, Promtail, rsyslog's ``imfile``, a ``tail -F``
|
||||
-- holds it too, and is no producer (NetOrk/netork#2092). ``/proc/<pid>/cgroup``
|
||||
says which systemd service or container the first writer runs in, as for
|
||||
listening sockets (:mod:`napalm_device_types.listening`). A file only readers
|
||||
hold is listed without a producer; one only readers in a container hold is left
|
||||
out, its path being the reader's view of a mount. journald's own files, files
|
||||
already deleted and a database's write-ahead log (LevelDB's ``000004.log``) are
|
||||
left out.
|
||||
|
||||
**Where they go** is read in the same round trip
|
||||
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
|
||||
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
|
||||
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
|
||||
reads them.
|
||||
|
||||
**One round trip, root, and without it.** As for listening sockets the script
|
||||
goes as one ``sh -c`` argument. Without root only the login user's processes
|
||||
show their files; the command then runs again unprivileged, and the reading
|
||||
says it is not ``attributed``. A report whose end is missing raises: a list cut
|
||||
short must never read as logs that went away, and so must a host whose
|
||||
``find`` cannot list open files (busybox): it raises
|
||||
:class:`LogSourcesUnavailable`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
|
||||
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class LogSourceDict(TypedDict):
|
||||
"""A log the host writes: a unit's journal, a file a process holds open, or a
|
||||
container's stream as the engine keeps it.
|
||||
|
||||
``path`` is as its writer sees it: a file a container writes is named inside
|
||||
that container. Whether anything reads the log is the consumer's question.
|
||||
"""
|
||||
|
||||
kind: str # "journald", "file" or "container"
|
||||
path: str | None # the file; None for a journal
|
||||
unit: str | None # the journal's unit, or the file writer's service; without ".service"
|
||||
container_id: str | None # the full container ID: the writer's, or the stream's
|
||||
process: str | None # the first process writing the file; None when only readers hold it
|
||||
size: int | None # bytes; None for a journal, or when the file could not be read
|
||||
modified: str | None # the last write, ISO 8601 UTC; None as for size
|
||||
|
||||
|
||||
class LogSourcesDict(TypedDict):
|
||||
"""What ``LogSourcesMixin.get_log_sources`` read.
|
||||
|
||||
``attributed`` is False when the reading ran without root: only the login
|
||||
user's own processes show their files then, so a missing log means "not
|
||||
told", not "not there". ``journald`` is False on a host without systemd's
|
||||
journal, where no unit has one.
|
||||
"""
|
||||
|
||||
attributed: bool
|
||||
journald: bool
|
||||
sources: list[LogSourceDict]
|
||||
forwarding: LogForwardingDict | None # where the logs go beyond the host
|
||||
|
||||
|
||||
_BEGIN = "LOGS_BEGIN"
|
||||
_END = "LOGS_END"
|
||||
|
||||
#: One line, POSIX ``sh`` with GNU ``find`` (``-lname``, ``-printf``), read-only.
|
||||
#: The frame markers are printed in two halves so that a transport which echoes
|
||||
#: the command does not show them early. ``-printf '%p %l'`` puts the descriptor
|
||||
#: before its target, so a target with a space in it stays whole.
|
||||
LOG_SOURCES_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' LOGS_ BEGIN; "
|
||||
"if [ -d /run/systemd/system ] && command -v journalctl >/dev/null 2>&1; then "
|
||||
"j=journalctl; command -v timeout >/dev/null 2>&1 && j='timeout 30 journalctl'; "
|
||||
"echo '[journald]'; $j -F _SYSTEMD_UNIT 2>/dev/null; fi; "
|
||||
"if find / -maxdepth 0 -lname x -printf '' >/dev/null 2>&1; then "
|
||||
"o=$(find /proc/[0-9]*/fd -mindepth 1 -maxdepth 1 -type l "
|
||||
"\\( -lname '*.log' -o -lname '*.log.[0-9]*' -o -lname '/var/log/*' \\) "
|
||||
"-printf '%p %l\\n' 2>/dev/null); "
|
||||
"echo '[open]'; printf '%s\\n' \"$o\"; "
|
||||
"echo '[stat]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 "
|
||||
"| xargs -r stat -L -c '%n %s %Y' 2>/dev/null; "
|
||||
"echo '[flags]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 | sed 's|/fd/|/fdinfo/|' "
|
||||
"| xargs -r grep -H '^flags:' 2>/dev/null; "
|
||||
"pids=$(printf '%s\\n' \"$o\" | cut -d/ -f3 | sort -un); "
|
||||
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
|
||||
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
|
||||
"else echo '[no-find]'; fi; "
|
||||
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
|
||||
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
|
||||
"| sed 's/^/| /'; "
|
||||
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
|
||||
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
|
||||
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
|
||||
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
|
||||
"printf '%s%s\\n' LOGS_ END"
|
||||
)
|
||||
|
||||
#: ``/proc/900/fd/5`` -- the process and its descriptor.
|
||||
_FD_RE = re.compile(r"^/proc/(\d+)/fd/\d+$")
|
||||
#: ``/proc/900/fdinfo/5:flags:<tab>0102001`` -- the descriptor's open flags, octal.
|
||||
_FLAGS_RE = re.compile(r"^/proc/(\d+)/fdinfo/(\d+):flags:\s*([0-7]+)$")
|
||||
#: The access mode bits of the open flags: 0 is O_RDONLY, 1 O_WRONLY, 2 O_RDWR.
|
||||
_ACCESS_MODE = 0o3
|
||||
#: The engine's file of a container's output: json-file and local drivers.
|
||||
_STREAM_RE = re.compile(r"^/var/lib/docker/containers/([0-9a-f]{64})/")
|
||||
#: Files that are no log to read: journald's store, a file already deleted, and
|
||||
#: the write-ahead log LevelDB and RocksDB name ``000004.log``.
|
||||
_NOT_A_SOURCE = re.compile(r"^/var/log/journal/|\.journal~?$| \(deleted\)$|/\d{6}\.log$")
|
||||
_SERVICE_SUFFIX = ".service"
|
||||
_NO_FIND = "no-find"
|
||||
|
||||
|
||||
class LogSourcesUnavailable(NotImplementedError):
|
||||
"""The host's ``find`` cannot list open files (busybox); nothing to read, nothing to retry."""
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact log source report in the output") from None
|
||||
return [line for line in lines[start + 1 : end] if line]
|
||||
|
||||
|
||||
def _stats(lines: list[str]) -> dict[str, tuple[int, str]]:
|
||||
"""Each descriptor's size and last write: ``/proc/900/fd/5 2048 1762250200``."""
|
||||
stats: dict[str, tuple[int, str]] = {}
|
||||
for line in lines:
|
||||
fd, _, rest = line.partition(" ")
|
||||
size, _, mtime = rest.partition(" ")
|
||||
if size.isdigit() and mtime.isdigit():
|
||||
stats[fd] = (int(size), datetime.fromtimestamp(int(mtime), UTC).isoformat())
|
||||
return stats
|
||||
|
||||
|
||||
def _names(lines: list[str]) -> dict[int, str]:
|
||||
names: dict[int, str] = {}
|
||||
for line in lines:
|
||||
pid, _, name = line.partition(" ")
|
||||
if pid.isdigit() and name:
|
||||
names[int(pid)] = name
|
||||
return names
|
||||
|
||||
|
||||
def _journals(units: list[str]) -> list[LogSourceDict]:
|
||||
services = {u[: -len(_SERVICE_SUFFIX)] for u in units if u.endswith(_SERVICE_SUFFIX)}
|
||||
return [
|
||||
{
|
||||
"kind": "journald",
|
||||
"path": None,
|
||||
"unit": unit,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": None,
|
||||
"modified": None,
|
||||
}
|
||||
for unit in sorted(services)
|
||||
]
|
||||
|
||||
|
||||
def _held(lines: list[str]) -> list[tuple[int, str, str]]:
|
||||
"""(pid, descriptor, target) for every log file a process holds, by pid."""
|
||||
held = []
|
||||
for line in lines:
|
||||
fd, _, target = line.partition(" ")
|
||||
match = _FD_RE.match(fd)
|
||||
if match and target.startswith("/") and not _NOT_A_SOURCE.search(target):
|
||||
held.append((int(match.group(1)), fd, target))
|
||||
return sorted(held)
|
||||
|
||||
|
||||
def _writes(lines: list[str]) -> dict[str, bool]:
|
||||
"""Whether each descriptor is open for writing, by its ``/proc/<pid>/fd/<fd>``."""
|
||||
writes: dict[str, bool] = {}
|
||||
for line in lines:
|
||||
match = _FLAGS_RE.match(line)
|
||||
if match:
|
||||
pid, fd, flags = match.groups()
|
||||
writes[f"/proc/{pid}/fd/{fd}"] = int(flags, 8) & _ACCESS_MODE != 0
|
||||
return writes
|
||||
|
||||
|
||||
_Key = tuple[str, str | None, str]
|
||||
|
||||
|
||||
def _holders(lines: list[str], paths: dict[int, str]) -> dict[_Key, list[tuple[int, str]]]:
|
||||
"""The (pid, descriptor) pairs holding each log, by kind, container and path."""
|
||||
holders: dict[_Key, list[tuple[int, str]]] = {}
|
||||
for pid, fd, target in _held(lines):
|
||||
stream = _STREAM_RE.match(target)
|
||||
kind, container = (
|
||||
("container", stream.group(1)) if stream else ("file", _container(paths.get(pid)))
|
||||
)
|
||||
holders.setdefault((kind, container, target), []).append((pid, fd))
|
||||
return holders
|
||||
|
||||
|
||||
def _writer(held: list[tuple[int, str]], writes: dict[str, bool] | None) -> int | None:
|
||||
"""The first process writing; the first holding for a report without flags,
|
||||
as an older command printed it."""
|
||||
if writes is None:
|
||||
return held[0][0]
|
||||
return next((pid for pid, fd in held if writes.get(fd)), None)
|
||||
|
||||
|
||||
def _producer(
|
||||
kind: str, writer: int | None, paths: dict[int, str], names: dict[int, str]
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""The writer's service and name; a stream's service is its container."""
|
||||
if writer is None:
|
||||
return None, None
|
||||
return (_unit(paths.get(writer)) if kind == "file" else None), names.get(writer)
|
||||
|
||||
|
||||
def _files(sections: dict[str, list[str]]) -> list[LogSourceDict]:
|
||||
stats = _stats(sections.get("stat", []))
|
||||
paths = _cgroup_paths(sections.get("cgroups", []))
|
||||
names = _names(sections.get("comm", []))
|
||||
writes = _writes(sections["flags"]) if "flags" in sections else None
|
||||
found: list[LogSourceDict] = []
|
||||
for (kind, container, target), held in _holders(sections.get("open", []), paths).items():
|
||||
writer = _writer(held, writes)
|
||||
if writer is None and kind == "file" and container is not None:
|
||||
continue # a reader's view of a mount, inside its container
|
||||
unit, process = _producer(kind, writer, paths, names)
|
||||
size, modified = next((stats[fd] for _, fd in held if fd in stats), (None, None))
|
||||
found.append(
|
||||
{
|
||||
"kind": kind,
|
||||
"path": target,
|
||||
"unit": unit,
|
||||
"container_id": container,
|
||||
"process": process,
|
||||
"size": size,
|
||||
"modified": modified,
|
||||
}
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
def _order(source: LogSourceDict) -> tuple[str, str, str]:
|
||||
return (source["kind"], source["container_id"] or "", source["path"] or source["unit"] or "")
|
||||
|
||||
|
||||
def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
|
||||
"""Parse what :data:`LOG_SOURCES_COMMAND` printed: whether the host has a
|
||||
journal, and every source, sorted by kind, container and name.
|
||||
|
||||
:raises LogSourcesUnavailable: when the host has no GNU ``find``.
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if _NO_FIND in sections:
|
||||
raise LogSourcesUnavailable("the host's find cannot list open files (no GNU find)")
|
||||
journald = "journald" in sections
|
||||
sources = _journals(sections.get("journald", [])) + _files(sections)
|
||||
return journald, sorted(sources, key=_order)
|
||||
|
||||
|
||||
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
|
||||
journald, sources = parse_log_sources(output)
|
||||
forwarding = parse_log_forwarding(output)
|
||||
return {
|
||||
"attributed": attributed,
|
||||
"journald": journald,
|
||||
"sources": sources,
|
||||
"forwarding": forwarding,
|
||||
}
|
||||
|
||||
|
||||
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
|
||||
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
|
||||
None for a report without the sections, as an older command printed it.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if "rsyslog" not in sections:
|
||||
return None
|
||||
config = "\n".join(
|
||||
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
|
||||
)
|
||||
running = "running" in sections["rsyslog"]
|
||||
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
|
||||
|
||||
|
||||
class LogSourcesMixin:
|
||||
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse
|
||||
are the same everywhere, so they are concrete here, and a driver supplies
|
||||
only :meth:`_run_log_sources_command` -- how a command reaches its host, and
|
||||
how it gains root there. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_log_sources")`` stays truthful.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation."""
|
||||
...
|
||||
|
||||
def get_log_sources(self) -> LogSourcesDict:
|
||||
"""
|
||||
Returns every log the host writes, with the service or container writing it.
|
||||
|
||||
* attributed (bool) - read as root, so every process's files are listed
|
||||
* journald (bool) - the host keeps systemd's journal
|
||||
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
|
||||
* forwarding (dict) - where the logs go beyond the host, see
|
||||
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"attributed": True,
|
||||
"journald": True,
|
||||
"sources": [
|
||||
{"kind": "journald", "path": None, "unit": "nginx",
|
||||
"container_id": None, "process": None, "size": None, "modified": None},
|
||||
{"kind": "file", "path": "/var/log/nginx/access.log", "unit": "nginx",
|
||||
"container_id": None, "process": "nginx", "size": 2048,
|
||||
"modified": "2026-11-04T09:56:40+00:00"},
|
||||
],
|
||||
}
|
||||
|
||||
:raises LogSourcesUnavailable: if the host has no GNU ``find``.
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
|
||||
try:
|
||||
output = self._run_log_sources_command(command, privileged=True)
|
||||
return _reading(output, attributed=True)
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_log_sources_command(command, privileged=False)
|
||||
return _reading(output, attributed=False)
|
||||
@@ -85,8 +85,8 @@ class InterruptedPackageDict(TypedDict):
|
||||
"""A package an interrupted package manager run left halfway."""
|
||||
|
||||
name: str
|
||||
#: dpkg's word for it: "half-installed", "unpacked", "half-configured",
|
||||
#: "triggers-awaited", "triggers-pending", or "reinstall-required".
|
||||
#: The manager's word for it: dpkg's and opkg's ("unpacked", "half-configured", ...),
|
||||
#: rpm's "duplicate", apk's "broken-files" and "broken-script" (``package_managers``).
|
||||
state: str
|
||||
|
||||
|
||||
@@ -111,9 +111,9 @@ class HostStatusDict(TypedDict):
|
||||
reboot_reason: str | None
|
||||
#: True when the host installs updates on its own (unattended-upgrades, dnf-automatic).
|
||||
auto_updates: bool | None
|
||||
#: True when a package manager run was cut off and has to be finished
|
||||
#: (``dpkg --configure -a``) before anything else installs; None when it
|
||||
#: cannot tell, or a package manager is running right now.
|
||||
#: The package manager read (dpkg, dnf, yum, apk, opkg; None: none) and whether a run
|
||||
#: of it was cut off, left halfway (None: cannot tell, or one is running right now).
|
||||
package_manager: NotRequired[str | None]
|
||||
package_manager_interrupted: NotRequired[bool | None]
|
||||
#: The packages that run left halfway; empty when only its journal says so.
|
||||
interrupted_packages: NotRequired[list[InterruptedPackageDict]]
|
||||
@@ -1440,15 +1440,3 @@ class ManagementProtocolsDict(TypedDict):
|
||||
snmp: ProtocolStateDict
|
||||
kvm: ProtocolStateDict
|
||||
virtual_media: ProtocolStateDict
|
||||
|
||||
|
||||
class SystemIdentityDict(TypedDict):
|
||||
"""A host's SMBIOS identity, what ties it to its BMC.
|
||||
|
||||
See :mod:`napalm_device_types.system_identity`.
|
||||
|
||||
Each value is empty when unknown or a vendor placeholder."""
|
||||
|
||||
system_uuid: str # lowercase
|
||||
product_serial: str
|
||||
board_serial: str
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Whether dnf or yum, apk or opkg was left halfway: the host status's answer for
|
||||
a host without dpkg (NetOrk/netork#851).
|
||||
|
||||
Read the same way as dpkg's state (:mod:`napalm_device_types.host_status`): in the
|
||||
one host status command, without root, and changing nothing. Each manager is asked
|
||||
only where it exists, and while one of its processes runs -- from
|
||||
``/proc/<pid>/comm``, which every Linux has and BusyBox's ``ps`` cannot narrow to --
|
||||
its packages are halfway on purpose and the answer is unknown.
|
||||
|
||||
**dnf and yum** (rpm). A transaction cut off after rpm installed the new version
|
||||
of a package and before it erased the old one leaves both in the rpm database:
|
||||
what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
|
||||
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
|
||||
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
|
||||
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
|
||||
per imported key. The defaults are the ones the host's libdnf was built with, asked
|
||||
of its Python binding, as a distribution may add its own: Oracle Linux adds
|
||||
``kernel-uek`` and its parts, whose packages provide nothing of upstream's list.
|
||||
Upstream's list is the floor where libdnf cannot be asked (dnf 5, yum). Also
|
||||
interrupted is a transaction that never ended: dnf 4's history (libdnf's ``trans``
|
||||
table) holds its row with ``state`` 0 until ``finish()`` writes 1 (done) or 2
|
||||
(error), so the newest row at 0 is one cut off;
|
||||
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
|
||||
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
|
||||
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
|
||||
has no ``sqlite3`` program and a ``dnf`` command loads every plugin and may reach
|
||||
for its repositories. dnf 5's history lives elsewhere and is not read; its
|
||||
duplicates are.
|
||||
|
||||
**apk** (Alpine, OpenWrt from 25.12). apk marks an installed package whose files
|
||||
or scripts failed with ``f:`` in ``/lib/apk/db/installed`` -- ``f`` broken files,
|
||||
``s`` a broken script; ``x`` (xattrs) and ``S`` (the checksum kind) are no fault --
|
||||
and those are exactly the packages ``apk fix`` without arguments picks. A run
|
||||
killed outright leaves no such mark; its next run usually repeats what it left.
|
||||
|
||||
**opkg** (OpenWrt before 25.12). ``/usr/lib/opkg/status`` holds
|
||||
``Status: <want> <flags> <state>`` per package. A run cut off, or a postinst that
|
||||
failed, leaves ``unpacked`` (or ``half-configured``, ``half-installed``), and the
|
||||
``reinstreq`` flag asks for a reinstall; ``opkg configure`` finishes every unpacked
|
||||
package.
|
||||
|
||||
What to do about it is not decided here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from napalm_device_types.models import InterruptedPackageDict
|
||||
|
||||
#: dnf's default ``installonlypkgs``: names and what such packages provide.
|
||||
_INSTALLONLY = (
|
||||
"kernel",
|
||||
"kernel-PAE",
|
||||
"installonlypkg(kernel)",
|
||||
"installonlypkg(kernel-module)",
|
||||
"installonlypkg(vm)",
|
||||
"multiversion(kernel)",
|
||||
)
|
||||
#: Where a host adds its own installonly packages, dnf's then yum's.
|
||||
_DNF_CONF = "/etc/dnf/dnf.conf /etc/yum.conf"
|
||||
DNF_HISTORY = "/var/lib/dnf/history.sqlite"
|
||||
YUM_DIR = "/var/lib/yum"
|
||||
APK_INSTALLED = "/lib/apk/db/installed"
|
||||
OPKG_STATUS = "/usr/lib/opkg/status"
|
||||
|
||||
#: manager -> the process names that leave its packages halfway while they run.
|
||||
_RUNNING = {
|
||||
"rpm": frozenset({"rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"}),
|
||||
"apk": frozenset({"apk"}),
|
||||
"opkg": frozenset({"opkg"}),
|
||||
}
|
||||
_UNFINISHED = "0"
|
||||
_YUM_JOURNAL = "transaction-all."
|
||||
_YUM_GIVEN_UP = ".disabled"
|
||||
_APK_FLAGS = (("f", "broken-files"), ("s", "broken-script"))
|
||||
_OPKG_HALFWAY = frozenset({"unpacked", "half-configured", "half-installed"})
|
||||
_OPKG_REINSTALL = "reinstreq"
|
||||
|
||||
#: The newest transaction in dnf's history as ``<id> <state>``; nothing when unreadable.
|
||||
#: One literal, with :data:`DNF_HISTORY` spelt out: a fixed query, nothing put into it.
|
||||
_DNF_HISTORY_PY = (
|
||||
"import sqlite3; "
|
||||
"c = sqlite3.connect('file:/var/lib/dnf/history.sqlite?mode=ro', uri=True); "
|
||||
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
|
||||
"r and print(*r)"
|
||||
)
|
||||
#: The ``installonlypkgs`` the host's libdnf (dnf 4) was built with, space-separated;
|
||||
#: nothing where there is no libdnf. ``ConfigMain`` is held in a name: an option
|
||||
#: taken from a temporary one outlives it, and reading it then crashes Python.
|
||||
_LIBDNF_INSTALLONLY_PY = (
|
||||
"import libdnf.conf; c = libdnf.conf.ConfigMain(); "
|
||||
"print(c.installonlypkgs().getValueString().replace(',', ' '))"
|
||||
)
|
||||
|
||||
_RUNNING_PART = (
|
||||
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
|
||||
+ "|".join(sorted(set().union(*_RUNNING.values())))
|
||||
+ "' | sort -u; "
|
||||
)
|
||||
_RPM_PART = (
|
||||
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
|
||||
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
|
||||
"rpmpy=; for p in /usr/libexec/platform-python python3; do "
|
||||
"if command -v $p >/dev/null 2>&1; then rpmpy=$p; break; fi; done; "
|
||||
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
|
||||
f'$([ -n "$rpmpy" ] && $rpmpy -c "{_LIBDNF_INSTALLONLY_PY}" 2>/dev/null) '
|
||||
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
|
||||
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
|
||||
"| grep -v ' ' | sort -u; "
|
||||
"echo '[rpm-dupes]'; rpm -qa --qf '%{NAME} %{ARCH}\\n' 2>/dev/null "
|
||||
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
|
||||
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
|
||||
"| head -n 20; "
|
||||
"echo '[dnf-history]'; "
|
||||
f'[ -n "$rpmpy" ] && $rpmpy -c "{_DNF_HISTORY_PY}" 2>/dev/null; fi; '
|
||||
)
|
||||
_APK_PART = (
|
||||
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
|
||||
"awk '/^P:/ {p = substr($0, 3)} /^f:/ {f = substr($0, 3)} "
|
||||
'/^$/ {if (f ~ /[fs]/) print f, p; p = ""; f = ""} END {if (f ~ /[fs]/) print f, p}\' '
|
||||
f"{APK_INSTALLED} | head -n 50; fi; "
|
||||
)
|
||||
_OPKG_PART = (
|
||||
f"if command -v opkg >/dev/null 2>&1 && [ -r {OPKG_STATUS} ]; then echo '[opkg-audit]'; "
|
||||
"awk '/^Package:/ {p = $2} /^Status:/ && "
|
||||
"($4 ~ /^(unpacked|half-configured|half-installed)$/ || $3 ~ /reinstreq/) "
|
||||
f"{{print $3, $4, p}}' {OPKG_STATUS} | head -n 50; fi; "
|
||||
)
|
||||
|
||||
#: The part of the host status command that reads them; POSIX ``sh``, BusyBox too.
|
||||
PACKAGE_MANAGERS_COMMAND = _RUNNING_PART + _RPM_PART + _APK_PART + _OPKG_PART
|
||||
|
||||
|
||||
class PackageManagerReading(NamedTuple):
|
||||
"""One package manager's state: halfway or not, None while it is running."""
|
||||
|
||||
manager: str | None
|
||||
interrupted: bool | None
|
||||
packages: list[InterruptedPackageDict]
|
||||
|
||||
|
||||
def _unfinished_transaction(sections: dict[str, list[str]]) -> bool:
|
||||
newest = (sections.get("dnf-history") or [""])[0].split()
|
||||
if len(newest) == 2 and newest[1] == _UNFINISHED:
|
||||
return True
|
||||
return any(
|
||||
name.startswith(_YUM_JOURNAL) and not name.endswith(_YUM_GIVEN_UP)
|
||||
for name in sections.get("yum-journal") or []
|
||||
)
|
||||
|
||||
|
||||
def _rpm(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "rpm-dupes" not in sections:
|
||||
return None
|
||||
manager = (sections.get("rpm-frontend") or ["dnf"])[0]
|
||||
if running & _RUNNING["rpm"]:
|
||||
return PackageManagerReading(manager, None, [])
|
||||
installonly = set(sections.get("rpm-installonly") or [])
|
||||
names = dict.fromkeys(line.split()[0] for line in sections["rpm-dupes"])
|
||||
packages: list[InterruptedPackageDict] = [
|
||||
{"name": name, "state": "duplicate"} for name in names if name not in installonly
|
||||
]
|
||||
return PackageManagerReading(
|
||||
manager, bool(packages) or _unfinished_transaction(sections), packages
|
||||
)
|
||||
|
||||
|
||||
def _apk(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "apk-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["apk"]:
|
||||
return PackageManagerReading("apk", None, [])
|
||||
packages: list[InterruptedPackageDict] = []
|
||||
for line in sections["apk-audit"]:
|
||||
flags, _, name = line.partition(" ")
|
||||
name = name.strip()
|
||||
packages += [
|
||||
{"name": name, "state": state} for flag, state in _APK_FLAGS if name and flag in flags
|
||||
]
|
||||
return PackageManagerReading("apk", bool(packages), packages)
|
||||
|
||||
|
||||
def _opkg_package(line: str) -> InterruptedPackageDict | None:
|
||||
parts = line.split()
|
||||
if len(parts) != 3:
|
||||
return None
|
||||
flags, state, name = parts
|
||||
if _OPKG_REINSTALL in flags.split(","):
|
||||
return {"name": name, "state": "reinstall-required"}
|
||||
return {"name": name, "state": state} if state in _OPKG_HALFWAY else None
|
||||
|
||||
|
||||
def _opkg(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "opkg-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["opkg"]:
|
||||
return PackageManagerReading("opkg", None, [])
|
||||
packages = [p for p in map(_opkg_package, sections["opkg-audit"]) if p is not None]
|
||||
return PackageManagerReading("opkg", bool(packages), packages)
|
||||
|
||||
|
||||
def parse_package_managers(sections: dict[str, list[str]]) -> list[PackageManagerReading]:
|
||||
"""What :data:`PACKAGE_MANAGERS_COMMAND` found, one reading per manager it asked."""
|
||||
running = set(sections.get("pm-running") or [])
|
||||
readings = (read(sections, running) for read in (_rpm, _apk, _opkg))
|
||||
return [reading for reading in readings if reading is not None]
|
||||
@@ -36,11 +36,24 @@ DNF_SECURITY_COMMAND = (
|
||||
" echo __DNF_RC=$?; } | cat"
|
||||
)
|
||||
|
||||
#: Read-only. Every advisory whose fix the installed packages carry, one line per
|
||||
#: CVE it names (``CVE SEVERITY/Sec. NEVRA``): the distribution's own word that a
|
||||
#: backport is in, where netOrk's knowledge base holds no records of the release
|
||||
#: (netOrk #2394). dnf 4 only: dnf 5 reads ``--with-cve`` as a filter, and
|
||||
#: Fedora's advisories reference bugs rather than CVEs. From the cache, as
|
||||
#: :data:`DNF_SECURITY_COMMAND` reads it; the output is long (90 000 lines on an
|
||||
#: Oracle Linux 9 host), so a caller reads it seldom.
|
||||
DNF_INSTALLED_CVES_COMMAND = (
|
||||
"{ LC_ALL=C dnf --cacheonly updateinfo list --installed --with-cve --quiet 2>/dev/null;"
|
||||
" echo __DNF_RC=$?; } | cat"
|
||||
)
|
||||
|
||||
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
|
||||
_APT_LINE = re.compile(r"^(\S+)/(\S+)\s+(\S+)\s+\S+\s+\[upgradable from:\s+(\S+)\]")
|
||||
_SECURITY_SUITE = "-security"
|
||||
_APT_STATUS = re.compile(r"^__APT_RC=(\d+)\s*$", re.MULTILINE)
|
||||
_DNF_STATUS = re.compile(r"^__DNF_RC=(\d+)\s*$", re.MULTILINE)
|
||||
_CVE_ID = re.compile(r"^CVE-\d{4}-\d{4,}$")
|
||||
|
||||
|
||||
def _joined_lines(output: str) -> list[str]:
|
||||
@@ -118,3 +131,21 @@ def parse_dnf_security(output: str) -> set[str]:
|
||||
if len(parts) >= 3 and parts[1].endswith("/Sec."):
|
||||
names.add(nevra_name(parts[-1]))
|
||||
return names
|
||||
|
||||
|
||||
def parse_dnf_installed_cves(output: str) -> dict[str, list[str]]:
|
||||
"""The CVEs the installed advisories close, per package name, each once and sorted.
|
||||
|
||||
Parses :data:`DNF_INSTALLED_CVES_COMMAND`'s ``CVE SEVERITY/Sec. NEVRA`` lines;
|
||||
a line that names no CVE (a key prompt, an advisory id) is no statement.
|
||||
|
||||
:raises ValueError: when dnf failed or its exit status never arrived -- "could
|
||||
not read" must not look like "no advisory installed".
|
||||
"""
|
||||
by_name: dict[str, set[str]] = {}
|
||||
listing = _listing(output, _DNF_STATUS, "dnf updateinfo list --installed --with-cve")
|
||||
for line in listing.splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) >= 3 and _CVE_ID.match(parts[0]):
|
||||
by_name.setdefault(nevra_name(parts[-1]), set()).add(parts[0])
|
||||
return {name: sorted(cves) for name, cves in by_name.items()}
|
||||
@@ -0,0 +1,176 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or the rule's parts.
|
||||
|
||||
An application names a zone the IANA way (``Europe/Berlin``). Not every device
|
||||
can: OpenWrt hands ``system.timezone`` to libc as ``TZ``, which wants the rule
|
||||
itself (``CET-1CEST,M3.5.0,M10.5.0/3``), and a switch wants a UTC offset and the
|
||||
summer-time rule. A driver translates with these helpers.
|
||||
|
||||
The rule is the footer of the zone's TZif file (version 2 and later): the line
|
||||
after the last transition that says how the zone goes on. OpenWrt's LuCI table is
|
||||
built from the same footers. The files come from the ``tzdata`` package, a
|
||||
declared dependency, so every driver reads the same database whatever its host
|
||||
ships.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from functools import lru_cache
|
||||
from importlib.resources import files
|
||||
from typing import TypedDict
|
||||
|
||||
#: An IANA name: letters, digits, ``_+-`` in ``/``-separated parts. No dots, so
|
||||
#: nothing reaches outside the database.
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
|
||||
_ABBR = r"(?:[A-Za-z]{3,}|<[A-Za-z0-9+-]{3,}>)"
|
||||
_OFFSET = r"[+-]?\d{1,2}(?::\d{1,2}){0,2}"
|
||||
_DATE = r"(?:J\d{1,3}|\d{1,3}|M\d{1,2}\.\d\.\d)(?:/[+-]?\d{1,3}(?::\d{1,2}){0,2})?"
|
||||
#: ``std offset [dst [offset] [,start,end]]`` (POSIX.1, 8.3), and a bare ``UTC`` or
|
||||
#: ``GMT``, which OpenWrt itself writes by default.
|
||||
_POSIX_TZ = re.compile(
|
||||
rf"(?:UTC|GMT)|{_ABBR}{_OFFSET}(?:{_ABBR}(?:{_OFFSET})?(?:,{_DATE},{_DATE})?)?"
|
||||
)
|
||||
|
||||
_PARTS = re.compile(
|
||||
rf"(?P<std>{_ABBR})(?P<offset>{_OFFSET})"
|
||||
rf"(?:(?P<dst>{_ABBR})(?P<dst_offset>{_OFFSET})?"
|
||||
r"(?:,(?P<start>[^,]+),(?P<end>[^,]+))?)?"
|
||||
)
|
||||
_TRANSITION = re.compile(r"M(\d{1,2})\.([1-5])\.([0-6])(?:/([+-]?\d{1,3}(?::\d{1,2}){0,2}))?")
|
||||
#: When a rule names no time of day for a transition: 02:00 local time.
|
||||
_DEFAULT_TRANSITION_TIME = 7200
|
||||
|
||||
|
||||
class DstTransitionDict(TypedDict):
|
||||
"""When summer time starts or ends: a weekday of a week of a month, and a time."""
|
||||
|
||||
month: int # 1-12
|
||||
week: int # 1-5; 5 is the month's last such weekday
|
||||
weekday: int # 0 = Sunday ... 6 = Saturday
|
||||
time: int # seconds after local midnight, in the time then in force; may be < 0 or > 24h
|
||||
|
||||
|
||||
class PosixTzDict(TypedDict):
|
||||
"""A POSIX TZ rule in parts (:func:`parse_posix_tz`)."""
|
||||
|
||||
std_name: str # "CET"; a quoted name without its <> ("-03")
|
||||
utc_offset: int # seconds east of UTC in standard time: CET 3600, EST -18000
|
||||
dst_name: str | None # "CEST"; None for a zone without summer time
|
||||
dst_offset: int | None # seconds east of UTC in summer time; None without it
|
||||
dst_start: DstTransitionDict | None # None without summer time or without a rule for it
|
||||
dst_end: DstTransitionDict | None
|
||||
|
||||
|
||||
class UnknownTimezone(ValueError):
|
||||
"""A zone that is not in the zone database (nor, where one is taken, a POSIX TZ rule)."""
|
||||
|
||||
def __init__(self, zone: str) -> None:
|
||||
super().__init__(zone)
|
||||
self.zone = zone
|
||||
|
||||
def __str__(self) -> str:
|
||||
return (
|
||||
f"unknown time zone {self.zone!r}: not a name of the IANA time zone database"
|
||||
" such as Europe/Berlin"
|
||||
)
|
||||
|
||||
|
||||
def is_posix_tz(value: str) -> bool:
|
||||
"""Whether *value* reads as a POSIX TZ string -- the form libc takes in ``TZ``."""
|
||||
return bool(_POSIX_TZ.fullmatch(value))
|
||||
|
||||
|
||||
@lru_cache(maxsize=512)
|
||||
def _footer(zone: str) -> str | None:
|
||||
"""The POSIX rule at the end of *zone*'s TZif file; None for no such zone."""
|
||||
if not _ZONE_NAME.fullmatch(zone):
|
||||
return None
|
||||
tzfile = files("tzdata").joinpath("zoneinfo", *zone.split("/"))
|
||||
if not tzfile.is_file():
|
||||
return None
|
||||
data = tzfile.read_bytes()
|
||||
if not data.startswith(b"TZif") or data[4:5] not in (b"2", b"3", b"4") or data[-1:] != b"\n":
|
||||
return None
|
||||
footer = data[:-1].rsplit(b"\n", 1)[-1].decode("ascii", "replace")
|
||||
return footer if is_posix_tz(footer) else None
|
||||
|
||||
|
||||
def is_iana_zone(zone: str) -> bool:
|
||||
"""Whether *zone* names a zone of the database (``Europe/Berlin``, ``UTC``)."""
|
||||
return _footer(zone) is not None
|
||||
|
||||
|
||||
def posix_tz(zone: str) -> str | None:
|
||||
"""The POSIX TZ string for *zone*: an IANA name's rule, a POSIX string as it is.
|
||||
|
||||
None when *zone* is neither -- a device given it would run on UTC.
|
||||
"""
|
||||
return _footer(zone) or (zone if is_posix_tz(zone) else None)
|
||||
|
||||
|
||||
def _seconds(value: str) -> int:
|
||||
"""``[+-]hh[:mm[:ss]]`` in seconds."""
|
||||
sign = -1 if value.startswith("-") else 1
|
||||
parts = [int(p) for p in value.lstrip("+-").split(":")]
|
||||
hours, minutes, seconds = (parts + [0, 0])[:3]
|
||||
return sign * (hours * 3600 + minutes * 60 + seconds)
|
||||
|
||||
|
||||
def _transition(value: str) -> DstTransitionDict | None:
|
||||
match = _TRANSITION.fullmatch(value)
|
||||
if match is None:
|
||||
return None
|
||||
month, week, weekday, time = match.groups()
|
||||
return {
|
||||
"month": int(month),
|
||||
"week": int(week),
|
||||
"weekday": int(weekday),
|
||||
"time": _DEFAULT_TRANSITION_TIME if time is None else _seconds(time),
|
||||
}
|
||||
|
||||
|
||||
def parse_posix_tz(rule: str) -> PosixTzDict | None:
|
||||
"""*rule* in parts; None for what is not a rule, or one with Julian-day transitions.
|
||||
|
||||
POSIX counts offsets west of UTC (``CET-1``); the parts count them east, as a
|
||||
device's "UTC+01:00" does. Summer time without an offset of its own is one hour
|
||||
ahead of standard time.
|
||||
"""
|
||||
if rule in ("UTC", "GMT"):
|
||||
rule += "0"
|
||||
match = _PARTS.fullmatch(rule)
|
||||
if match is None or not is_posix_tz(rule):
|
||||
return None
|
||||
utc_offset = -_seconds(match["offset"])
|
||||
parts: PosixTzDict = {
|
||||
"std_name": match["std"].strip("<>"),
|
||||
"utc_offset": utc_offset,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
if match["dst"] is None:
|
||||
return parts
|
||||
parts["dst_name"] = match["dst"].strip("<>")
|
||||
parts["dst_offset"] = (
|
||||
utc_offset + 3600 if match["dst_offset"] is None else -_seconds(match["dst_offset"])
|
||||
)
|
||||
if match["start"] is not None:
|
||||
start, end = _transition(match["start"]), _transition(match["end"])
|
||||
if start is None or end is None:
|
||||
return None
|
||||
parts["dst_start"], parts["dst_end"] = start, end
|
||||
return parts
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""The backups running on a device right now.
|
||||
|
||||
A restart aborts every backup that runs on the device it restarts: a
|
||||
hypervisor's backup of its guests, a backup server's backup, sync, verify or
|
||||
garbage collection, a host's own file-level backup (restic, borg and the like).
|
||||
A guest's restart breaks the backup its hypervisor is taking of it. So an
|
||||
application that restarts a device asks first, right before the restart.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a driver mixes this in when it can tell;
|
||||
``hasattr`` tells a caller whether it did. No role base carries it, and a driver
|
||||
without it cannot tell -- which is not the same as "nothing runs".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Literal, TypedDict, get_args
|
||||
|
||||
#: What a running backup does: the backup itself, or one of a backup server's
|
||||
#: jobs (``napalm_device_types.backups.BackupJobKind``).
|
||||
RunningBackupKind = Literal["backup", "sync", "verify", "gc", "prune", "tape"]
|
||||
RUNNING_BACKUP_KINDS: tuple[str, ...] = get_args(RunningBackupKind)
|
||||
|
||||
|
||||
class RunningBackupDict(TypedDict):
|
||||
"""One backup running on the device (``get_running_backups``)."""
|
||||
|
||||
kind: RunningBackupKind
|
||||
job: str | None # the job it runs for, as the platform names it; None: started by hand
|
||||
task: str | None # this run's id on the platform: a task id, a unit, a process id
|
||||
guests: list[str] # the guests (get_vms' ids) it is backing up now; [] for the host's own
|
||||
since: int | None # unix seconds it started; None when the platform does not say
|
||||
|
||||
|
||||
class RunningBackupsMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_running_backups(self) -> list[RunningBackupDict]:
|
||||
"""
|
||||
Returns the backups running on the device now; ``[]`` when none is.
|
||||
|
||||
Every entry runs on this device: restarting the device aborts it.
|
||||
A hypervisor names in ``guests`` the guests a backup is taking right
|
||||
now -- a guest that is locked for its backup counts, with or without
|
||||
a task the platform still shows -- so the application can tell a
|
||||
guest's own restart not to break it.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"kind": "backup",
|
||||
"job": "backup-3f2a9c1e-77d1",
|
||||
"task": "UPID:pve1:0001F2A3:00C4B1D2:6731A0C0:vzdump::root@pam:",
|
||||
"guests": ["101"],
|
||||
"since": 1793660400,
|
||||
},
|
||||
{
|
||||
"kind": "gc",
|
||||
"job": "store1",
|
||||
"task": "UPID:localhost:000004D2:...:garbage_collection:store1:root@pam:",
|
||||
"guests": [],
|
||||
"since": 1793664000,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the device could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
@@ -53,6 +53,8 @@ from typing import (
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
from napalm_device_types.ssh_server_auth import SSHD_EFFECTIVE_COMMAND, SshServerAuthMixin
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
@@ -420,7 +422,7 @@ def _unchanged(
|
||||
return result
|
||||
|
||||
|
||||
class OpenSshServerMixin:
|
||||
class OpenSshServerMixin(SshServerAuthMixin):
|
||||
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH, not by a base class,
|
||||
@@ -430,6 +432,9 @@ class OpenSshServerMixin:
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
|
||||
It carries :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`:
|
||||
a host whose sshd can be hardened can also say how it lets people log in.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
@@ -513,7 +518,7 @@ class OpenSshServerMixin:
|
||||
return parse_sshd_effective(self._sshd_effective_text())
|
||||
|
||||
def _sshd_effective_text(self) -> str:
|
||||
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -T")
|
||||
result = self._ssh_run(SSHD_EFFECTIVE_COMMAND)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
return result.stdout
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
"""How an OpenSSH server lets people log in: the effective settings and sshd's PAM stack.
|
||||
|
||||
Whether a host takes a password, only a key, or a key *and* a second factor is
|
||||
the same question on every host that runs OpenSSH, so the reading lives here,
|
||||
and a driver only mixes in :class:`SshServerAuthMixin` and supplies
|
||||
``run_command``.
|
||||
|
||||
**What sshd uses, not what a file says.** ``sshd -T`` prints the settings after
|
||||
sshd's defaults, ``sshd_config``, every ``Include`` and first-match-wins -- the
|
||||
same reading :class:`~napalm_device_types.ssh_server.OpenSshServerMixin` plans
|
||||
its hardening on, by the same fixed command (:data:`SSHD_EFFECTIVE_COMMAND`). It
|
||||
needs root: without it sshd cannot load its host keys and exits.
|
||||
|
||||
**PAM, as far as it can be read.** With ``UsePAM yes``, keyboard-interactive
|
||||
asks whatever PAM's ``auth`` stack for sshd asks: a password from
|
||||
``pam_unix.so``, a one-time code from an OTP module, or both. The stack is read
|
||||
from ``/etc/pam.d`` in one listing as the login user (the files are
|
||||
world-readable), and followed through ``@include``, ``include`` and
|
||||
``substack``. Each module is reported with its control and arguments as
|
||||
written; which of them is a second factor, and whether ``nullok`` lets a user
|
||||
without one through, is the consumer's call.
|
||||
|
||||
Not read: ``Match`` blocks (``sshd -T`` without ``-C`` prints the global
|
||||
settings), and what a PAM module's own configuration asks for.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: sshd's effective configuration. ``sshd`` lives in ``/usr/sbin``, outside a login
|
||||
#: user's ``PATH``. One fixed command, run as root.
|
||||
SSHD_EFFECTIVE_COMMAND = "PATH=$PATH:/usr/sbin:/sbin sshd -T"
|
||||
|
||||
PAM_DIR = "/etc/pam.d"
|
||||
#: Every file in ``/etc/pam.d``, each after a ``==> path <==`` header (GNU, BSD and
|
||||
#: busybox ``tail`` alike). One fixed command, as the login user.
|
||||
PAM_FILES_COMMAND = f"tail -n +1 -- {PAM_DIR}/*"
|
||||
#: The PAM service sshd runs as.
|
||||
PAM_SERVICE = "sshd"
|
||||
|
||||
_TIMEOUT = 30
|
||||
_HEADER_RE = re.compile(rf"^==> {re.escape(PAM_DIR)}/(?P<name>[^/]+) <==$")
|
||||
#: A service name an include may name: a file in ``/etc/pam.d``, never a path out of it.
|
||||
_SERVICE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
|
||||
_INCLUDE_CONTROLS = ("include", "substack")
|
||||
|
||||
|
||||
class PamAuthEntryDict(TypedDict):
|
||||
"""One line of a PAM ``auth`` stack: ``auth required pam_unix.so nullok``."""
|
||||
|
||||
#: The module's file name, without its directory: ``pam_unix.so``.
|
||||
module: str
|
||||
#: ``required``, ``requisite``, ``sufficient``, ``optional``, or a bracketed
|
||||
#: ``[value=action ...]`` as written.
|
||||
control: str
|
||||
args: list[str]
|
||||
|
||||
|
||||
class SshServerAuthDict(TypedDict):
|
||||
"""How an OpenSSH server lets people log in, as ``get_ssh_server_auth`` read it.
|
||||
|
||||
The global settings ``sshd -T`` prints. A ``Match`` block can change them for
|
||||
some users or addresses; ``sshd -T`` without ``-C`` does not show that.
|
||||
|
||||
``authentication_methods`` is ``AuthenticationMethods``: each inner list is one
|
||||
way in, every method of it required in turn (``publickey,keyboard-interactive``
|
||||
-> ``["publickey", "keyboard-interactive"]``); empty for ``any``, where any one
|
||||
enabled method is enough. A method keeps its submethod
|
||||
(``keyboard-interactive:pam``).
|
||||
|
||||
``use_pam`` is None where sshd has no PAM (OpenBSD). ``pam_auth_stack`` is the
|
||||
``auth`` stack PAM runs for sshd, includes and substacks followed; None when
|
||||
PAM is off or its files gave nothing for sshd.
|
||||
"""
|
||||
|
||||
password_authentication: bool
|
||||
#: ``KbdInteractiveAuthentication``; before OpenSSH 8.7, either it or
|
||||
#: ``ChallengeResponseAuthentication``.
|
||||
kbd_interactive_authentication: bool
|
||||
pubkey_authentication: bool
|
||||
hostbased_authentication: bool
|
||||
#: False also where sshd was built without GSSAPI.
|
||||
gssapi_authentication: bool
|
||||
authentication_methods: list[list[str]]
|
||||
#: ``yes``, ``prohibit-password`` (``without-password``), ``forced-commands-only`` or ``no``.
|
||||
permit_root_login: str
|
||||
use_pam: bool | None
|
||||
pam_auth_stack: list[PamAuthEntryDict] | None
|
||||
|
||||
|
||||
def _flag(settings: Mapping[str, str], keyword: str) -> bool:
|
||||
return settings.get(keyword, "no") == "yes"
|
||||
|
||||
|
||||
def _settings(text: str) -> dict[str, str]:
|
||||
"""``keyword value`` per line, the first one of each keyword."""
|
||||
found: dict[str, str] = {}
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if len(parts) == 2:
|
||||
found.setdefault(parts[0].lower(), parts[1].strip())
|
||||
return found
|
||||
|
||||
|
||||
def _methods(value: str) -> list[list[str]]:
|
||||
if value == "any":
|
||||
return []
|
||||
return [[m for m in alternative.split(",") if m] for alternative in value.split()]
|
||||
|
||||
|
||||
def parse_sshd_auth(text: str) -> SshServerAuthDict:
|
||||
"""The authentication settings from ``sshd -T`` output; ``pam_auth_stack`` left None.
|
||||
|
||||
:raises ValueError: if a setting every OpenSSH prints is missing -- not sshd -T
|
||||
output, or cut short.
|
||||
"""
|
||||
settings = _settings(text)
|
||||
missing = [
|
||||
k
|
||||
for k in ("passwordauthentication", "pubkeyauthentication", "permitrootlogin")
|
||||
if k not in settings
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(f"sshd -T printed no {', '.join(missing)}")
|
||||
use_pam = settings.get("usepam")
|
||||
return {
|
||||
"password_authentication": _flag(settings, "passwordauthentication"),
|
||||
"kbd_interactive_authentication": _flag(settings, "kbdinteractiveauthentication")
|
||||
or _flag(settings, "challengeresponseauthentication"),
|
||||
"pubkey_authentication": _flag(settings, "pubkeyauthentication"),
|
||||
"hostbased_authentication": _flag(settings, "hostbasedauthentication"),
|
||||
"gssapi_authentication": _flag(settings, "gssapiauthentication"),
|
||||
"authentication_methods": _methods(settings.get("authenticationmethods", "any")),
|
||||
"permit_root_login": settings["permitrootlogin"],
|
||||
"use_pam": None if use_pam is None else use_pam == "yes",
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def _pam_tokens(line: str) -> list[str]:
|
||||
"""A PAM line's tokens, a bracketed control (``[success=1 default=ignore]``) as one."""
|
||||
tokens: list[str] = []
|
||||
for part in line.split():
|
||||
if tokens and tokens[-1].startswith("[") and not tokens[-1].endswith("]"):
|
||||
tokens[-1] += " " + part
|
||||
else:
|
||||
tokens.append(part)
|
||||
return tokens
|
||||
|
||||
|
||||
def _pam_entry(line: str) -> PamAuthEntryDict | None:
|
||||
tokens = _pam_tokens(line.split("#", 1)[0])
|
||||
if len(tokens) >= 2 and tokens[0] == "@include":
|
||||
tokens = ["auth", "include", tokens[1]]
|
||||
if len(tokens) < 3 or tokens[0].lstrip("-") != "auth":
|
||||
return None
|
||||
control, target, args = tokens[1], tokens[2], tokens[3:]
|
||||
if control in _INCLUDE_CONTROLS:
|
||||
if not _SERVICE_RE.match(target):
|
||||
return None
|
||||
return {"module": target, "control": control, "args": []}
|
||||
return {"module": target.rsplit("/", 1)[-1], "control": control, "args": args}
|
||||
|
||||
|
||||
def parse_pam_auth(text: str) -> list[PamAuthEntryDict]:
|
||||
"""One PAM file's ``auth`` stack, in order.
|
||||
|
||||
A module by its file name (``pam_unix.so``), with its control and arguments.
|
||||
An include is an entry whose control is ``include`` or ``substack`` and whose
|
||||
module is the service it names: ``@include common-auth`` brings in a whole
|
||||
file, ``auth include`` and ``auth substack`` its ``auth`` lines; both are
|
||||
followed by :func:`pam_auth_stack`. An include that names a path rather than
|
||||
a service is left out.
|
||||
"""
|
||||
return [entry for line in text.splitlines() if (entry := _pam_entry(line.strip()))]
|
||||
|
||||
|
||||
def parse_pam_files(text: str) -> dict[str, str]:
|
||||
""":data:`PAM_FILES_COMMAND`'s output, by file name."""
|
||||
files: dict[str, list[str]] = {}
|
||||
current: list[str] | None = None
|
||||
for line in text.splitlines():
|
||||
header = _HEADER_RE.match(line.strip())
|
||||
if header:
|
||||
current = files.setdefault(header["name"], [])
|
||||
elif current is not None:
|
||||
current.append(line)
|
||||
return {name: "\n".join(lines).strip("\n") + "\n" for name, lines in files.items()}
|
||||
|
||||
|
||||
def pam_auth_stack(
|
||||
files: Mapping[str, str], service: str = PAM_SERVICE
|
||||
) -> list[PamAuthEntryDict] | None:
|
||||
"""*service*'s ``auth`` modules, includes and substacks followed, in stack order.
|
||||
|
||||
A file included twice -- or in a loop -- counts once; an include of a file
|
||||
that is not there adds nothing, as with PAM. None without a file for *service*.
|
||||
"""
|
||||
if service not in files:
|
||||
return None
|
||||
stack: list[PamAuthEntryDict] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def walk(name: str) -> None:
|
||||
if name in seen or name not in files:
|
||||
return
|
||||
seen.add(name)
|
||||
for entry in parse_pam_auth(files[name]):
|
||||
if entry["control"] in _INCLUDE_CONTROLS:
|
||||
walk(entry["module"])
|
||||
else:
|
||||
stack.append(entry)
|
||||
|
||||
walk(service)
|
||||
return stack
|
||||
|
||||
|
||||
class SshServerAuthMixin:
|
||||
"""Adds :meth:`get_ssh_server_auth` to a driver that can run commands on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH --
|
||||
:class:`~napalm_device_types.ssh_server.OpenSshServerMixin` carries it -- so
|
||||
``hasattr(driver, "get_ssh_server_auth")`` stays a truthful answer. Reading,
|
||||
not writing: a platform that generates sshd_config itself can carry it too.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def get_ssh_server_auth(self) -> SshServerAuthDict:
|
||||
"""
|
||||
Returns how sshd lets people log in, from ``sshd -T`` and PAM's sshd stack.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"password_authentication": False,
|
||||
"kbd_interactive_authentication": True,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [["publickey", "keyboard-interactive"]],
|
||||
"permit_root_login": "no",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": [
|
||||
{"module": "pam_google_authenticator.so", "control": "required", "args": []}
|
||||
],
|
||||
}
|
||||
|
||||
:raises RuntimeError: if ``sshd -T`` fails -- no OpenSSH, or no root.
|
||||
"""
|
||||
result = self.run_command(SSHD_EFFECTIVE_COMMAND, privileged=True, timeout=_TIMEOUT)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
auth = parse_sshd_auth(result.stdout)
|
||||
if auth["use_pam"]:
|
||||
auth["pam_auth_stack"] = self._pam_auth_stack()
|
||||
return auth
|
||||
|
||||
def _pam_auth_stack(self) -> list[PamAuthEntryDict] | None:
|
||||
"""sshd's PAM auth stack; None when ``/etc/pam.d`` gave nothing for sshd.
|
||||
|
||||
``tail`` goes on past a file it cannot open and exits 1, so what it did
|
||||
print is used whatever the exit status.
|
||||
"""
|
||||
result = self.run_command(PAM_FILES_COMMAND, privileged=False, timeout=_TIMEOUT)
|
||||
return pam_auth_stack(parse_pam_files(result.stdout))
|
||||
@@ -1,6 +1,7 @@
|
||||
"""The SMBIOS identity of a host: system UUID, product serial, board serial.
|
||||
"""The SMBIOS identity of a host: system UUID, product serial, board serial -- and
|
||||
its system firmware (BIOS or UEFI): vendor, version, release date.
|
||||
|
||||
What ties a server to its BMC. The BMC reports the server's serial number and
|
||||
The identity is what ties a server to its BMC. The BMC reports the server's serial number and
|
||||
system UUID; the host reads the same values from its firmware tables. Both sides
|
||||
read them the same way on every Linux host, so the command, its parse and the
|
||||
placeholder filter live here once, and a driver only carries the command across.
|
||||
@@ -13,27 +14,54 @@ Matching on those would join unrelated machines, so they read as empty.
|
||||
|
||||
**Root.** ``product_uuid`` and ``product_serial`` are readable by root only; without
|
||||
it they come back empty, which is "not known", not a mismatch.
|
||||
|
||||
**The firmware** comes from the same tables in the same round trip, so a driver that
|
||||
reads the identity reads it too. It is a fact about the host, not part of its identity:
|
||||
its placeholders are the vendors' text defaults, never a short value (a version of
|
||||
``1`` is a version), and sysfs's ``MM/DD/YYYY`` release date is given as ISO 8601.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
from datetime import date
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.models import SystemIdentityDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class SystemIdentityDict(TypedDict):
|
||||
"""A host's SMBIOS identity, what ties it to its BMC, and its system firmware.
|
||||
|
||||
Each value is empty when unknown or a vendor placeholder."""
|
||||
|
||||
system_uuid: str # lowercase
|
||||
product_serial: str
|
||||
board_serial: str
|
||||
bios_vendor: str # the system firmware (BIOS or UEFI): its vendor,
|
||||
bios_version: str # its version,
|
||||
bios_date: str # its release date, YYYY-MM-DD
|
||||
|
||||
|
||||
_BEGIN = "IDBEGIN"
|
||||
_END = "IDEND"
|
||||
_DMI = "/sys/class/dmi/id"
|
||||
|
||||
#: Reads the three files in one round trip, framed so a banner or prompt around the
|
||||
#: output cannot shift a field.
|
||||
SYSTEM_IDENTITY_COMMAND = (
|
||||
f"printf '{_BEGIN}\\n%s\\n%s\\n%s\\n{_END}\\n' "
|
||||
f'"$(cat {_DMI}/product_uuid 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/product_serial 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/board_serial 2>/dev/null)"'
|
||||
#: The files read, in the order the output carries them.
|
||||
_FILES = (
|
||||
"product_uuid",
|
||||
"product_serial",
|
||||
"board_serial",
|
||||
"bios_vendor",
|
||||
"bios_version",
|
||||
"bios_date",
|
||||
)
|
||||
|
||||
#: Reads the files in one round trip, framed so a banner or prompt around the output
|
||||
#: cannot shift a field.
|
||||
_FIELDS = "%s\\n" * len(_FILES)
|
||||
SYSTEM_IDENTITY_COMMAND = f"printf '{_BEGIN}\\n{_FIELDS}{_END}\\n' " + " ".join(
|
||||
f'"$(cat {_DMI}/{name} 2>/dev/null)"' for name in _FILES
|
||||
)
|
||||
|
||||
#: Lowercased defaults board vendors leave in place.
|
||||
@@ -80,6 +108,22 @@ def _clean(value: str) -> str:
|
||||
return "" if is_placeholder_identity(value) else value
|
||||
|
||||
|
||||
def _clean_text(value: str) -> str:
|
||||
"""A firmware text, empty for a vendor's text default; a short value stays."""
|
||||
value = value.strip()
|
||||
return "" if value.lower() in _PLACEHOLDERS else value
|
||||
|
||||
|
||||
def _iso_date(value: str) -> str:
|
||||
"""sysfs's ``MM/DD/YYYY`` as ``YYYY-MM-DD``; anything else is no date."""
|
||||
month, _, rest = value.strip().partition("/")
|
||||
day, _, year = rest.partition("/")
|
||||
try:
|
||||
return date(int(year), int(month), int(day)).isoformat()
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
|
||||
def parse_system_identity(output: str) -> SystemIdentityDict:
|
||||
"""The identity from :data:`SYSTEM_IDENTITY_COMMAND`'s output, placeholders empty.
|
||||
|
||||
@@ -91,12 +135,16 @@ def parse_system_identity(output: str) -> SystemIdentityDict:
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("system identity: no intact IDBEGIN/IDEND frame") from None
|
||||
fields = (lines[start:end] + ["", "", ""])[:3]
|
||||
uuid, product_serial, board_serial = (_clean(f) for f in fields)
|
||||
fields = (lines[start:end] + [""] * len(_FILES))[: len(_FILES)]
|
||||
uuid, product_serial, board_serial = (_clean(f) for f in fields[:3])
|
||||
bios_vendor, bios_version, bios_date = fields[3:]
|
||||
return {
|
||||
"system_uuid": uuid.lower(),
|
||||
"product_serial": product_serial,
|
||||
"board_serial": board_serial,
|
||||
"bios_vendor": _clean_text(bios_vendor),
|
||||
"bios_version": _clean_text(bios_version),
|
||||
"bios_date": _iso_date(bios_date),
|
||||
}
|
||||
|
||||
|
||||
@@ -119,12 +167,15 @@ class SystemIdentityMixin:
|
||||
|
||||
def get_system_identity(self) -> SystemIdentityDict:
|
||||
"""
|
||||
Returns the host's SMBIOS identity, each value empty when unknown or a vendor
|
||||
placeholder:
|
||||
Returns the host's SMBIOS identity and system firmware, each value empty when
|
||||
unknown or a vendor placeholder:
|
||||
|
||||
* system_uuid (str) - lowercase
|
||||
* product_serial (str)
|
||||
* board_serial (str)
|
||||
* bios_vendor (str) - the system firmware's vendor
|
||||
* bios_version (str)
|
||||
* bios_date (str) - its release date, ``YYYY-MM-DD``
|
||||
|
||||
:raises ValueError: if the output carried no intact reading.
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
"""A device's time sources and time zone: read them, and set them to what the caller says.
|
||||
|
||||
An application keeps one list of NTP servers and a time zone per site, and aligns a
|
||||
device that differs. :class:`TimeSyncMixin` is the contract for that:
|
||||
|
||||
* ``get_time_sync()`` -- declared only, so ``callable(getattr(driver,
|
||||
"get_time_sync", None))`` is the truthful answer to "can this driver tell";
|
||||
* ``configure_time_sync(ntp_servers=None, timezone=None)`` -- a fix template
|
||||
(:mod:`napalm_device_types.fixes`): ``None`` leaves that value as it is, and the
|
||||
device ends up configured with exactly the servers given, in their order. A driver
|
||||
implements the steps; one whose device takes the change in a single call
|
||||
overrides the method;
|
||||
* ``time_sync_writes`` -- which of the two values the driver can write. A BMC that
|
||||
runs on UTC only writes ``{"ntp_servers"}``; the application never asks it for a
|
||||
zone.
|
||||
|
||||
A driver that inherits the mixin but cannot run it sets ``get_time_sync`` (and
|
||||
``configure_time_sync``) to ``None``.
|
||||
|
||||
**Servers from DHCP count** (NetOrk/netork#2380). A device that takes its time
|
||||
servers from a DHCP lease (option 42) gets them from whoever serves the subnet --
|
||||
usually the site's time giver. ``ntp_servers`` are only the servers configured on
|
||||
the device; ``dhcp_ntp_servers`` are those the device got by DHCP and uses. A write
|
||||
replaces the configured servers and leaves DHCP's alone: it never switches the use of
|
||||
DHCP-provided servers off, and the ``verify`` step does not count them.
|
||||
|
||||
A zone is an IANA name on the wire. A device that wants a POSIX rule or an offset
|
||||
with a summer-time rule gets it from :mod:`napalm_device_types.posix_tz`.
|
||||
"""
|
||||
|
||||
# No ``from __future__ import annotations`` here: TypedDict tells a NotRequired key
|
||||
# only from an evaluated annotation (``TimeSyncDict.__optional_keys__``).
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.posix_tz import UnknownTimezone, is_iana_zone
|
||||
|
||||
#: The values a driver can write; ``time_sync_writes`` is a subset.
|
||||
TIME_SYNC_VALUES: frozenset[str] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
_HOST_NAME = re.compile(
|
||||
r"(?=.{1,253}$)[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?"
|
||||
r"(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*\.?"
|
||||
)
|
||||
|
||||
|
||||
class TimeSyncDict(TypedDict):
|
||||
"""How a device keeps its clock (``get_time_sync``)."""
|
||||
|
||||
ntp_servers: list[str] # configured time sources, in the device's order; [] = none configured
|
||||
timezone: str | None # IANA name ("Europe/Berlin"); None = not readable or no named zone
|
||||
synchronized: bool | None # the clock is synchronised to a source right now; None = unknown
|
||||
daemon: str | None # "chrony", "systemd-timesyncd", "ntpd", "openntpd", ...; None = own client
|
||||
# The servers the device got by DHCP (option 42) and uses, in its order; never in
|
||||
# ntp_servers. [] = none; absent or None = the driver cannot tell.
|
||||
dhcp_ntp_servers: NotRequired[list[str] | None]
|
||||
|
||||
|
||||
def _is_server(name: str) -> bool:
|
||||
try:
|
||||
ipaddress.ip_address(name)
|
||||
except ValueError:
|
||||
return bool(_HOST_NAME.fullmatch(name))
|
||||
return True
|
||||
|
||||
|
||||
def check_ntp_servers(servers: Sequence[str]) -> list[str]:
|
||||
"""*servers* as a driver writes them: stripped, in their order, each once.
|
||||
|
||||
:raises ValueError: for no server at all, or one that is not a host name or
|
||||
an IP address -- before anything reaches a configuration file.
|
||||
:raises TypeError: for a single string instead of a list of them.
|
||||
"""
|
||||
if isinstance(servers, str):
|
||||
raise TypeError("ntp_servers is a list of servers, not one string")
|
||||
kept: list[str] = []
|
||||
for server in (str(s).strip() for s in servers):
|
||||
if not server:
|
||||
continue
|
||||
if not _is_server(server):
|
||||
raise ValueError(f"{server!r} is not a host name or address of an NTP server")
|
||||
if server.lower() not in (k.lower() for k in kept):
|
||||
kept.append(server)
|
||||
if not kept:
|
||||
raise ValueError("name at least one NTP server; None leaves the servers as they are")
|
||||
return kept
|
||||
|
||||
|
||||
class TimeSyncMixin:
|
||||
"""Read and set a device's time sources and time zone; see the module."""
|
||||
|
||||
#: Which of the two values this driver can write: a subset of {"ntp_servers", "timezone"}.
|
||||
#: A BMC that only runs on UTC writes {"ntp_servers"}; an application never asks it for a
|
||||
#: zone. Declared only: a driver that names nothing writes nothing.
|
||||
time_sync_writes: ClassVar[frozenset[str]]
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""How the device keeps its clock now.
|
||||
|
||||
``ntp_servers`` are the sources its configuration names, in its order
|
||||
(a pool by its name, not the addresses it resolved to);
|
||||
``dhcp_ntp_servers`` those a DHCP lease gave it and it uses, apart from
|
||||
them -- left out where the driver cannot tell. ``timezone``
|
||||
is an IANA name, or None where the device keeps an offset and a rule
|
||||
instead of a name. ``daemon`` names the client that keeps the clock on
|
||||
a host that has a choice; None for a device's own.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"ntp_servers": ["ntp1.example.org", "10.0.0.1"],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
"dhcp_ntp_servers": ["192.168.1.1"],
|
||||
}
|
||||
|
||||
:raises Exception: when the device could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# The steps a driver implements:
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
"""Leave the device configured with exactly *servers*, in this order:
|
||||
sources it had configured elsewhere stop counting (commented out,
|
||||
replaced). Servers it gets by DHCP stay, and so does its use of them."""
|
||||
...
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
"""Set the device's zone to *zone*, an IANA name of the database."""
|
||||
...
|
||||
|
||||
def configure_time_sync(
|
||||
self, ntp_servers: Sequence[str] | None = None, timezone: str | None = None
|
||||
) -> FixResultDict:
|
||||
"""Give the device exactly *ntp_servers*, in order, and *timezone*; None leaves one alone.
|
||||
|
||||
What the device gets by DHCP is not touched: it keeps using those servers too.
|
||||
|
||||
The steps: ``ntp_servers``, ``timezone``, then ``verify``, which reads the
|
||||
device again.
|
||||
|
||||
:raises UnknownTimezone: for a *timezone* that is not an IANA name of the
|
||||
database, before anything is written.
|
||||
:raises ValueError: for a value this driver does not write
|
||||
(:attr:`time_sync_writes`), for no server or a malformed one, or for
|
||||
nothing asked -- before anything is written.
|
||||
"""
|
||||
servers = None if ntp_servers is None else check_ntp_servers(ntp_servers)
|
||||
if timezone is not None and not is_iana_zone(timezone):
|
||||
raise UnknownTimezone(timezone)
|
||||
asked = {name for name, v in (("ntp_servers", servers), ("timezone", timezone)) if v}
|
||||
if not asked:
|
||||
raise ValueError("nothing to configure: name ntp_servers, timezone or both")
|
||||
refused = asked - getattr(self, "time_sync_writes", frozenset())
|
||||
if refused:
|
||||
raise ValueError(f"this driver does not write {', '.join(sorted(refused))}")
|
||||
steps: list[FixStep] = []
|
||||
if servers is not None:
|
||||
steps.append(("ntp_servers", lambda: self._write_ntp_servers(servers)))
|
||||
if timezone is not None:
|
||||
steps.append(("timezone", lambda: self._write_time_zone(timezone)))
|
||||
steps.append(("verify", lambda: self._verify_time_sync(servers, timezone)))
|
||||
return run_fix_steps(steps)
|
||||
|
||||
def _verify_time_sync(self, servers: list[str] | None, zone: str | None) -> FixStepOutcomeDict:
|
||||
"""Read the device again: does it now have what was written?
|
||||
|
||||
Servers compare without case; the servers DHCP brought are not among
|
||||
them. A zone is compared where the device reads one back by name.
|
||||
"""
|
||||
now = self.get_time_sync()
|
||||
wrong: list[str] = []
|
||||
if servers is not None and [s.lower() for s in now["ntp_servers"]] != [
|
||||
s.lower() for s in servers
|
||||
]:
|
||||
wrong.append(f"the device has the servers {', '.join(now['ntp_servers']) or 'none'}")
|
||||
if zone is not None and now["timezone"] is not None and now["timezone"] != zone:
|
||||
wrong.append(f"the device is in the time zone {now['timezone']}")
|
||||
if wrong:
|
||||
return {"ok": False, "output": "; ".join(wrong)}
|
||||
return {"ok": True, "output": "the device reads back what was written"}
|
||||
|
||||
|
||||
__all__ = ["TIME_SYNC_VALUES", "TimeSyncDict", "TimeSyncMixin", "check_ntp_servers"]
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
:class:`PosixTimeSyncMixin` implements
|
||||
:class:`~napalm_device_types.time_sync.TimeSyncMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The defaults
|
||||
are Debian's paths; a driver sets its host's.
|
||||
|
||||
**Reading** is one round trip as the login user
|
||||
(:mod:`napalm_device_types._time_sync_reading`): ``timedatectl show`` for the zone
|
||||
(``/etc/timezone`` or ``/etc/localtime``'s target without it), which time sync
|
||||
service is active, and that client's configuration -- chrony's files as chronyd
|
||||
reads them, ``timedatectl show-timesync`` for systemd-timesyncd, ``ntp.conf``,
|
||||
OpenNTPD's ``ntpd.conf``. Synchronised is chronyd's own word where it runs (a
|
||||
selected source in ``chronyc -n sources``), else the kernel's (``NTPSynchronized``).
|
||||
|
||||
What DHCP brought (option 42) is read apart, as ``dhcp_ntp_servers``, and never as a
|
||||
configured server: chrony's sources from the DHCP client's ``sourcedir``
|
||||
(:attr:`~PosixTimeSyncMixin.CHRONY_DHCP_DIRS`, the files as chronyd reads them),
|
||||
systemd-timesyncd's ``LinkNTPServers``. Where ntpd or OpenNTPD runs it is unknown.
|
||||
|
||||
**Writing** runs as root through ``run_command(privileged=True)``; root never reads
|
||||
stdin -- a file is staged by the login user and installed by root.
|
||||
|
||||
* The zone: ``timedatectl set-timezone``.
|
||||
* chrony: the servers go to a file of their own and every other configured source is
|
||||
commented out (:mod:`napalm_device_types._chrony`); the DHCP client's ``sourcedir``
|
||||
stays, and one 4.35.0 commented out comes back. Then ``chronyc reload sources``, or
|
||||
a restart of chronyd where its main configuration changed or it cannot reload.
|
||||
* systemd-timesyncd: :attr:`~PosixTimeSyncMixin.TIMESYNCD_DROPIN` resets ``NTP=``,
|
||||
sets the servers and empties ``FallbackNTP=``, then the service restarts. Servers a
|
||||
network link brings (DHCP through systemd-networkd) stay and count.
|
||||
* ntpd, OpenNTPD, or no client: the servers are read, not written -- the step fails
|
||||
and says so.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._chrony import (
|
||||
COMMENT_MARK,
|
||||
comment_out_command,
|
||||
include_command,
|
||||
plan_chrony,
|
||||
restore_command,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import TIME_ZONE_SH
|
||||
from napalm_device_types._time_sync_reading import (
|
||||
CHRONY_DHCP_DIRS,
|
||||
TIME_SYNC_UNITS,
|
||||
Reading,
|
||||
ReadPaths,
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
parse_reading,
|
||||
parse_time_sync,
|
||||
reading_command,
|
||||
time_sync_of,
|
||||
timesyncd_link_servers,
|
||||
timesyncd_servers,
|
||||
)
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
from napalm_device_types.time_sync import TIME_SYNC_VALUES, TimeSyncDict, TimeSyncMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
_WRITERS = "chrony and systemd-timesyncd"
|
||||
|
||||
#: The head of the systemd-timesyncd drop-in.
|
||||
TIMESYNCD_HEADER = (
|
||||
"# The time sources, exactly these: written by napalm-device-types'\n"
|
||||
"# PosixTimeSyncMixin and replaced on every change.\n"
|
||||
)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)
|
||||
|
||||
|
||||
def timesyncd_dropin(servers: Sequence[str]) -> str:
|
||||
"""The drop-in that leaves systemd-timesyncd with exactly *servers*."""
|
||||
return f"{TIMESYNCD_HEADER}[Time]\nNTP=\nNTP={' '.join(servers)}\nFallbackNTP=\n"
|
||||
|
||||
|
||||
class PosixTimeSyncMixin(TimeSyncMixin):
|
||||
"""Time sync of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = TIME_SYNC_VALUES
|
||||
|
||||
#: ``(unit, daemon)``: the services that keep the clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: ClassVar[tuple[tuple[str, str], ...]] = TIME_SYNC_UNITS
|
||||
#: chrony's main configuration: the first that exists.
|
||||
CHRONY_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/chrony/chrony.conf", "/etc/chrony.conf")
|
||||
#: The servers' file where the main configuration has a ``sourcedir`` of its directory.
|
||||
CHRONY_SOURCES_FILE: ClassVar[str] = "/etc/chrony/sources.d/netork.sources"
|
||||
#: The servers' file otherwise (its name also in a ``confdir``); included if need be.
|
||||
CHRONY_INCLUDE_FILE: ClassVar[str] = "/etc/chrony.d/netork.conf"
|
||||
#: The ``sourcedir`` a DHCP client writes chrony's sources to: read as
|
||||
#: ``dhcp_ntp_servers``, never commented out.
|
||||
CHRONY_DHCP_DIRS: ClassVar[tuple[str, ...]] = CHRONY_DHCP_DIRS
|
||||
TIMESYNCD_DROPIN: ClassVar[str] = "/etc/systemd/timesyncd.conf.d/netork.conf"
|
||||
NTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/ntpsec/ntp.conf", "/etc/ntp.conf")
|
||||
OPENNTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/openntpd/ntpd.conf", "/etc/ntpd.conf")
|
||||
#: ``sh`` that prints the zone without timedatectl: a name, or a path ending in it.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_read(self) -> Reading:
|
||||
paths = ReadPaths(
|
||||
units=self.TIME_SYNC_UNITS,
|
||||
chrony_configs=self.CHRONY_CONFIGS,
|
||||
ntpd_configs=self.NTPD_CONFIGS,
|
||||
openntpd_configs=self.OPENNTPD_CONFIGS,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
return parse_reading(self.run_command(reading_command(paths), timeout=60).stdout)
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""The host's time sources, zone and client, in one round trip as the login user.
|
||||
|
||||
:raises ValueError: when the reading came back cut short.
|
||||
"""
|
||||
return time_sync_of(self._ts_read(), self.CHRONY_DHCP_DIRS)
|
||||
|
||||
# -- writing ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_root(self, command: str) -> CommandResult:
|
||||
return self.run_command(command, privileged=True, timeout=60)
|
||||
|
||||
def _ts_checked(self, command: str) -> str:
|
||||
result = self._ts_root(command)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{command}: {_said(result) or f'exit {result.exit_code}'}")
|
||||
return _said(result)
|
||||
|
||||
def _ts_install(self, path: str, content: str) -> None:
|
||||
"""*content* to *path* as root, staged by the login user: root reads no stdin."""
|
||||
staged = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
tmp = (staged.stdout.strip().splitlines() or [""])[-1]
|
||||
if staged.exit_code != 0 or not tmp.startswith("/"):
|
||||
raise RuntimeError(f"cannot stage {path}: {_said(staged)}")
|
||||
try:
|
||||
self._ts_checked(shlex.join(["install", "-D", "-m", "644", "--", tmp, path]))
|
||||
finally:
|
||||
self.run_command(shlex.join(["rm", "-f", "--", tmp]), timeout=30)
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
result = self._ts_root(shlex.join(["timedatectl", "set-timezone", zone]))
|
||||
if result.exit_code != 0:
|
||||
return {"ok": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"ok": True, "output": f"time zone set to {zone}"}
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
reading = self._ts_read()
|
||||
if reading.daemon == "chrony":
|
||||
return self._ts_chrony(reading, servers)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return self._ts_timesyncd(servers)
|
||||
runs = f"this host runs {reading.daemon}" if reading.daemon else "none runs here"
|
||||
return {"ok": False, "output": f"time sources are written for {_WRITERS}; {runs}"}
|
||||
|
||||
def _ts_timesyncd(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
self._ts_install(self.TIMESYNCD_DROPIN, timesyncd_dropin(servers))
|
||||
self._ts_checked("systemctl restart systemd-timesyncd.service")
|
||||
return {"ok": True, "output": f"wrote {self.TIMESYNCD_DROPIN}; restarted systemd-timesyncd"}
|
||||
|
||||
def _ts_chrony(self, reading: Reading, servers: list[str]) -> FixStepOutcomeDict:
|
||||
plan = plan_chrony(
|
||||
reading.chrony,
|
||||
servers,
|
||||
sources_file=self.CHRONY_SOURCES_FILE,
|
||||
include_file=self.CHRONY_INCLUDE_FILE,
|
||||
dhcp_dirs=self.CHRONY_DHCP_DIRS,
|
||||
)
|
||||
said = [f"wrote {plan.target}"]
|
||||
self._ts_install(plan.target, plan.content)
|
||||
for path, numbers in plan.restore.items():
|
||||
self._ts_checked(restore_command(path, numbers))
|
||||
said.append(f"gave DHCP's sourcedir back in {path}:{','.join(map(str, numbers))}")
|
||||
for path, numbers in plan.comment_out.items():
|
||||
self._ts_checked(comment_out_command(path, numbers))
|
||||
said.append(f"commented out {path}:{','.join(map(str, numbers))}")
|
||||
if plan.include_in is not None:
|
||||
self._ts_checked(include_command(plan.include_in, plan.target))
|
||||
said.append(f"included it from {plan.include_in}")
|
||||
if not (plan.reload and self._ts_root("chronyc reload sources").exit_code == 0):
|
||||
self._ts_checked(shlex.join(["systemctl", "restart", f"{reading.unit}.service"]))
|
||||
said.append(f"restarted {reading.unit}")
|
||||
else:
|
||||
said.append("reloaded the sources")
|
||||
return {"ok": True, "output": "; ".join(said)}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"COMMENT_MARK",
|
||||
"PosixTimeSyncMixin",
|
||||
"chrony_synchronized",
|
||||
"parse_key_values",
|
||||
"parse_time_sync",
|
||||
"timesyncd_dropin",
|
||||
"timesyncd_link_servers",
|
||||
"timesyncd_servers",
|
||||
]
|
||||
@@ -17,6 +17,15 @@ from typing import TYPE_CHECKING, Any
|
||||
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict, UpdateProgressDict
|
||||
|
||||
|
||||
class PackageIndexMissing(RuntimeError):
|
||||
"""The host holds no package index to read pending updates from.
|
||||
|
||||
Not a fault: OpenWrt keeps its index in RAM and loses it on every reboot.
|
||||
:meth:`UpdateMixin.refresh_available_updates` loads it again, so a caller
|
||||
that sees this can refresh instead of reporting a failed read.
|
||||
"""
|
||||
|
||||
|
||||
class UpdateMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
@@ -37,6 +46,8 @@ class UpdateMixin:
|
||||
read -- no package index yet, an API that did not answer -- raises
|
||||
instead: netOrk keeps "pending since" per package, and an empty
|
||||
list for "don't know" would reset every one of those clocks.
|
||||
Without a package index it raises :exc:`PackageIndexMissing`, which
|
||||
:meth:`refresh_available_updates` cures.
|
||||
|
||||
Example::
|
||||
|
||||
@@ -50,6 +61,28 @@ class UpdateMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_installed_advisory_cves(self) -> dict[str, list[str]]:
|
||||
"""
|
||||
The CVEs the host's installed security advisories close, per package
|
||||
name: the distribution's own word that a fix is in the installed build,
|
||||
although its upstream version did not move (a backport).
|
||||
|
||||
netOrk asks for it only where its knowledge base holds no records of
|
||||
the host's release (netOrk #2394), since the answer can be long. A
|
||||
package manager whose advisories name no CVEs has nothing to say here.
|
||||
|
||||
**An empty dict means no installed advisory names a CVE.** A reader
|
||||
that cannot read raises, as :meth:`get_available_updates` does.
|
||||
``NotImplementedError`` where the host's package manager keeps no such
|
||||
advisories (:data:`~napalm_device_types.package_updates.DNF_INSTALLED_CVES_COMMAND`
|
||||
is dnf 4's).
|
||||
|
||||
Example::
|
||||
|
||||
{"glibc-common": ["CVE-2023-4911"], "curl": ["CVE-2023-38545"]}
|
||||
"""
|
||||
...
|
||||
|
||||
def refresh_available_updates(self) -> dict[str, Any]:
|
||||
"""
|
||||
Refreshes the host's package index, so that :meth:`get_available_updates`
|
||||
|
||||
+2
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.27.0"
|
||||
version = "4.39.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -40,6 +40,7 @@ dependencies = [
|
||||
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
|
||||
"requests>=2.25.0",
|
||||
"typing-extensions>=4.0",
|
||||
"tzdata>=2024.1", # posix_tz reads the zones' TZif files from it
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
},
|
||||
"function_lines": {
|
||||
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 119,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_reservations": 54,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_subnets": 60,
|
||||
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
|
||||
@@ -32,7 +31,7 @@
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1454
|
||||
"napalm_device_types/models.py": 1442
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
"""ConfigRevisionsMixin: a device's own configuration history (NetOrk/netork#2511).
|
||||
|
||||
Some devices keep their earlier configurations themselves. An application restores a
|
||||
configuration there by reverting to the revision that holds it -- on OPNsense the only
|
||||
way, since no API takes an uploaded config.xml. Listing and reading are declared only,
|
||||
so ``callable(getattr(driver, "get_config_revisions", None))`` is the truthful answer;
|
||||
``revert_config_revision`` is a template: it refuses an id that is not in the history,
|
||||
lets the driver revert, and checks that the configuration in place now is the
|
||||
revision's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
ConfigRevertFailed,
|
||||
ConfigRevisionDict,
|
||||
ConfigRevisionsMixin,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
OSDriver,
|
||||
SwitchDriver,
|
||||
UnknownConfigRevision,
|
||||
)
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
assert get_type_hints(ConfigRevisionDict) == {
|
||||
"id": str,
|
||||
"time": str | None,
|
||||
"description": str | None,
|
||||
"user": str | None,
|
||||
"size": int | None,
|
||||
}
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, ConfigRevisionsMixin)
|
||||
|
||||
|
||||
def test_listing_and_reading_are_declared_not_implemented():
|
||||
assert not hasattr(ConfigRevisionsMixin, "get_config_revisions")
|
||||
assert not hasattr(ConfigRevisionsMixin, "get_config_revision")
|
||||
source = inspect.getsource(ConfigRevisionsMixin)
|
||||
assert "def get_config_revisions(self) -> list[ConfigRevisionDict]" in source
|
||||
assert "def get_config_revision(self, revision_id: str) -> str" in source
|
||||
assert "def _revert_to_config_revision(self, revision_id: str) -> None" in source
|
||||
|
||||
|
||||
def test_an_unknown_revision_is_a_value_error_that_names_it():
|
||||
error = UnknownConfigRevision("r9")
|
||||
|
||||
assert isinstance(error, ValueError)
|
||||
assert "'r9'" in str(error)
|
||||
|
||||
|
||||
def test_a_failed_revert_is_a_runtime_error():
|
||||
assert issubclass(ConfigRevertFailed, RuntimeError)
|
||||
|
||||
|
||||
def _revision(revision_id: str, time: str) -> ConfigRevisionDict:
|
||||
return {"id": revision_id, "time": time, "description": None, "user": None, "size": None}
|
||||
|
||||
|
||||
class _Device(ConfigRevisionsMixin):
|
||||
"""A device that keeps its history in memory, newest first, and saves a revert as a
|
||||
new newest revision with a stamp of its own on the first line."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.history = [
|
||||
(_revision("r3", "2026-11-13T21:00:00+00:00"), "#saved 3\nb\n"),
|
||||
(_revision("r2", "2026-11-12T21:00:00+00:00"), "#saved 2\na\n"),
|
||||
(_revision("r1", "2026-11-11T21:00:00+00:00"), "#saved 1\na\n"),
|
||||
]
|
||||
self.calls: list[str] = []
|
||||
|
||||
def get_config_revisions(self) -> list[ConfigRevisionDict]:
|
||||
self.calls.append("list")
|
||||
return [revision for revision, _ in self.history]
|
||||
|
||||
def get_config_revision(self, revision_id: str) -> str:
|
||||
self.calls.append(f"read {revision_id}")
|
||||
for revision, text in self.history:
|
||||
if revision["id"] == revision_id:
|
||||
return text
|
||||
raise UnknownConfigRevision(revision_id)
|
||||
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.calls.append(f"revert {revision_id}")
|
||||
body = self.get_config_revision(revision_id).split("\n", 1)[1]
|
||||
self.history.insert(0, (_revision("r4", "2026-11-14T08:00:00+00:00"), f"#saved 4\n{body}"))
|
||||
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
return text.split("\n", 1)[1]
|
||||
|
||||
|
||||
class TestRevertConfigRevision:
|
||||
def test_reverts_and_answers_with_the_revision_now_in_place(self):
|
||||
device = _Device()
|
||||
|
||||
now = device.revert_config_revision("r1")
|
||||
|
||||
assert now["id"] == "r4"
|
||||
assert device.calls == ["list", "read r1", "revert r1", "read r1", "list", "read r4"]
|
||||
|
||||
def test_the_revision_is_read_before_the_revert_which_may_rotate_it_out(self):
|
||||
class Rotating(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
super()._revert_to_config_revision(revision_id)
|
||||
self.history.pop() # the oldest, r1, drops out
|
||||
|
||||
assert Rotating().revert_config_revision("r1")["id"] == "r4"
|
||||
|
||||
def test_a_revision_not_in_the_history_is_refused_before_anything_changes(self):
|
||||
device = _Device()
|
||||
|
||||
with pytest.raises(UnknownConfigRevision, match="r9"):
|
||||
device.revert_config_revision("r9")
|
||||
|
||||
assert device.calls == ["list"]
|
||||
|
||||
def test_a_revert_the_device_did_not_take_fails(self):
|
||||
# OPNsense answers "reverted" even when it fell back to the previous file.
|
||||
class Deaf(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.calls.append(f"revert {revision_id}")
|
||||
|
||||
with pytest.raises(ConfigRevertFailed, match="r1"):
|
||||
Deaf().revert_config_revision("r1")
|
||||
|
||||
def test_a_history_that_is_empty_afterwards_fails(self):
|
||||
class Forgetful(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.history.clear()
|
||||
|
||||
with pytest.raises(ConfigRevertFailed):
|
||||
Forgetful().revert_config_revision("r1")
|
||||
|
||||
def test_what_a_save_stamps_anew_does_not_count(self):
|
||||
# r2 and r1 differ only in their stamp: reverting to r2 leaves r1's content.
|
||||
device = _Device()
|
||||
|
||||
assert device.revert_config_revision("r2")["id"] == "r4"
|
||||
|
||||
def test_a_refusal_of_the_device_reaches_the_caller(self):
|
||||
class Refusing(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
raise ConfigRevertFailed("read-only user")
|
||||
|
||||
with pytest.raises(ConfigRevertFailed, match="read-only"):
|
||||
Refusing().revert_config_revision("r1")
|
||||
|
||||
|
||||
class TestConfigRevisionContent:
|
||||
def test_by_default_the_whole_text(self):
|
||||
class Plain(ConfigRevisionsMixin):
|
||||
pass
|
||||
|
||||
assert Plain().config_revision_content("a\nb\n") == "a\nb\n"
|
||||
|
||||
def test_without_a_stamp_every_save_differs(self):
|
||||
class Plain(_Device):
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
return ConfigRevisionsMixin.config_revision_content(self, text)
|
||||
|
||||
with pytest.raises(ConfigRevertFailed):
|
||||
Plain().revert_config_revision("r1")
|
||||
@@ -14,7 +14,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin, FirewallDriver, ResidentialGatewayDriver
|
||||
from napalm_device_types.dhcp import normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, normalize_mac
|
||||
from napalm_device_types.models import DhcpReservationDict
|
||||
|
||||
|
||||
@@ -209,6 +209,245 @@ class TestApplyReservationSet:
|
||||
assert lines == ["[commit] no changes"]
|
||||
|
||||
|
||||
class _RefusingDhcpServer(_FakeDhcpServer):
|
||||
"""A server that refuses the reservations for the IPs in *refused*, as Kea does
|
||||
for one whose subnet it does not manage."""
|
||||
|
||||
def __init__(self, live: list[DhcpReservationDict] | None = None, *, refused: set[str]) -> None:
|
||||
super().__init__(live)
|
||||
self.refused = refused
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if reservation["ip"] in self.refused:
|
||||
raise ValueError(f"No Kea-managed subnet contains {reservation['ip']}")
|
||||
return super().apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
|
||||
|
||||
def _three() -> list[DhcpReservationDict]:
|
||||
return [
|
||||
_reservation(mac="aa:bb:cc:dd:ee:01", ip="10.10.20.50"),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:02", ip="10.10.99.5", subnet=""),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:03", ip="10.10.20.52"),
|
||||
]
|
||||
|
||||
|
||||
class TestApplyWithARefusedReservation:
|
||||
"""One reservation the server refuses must not hold back the others, and what was
|
||||
written must take effect: a server that saved a write and was never committed shows
|
||||
it as configured, so the next apply finds nothing to do and never commits it."""
|
||||
|
||||
def test_the_others_are_still_written(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert [entry["reservation"]["ip"] for entry in driver.applied] == [
|
||||
"10.10.20.50",
|
||||
"10.10.20.52",
|
||||
]
|
||||
|
||||
def test_what_was_written_is_committed(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_a_refused_set_still_commits(self):
|
||||
# An earlier apply that stopped before its commit left saved writes behind; a
|
||||
# run whose every write is refused commits too, so those take effect.
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset([_three()[1]]))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_progress_names_the_refused_one_and_why(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError) as raised:
|
||||
for line in driver.apply_dhcp_reservationset(_three()):
|
||||
lines.append(line)
|
||||
|
||||
failed = [line for line in lines if line.startswith("[failed]")]
|
||||
assert failed == [
|
||||
"[failed] aa:bb:cc:dd:ee:02 -> 10.10.99.5: No Kea-managed subnet contains 10.10.99.5"
|
||||
]
|
||||
assert lines[-1] == "[commit] applied 2 add(s), 0 update(s)"
|
||||
assert "1 of 3" in str(raised.value)
|
||||
|
||||
def test_a_refused_update_is_reported_like_an_add(self):
|
||||
driver = _RefusingDhcpServer(
|
||||
[_reservation(uuid="u1", ip="10.10.20.9")], refused={"10.10.20.50"}
|
||||
)
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 1"):
|
||||
list(driver.apply_dhcp_reservationset([_reservation()]))
|
||||
|
||||
assert driver.applied == []
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_the_error_is_a_runtime_error(self):
|
||||
# Callers that catch RuntimeError for a failed apply keep working.
|
||||
assert issubclass(DhcpApplyError, RuntimeError)
|
||||
|
||||
|
||||
#: What Kea answers through OPNsense's API for an address another reservation holds.
|
||||
_KEA_DUPLICATE = (
|
||||
"{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}"
|
||||
)
|
||||
|
||||
|
||||
class _KeaLikeServer(_FakeDhcpServer):
|
||||
"""A server that keeps each address unique at every write, as Kea does: a write
|
||||
taking an address another reservation still holds is refused."""
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if any(r["ip"] == reservation["ip"] and r["uuid"] != uuid for r in self.live):
|
||||
raise RuntimeError(_KEA_DUPLICATE)
|
||||
stored = {**reservation, "uuid": uuid or f"new-{len(self.live)}"}
|
||||
self.live = [r for r in self.live if r["uuid"] != uuid] + [stored] # type: ignore[list-item]
|
||||
return super().apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
|
||||
def ip_of(self, mac: str) -> str:
|
||||
return next(r["ip"] for r in self.live if r["mac"] == mac)
|
||||
|
||||
|
||||
_A, _B, _C, _N = (f"02:00:00:00:00:0{i}" for i in "abcd")
|
||||
|
||||
|
||||
def _at(mac: str, ip: str, uuid: str = "") -> DhcpReservationDict:
|
||||
return _reservation(mac=mac, ip=ip, uuid=uuid, subnet="192.0.2.0/24")
|
||||
|
||||
|
||||
class TestApplyOrderReleasesBeforeTaking:
|
||||
"""An address is released before another change takes it (#106)."""
|
||||
|
||||
def test_an_update_that_frees_an_address_goes_first(self):
|
||||
# The issue: B takes A's address, A moves on; desired lists B first.
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")])
|
||||
|
||||
lines = list(
|
||||
driver.apply_dhcp_reservationset([_at(_B, "192.0.2.10"), _at(_A, "192.0.2.20")])
|
||||
)
|
||||
|
||||
assert driver.ip_of(_A) == "192.0.2.20"
|
||||
assert driver.ip_of(_B) == "192.0.2.10"
|
||||
assert lines == [
|
||||
f"[update] {_A} -> 192.0.2.20 (ip)",
|
||||
f"[update] {_B} -> 192.0.2.10 (ip)",
|
||||
"[commit] applied 0 add(s), 2 update(s)",
|
||||
]
|
||||
|
||||
def test_a_chain_is_written_from_its_free_end(self):
|
||||
live = [
|
||||
_at(_A, "192.0.2.10", "ua"),
|
||||
_at(_B, "192.0.2.11", "ub"),
|
||||
_at(_C, "192.0.2.12", "uc"),
|
||||
]
|
||||
driver = _KeaLikeServer(live)
|
||||
|
||||
list(
|
||||
driver.apply_dhcp_reservationset(
|
||||
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.13")]
|
||||
)
|
||||
)
|
||||
|
||||
assert [e["reservation"]["mac"] for e in driver.applied] == [_C, _B, _A]
|
||||
assert [driver.ip_of(m) for m in (_A, _B, _C)] == ["192.0.2.11", "192.0.2.12", "192.0.2.13"]
|
||||
|
||||
def test_an_add_comes_after_the_update_that_frees_its_address(self):
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
|
||||
|
||||
lines = list(
|
||||
driver.apply_dhcp_reservationset([_at(_N, "192.0.2.10"), _at(_A, "192.0.2.20")])
|
||||
)
|
||||
|
||||
assert driver.ip_of(_N) == "192.0.2.10"
|
||||
assert driver.ip_of(_A) == "192.0.2.20"
|
||||
assert lines[-1] == "[commit] applied 1 add(s), 1 update(s)"
|
||||
|
||||
def test_addresses_match_however_they_are_spelled(self):
|
||||
driver = _FakeDhcpServer([_at(_A, "2001:DB8::10", "ua"), _at(_B, "2001:DB8::11", "ub")])
|
||||
|
||||
list(driver.apply_dhcp_reservationset([_at(_B, "2001:db8::10"), _at(_A, "2001:db8::20")]))
|
||||
|
||||
assert [e["reservation"]["mac"] for e in driver.applied] == [_A, _B]
|
||||
|
||||
def test_an_update_that_keeps_its_address_frees_nothing(self):
|
||||
# A hostname change is no release: the add for A's address stays refused.
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 2"):
|
||||
list(
|
||||
driver.apply_dhcp_reservationset(
|
||||
[_at(_N, "192.0.2.10"), {**_at(_A, "192.0.2.10"), "hostname": "renamed"}]
|
||||
)
|
||||
)
|
||||
|
||||
assert driver.ip_of(_A) == "192.0.2.10"
|
||||
|
||||
|
||||
class TestApplyOrderWithASwap:
|
||||
"""Two reservations trading addresses: no order frees one first. Each is still
|
||||
tried -- a server that checks only at the commit takes them -- and a refusal says it
|
||||
was a swap. Nothing is moved to a temporary address: the contract has no way to know
|
||||
a free one, and a second write that failed would leave the reservation there."""
|
||||
|
||||
def _swap(self) -> list[DhcpReservationDict]:
|
||||
return [_at(_A, "192.0.2.11"), _at(_B, "192.0.2.10"), _at(_N, "192.0.2.30")]
|
||||
|
||||
def _live(self) -> list[DhcpReservationDict]:
|
||||
return [_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")]
|
||||
|
||||
def test_a_refused_swap_is_reported_as_one_and_the_rest_is_written(self):
|
||||
driver = _KeaLikeServer(self._live())
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="2 of 3"):
|
||||
for line in driver.apply_dhcp_reservationset(self._swap()):
|
||||
lines.append(line)
|
||||
|
||||
failed = [line for line in lines if line.startswith("[failed]")]
|
||||
assert len(failed) == 2
|
||||
assert all("Duplicate entry exists" in line for line in failed)
|
||||
assert f"swap with {_B}" in failed[0] and f"swap with {_A}" in failed[1]
|
||||
assert driver.ip_of(_A) == "192.0.2.10" and driver.ip_of(_B) == "192.0.2.11"
|
||||
assert driver.ip_of(_N) == "192.0.2.30"
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_a_server_that_takes_the_swap_gets_it(self):
|
||||
driver = _FakeDhcpServer(self._live())
|
||||
|
||||
lines = list(driver.apply_dhcp_reservationset(self._swap()))
|
||||
|
||||
assert len(driver.applied) == 3
|
||||
assert not any("swap" in line for line in lines)
|
||||
|
||||
def test_a_rotation_of_three_is_a_swap_too(self):
|
||||
driver = _KeaLikeServer([*self._live(), _at(_C, "192.0.2.12", "uc")])
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="3 of 3"):
|
||||
for line in driver.apply_dhcp_reservationset(
|
||||
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.10")]
|
||||
):
|
||||
lines.append(line)
|
||||
|
||||
by_mac = {line.split()[1]: line for line in lines if line.startswith("[failed]")}
|
||||
assert f"swap with {_B}, {_C}" in by_mac[_A]
|
||||
assert f"swap with {_A}, {_C}" in by_mac[_B]
|
||||
assert f"swap with {_A}, {_B}" in by_mac[_C]
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneReservation:
|
||||
def test_a_live_reservation_without_mac_is_never_matched(self):
|
||||
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
|
||||
|
||||
@@ -16,6 +16,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin
|
||||
from napalm_device_types.dhcp import DhcpApplyError
|
||||
from napalm_device_types.models import DhcpSubnetDict
|
||||
|
||||
|
||||
@@ -235,3 +236,92 @@ def test_calling_a_missing_device_method_fails_loudly() -> None:
|
||||
def test_a_subnet_that_cannot_name_one_raises(desired):
|
||||
with pytest.raises(ValueError):
|
||||
_FakeDhcpServer([]).diff_dhcp_subnets(desired)
|
||||
|
||||
|
||||
# ── apply: a refused subnet does not hold back the others (#113) ──────────────
|
||||
|
||||
|
||||
class _RefusingDhcpServer(_FakeDhcpServer):
|
||||
"""A server that refuses the subnets in *refused*, as Kea refuses one that
|
||||
overlaps a subnet it already serves."""
|
||||
|
||||
def __init__(self, live: list[DhcpSubnetDict] | None = None, *, refused: set[str]) -> None:
|
||||
super().__init__(live)
|
||||
self.refused = refused
|
||||
|
||||
def apply_dhcp_subnet(
|
||||
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if subnet["subnet"] in self.refused:
|
||||
raise RuntimeError(f"{subnet['subnet']} overlaps a subnet the server has")
|
||||
return super().apply_dhcp_subnet(subnet, uuid=uuid)
|
||||
|
||||
|
||||
def _three_subnets() -> list[DhcpSubnetDict]:
|
||||
return [
|
||||
_subnet(subnet="192.0.2.0/26"),
|
||||
_subnet(subnet="192.0.2.64/26"),
|
||||
_subnet(subnet="192.0.2.128/26"),
|
||||
]
|
||||
|
||||
|
||||
def test_a_refused_subnet_does_not_stop_the_others() -> None:
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset(_three_subnets()))
|
||||
|
||||
assert [e["subnet"]["subnet"] for e in fake.applied] == ["192.0.2.0/26", "192.0.2.128/26"]
|
||||
|
||||
|
||||
def test_what_was_written_is_committed() -> None:
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset(_three_subnets()))
|
||||
|
||||
assert fake.commits == 1
|
||||
|
||||
|
||||
def test_a_set_whose_every_write_is_refused_still_commits() -> None:
|
||||
# A run that stopped before its commit left saved writes behind, which the device
|
||||
# reports as configured: no later diff would commit them.
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset([_three_subnets()[1]]))
|
||||
|
||||
assert fake.commits == 1
|
||||
|
||||
|
||||
def test_progress_names_the_refused_subnet_and_why() -> None:
|
||||
fake = _RefusingDhcpServer(
|
||||
[_subnet(uuid="dev-1", subnet="192.0.2.128/26")], refused={"192.0.2.64/26"}
|
||||
)
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError) as raised:
|
||||
for line in fake.apply_dhcp_subnetset(
|
||||
[*_three_subnets()[:2], _subnet(subnet="192.0.2.128/26", description="Office")]
|
||||
):
|
||||
lines.append(line)
|
||||
|
||||
assert lines == [
|
||||
"[add] 192.0.2.0/26",
|
||||
"[failed] 192.0.2.64/26: 192.0.2.64/26 overlaps a subnet the server has",
|
||||
"[update] 192.0.2.128/26 (description)",
|
||||
"[commit] applied 1 add(s), 1 update(s)",
|
||||
]
|
||||
assert (raised.value.failed, raised.value.attempted) == (1, 3)
|
||||
assert str(raised.value) == "1 of 3 DHCP subnet change(s) were refused"
|
||||
|
||||
|
||||
def test_a_refused_update_is_counted_like_an_add() -> None:
|
||||
live = _subnet(uuid="dev-1", subnet="192.0.2.0/24")
|
||||
fake = _RefusingDhcpServer([live], refused={"192.0.2.0/24"})
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 1"):
|
||||
list(fake.apply_dhcp_subnetset([_subnet(subnet="192.0.2.0/24", description="Office")]))
|
||||
|
||||
assert fake.applied == []
|
||||
assert fake.commits == 1
|
||||
@@ -13,6 +13,8 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
@@ -25,6 +27,12 @@ from napalm_device_types.host_status import (
|
||||
kernel_reboot_pending,
|
||||
parse_host_status,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
APK_INSTALLED,
|
||||
DNF_HISTORY,
|
||||
OPKG_STATUS,
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
)
|
||||
|
||||
|
||||
def _wire(
|
||||
@@ -146,6 +154,24 @@ class TestKernelRebootPending:
|
||||
"5.14.0-503.11.1.el9_5.x86_64",
|
||||
),
|
||||
("6.9.7-200.fc40.x86_64", ["6.9.7-200.fc40.x86_64"], None),
|
||||
# Oracle UEK: 112.5.1 is newer than 112.5, as rpm counts -- a number after
|
||||
# 5 beats the letters ``el`` (#109, build-arm-01).
|
||||
(
|
||||
"6.12.0-208.112.5.1.el9uek.aarch64",
|
||||
[
|
||||
"6.12.0-207.111.5.6.el9uek.aarch64",
|
||||
"6.12.0-208.112.5.el9uek.aarch64",
|
||||
"6.12.0-208.112.5.1.el9uek.aarch64",
|
||||
],
|
||||
None,
|
||||
),
|
||||
(
|
||||
"6.12.0-208.112.5.el9uek.aarch64",
|
||||
["6.12.0-208.112.5.el9uek.aarch64", "6.12.0-208.112.5.1.el9uek.aarch64"],
|
||||
"6.12.0-208.112.5.1.el9uek.aarch64",
|
||||
),
|
||||
# A release candidate (``~``) comes before its release, in rpm and in dpkg.
|
||||
("6.13~rc5-1-amd64", ["6.13~rc5-1-amd64", "6.13-1-amd64"], "6.13-1-amd64"),
|
||||
],
|
||||
)
|
||||
def test_the_newer_kernel_of_the_running_flavour(self, running, installed, newer):
|
||||
@@ -570,3 +596,411 @@ class TestFilesystems:
|
||||
|
||||
mountpoints = [fs["mountpoint"] for fs in parse_host_status(out.stdout)["filesystems"]]
|
||||
assert "/" in mountpoints
|
||||
|
||||
|
||||
def _other(*sections: tuple[str, tuple[str, ...]], running: tuple[str, ...] = ()) -> str:
|
||||
"""A report from a host without dpkg: the given sections before ``[timers]``."""
|
||||
lines = ["[pm-running]", *running]
|
||||
for name, body in sections:
|
||||
lines += [f"[{name}]", *body]
|
||||
return _wire().replace("[timers]", "\n".join([*lines, "[timers]"]))
|
||||
|
||||
|
||||
class TestInterruptedRpm:
|
||||
"""dnf and yum hosts: a transaction rpm was cut off in leaves the old and the
|
||||
new version of a package both installed (``dnf check --duplicates``,
|
||||
``package-cleanup --dupes``), and dnf's history records a transaction that
|
||||
never ended. netOrk only reports it: there is no one repair that is safe on
|
||||
every host (NetOrk/netork#851, decided 2026-11-07).
|
||||
|
||||
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
|
||||
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
|
||||
1 or ERROR 2), yum's ``transaction-all.*`` journal that
|
||||
``yum-complete-transaction`` reads. Checked against Oracle Linux 9 (build-arm-01,
|
||||
2026-11-12): its UEK kernels are installonly only by libdnf's built-in defaults.
|
||||
"""
|
||||
|
||||
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
|
||||
#: An upgrade of glibc and openssl-libs cut off after the new versions were in.
|
||||
CUT_OFF = ("glibc x86_64", "kernel-uek-core x86_64", "openssl-libs x86_64")
|
||||
|
||||
def _rpm(self, dupes=(), history=("42 1",), journal=(), frontend="dnf", running=()):
|
||||
return _other(
|
||||
("rpm-frontend", (frontend,)),
|
||||
("rpm-installonly", self.INSTALLONLY),
|
||||
("rpm-dupes", dupes),
|
||||
("yum-journal", journal),
|
||||
("dnf-history", history),
|
||||
running=running,
|
||||
)
|
||||
|
||||
def test_a_clean_rpm_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._rpm())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_two_versions_of_a_package_left_by_a_cut_off_transaction(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "glibc", "state": "duplicate"},
|
||||
{"name": "openssl-libs", "state": "duplicate"},
|
||||
]
|
||||
|
||||
def test_kernels_are_installed_side_by_side_on_purpose(self):
|
||||
"""dnf's installonly packages: several versions are the rule, not a fault."""
|
||||
status = parse_host_status(self._rpm(dupes=("kernel-uek-core x86_64",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
def test_a_package_duplicated_in_two_arches_is_named_once(self):
|
||||
status = parse_host_status(self._rpm(dupes=("glibc i686", "glibc x86_64")))
|
||||
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_the_last_transaction_never_finished(self):
|
||||
status = parse_host_status(self._rpm(history=("43 0",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
@pytest.mark.parametrize("history", [("44 1",), ("45 2",), ()])
|
||||
def test_a_finished_or_unreadable_history_says_nothing(self, history):
|
||||
"""A failed transaction (ERROR) did end; an unreadable history is no answer."""
|
||||
assert parse_host_status(self._rpm(history=history))["package_manager_interrupted"] is False
|
||||
|
||||
def test_yum_left_its_transaction_journal(self):
|
||||
journal = ("transaction-all.2026-11-01.10:12.33", "transaction-done.2026-11-01.10:12.33")
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["package_manager"] == "yum"
|
||||
|
||||
def test_a_journal_yum_gave_up_on_is_none(self):
|
||||
"""yum-complete-transaction skips a journal it renamed ``.disabled``."""
|
||||
journal = ("transaction-all.2026-11-01.10:12.33.disabled",)
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
@pytest.mark.parametrize("name", ["rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"])
|
||||
def test_a_running_transaction_makes_it_unknown(self, name):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=(name,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_another_managers_run_does_not(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
|
||||
def test_the_command_asks_rpm_only_where_dnf_or_yum_exists(self):
|
||||
assert "command -v rpm" in HOST_STATUS_COMMAND
|
||||
assert "rpm -qa --qf" in HOST_STATUS_COMMAND
|
||||
assert f"file:{DNF_HISTORY}?mode=ro" in HOST_STATUS_COMMAND
|
||||
|
||||
def test_it_never_repairs(self):
|
||||
for word in ("distro-sync", "--cleandupes", "remove", "yum-complete-transaction", "rpm -e"):
|
||||
assert word not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedApk:
|
||||
"""Alpine and OpenWrt 25.12: apk marks a package whose files or scripts failed
|
||||
in ``/lib/apk/db/installed`` (``f:`` with ``f`` broken files, ``s`` a broken
|
||||
script; ``x`` xattrs and ``S`` the checksum kind are no fault), and ``apk fix``
|
||||
with no arguments is what picks exactly those (apk-tools ``app_fix.c``).
|
||||
netOrk only reports it (NetOrk/netork#851). Shaped after apk-tools'
|
||||
``database.c``; to be checked against a real OpenWrt 25.12 host."""
|
||||
|
||||
def _apk(self, audit=(), running=()):
|
||||
return _other(("apk-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._apk())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "apk"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("flags", "states"),
|
||||
[
|
||||
("s", ["broken-script"]),
|
||||
("f", ["broken-files"]),
|
||||
("fsS", ["broken-files", "broken-script"]),
|
||||
],
|
||||
)
|
||||
def test_a_package_apk_marked_broken(self, flags, states):
|
||||
status = parse_host_status(self._apk(audit=(f"{flags} busybox",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "busybox", "state": state} for state in states
|
||||
]
|
||||
|
||||
def test_a_running_apk_makes_it_unknown(self):
|
||||
status = parse_host_status(self._apk(audit=("s busybox",), running=("apk",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_installed_database(self):
|
||||
assert "/lib/apk/db/installed" in HOST_STATUS_COMMAND
|
||||
assert "apk fix" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedOpkg:
|
||||
"""OpenWrt before 25.12: opkg writes ``Status: <want> <flags> <state>`` per
|
||||
package to ``/usr/lib/opkg/status``; a package whose run was cut off or whose
|
||||
postinst failed stays ``unpacked``, and ``opkg configure`` finishes every
|
||||
unpacked package (opkg-lede ``opkg_cmd.c``, ``pkg.c``). netOrk's update run
|
||||
does that first (NetOrk/netork#851). To be checked against a real router."""
|
||||
|
||||
def _opkg(self, audit=(), running=()):
|
||||
return _other(("opkg-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_status_file_is_not_interrupted(self):
|
||||
status = parse_host_status(self._opkg())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "opkg"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("line", "state"),
|
||||
[
|
||||
("user unpacked luci-app-sqm", "unpacked"),
|
||||
("ok half-configured luci-app-sqm", "half-configured"),
|
||||
("ok half-installed luci-app-sqm", "half-installed"),
|
||||
("reinstreq,user installed luci-app-sqm", "reinstall-required"),
|
||||
],
|
||||
)
|
||||
def test_every_state_an_interrupted_run_leaves(self, line, state):
|
||||
status = parse_host_status(self._opkg(audit=(line,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": state}]
|
||||
|
||||
def test_a_running_opkg_makes_it_unknown(self):
|
||||
status = parse_host_status(self._opkg(audit=("ok unpacked x",), running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_status_file_and_never_configures(self):
|
||||
assert "/usr/lib/opkg/status" in HOST_STATUS_COMMAND
|
||||
assert "opkg configure" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestPackageManager:
|
||||
def test_dpkg_says_so(self):
|
||||
assert parse_host_status(_wire(dpkg=()))["package_manager"] == "dpkg"
|
||||
|
||||
def test_a_busy_dpkg_still_says_which(self):
|
||||
status = parse_host_status(_wire(dpkg=(), busy=True))
|
||||
|
||||
assert (status["package_manager"], status["package_manager_interrupted"]) == (
|
||||
"dpkg",
|
||||
None,
|
||||
)
|
||||
|
||||
def test_no_package_manager_read_is_none(self):
|
||||
status = parse_host_status(_wire())
|
||||
|
||||
assert status["package_manager"] is None
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
|
||||
#: An rpm that answers the two queries the command asks, as on an EL host whose
|
||||
#: glibc upgrade was cut off; and a dnf that only has to exist.
|
||||
_RPM_STUB = """#!/bin/sh
|
||||
case "$*" in
|
||||
*--whatprovides*) printf '%s\\n' kernel-core kernel-core 'no package provides kernel-PAE' ;;
|
||||
-qa*) printf '%s\\n' 'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' \\
|
||||
'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' 'bash x86_64' ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
#: An rpm on an Oracle Linux host: its UEK kernel ``vendor-kernel`` provides nothing
|
||||
#: of dnf's upstream defaults and sits in the database three times, like glibc twice.
|
||||
_RPM_VENDOR_KERNEL_STUB = """#!/bin/sh
|
||||
case "$*" in
|
||||
*--whatprovides*) for a; do [ "$a" = vendor-kernel ] && echo vendor-kernel; done
|
||||
echo 'no package provides kernel-PAE' ;;
|
||||
-qa*) printf '%s\\n' 'vendor-kernel aarch64' 'vendor-kernel aarch64' 'vendor-kernel aarch64' \\
|
||||
'glibc aarch64' 'glibc aarch64' ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
#: libdnf's Python binding, as far as the command asks it: the installonlypkgs a
|
||||
#: distribution built into its libdnf -- here one more than upstream's.
|
||||
_LIBDNF_CONF = """class _Option:
|
||||
def getValueString(self):
|
||||
return "kernel, installonlypkg(kernel), vendor-kernel"
|
||||
|
||||
|
||||
class ConfigMain:
|
||||
def installonlypkgs(self):
|
||||
return _Option()
|
||||
"""
|
||||
|
||||
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
|
||||
_APK_DB = """C:Q1abc=
|
||||
P:alpine-baselayout
|
||||
V:3.6.8-r1
|
||||
A:x86_64
|
||||
F:etc
|
||||
R:fstab
|
||||
|
||||
C:Q1def=
|
||||
P:busybox
|
||||
V:1.37.0-r12
|
||||
A:x86_64
|
||||
f:s
|
||||
F:bin
|
||||
R:busybox
|
||||
|
||||
C:Q1ghi=
|
||||
P:musl
|
||||
V:1.2.5-r9
|
||||
f:S
|
||||
"""
|
||||
|
||||
#: /usr/lib/opkg/status as opkg writes it: luci-app-sqm's postinst never ran.
|
||||
_OPKG_STATUS = """Package: libc
|
||||
Version: 1.2.4-4
|
||||
Depends: libgcc1
|
||||
Status: install hold installed
|
||||
Architecture: mipsel_24kc
|
||||
Installed-Time: 1717000000
|
||||
|
||||
Package: luci-app-sqm
|
||||
Version: 1.5.3-r1
|
||||
Depends: libc, sqm-scripts
|
||||
Status: install user unpacked
|
||||
Architecture: all
|
||||
Installed-Time: 1730000000
|
||||
"""
|
||||
|
||||
|
||||
def _stubs(tmp_path, *names: str, rpm: bool = False) -> dict:
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
for name in names:
|
||||
stub = bin_dir / name
|
||||
stub.write_text(_RPM_STUB if name == "rpm" else "#!/bin/sh\nexit 0\n")
|
||||
stub.chmod(0o755)
|
||||
return {**os.environ, "PATH": f"{bin_dir}:/usr/bin:/bin"}
|
||||
|
||||
|
||||
def _run(command: str, env: dict, shell: tuple[str, ...]) -> str:
|
||||
report = f"printf '%s%s\\n' HSTAT_ BEGIN; {command} printf '%s%s\\n' HSTAT_ END"
|
||||
program = shutil.which(shell[0]) or shell[0]
|
||||
return subprocess.run(
|
||||
[program, *shell[1:], report], capture_output=True, text=True, env=env, timeout=30
|
||||
).stdout
|
||||
|
||||
|
||||
_SHELLS = [("sh", "-c")] + ([("busybox", "sh", "-c")] if shutil.which("busybox") else [])
|
||||
|
||||
|
||||
def _skip_busybox_rpm(shell: tuple[str, ...]) -> None:
|
||||
"""A BusyBox built with its own ``rpm`` applet runs that before any stand-in."""
|
||||
if shell[0] == "busybox":
|
||||
applets = subprocess.run(["busybox", "--list"], capture_output=True, text=True).stdout
|
||||
if "rpm" in applets.split():
|
||||
pytest.skip("this busybox has an rpm applet of its own")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shell", _SHELLS)
|
||||
class TestThePackageManagerCommandOnARealShell:
|
||||
"""The command part run for real, with stand-in programs and files."""
|
||||
|
||||
def test_rpm_duplicates_without_the_kernel_and_the_keys(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "dnf"
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_the_installonly_packages_libdnf_was_built_with(self, tmp_path, shell):
|
||||
"""Oracle builds kernel-uek into libdnf's defaults; its UEK packages provide
|
||||
nothing of upstream's, so only libdnf itself tells (#107)."""
|
||||
_skip_busybox_rpm(shell)
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
(tmp_path / "bin" / "rpm").write_text(_RPM_VENDOR_KERNEL_STUB)
|
||||
libdnf = tmp_path / "py" / "libdnf"
|
||||
libdnf.mkdir(parents=True)
|
||||
(libdnf / "__init__.py").write_text("")
|
||||
(libdnf / "conf.py").write_text(_LIBDNF_CONF)
|
||||
env["PYTHONPATH"] = str(tmp_path / "py")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
history = tmp_path / "history.sqlite"
|
||||
with sqlite3.connect(history) as db:
|
||||
db.execute("CREATE TABLE trans (id INTEGER PRIMARY KEY, state INTEGER NOT NULL)")
|
||||
db.executemany("INSERT INTO trans VALUES (?, ?)", [(41, 1), (42, 0)])
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(DNF_HISTORY, str(history)), env, shell)
|
||||
|
||||
assert "[dnf-history]\n42 0\n" in out
|
||||
|
||||
def test_apk_names_the_package_it_marked(self, tmp_path, shell):
|
||||
installed = tmp_path / "installed"
|
||||
installed.write_text(_APK_DB)
|
||||
env = _stubs(tmp_path, "apk")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(APK_INSTALLED, str(installed)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "apk"
|
||||
assert status["interrupted_packages"] == [{"name": "busybox", "state": "broken-script"}]
|
||||
|
||||
def test_opkg_names_the_unpacked_package(self, tmp_path, shell):
|
||||
status_file = tmp_path / "status"
|
||||
status_file.write_text(_OPKG_STATUS)
|
||||
env = _stubs(tmp_path, "opkg")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(OPKG_STATUS, str(status_file)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "opkg"
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": "unpacked"}]
|
||||
|
||||
def test_a_host_with_none_of_them_reads_none(self, tmp_path, shell):
|
||||
env = _stubs(tmp_path)
|
||||
env["PATH"] = str(tmp_path / "bin")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
assert parse_host_status(out)["package_manager"] is None
|
||||
|
||||
def test_the_running_processes_come_from_proc(self, tmp_path, shell):
|
||||
"""The test runner's own process is there under its own name; no manager is."""
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, _stubs(tmp_path), shell)
|
||||
|
||||
assert "[pm-running]\n" in out
|
||||
|
||||
|
||||
class TestTheWholeCommand:
|
||||
def test_it_runs_on_busybox(self):
|
||||
"""OpenWrt's and Alpine's shell: still one intact report."""
|
||||
if not shutil.which("busybox"):
|
||||
pytest.skip("no busybox here")
|
||||
out = subprocess.run(
|
||||
["busybox", "sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, timeout=30
|
||||
).stdout
|
||||
|
||||
assert parse_host_status(out)["filesystems"]
|
||||
|
||||
def test_the_package_managers_are_part_of_it(self):
|
||||
assert PACKAGE_MANAGERS_COMMAND in HOST_STATUS_COMMAND
|
||||
@@ -23,6 +23,8 @@ from napalm_device_types.local_accounts import (
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
|
||||
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
|
||||
#: The key Alice had before, on the laptop she lost.
|
||||
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOld alice@lost-laptop"
|
||||
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
|
||||
HASH = "$6$salt$" + "x" * 86
|
||||
#: Shell syntax a command for root must not hold: each is one fixed command.
|
||||
@@ -61,6 +63,10 @@ class _Host:
|
||||
self.ran: list[tuple[str, bool]] = []
|
||||
self.staged = 0
|
||||
|
||||
def ran_words(self, first: str) -> list[str]:
|
||||
"""The words of the last command run whose first word is *first*."""
|
||||
return next(w for w in (shlex.split(c) for c, _ in reversed(self.ran)) if w[0] == first)
|
||||
|
||||
def is_admin(self, name: str) -> bool:
|
||||
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
|
||||
return name in self.other_rules or (self.includes_dropins and rule is not None)
|
||||
@@ -200,6 +206,11 @@ class _Host:
|
||||
self.accounts[args[0]].groups.remove(args[1])
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _clearpw(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
self.accounts[args[0]].password = None
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
|
||||
class _Driver(LocalAccountsMixin):
|
||||
ACCOUNT_SHELL = "/bin/sh"
|
||||
@@ -238,6 +249,9 @@ class _Driver(LocalAccountsMixin):
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
return ["leavegroup", name, group]
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
return [["clearpw", name]]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host() -> _Host:
|
||||
@@ -362,7 +376,9 @@ def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driv
|
||||
|
||||
|
||||
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", OLD_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY], clear_password=True)
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
driver.lock_local_account("alice")
|
||||
|
||||
for command in _root_commands(host):
|
||||
@@ -690,3 +706,344 @@ def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
|
||||
[account] = _ExtraFile(host).get_local_accounts(["alice"])
|
||||
|
||||
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
# -- replacing a key (NetOrk/netork#2013) ----------------------------------------------
|
||||
|
||||
|
||||
def _alice_keys(host: _Host) -> str:
|
||||
return host.files.get("/home/alice/.ssh/authorized_keys", "")
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_and_the_persons_own_keys_stay(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n{ALICE_KEY}\n"
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_from_every_key_file_with_any_options_or_comment(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
# The same key with options in front and another comment is the same key.
|
||||
blob = OLD_KEY.rsplit(" ", 1)[0]
|
||||
_add_alice(host, keys=f'from="10.0.0.0/8" {blob} renamed\n# the laptop\n')
|
||||
host.files["/home/alice/.ssh/authorized_keys2"] = f"{OWN_KEY}\n{blob}\n"
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"# the laptop\n{ALICE_KEY}\n"
|
||||
assert host.files["/home/alice/.ssh/authorized_keys2"] == f"{OWN_KEY}\n"
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_a_replaced_key_that_is_not_there_writes_nothing(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.staged == 0
|
||||
assert "none of the keys" in result["steps"][-1]["output"]
|
||||
|
||||
|
||||
def test_the_new_key_is_never_taken_for_a_replaced_one(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY, ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_without_replaces_no_key_goes(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert _alice_keys(host) == f"{OLD_KEY}\n{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("replaces", [["not a key"], [OLD_KEY, ""], OLD_KEY])
|
||||
def test_a_replaced_key_that_is_none_is_refused(
|
||||
host: _Host, driver: _Driver, replaces: list[str]
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=replaces)
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_the_old_key_goes_even_when_the_new_one_could_not_be_added(host: _Host) -> None:
|
||||
class _AddFails(_Driver):
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
return {"ok": False, "output": "disk full"}
|
||||
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = _AddFails(host).add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert next(s for s in result["steps"] if s["step"] == "old keys")["ok"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failing", "ignored", "said"),
|
||||
[
|
||||
({"as cat"}, set(), "cannot read"),
|
||||
({"as install"}, set(), "authorized_keys"),
|
||||
(set(), {"as install"}, "still has"),
|
||||
],
|
||||
)
|
||||
def test_a_replaced_key_that_did_not_go_says_so(
|
||||
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], said: str
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{ALICE_KEY}\n")
|
||||
host.failing |= failing
|
||||
host.ignored |= ignored
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
old = next(s for s in result["steps"] if s["step"] == "old keys")
|
||||
assert not old["ok"] and said in old["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
# -- removing the password (NetOrk/netork#2013) ----------------------------------------
|
||||
|
||||
|
||||
def test_clearing_the_password_leaves_a_key_only_account_without_sudo(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "password", "keys"]
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_new_account_without_a_password_gets_it_cleared_too(host: _Host, driver: _Driver) -> None:
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert ["clearpw", "alice"] == host.ran_words("clearpw")
|
||||
|
||||
|
||||
def test_without_clear_password_the_password_stays(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert "password" not in [s["step"] for s in result["steps"]]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs",
|
||||
[{"password_hash": HASH}, {"admin": True}],
|
||||
)
|
||||
def test_clearing_the_password_refuses_a_password_or_sudo(
|
||||
host: _Host, driver: _Driver, kwargs: dict[str, object]
|
||||
) -> None:
|
||||
# The sudo rule asks for the password: an account without one cannot use it.
|
||||
with pytest.raises(ValueError, match="password"):
|
||||
driver.add_local_account("alice", ALICE_KEY, clear_password=True, **kwargs) # type: ignore[arg-type]
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_password_that_cannot_be_cleared_fails_its_step_only(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "clearpw: failed" in password["output"]
|
||||
# sudo and the key do not wait for it.
|
||||
assert next(s for s in result["steps"] if s["step"] == "sudo")["ok"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host)
|
||||
|
||||
result = _NoClear(host).add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
# -- removing keys without a new one (NetOrk/netork#2013) -------------------------------
|
||||
|
||||
|
||||
def test_removing_keys_takes_only_those_and_leaves_the_account_open(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert not a.locked and a.password == HASH and a.groups == ["alice", "wheel"]
|
||||
assert "/etc/sudoers.d/80-account-alice" in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY]
|
||||
|
||||
|
||||
def test_removing_the_last_key_leaves_an_empty_key_file(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == ""
|
||||
assert not host.accounts["alice"].locked
|
||||
|
||||
|
||||
def test_removing_keys_changes_nothing_as_root(host: _Host, driver: _Driver) -> None:
|
||||
# Root only asks whether a key file is there, and reads the account back
|
||||
# (its lock, sudo -l); the files are written as the person.
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
for command in _root_commands(host):
|
||||
words = shlex.split(command)
|
||||
assert words[0] in {"test", "islocked", "sudo"}, command
|
||||
if words[0] == "sudo":
|
||||
assert words[:3] in (["sudo", "-n", "-u"], ["sudo", "-l", "-U"]), command
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_fails(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert "no account" in result["output"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("keys", [[], ["not a key"], ALICE_KEY])
|
||||
def test_removing_keys_refuses_what_is_no_list_of_keys(
|
||||
host: _Host, driver: _Driver, keys: list[str]
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.remove_local_account_keys("alice", keys)
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("name", "said"), [("netadmin", "logs in"), ("daemon", "system")])
|
||||
def test_removing_keys_never_touches_the_login_user_or_a_system_account(
|
||||
host: _Host, driver: _Driver, name: str, said: str
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=said):
|
||||
driver.remove_local_account_keys(name, [ALICE_KEY])
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_driver_with_its_own_key_store_drops_keys_there(host: _Host) -> None:
|
||||
class _Store(_OwnKeyStore):
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
gone = {k.split()[1] for k in keys}
|
||||
self.store[name] = [k for k in self.store.get(name, []) if k.split()[2] not in gone]
|
||||
return {"ok": True, "output": "dropped some"}
|
||||
|
||||
driver = _Store(host)
|
||||
driver.add_local_account("alice", OLD_KEY)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
|
||||
|
||||
removed = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
assert removed["success"], removed["output"]
|
||||
assert driver.store == {"alice": []}
|
||||
|
||||
|
||||
# -- removing keys and the password at once (NetOrk/netork#2033) ------------------------
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_takes_the_password_and_sudo_too(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
"""A person who deleted their key: the password netOrk no longer knows goes too, and
|
||||
sudo, which asks for it. The account stays open, its other keys stay."""
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys", "password", "sudo"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and not account["admin"] and not account["locked"]
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_runs_each_step_when_one_failed(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
steps = {s["step"]: s for s in result["steps"]}
|
||||
assert steps["keys"]["ok"] and not steps["password"]["ok"]
|
||||
# sudo does not wait for the password: without a known password it has to go anyway.
|
||||
assert steps["sudo"]["ok"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == ""
|
||||
|
||||
|
||||
def test_removing_keys_with_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = _NoClear(host).remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_clears_nothing(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"] and "no account" in result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
|
||||
|
||||
def test_clear_password_is_a_keyword(driver: _Driver) -> None:
|
||||
with pytest.raises(TypeError):
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY], True) # type: ignore[misc]
|
||||
@@ -0,0 +1,189 @@
|
||||
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
|
||||
whether it gets the journal (NetOrk/netork#2053)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
|
||||
|
||||
DEBIAN = """\
|
||||
module(load="imuxsock")
|
||||
module(load="imklog")
|
||||
$FileOwner root
|
||||
$IncludeConfig /etc/rsyslog.d/*.conf
|
||||
*.*;auth,authpriv.none\t\t-/var/log/syslog
|
||||
auth,authpriv.*\t\t\t/var/log/auth.log
|
||||
*.emerg\t\t\t\t:omusrmsg:*
|
||||
"""
|
||||
|
||||
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
|
||||
|
||||
|
||||
def _rules(text: str) -> list[dict]:
|
||||
return parse_rsyslog(text)["rules"]
|
||||
|
||||
|
||||
class TestRules:
|
||||
def test_a_debian_host_forwards_nothing(self):
|
||||
assert _rules(DEBIAN) == []
|
||||
|
||||
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
|
||||
assert _rules(DEBIAN + GRAYLOG) == [
|
||||
{
|
||||
"protocol": "udp",
|
||||
"target": "graylog.example.net",
|
||||
"port": 1514,
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
"template": "RSYSLOG_SyslogProtocol23Format",
|
||||
},
|
||||
]
|
||||
|
||||
def test_a_rule_without_a_template_sends_rsyslogs_default(self):
|
||||
"""None: RFC 3164, the sender's local time without a zone (NetOrk/netork#493)."""
|
||||
(rule,) = _rules("*.* @graylog.example.net:514\n")
|
||||
assert rule["template"] is None
|
||||
|
||||
def test_an_actions_template_parameter(self):
|
||||
(rule,) = _rules(
|
||||
'action(type="omfwd" target="10.0.0.7" Template="RSYSLOG_ForwardFormat")\n'
|
||||
)
|
||||
assert rule["template"] == "RSYSLOG_ForwardFormat"
|
||||
|
||||
def test_the_legacy_default_template_holds_for_the_rules_after_it(self):
|
||||
rules = _rules(
|
||||
"*.* @a\n$ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format\n"
|
||||
"*.* @@b:514\n*.* @c;RSYSLOG_TraditionalForwardFormat\n"
|
||||
)
|
||||
assert [(r["target"], r["template"]) for r in rules] == [
|
||||
("a", None),
|
||||
("b", "RSYSLOG_SyslogProtocol23Format"),
|
||||
("c", "RSYSLOG_TraditionalForwardFormat"),
|
||||
]
|
||||
|
||||
def test_the_forward_modules_template_holds_for_its_actions(self):
|
||||
text = (
|
||||
'module(load="builtin:omfwd" template="RSYSLOG_SyslogProtocol23Format")\n'
|
||||
'action(type="omfwd" target="10.0.0.7")\n'
|
||||
)
|
||||
(rule,) = _rules(text)
|
||||
assert rule["template"] == "RSYSLOG_SyslogProtocol23Format"
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
def test_an_ipv6_target_keeps_its_brackets_off(self):
|
||||
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
|
||||
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
|
||||
|
||||
def test_a_narrower_selector_is_partial_and_says_which(self):
|
||||
(rule,) = _rules("authpriv.* @10.0.0.5\n")
|
||||
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
|
||||
|
||||
def test_excluding_a_facility_is_partial_too(self):
|
||||
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
|
||||
assert rule["complete"] is False
|
||||
|
||||
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
|
||||
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
|
||||
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
|
||||
|
||||
def test_relp_is_its_own_protocol(self):
|
||||
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
|
||||
|
||||
def test_an_action_over_several_lines_is_read_whole(self):
|
||||
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
|
||||
"tcp",
|
||||
"10.0.0.7",
|
||||
6514,
|
||||
True,
|
||||
)
|
||||
|
||||
def test_an_action_on_its_own_takes_every_message(self):
|
||||
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
|
||||
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
|
||||
|
||||
def test_an_action_under_a_condition_is_partial_and_names_it(self):
|
||||
(rule,) = _rules(
|
||||
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
|
||||
)
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == "$programname == 'nginx'"
|
||||
|
||||
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
|
||||
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
|
||||
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
|
||||
|
||||
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
|
||||
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
|
||||
rules = _rules(text)
|
||||
assert [(r["target"], r["complete"]) for r in rules] == [
|
||||
("a", False),
|
||||
("b", False),
|
||||
("c", True),
|
||||
]
|
||||
|
||||
def test_a_property_filter_is_a_partial_selector(self):
|
||||
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == ':programname, isequal, "sshd"'
|
||||
|
||||
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
|
||||
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
|
||||
|
||||
def test_comments_are_no_rules(self):
|
||||
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
|
||||
|
||||
def test_a_hash_inside_quotes_is_no_comment(self):
|
||||
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
|
||||
assert rule["target"] == "10.0.0.13"
|
||||
|
||||
def test_a_local_action_is_no_forwarding(self):
|
||||
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
|
||||
|
||||
|
||||
class TestInputs:
|
||||
def test_an_imfile_input_names_its_file_and_ruleset(self):
|
||||
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
|
||||
assert parse_rsyslog(text)["file_inputs"] == [
|
||||
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
|
||||
]
|
||||
|
||||
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
|
||||
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
|
||||
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
|
||||
|
||||
|
||||
class TestModules:
|
||||
def test_imjournal_reads_the_journal(self):
|
||||
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
|
||||
|
||||
def test_legacy_modload_counts(self):
|
||||
modules = parse_rsyslog("$ModLoad imuxsock\n")
|
||||
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
|
||||
|
||||
|
||||
class TestJournalToSyslog:
|
||||
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
|
||||
the distribution prints commented out in its ``journald.conf``."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("lines", "expected"),
|
||||
[
|
||||
(["#ForwardToSyslog=yes"], True),
|
||||
(["#ForwardToSyslog=no"], False),
|
||||
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
|
||||
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
|
||||
([], None),
|
||||
],
|
||||
)
|
||||
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
|
||||
assert journal_to_syslog(lines) is expected
|
||||
@@ -0,0 +1,394 @@
|
||||
"""get_log_sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person in a log window, CrowdSec,
|
||||
a syslog forwarder -- something has to say which logs there are. Reading that
|
||||
is the same on every Linux host: journald names the units it holds entries for,
|
||||
``/proc/<pid>/fd`` shows the log files running processes have open, and
|
||||
``/proc/<pid>/cgroup`` says which systemd service or container a writer runs in.
|
||||
So all of it lives here once, and a driver only carries the command across
|
||||
(NetOrk/netork#2049).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
|
||||
CID = "4f1c" + "0" * 60
|
||||
OTHER = "9a2b" + "1" * 60
|
||||
JOURNALD = """nginx.service
|
||||
ssh.service
|
||||
postgresql@16-main.service
|
||||
session-3.scope
|
||||
init.scope
|
||||
"""
|
||||
OPEN = f"""/proc/610/fd/7 /var/log/syslog
|
||||
/proc/610/fd/8 /var/log/auth.log
|
||||
/proc/620/fd/9 /var/log/journal/0123abcd/system.journal
|
||||
/proc/900/fd/5 /var/log/nginx/access.log
|
||||
/proc/900/fd/6 /var/log/nginx/error.log
|
||||
/proc/901/fd/5 /var/log/nginx/access.log
|
||||
/proc/700/fd/4 /var/log/old.log (deleted)
|
||||
/proc/1100/fd/30 /var/lib/docker/containers/{CID}/{CID}-json.log
|
||||
/proc/1100/fd/31 /var/lib/docker/containers/{OTHER}/local-logs/container.log
|
||||
/proc/1400/fd/3 /usr/src/paperless/data/log/paperless.log
|
||||
/proc/1500/fd/3 /srv/app/log with space.log
|
||||
/proc/1500/fd/4 /srv/app/Local Storage/leveldb/000004.log
|
||||
/proc/1500/fd/5 /var/lib/rocksdb/012345.log
|
||||
/proc/300/fd/11 /var/log/nginx/access.log
|
||||
/proc/300/fd/12 /mnt/docroot/data/log/paperless.log
|
||||
/proc/300/fd/13 socket:[48213]
|
||||
/proc/300/fd/14 pipe:[48214]
|
||||
/proc/1600/fd/6 /var/log/host/syslog
|
||||
"""
|
||||
#: Each descriptor's ``flags`` from ``/proc/<pid>/fdinfo/<fd>``, octal: the access
|
||||
#: mode (``& 3``) is 1 or 2 for a writer -- 0102001 is O_WRONLY|O_APPEND|O_LARGEFILE
|
||||
#: -- and 0 for a reader (0100000, O_RDONLY|O_LARGEFILE). CrowdSec (300) reads
|
||||
#: nginx's log and a log on a mount; a promtail in a container (1600) reads the
|
||||
#: host's syslog as its own /var/log/host/syslog. 1500 opens its log O_RDWR.
|
||||
FLAGS = """/proc/610/fdinfo/7:flags:\t0102001
|
||||
/proc/610/fdinfo/8:flags:\t0102001
|
||||
/proc/620/fdinfo/9:flags:\t02100002
|
||||
/proc/900/fdinfo/5:flags:\t0102001
|
||||
/proc/900/fdinfo/6:flags:\t0102001
|
||||
/proc/901/fdinfo/5:flags:\t0102001
|
||||
/proc/700/fdinfo/4:flags:\t0102001
|
||||
/proc/1100/fdinfo/30:flags:\t0102001
|
||||
/proc/1100/fdinfo/31:flags:\t0102001
|
||||
/proc/1400/fdinfo/3:flags:\t0102001
|
||||
/proc/1500/fdinfo/3:flags:\t0100002
|
||||
/proc/1500/fdinfo/4:flags:\t0100002
|
||||
/proc/1500/fdinfo/5:flags:\t0100002
|
||||
/proc/300/fdinfo/11:flags:\t0100000
|
||||
/proc/300/fdinfo/12:flags:\t0100000
|
||||
/proc/300/fdinfo/13:flags:\t02004002
|
||||
/proc/300/fdinfo/14:flags:\t02004001
|
||||
/proc/1600/fdinfo/6:flags:\t0100000
|
||||
"""
|
||||
STAT = """/proc/610/fd/7 99000 1762250000
|
||||
/proc/610/fd/8 4100 1762250100
|
||||
/proc/900/fd/5 2048 1762250200
|
||||
/proc/900/fd/6 512 1762250300
|
||||
/proc/901/fd/5 2048 1762250200
|
||||
/proc/1100/fd/30 7000 1762250400
|
||||
/proc/1100/fd/31 300 1762250500
|
||||
/proc/1400/fd/3 0 1762250600
|
||||
/proc/300/fd/11 2048 1762250200
|
||||
/proc/300/fd/12 81920 1762250700
|
||||
"""
|
||||
CGROUPS = f"""610 0::/system.slice/rsyslog.service
|
||||
900 0::/system.slice/nginx.service
|
||||
901 0::/system.slice/nginx.service
|
||||
1100 0::/system.slice/docker.service
|
||||
1400 0::/system.slice/docker-{CID}.scope
|
||||
1500 0::/user.slice/user-1000.slice/session-3.scope
|
||||
300 0::/system.slice/crowdsec.service
|
||||
1600 0::/system.slice/docker-{OTHER}.scope
|
||||
"""
|
||||
COMM = """610 rsyslogd
|
||||
900 nginx
|
||||
901 nginx
|
||||
1100 dockerd
|
||||
1400 python3
|
||||
1500 app
|
||||
300 crowdsec
|
||||
1600 promtail
|
||||
"""
|
||||
|
||||
|
||||
def _wire(
|
||||
*,
|
||||
journald: str | None = JOURNALD,
|
||||
open_: str = OPEN,
|
||||
stat: str = STAT,
|
||||
cgroups: str = CGROUPS,
|
||||
comm: str = COMM,
|
||||
flags: str | None = FLAGS,
|
||||
noise: str = "",
|
||||
forwarding: str = "",
|
||||
) -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
parts = [noise, "LOGS_BEGIN\n"]
|
||||
if journald is not None:
|
||||
parts += ["[journald]\n", journald]
|
||||
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
|
||||
if flags is not None:
|
||||
parts += ["[flags]\n", flags]
|
||||
parts += [forwarding, "LOGS_END\n"]
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _sources(output: str | None = None) -> list:
|
||||
return parse_log_sources(_wire() if output is None else output)[1]
|
||||
|
||||
|
||||
def _by_key(sources: list) -> dict:
|
||||
return {(s["kind"], s["container_id"], s["path"] or s["unit"]): s for s in sources}
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_journald_names_each_service_unit_once_without_its_suffix(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire())
|
||||
units = [s["unit"] for s in sources if s["kind"] == "journald"]
|
||||
assert units == ["nginx", "postgresql@16-main", "ssh"]
|
||||
assert journald is True
|
||||
|
||||
def test_a_host_without_journald_says_so(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire(journald=None))
|
||||
assert journald is False
|
||||
assert not [s for s in sources if s["kind"] == "journald"]
|
||||
|
||||
def test_an_open_file_names_its_writer_size_and_last_write(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
access = sources[("file", None, "/var/log/nginx/access.log")]
|
||||
assert access == {
|
||||
"kind": "file",
|
||||
"path": "/var/log/nginx/access.log",
|
||||
"unit": "nginx",
|
||||
"container_id": None,
|
||||
"process": "nginx",
|
||||
"size": 2048,
|
||||
"modified": "2025-11-04T09:56:40+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_two_processes_hold_is_one_source(self) -> None:
|
||||
paths = [s["path"] for s in _sources() if s["kind"] == "file"]
|
||||
assert paths.count("/var/log/nginx/access.log") == 1
|
||||
|
||||
def test_a_file_without_a_suffix_under_var_log_counts(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
assert sources[("file", None, "/var/log/syslog")]["unit"] == "rsyslog"
|
||||
|
||||
def test_a_file_a_container_writes_carries_the_container_and_its_own_path(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
paperless = sources[("file", CID, "/usr/src/paperless/data/log/paperless.log")]
|
||||
assert paperless["unit"] is None
|
||||
assert paperless["process"] == "python3"
|
||||
assert paperless["size"] == 0
|
||||
|
||||
def test_the_engines_own_log_file_is_the_containers_stream(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
json_file = sources[("container", CID, f"/var/lib/docker/containers/{CID}/{CID}-json.log")]
|
||||
assert json_file["process"] == "dockerd"
|
||||
assert json_file["size"] == 7000
|
||||
local = sources[
|
||||
("container", OTHER, f"/var/lib/docker/containers/{OTHER}/local-logs/container.log")
|
||||
]
|
||||
assert local["modified"] == "2025-11-04T10:01:40+00:00"
|
||||
assert not [s for s in _sources() if "docker" in (s["unit"] or "")]
|
||||
|
||||
def test_journal_files_and_deleted_files_are_not_sources(self) -> None:
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if "/var/log/journal/" in p or p.startswith("/var/log/old")]
|
||||
|
||||
def test_a_databases_write_ahead_log_is_not_a_source(self) -> None:
|
||||
# LevelDB and RocksDB name theirs 000004.log; it holds records, not lines.
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if p.endswith(("000004.log", "012345.log"))]
|
||||
|
||||
def test_a_path_with_a_space_survives(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
spaced = sources[("file", None, "/srv/app/log with space.log")]
|
||||
# The writer runs in a login session, not in a service; stat did not answer.
|
||||
assert spaced["unit"] is None
|
||||
assert spaced["process"] == "app"
|
||||
assert spaced["size"] is None
|
||||
assert spaced["modified"] is None
|
||||
|
||||
def test_sources_come_sorted_by_kind_then_container_then_name(self) -> None:
|
||||
sources = _sources()
|
||||
order = [(s["kind"], s["container_id"] or "", s["path"] or s["unit"]) for s in sources]
|
||||
assert order == sorted(order)
|
||||
|
||||
def test_noise_before_the_frame_is_ignored(self) -> None:
|
||||
assert _sources(_wire(noise="Last login: today\r\n\x1b[0m"))
|
||||
|
||||
def test_a_host_without_gnu_find_raises_rather_than_reading_as_no_files(self) -> None:
|
||||
# busybox find has neither -lname nor -printf: no files would read as none.
|
||||
with pytest.raises(LogSourcesUnavailable, match="find"):
|
||||
parse_log_sources("LOGS_BEGIN\n[journald]\nnginx.service\n[no-find]\nLOGS_END\n")
|
||||
|
||||
def test_a_report_cut_short_raises_rather_than_reading_as_fewer_logs(self) -> None:
|
||||
cut = _wire().replace("LOGS_END\n", "")
|
||||
with pytest.raises(ValueError, match="log source report"):
|
||||
parse_log_sources(cut)
|
||||
|
||||
|
||||
class TestWriters:
|
||||
"""A file is attributed to the process writing it, not to one reading it
|
||||
(NetOrk/netork#2092): CrowdSec, Promtail or a ``tail -F`` holding a log is
|
||||
no producer of it."""
|
||||
|
||||
def test_a_reader_with_the_lower_pid_does_not_take_the_writers_file(self) -> None:
|
||||
access = _by_key(_sources())[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("nginx", "nginx")
|
||||
|
||||
def test_a_file_only_readers_hold_is_listed_without_a_producer(self) -> None:
|
||||
paperless = _by_key(_sources())[("file", None, "/mnt/docroot/data/log/paperless.log")]
|
||||
assert paperless == {
|
||||
"kind": "file",
|
||||
"path": "/mnt/docroot/data/log/paperless.log",
|
||||
"unit": None,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": 81920,
|
||||
"modified": "2025-11-04T10:05:00+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_open_for_reading_and_writing_is_written(self) -> None:
|
||||
spaced = _by_key(_sources())[("file", None, "/srv/app/log with space.log")]
|
||||
assert spaced["process"] == "app"
|
||||
|
||||
def test_a_file_only_a_reader_in_a_container_holds_is_left_out(self) -> None:
|
||||
# Its path is the reader's view of a mount: it names nothing on the host,
|
||||
# and the container does not write it.
|
||||
assert not [s for s in _sources() if s["path"] == "/var/log/host/syslog"]
|
||||
|
||||
def test_sockets_and_pipes_are_no_sources(self) -> None:
|
||||
assert not [s for s in _sources() if not (s["path"] or "/").startswith("/")]
|
||||
|
||||
def test_a_deleted_file_stays_out_though_it_is_written(self) -> None:
|
||||
assert not [s for s in _sources() if (s["path"] or "").startswith("/var/log/old")]
|
||||
|
||||
def test_a_holder_whose_flags_did_not_answer_is_no_writer(self) -> None:
|
||||
# The process ended between the listing and the read of its fdinfo.
|
||||
flags = FLAGS.replace("/proc/900/fdinfo/5:flags:\t0102001\n", "")
|
||||
flags = flags.replace("/proc/901/fdinfo/5:flags:\t0102001\n", "")
|
||||
access = _by_key(_sources(_wire(flags=flags)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == (None, None)
|
||||
|
||||
def test_a_report_without_flags_names_the_first_holder_as_before(self) -> None:
|
||||
# An older command printed no [flags]: who writes cannot be told apart.
|
||||
access = _by_key(_sources(_wire(flags=None)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("crowdsec", "crowdsec")
|
||||
|
||||
|
||||
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
|
||||
FORWARDING = """[rsyslog]
|
||||
running
|
||||
[rsyslog-conf]
|
||||
| module(load="imuxsock")
|
||||
| [comment-looking line]
|
||||
|
|
||||
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
|
||||
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
|
||||
[journald-conf]
|
||||
#ForwardToSyslog=yes
|
||||
"""
|
||||
|
||||
|
||||
class TestForwarding:
|
||||
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
|
||||
|
||||
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None
|
||||
assert forwarding["rsyslog"] is True
|
||||
assert forwarding["journal"] is True
|
||||
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
|
||||
("graylog.lan", 1514, True)
|
||||
]
|
||||
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
|
||||
|
||||
def test_a_config_line_in_brackets_is_no_section(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None and forwarding["rules"]
|
||||
|
||||
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
|
||||
stopped = FORWARDING.replace("running\n", "")
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
|
||||
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
|
||||
def test_a_report_without_the_sections_says_nothing(self) -> None:
|
||||
assert parse_log_forwarding(_wire()) is None
|
||||
|
||||
def test_the_reading_carries_it(self) -> None:
|
||||
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
|
||||
assert reading["forwarding"] is not None
|
||||
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
|
||||
|
||||
|
||||
class _Host(LogSourcesMixin):
|
||||
"""Carries the command the way a driver does; *answers* per privilege."""
|
||||
|
||||
def __init__(self, answers: dict[bool, str]) -> None:
|
||||
self.answers = answers
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
self.calls.append((command, privileged))
|
||||
return self.answers[privileged]
|
||||
|
||||
|
||||
class TestMixin:
|
||||
def test_it_reads_as_root_in_one_sh_call(self) -> None:
|
||||
host = _Host({True: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is True
|
||||
command, privileged = host.calls[0]
|
||||
assert privileged is True
|
||||
assert command.startswith("sh -c ")
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_without_root_it_reads_again_and_says_so(self) -> None:
|
||||
host = _Host({True: "sudo: a password is required\n", False: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is False
|
||||
assert [privileged for _, privileged in host.calls] == [True, False]
|
||||
assert reading["sources"]
|
||||
|
||||
def test_a_host_without_gnu_find_is_not_read_again_unprivileged(self) -> None:
|
||||
host = _Host({True: "LOGS_BEGIN\n[no-find]\nLOGS_END\n"})
|
||||
with pytest.raises(LogSourcesUnavailable):
|
||||
host.get_log_sources()
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_a_host_that_answers_neither_way_raises(self) -> None:
|
||||
host = _Host({True: "", False: "garbage"})
|
||||
with pytest.raises(ValueError):
|
||||
host.get_log_sources()
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_it_is_valid_posix_sh(self) -> None:
|
||||
assert subprocess.run(["sh", "-n", "-c", LOG_SOURCES_COMMAND]).returncode == 0
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_here_a_file_held_for_writing_has_a_producer_and_one_held_for_reading_none(
|
||||
self, tmp_path
|
||||
) -> None:
|
||||
written, read = tmp_path / "written.log", tmp_path / "read.log"
|
||||
read.write_text("one line\n")
|
||||
with written.open("a") as w, read.open() as r:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert w.writable() and r.readable()
|
||||
sources = {s["path"]: s for s in _sources(out)}
|
||||
assert sources[str(written)]["process"] is not None
|
||||
assert sources[str(read)]["process"] is None
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_it_runs_here_and_parses(self, tmp_path) -> None:
|
||||
log = tmp_path / "held.log"
|
||||
log.write_text("one line\n")
|
||||
with log.open() as held:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert held.readable()
|
||||
# This test's own process holds the file, so it must be among the sources.
|
||||
mine = [s for s in _sources(out) if s["path"] == str(log)]
|
||||
assert mine and mine[0]["size"] == len("one line\n")
|
||||
# Whether rsyslog runs here or not, the report says where logs go.
|
||||
assert parse_log_forwarding(out) is not None
|
||||
@@ -18,9 +18,11 @@ import pytest
|
||||
|
||||
from napalm_device_types.package_updates import (
|
||||
APT_UPGRADABLE_COMMAND,
|
||||
DNF_INSTALLED_CVES_COMMAND,
|
||||
DNF_SECURITY_COMMAND,
|
||||
nevra_name,
|
||||
parse_apt_upgradable,
|
||||
parse_dnf_installed_cves,
|
||||
parse_dnf_security,
|
||||
)
|
||||
|
||||
@@ -198,3 +200,75 @@ __DNF_RC=0
|
||||
)
|
||||
def test_the_name_of_a_nevra(self, nevra, name):
|
||||
assert nevra_name(nevra) == name
|
||||
|
||||
|
||||
class TestDnfInstalledCves:
|
||||
"""The CVEs the host's installed advisories close, per package (netOrk #2394).
|
||||
|
||||
Where netOrk's knowledge base holds no records of a release, the host's own
|
||||
advisories are the distribution's word that a backport is in. Lines from real
|
||||
hosts: Oracle Linux 9.8 (dnf 4.14, aarch64) and Amazon Linux 2023 (dnf 4.14).
|
||||
"""
|
||||
|
||||
ORACLE = """\
|
||||
CVE-2024-6501 Low/Sec. NetworkManager-1:1.48.10-2.0.1.el9_5.aarch64
|
||||
CVE-2024-3661 Moderate/Sec. NetworkManager-1:1.48.10-5.0.1.el9_5.aarch64
|
||||
CVE-2023-38545 Important/Sec. curl-7.76.1-23.el9_2.4.aarch64
|
||||
CVE-2023-38545 Important/Sec. curl-7.76.1-26.el9_3.2.aarch64
|
||||
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9.aarch64
|
||||
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9_2.7.aarch64
|
||||
__DNF_RC=0
|
||||
"""
|
||||
|
||||
AMAZON = """\
|
||||
CVE-2022-3715 Medium/Sec. bash-5.2.15-1.amzn2023.0.2.x86_64
|
||||
CVE-2022-23491 Important/Sec. ca-certificates-2021.2.50-1.0.amzn2023.0.6.noarch
|
||||
CVE-2023-32803 Important/Sec. ca-certificates-2023.2.60-1.0.amzn2023.0.2.noarch
|
||||
__DNF_RC=0
|
||||
"""
|
||||
|
||||
def test_cves_per_package_once_each_and_sorted(self):
|
||||
assert parse_dnf_installed_cves(self.ORACLE) == {
|
||||
"NetworkManager": ["CVE-2024-3661", "CVE-2024-6501"],
|
||||
"curl": ["CVE-2023-38545"],
|
||||
"glibc-common": ["CVE-2023-4911"],
|
||||
}
|
||||
|
||||
def test_amazon_linux(self):
|
||||
assert parse_dnf_installed_cves(self.AMAZON) == {
|
||||
"bash": ["CVE-2022-3715"],
|
||||
"ca-certificates": ["CVE-2022-23491", "CVE-2023-32803"],
|
||||
}
|
||||
|
||||
def test_lines_that_name_no_cve_are_left_out(self):
|
||||
"""A GPG prompt a repository without its key prints, an advisory id."""
|
||||
out = (
|
||||
"Importing GPG key 0x6E93CD0C:\n"
|
||||
' Userid : "https://packagecloud.io/crowdsec/crowdsec"\n'
|
||||
"ELSA-2023-5453 Important/Sec. glibc-2.34-60.0.3.el9_2.7.aarch64\n"
|
||||
"__DNF_RC=0\n"
|
||||
)
|
||||
assert parse_dnf_installed_cves(out) == {}
|
||||
|
||||
def test_a_failed_dnf_raises_instead_of_reading_as_no_advisories(self):
|
||||
with pytest.raises(ValueError, match="exit status 1"):
|
||||
parse_dnf_installed_cves("__DNF_RC=1\n")
|
||||
|
||||
def test_output_cut_short_raises(self):
|
||||
with pytest.raises(ValueError, match="no exit status"):
|
||||
parse_dnf_installed_cves(self.ORACLE.replace("__DNF_RC=0\n", ""))
|
||||
|
||||
def test_the_command_lists_installed_advisories_with_their_cves_from_the_cache(self, tmp_path):
|
||||
args = tmp_path / "args"
|
||||
dnf = tmp_path / "dnf"
|
||||
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
|
||||
dnf.chmod(0o755)
|
||||
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
|
||||
out = subprocess.run(
|
||||
["sh", "-c", DNF_INSTALLED_CVES_COMMAND], capture_output=True, text=True, env=env
|
||||
).stdout
|
||||
|
||||
assert {"--cacheonly", "updateinfo", "list", "--installed", "--with-cve"} <= set(
|
||||
args.read_text().split()
|
||||
)
|
||||
assert out == "__DNF_RC=0\n"
|
||||
@@ -0,0 +1,161 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or its parts.
|
||||
|
||||
The wire carries IANA names (``Europe/Berlin``). OpenWrt hands ``TZ`` to libc and
|
||||
wants the rule itself (``CET-1CEST,M3.5.0,M10.5.0/3``); a switch wants a UTC
|
||||
offset and the summer-time rule. The rule is the footer of the zone's TZif file
|
||||
from the ``tzdata`` package (moved here from netOrk's ``core/posix_tz.py`` for
|
||||
NetOrk/netork#2361).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib.resources import files
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
|
||||
BERLIN = "CET-1CEST,M3.5.0,M10.5.0/3"
|
||||
|
||||
|
||||
def _zones() -> list[str]:
|
||||
return files("tzdata").joinpath("zones").read_text().split()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("zone", "rule"),
|
||||
[
|
||||
("Europe/Berlin", BERLIN),
|
||||
("America/New_York", "EST5EDT,M3.2.0,M11.1.0"),
|
||||
("America/Sao_Paulo", "<-03>3"),
|
||||
("Australia/Lord_Howe", "<+1030>-10:30<+11>-11,M10.1.0,M4.1.0"),
|
||||
("Asia/Kolkata", "IST-5:30"),
|
||||
("UTC", "UTC0"),
|
||||
],
|
||||
)
|
||||
def test_an_iana_name_becomes_its_posix_rule(zone, rule):
|
||||
assert posix_tz(zone) == rule
|
||||
assert is_iana_zone(zone)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("rule", [BERLIN, "<+03>-3", "EST5EDT,M3.2.0/2,M11.1.0/2", "UTC0"])
|
||||
def test_a_posix_string_is_taken_as_it_is(rule):
|
||||
assert posix_tz(rule) == rule
|
||||
assert not is_iana_zone(rule)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
["Mars/Olympus_Mons", "Europe", "../../etc/passwd", "/etc/localtime", "Europe/../UTC", ""],
|
||||
)
|
||||
def test_neither_a_zone_nor_a_rule_has_none(value):
|
||||
assert posix_tz(value) is None
|
||||
assert not is_iana_zone(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["UTC", "GMT0", BERLIN, "<-03>3", "IST-5:30"])
|
||||
def test_posix_strings(value):
|
||||
assert is_posix_tz(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["Europe/Berlin", "America/Argentina/Buenos_Aires", "", "CET-"])
|
||||
def test_not_posix_strings(value):
|
||||
assert not is_posix_tz(value)
|
||||
|
||||
|
||||
def test_every_zone_of_the_database_has_a_rule_the_check_accepts():
|
||||
# A caller tells an IANA name from a rule with is_posix_tz; it must never
|
||||
# take the database's own rule for a name.
|
||||
zones = _zones()
|
||||
missing = [z for z in zones if not is_posix_tz(posix_tz(z) or "")]
|
||||
assert len(zones) > 500
|
||||
assert missing == []
|
||||
|
||||
|
||||
def test_the_refusal_names_the_zone_in_plain_english():
|
||||
exc = UnknownTimezone("Mars/Olympus_Mons")
|
||||
|
||||
assert isinstance(exc, ValueError)
|
||||
assert exc.zone == "Mars/Olympus_Mons"
|
||||
assert "Mars/Olympus_Mons" in str(exc)
|
||||
assert "Europe/Berlin" in str(exc)
|
||||
|
||||
|
||||
class TestTheRuleInParts:
|
||||
"""What a switch takes: the standard offset and, where there is one, the summer-time rule."""
|
||||
|
||||
def test_berlin(self):
|
||||
assert parse_posix_tz(BERLIN) == PosixTzDict(
|
||||
std_name="CET",
|
||||
utc_offset=3600,
|
||||
dst_name="CEST",
|
||||
dst_offset=7200,
|
||||
dst_start={"month": 3, "week": 5, "weekday": 0, "time": 7200},
|
||||
dst_end={"month": 10, "week": 5, "weekday": 0, "time": 10800},
|
||||
)
|
||||
|
||||
def test_utc_has_no_summer_time(self):
|
||||
assert parse_posix_tz("UTC0") == {
|
||||
"std_name": "UTC",
|
||||
"utc_offset": 0,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
|
||||
@pytest.mark.parametrize("bare", ["UTC", "GMT"])
|
||||
def test_the_bare_names_openwrt_writes(self, bare):
|
||||
parts = parse_posix_tz(bare)
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_name"]) == (bare, 0, None)
|
||||
|
||||
def test_sao_paulo_has_given_up_summer_time(self):
|
||||
parts = parse_posix_tz(posix_tz("America/Sao_Paulo") or "")
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_offset"]) == (
|
||||
"-03",
|
||||
-10800,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_lord_howe_moves_by_half_an_hour(self):
|
||||
parts = parse_posix_tz(posix_tz("Australia/Lord_Howe") or "")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (37800, 39600)
|
||||
assert parts["dst_start"] == {"month": 10, "week": 1, "weekday": 0, "time": 7200}
|
||||
assert parts["dst_end"] == {"month": 4, "week": 1, "weekday": 0, "time": 7200}
|
||||
|
||||
def test_new_york_is_west_of_utc_and_gains_an_hour_by_default(self):
|
||||
parts = parse_posix_tz("EST5EDT,M3.2.0,M11.1.0")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (-18000, -14400)
|
||||
|
||||
def test_a_transition_time_may_lie_outside_the_day(self):
|
||||
# Greenland's rule: a day's -1:00 and 0:00 (RFC 8536 3.3.1 allows -167..167 hours).
|
||||
parts = parse_posix_tz("<-02>2<-01>,M3.5.0/-1,M10.5.0/0")
|
||||
assert parts is not None
|
||||
assert parts["dst_start"] == {"month": 3, "week": 5, "weekday": 0, "time": -3600}
|
||||
assert parts["dst_end"] == {"month": 10, "week": 5, "weekday": 0, "time": 0}
|
||||
|
||||
def test_offsets_with_minutes_and_seconds(self):
|
||||
parts = parse_posix_tz("<+1245>-12:45<+1345>,M9.5.0/2:45,M4.1.0/3:45")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (45900, 49500)
|
||||
assert parts["dst_start"] == {"month": 9, "week": 5, "weekday": 0, "time": 9900}
|
||||
|
||||
@pytest.mark.parametrize("rule", ["Europe/Berlin", "", "CET-", "CET-1CEST,J60,J300"])
|
||||
def test_what_it_cannot_take_apart_is_none(self, rule):
|
||||
# Julian-day rules are POSIX, but no zone of the database uses them.
|
||||
assert parse_posix_tz(rule) is None
|
||||
|
||||
def test_every_zone_of_the_database_comes_apart(self):
|
||||
broken = [z for z in _zones() if parse_posix_tz(posix_tz(z) or "") is None]
|
||||
assert broken == []
|
||||
@@ -0,0 +1,51 @@
|
||||
"""get_running_backups: the backups running on a device right now.
|
||||
|
||||
An application that restarts a device asks first, right before the restart: a
|
||||
restart aborts a backup that runs on the device -- a hypervisor's vzdump, a
|
||||
backup server's backup, sync, verify or garbage collection, a host's own restic
|
||||
or borg run -- and a guest's restart breaks the backup its hypervisor is taking
|
||||
of it (NetOrk/netork#2318). A driver mixes it in when it can tell; no role
|
||||
demands it, and a driver without it cannot tell.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
HypervisorDriver,
|
||||
OSDriver,
|
||||
RunningBackupDict,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (BackupServerDriver, HypervisorDriver, OSDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, RunningBackupsMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
"""hasattr is the caller's probe: only a driver that can tell answers True."""
|
||||
assert not hasattr(RunningBackupsMixin, "get_running_backups")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(RunningBackupsMixin)
|
||||
assert "def get_running_backups(self) -> list[RunningBackupDict]" in source
|
||||
assert "empty list for a failed read" in source
|
||||
|
||||
|
||||
def test_the_kinds_cover_a_backup_servers_jobs_and_the_backup_itself():
|
||||
assert set(RUNNING_BACKUP_KINDS) == {"backup", "sync", "verify", "gc", "prune", "tape"}
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(RunningBackupDict)
|
||||
assert set(hints) == {"kind", "job", "task", "guests", "since"}
|
||||
assert hints["guests"] == list[str]
|
||||
assert hints["since"] == int | None
|
||||
@@ -299,3 +299,52 @@ def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
assert not _root_commands(host)
|
||||
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
|
||||
|
||||
|
||||
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOldNetork netork@central"
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_once_the_new_one_is_in(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{OWN_KEY}\n{OLD_KEY}\n"
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
|
||||
|
||||
|
||||
def test_the_new_key_is_never_one_that_goes(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{NETORK_KEY}\n"
|
||||
|
||||
result = driver.add_service_account(
|
||||
"netork", NETORK_KEY, replaces=[f"{NETORK_KEY.split()[0]} {NETORK_KEY.split()[1]}"]
|
||||
)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{NETORK_KEY}\n"
|
||||
|
||||
|
||||
def test_a_replaced_key_stays_while_the_new_one_is_not_in(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{OLD_KEY}\n"
|
||||
host.failing.add("as install")
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert "old keys" not in [s["step"] for s in result["steps"]]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OLD_KEY}\n"
|
||||
|
||||
|
||||
def test_replaced_keys_are_a_list_not_one_string(driver: _Driver) -> None:
|
||||
with pytest.raises(ValueError, match="list"):
|
||||
driver.add_service_account("netork", NETORK_KEY, replaces=OLD_KEY)
|
||||
@@ -0,0 +1,353 @@
|
||||
"""How an OpenSSH server lets people in: the effective settings and sshd's PAM auth stack.
|
||||
|
||||
The parsers are pure. The getter runs against a fake host that answers ``sshd -T``
|
||||
and the dump of ``/etc/pam.d`` from a dict, so the root read, the PAM includes
|
||||
and a failed read are covered without a machine.
|
||||
|
||||
The ``sshd -T`` texts are abridged from what OpenSSH 9.2p1 on Debian 12 prints
|
||||
(the keywords and their order as sshd writes them; the lines that say nothing
|
||||
about authentication are left out). OpenSSH before 8.7 also prints
|
||||
``challengeresponseauthentication``; OpenBSD's sshd has no ``usepam``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import OpenSshServerMixin
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
PAM_FILES_COMMAND,
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_pam_auth,
|
||||
parse_pam_files,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
|
||||
#: Debian 12 as installed: password login on, PAM on, no AuthenticationMethods.
|
||||
SSHD_T_DEBIAN = """\
|
||||
port 22
|
||||
addressfamily any
|
||||
listenaddress [::]:22
|
||||
listenaddress 0.0.0.0:22
|
||||
usepam yes
|
||||
logingracetime 120
|
||||
x11displayoffset 10
|
||||
maxauthtries 6
|
||||
maxsessions 10
|
||||
clientaliveinterval 0
|
||||
clientalivecountmax 3
|
||||
permitrootlogin without-password
|
||||
ignorerhosts yes
|
||||
ignoreuserknownhosts no
|
||||
hostbasedauthentication no
|
||||
hostbasedusesnamefrompacketonly no
|
||||
pubkeyauthentication yes
|
||||
kerberosauthentication no
|
||||
kerberosorlocalpasswd yes
|
||||
kerberosticketcleanup yes
|
||||
gssapiauthentication no
|
||||
gssapicleanupcredentials yes
|
||||
gssapikeyexchange no
|
||||
gssapistrictacceptorcheck yes
|
||||
gssapistorecredentialsonrotate no
|
||||
passwordauthentication yes
|
||||
kbdinteractiveauthentication no
|
||||
printmotd no
|
||||
x11forwarding yes
|
||||
authenticationmethods any
|
||||
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256
|
||||
"""
|
||||
|
||||
#: Key only: password and keyboard-interactive off.
|
||||
SSHD_T_KEY_ONLY = SSHD_T_DEBIAN.replace(
|
||||
"passwordauthentication yes", "passwordauthentication no"
|
||||
).replace("permitrootlogin without-password", "permitrootlogin no")
|
||||
|
||||
#: A key and then an OTP through PAM, for everyone.
|
||||
SSHD_T_KEY_AND_OTP = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no", "kbdinteractiveauthentication yes"
|
||||
).replace("authenticationmethods any", "authenticationmethods publickey,keyboard-interactive")
|
||||
|
||||
#: OpenSSH 8.4p1 (Debian 11): the old keyword beside the new one.
|
||||
SSHD_T_OLD = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication yes\nchallengeresponseauthentication no",
|
||||
)
|
||||
|
||||
#: OpenBSD 7.6: no usepam line.
|
||||
SSHD_T_OPENBSD = SSHD_T_KEY_ONLY.replace("usepam yes\n", "")
|
||||
|
||||
#: Debian 12's /etc/pam.d/sshd, abridged: the auth stack comes from common-auth.
|
||||
PAM_SSHD_DEBIAN = """\
|
||||
# PAM configuration for the Secure Shell service
|
||||
|
||||
# Standard Un*x authentication.
|
||||
@include common-auth
|
||||
|
||||
# Disallow non-root logins when /etc/nologin exists.
|
||||
account required pam_nologin.so
|
||||
|
||||
# Standard Un*x authorization.
|
||||
@include common-account
|
||||
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
|
||||
@include common-session
|
||||
@include common-password
|
||||
"""
|
||||
|
||||
PAM_COMMON_AUTH = """\
|
||||
auth\t[success=1 default=ignore]\tpam_unix.so nullok
|
||||
auth\trequisite\t\t\tpam_deny.so
|
||||
auth\trequired\t\t\tpam_permit.so
|
||||
"""
|
||||
|
||||
PAM_COMMON_ACCOUNT = "account [success=1 new_authtok_reqd=done default=ignore] pam_unix.so\n"
|
||||
|
||||
#: The usual way to add a TOTP: a line of its own in sshd's stack.
|
||||
PAM_SSHD_OTP = PAM_SSHD_DEBIAN.replace(
|
||||
"@include common-auth",
|
||||
"#@include common-auth\nauth required pam_google_authenticator.so nullok",
|
||||
)
|
||||
|
||||
#: Red Hat's form: ``auth substack password-auth``, a module by its full path.
|
||||
PAM_SSHD_RHEL = """\
|
||||
#%PAM-1.0
|
||||
auth substack password-auth
|
||||
auth include postlogin
|
||||
-auth optional /usr/lib64/security/pam_reauthorize.so prepare
|
||||
account required pam_sepermit.so
|
||||
"""
|
||||
PAM_PASSWORD_AUTH = "auth required pam_env.so\nauth sufficient pam_unix.so try_first_pass nullok\n"
|
||||
PAM_POSTLOGIN = "session optional pam_umask.so silent\n"
|
||||
|
||||
|
||||
# ── parse_sshd_auth ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_parse_reads_the_settings_that_decide_how_people_log_in():
|
||||
auth = parse_sshd_auth(SSHD_T_DEBIAN)
|
||||
assert auth == {
|
||||
"password_authentication": True,
|
||||
"kbd_interactive_authentication": False,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [],
|
||||
"permit_root_login": "without-password",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def test_parse_splits_authentication_methods_into_lists_of_methods():
|
||||
auth = parse_sshd_auth(
|
||||
SSHD_T_KEY_ONLY.replace(
|
||||
"authenticationmethods any",
|
||||
"authenticationmethods publickey,keyboard-interactive:pam publickey,password",
|
||||
)
|
||||
)
|
||||
assert auth["authentication_methods"] == [
|
||||
["publickey", "keyboard-interactive:pam"],
|
||||
["publickey", "password"],
|
||||
]
|
||||
|
||||
|
||||
def test_parse_counts_the_old_challenge_response_keyword_as_keyboard_interactive():
|
||||
"""Before 8.7 either keyword could switch PAM's prompts on: either one is enough."""
|
||||
assert parse_sshd_auth(SSHD_T_OLD)["kbd_interactive_authentication"] is True
|
||||
old_only = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication no\nchallengeresponseauthentication yes",
|
||||
)
|
||||
assert parse_sshd_auth(old_only)["kbd_interactive_authentication"] is True
|
||||
|
||||
|
||||
def test_parse_says_none_for_usepam_where_sshd_has_no_pam():
|
||||
assert parse_sshd_auth(SSHD_T_OPENBSD)["use_pam"] is None
|
||||
|
||||
|
||||
def test_parse_takes_a_method_sshd_was_built_without_as_off():
|
||||
"""sshd built without GSSAPI prints no gssapiauthentication line: that method cannot be used."""
|
||||
text = SSHD_T_DEBIAN.replace("gssapiauthentication no\n", "")
|
||||
assert parse_sshd_auth(text)["gssapi_authentication"] is False
|
||||
|
||||
|
||||
def test_parse_refuses_output_that_is_not_sshd_t():
|
||||
with pytest.raises(ValueError, match="passwordauthentication"):
|
||||
parse_sshd_auth("sshd: no hostkeys available -- exiting.\n")
|
||||
|
||||
|
||||
# ── PAM ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _pairs(entries) -> list[tuple[str, str]]:
|
||||
return [(e["control"], e["module"]) for e in entries]
|
||||
|
||||
|
||||
def _modules(stack) -> list[str]:
|
||||
return [e["module"] for e in stack]
|
||||
|
||||
|
||||
def test_pam_auth_lists_modules_and_includes_in_stack_order():
|
||||
# Every @include: which of them hold auth lines is for the included file to say.
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_DEBIAN)) == [
|
||||
("include", "common-auth"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_reads_bracketed_controls_paths_and_optional_lines():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_RHEL)) == [
|
||||
("substack", "password-auth"),
|
||||
("include", "postlogin"),
|
||||
("optional", "pam_reauthorize.so"),
|
||||
]
|
||||
assert parse_pam_auth(PAM_COMMON_AUTH) == [
|
||||
{"module": "pam_unix.so", "control": "[success=1 default=ignore]", "args": ["nullok"]},
|
||||
{"module": "pam_deny.so", "control": "requisite", "args": []},
|
||||
{"module": "pam_permit.so", "control": "required", "args": []},
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_keeps_a_modules_arguments():
|
||||
"""``nullok`` lets a user without an OTP secret through: the consumer has to see it."""
|
||||
assert parse_pam_auth(PAM_SSHD_OTP)[0] == {
|
||||
"module": "pam_google_authenticator.so",
|
||||
"control": "required",
|
||||
"args": ["nullok"],
|
||||
}
|
||||
|
||||
|
||||
def test_pam_auth_skips_comments_and_other_types():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_OTP)) == [
|
||||
("required", "pam_google_authenticator.so"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_ignores_an_include_that_is_not_a_service_name():
|
||||
assert parse_pam_auth("auth include ../../etc/shadow\n@include\nauth required\n") == []
|
||||
|
||||
|
||||
def _dump(files: dict[str, str]) -> str:
|
||||
"""What ``tail -n +1`` prints for several files: a header before each."""
|
||||
return "\n".join(f"==> /etc/pam.d/{name} <==\n{text}" for name, text in files.items())
|
||||
|
||||
|
||||
def test_pam_files_splits_the_dump_by_file():
|
||||
files = {"sshd": PAM_SSHD_DEBIAN, "common-auth": PAM_COMMON_AUTH}
|
||||
assert parse_pam_files(_dump(files)) == files
|
||||
|
||||
|
||||
def test_auth_stack_follows_includes_and_substacks_in_stack_order():
|
||||
files = {
|
||||
"sshd": PAM_SSHD_RHEL + "auth include password-auth\n",
|
||||
"password-auth": PAM_PASSWORD_AUTH,
|
||||
"postlogin": PAM_POSTLOGIN,
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_env.so", "pam_unix.so", "pam_reauthorize.so"]
|
||||
|
||||
|
||||
def test_auth_stack_stops_at_an_include_loop_and_skips_a_missing_file():
|
||||
files = {
|
||||
"sshd": "@include a\n@include gone\n",
|
||||
"a": "@include sshd\nauth required pam_unix.so\n",
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_auth_stack_is_none_without_a_file_for_the_service():
|
||||
assert pam_auth_stack({"login": PAM_COMMON_AUTH}) is None
|
||||
|
||||
|
||||
# ── the getter ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class _Host(SshServerAuthMixin):
|
||||
"""Answers ``sshd -T`` (root only) and the dump of /etc/pam.d."""
|
||||
|
||||
def __init__(self, sshd_t: str | None, pam: dict[str, str] | None = None) -> None:
|
||||
self.sshd_t = sshd_t
|
||||
self.pam = pam
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
self.calls.append((command, privileged))
|
||||
if command == SSHD_EFFECTIVE_COMMAND:
|
||||
if self.sshd_t is None or not privileged:
|
||||
return CommandResult("", "sshd: no hostkeys available -- exiting.", 1)
|
||||
return CommandResult(self.sshd_t, "", 0)
|
||||
assert command == PAM_FILES_COMMAND, command
|
||||
if self.pam is None:
|
||||
return CommandResult("", "tail: cannot open '/etc/pam.d/*' for reading", 1)
|
||||
return CommandResult(_dump(self.pam), "", 0)
|
||||
|
||||
|
||||
def test_getter_reads_sshd_t_as_root_and_pam_without():
|
||||
host = _Host(
|
||||
SSHD_T_DEBIAN,
|
||||
{
|
||||
"sshd": PAM_SSHD_DEBIAN,
|
||||
"common-auth": PAM_COMMON_AUTH,
|
||||
"common-account": PAM_COMMON_ACCOUNT,
|
||||
},
|
||||
)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["password_authentication"] is True
|
||||
assert _modules(auth["pam_auth_stack"]) == ["pam_unix.so", "pam_deny.so", "pam_permit.so"]
|
||||
# sshd -T once, as root; PAM's files once, as the login user.
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True), (PAM_FILES_COMMAND, False)]
|
||||
|
||||
|
||||
def test_getter_names_an_otp_module_in_sshds_own_stack():
|
||||
host = _Host(SSHD_T_KEY_AND_OTP, {"sshd": PAM_SSHD_OTP})
|
||||
stack = host.get_ssh_server_auth()["pam_auth_stack"]
|
||||
assert _modules(stack) == ["pam_google_authenticator.so"]
|
||||
|
||||
|
||||
def test_getter_keeps_what_it_read_when_one_pam_file_is_unreadable():
|
||||
"""tail goes on past a file it cannot open and exits 1: the rest is still read."""
|
||||
|
||||
class _Partial(_Host):
|
||||
def run_command(self, command: str, **kwargs) -> CommandResult: # type: ignore[override]
|
||||
result = super().run_command(command, **kwargs)
|
||||
if command == PAM_FILES_COMMAND:
|
||||
return CommandResult(result.stdout, "tail: cannot open 'x'", 1)
|
||||
return result
|
||||
|
||||
host = _Partial(SSHD_T_DEBIAN, {"sshd": "auth required pam_unix.so\n"})
|
||||
assert _modules(host.get_ssh_server_auth()["pam_auth_stack"]) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_getter_leaves_pam_unread_without_usepam():
|
||||
host = _Host(SSHD_T_OPENBSD)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["pam_auth_stack"] is None
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True)]
|
||||
|
||||
|
||||
def test_getter_says_none_when_pam_cannot_be_read():
|
||||
assert _Host(SSHD_T_DEBIAN).get_ssh_server_auth()["pam_auth_stack"] is None
|
||||
|
||||
|
||||
def test_getter_raises_when_sshd_t_fails():
|
||||
with pytest.raises(RuntimeError, match="no hostkeys"):
|
||||
_Host(None).get_ssh_server_auth()
|
||||
|
||||
|
||||
def test_the_hardening_mixin_carries_the_reader():
|
||||
"""Every driver that can harden sshd can also say how it lets people in."""
|
||||
assert issubclass(OpenSshServerMixin, SshServerAuthMixin)
|
||||
@@ -14,8 +14,19 @@ from napalm_device_types import (
|
||||
UUID = "31303043-534D-C40C-7A72-6F1300000000"
|
||||
|
||||
|
||||
def _out(uuid: str, product_serial: str, board_serial: str) -> str:
|
||||
return f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\nIDEND\n"
|
||||
def _out(
|
||||
uuid: str,
|
||||
product_serial: str,
|
||||
board_serial: str,
|
||||
bios: tuple[str, str, str] = ("", "", ""),
|
||||
) -> str:
|
||||
vendor, version, date = bios
|
||||
return (
|
||||
f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\n{vendor}\n{version}\n{date}\nIDEND\n"
|
||||
)
|
||||
|
||||
|
||||
_NO_FIRMWARE = {"bios_vendor": "", "bios_version": "", "bios_date": ""}
|
||||
|
||||
|
||||
# ── parsing ───────────────────────────────────────────────────────────────────
|
||||
@@ -26,6 +37,7 @@ def test_reads_uuid_and_both_serials():
|
||||
"system_uuid": "31303043-534d-c40c-7a72-6f1300000000",
|
||||
"product_serial": "S123",
|
||||
"board_serial": "ZM155S026022",
|
||||
**_NO_FIRMWARE,
|
||||
}
|
||||
|
||||
|
||||
@@ -81,9 +93,57 @@ def test_unreadable_files_read_as_empty():
|
||||
"system_uuid": "",
|
||||
"product_serial": "",
|
||||
"board_serial": "BS1",
|
||||
**_NO_FIRMWARE,
|
||||
}
|
||||
|
||||
|
||||
# ── the system firmware ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_reads_the_system_firmware():
|
||||
# pve-dual, a Supermicro X10DRL-i
|
||||
out = _out(UUID, "", "ZM155S026022", ("American Megatrends Inc.", "3.4", "11/22/2021"))
|
||||
identity = parse_system_identity(out)
|
||||
assert identity["bios_vendor"] == "American Megatrends Inc."
|
||||
assert identity["bios_version"] == "3.4"
|
||||
assert identity["bios_date"] == "2021-11-22"
|
||||
|
||||
|
||||
def test_a_virtual_machines_firmware_is_read_like_any_other():
|
||||
out = _out(UUID, "", "", ("EDK II", "4.2023.08-4", "02/15/2024"))
|
||||
assert parse_system_identity(out)["bios_version"] == "4.2023.08-4"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("version", ["1", "0", "1.0", "A"])
|
||||
def test_a_short_firmware_version_is_no_placeholder(version):
|
||||
# Serials of one repeated character are placeholders, versions are not.
|
||||
out = _out(UUID, "", "", ("Vendor", version, ""))
|
||||
assert parse_system_identity(out)["bios_version"] == version
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["", "To be filled by O.E.M.", "Default string", "Not Specified"])
|
||||
def test_a_placeholder_firmware_version_reads_as_empty(value):
|
||||
out = _out(UUID, "", "", (value, value, ""))
|
||||
identity = parse_system_identity(out)
|
||||
assert identity["bios_vendor"] == ""
|
||||
assert identity["bios_version"] == ""
|
||||
|
||||
|
||||
@pytest.mark.parametrize("date", ["", "not a date", "13/45/2021", "2021-11-22"])
|
||||
def test_a_firmware_date_that_is_no_smbios_date_reads_as_empty(date):
|
||||
# sysfs writes the SMBIOS date as MM/DD/YYYY; anything else is no reading of it.
|
||||
out = _out(UUID, "", "", ("Vendor", "1.2", date))
|
||||
assert parse_system_identity(out)["bios_date"] == ""
|
||||
|
||||
|
||||
def test_output_of_the_identity_alone_has_no_firmware():
|
||||
# What a host answered before the firmware was read with it: three fields.
|
||||
old = f"IDBEGIN\n{UUID}\nS1\nB1\nIDEND\n"
|
||||
identity = parse_system_identity(old)
|
||||
assert identity["board_serial"] == "B1"
|
||||
assert {k: identity[k] for k in _NO_FIRMWARE} == _NO_FIRMWARE
|
||||
|
||||
|
||||
def test_output_without_its_frame_is_not_a_reading():
|
||||
with pytest.raises(ValueError):
|
||||
parse_system_identity("sudo: a password is required\n")
|
||||
@@ -115,6 +175,13 @@ def test_the_driver_only_carries_the_command_across():
|
||||
assert privileged is True
|
||||
|
||||
|
||||
def test_the_command_reads_the_three_dmi_files():
|
||||
for name in ("product_uuid", "product_serial", "board_serial"):
|
||||
def test_the_command_reads_the_dmi_files():
|
||||
for name in (
|
||||
"product_uuid",
|
||||
"product_serial",
|
||||
"board_serial",
|
||||
"bios_vendor",
|
||||
"bios_version",
|
||||
"bios_date",
|
||||
):
|
||||
assert f"/sys/class/dmi/id/{name}" in SYSTEM_IDENTITY_COMMAND
|
||||
@@ -0,0 +1,245 @@
|
||||
"""TimeSyncMixin: a device's time sources and time zone, read and set (NetOrk/netork#2361).
|
||||
|
||||
The application keeps one list of NTP servers and a zone per site, and aligns a
|
||||
device that differs. ``get_time_sync`` is declared only, so ``callable(getattr(
|
||||
driver, "get_time_sync", None))`` is the truthful answer; ``configure_time_sync``
|
||||
is a template of steps like every fix, and an API driver overrides it whole.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import ClassVar, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
TIME_SYNC_VALUES,
|
||||
AccessPointDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
OSDriver,
|
||||
SwitchDriver,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
UnknownTimezone,
|
||||
check_ntp_servers,
|
||||
)
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(TimeSyncDict)
|
||||
assert hints == {
|
||||
"ntp_servers": list[str],
|
||||
"timezone": str | None,
|
||||
"synchronized": bool | None,
|
||||
"daemon": str | None,
|
||||
"dhcp_ntp_servers": list[str] | None,
|
||||
}
|
||||
|
||||
|
||||
def test_the_dhcp_servers_are_optional():
|
||||
# A driver that cannot tell what DHCP brought leaves the key out (NetOrk/netork#2380).
|
||||
assert TimeSyncDict.__required_keys__ == {"ntp_servers", "timezone", "synchronized", "daemon"}
|
||||
assert TimeSyncDict.__optional_keys__ == {"dhcp_ntp_servers"}
|
||||
|
||||
|
||||
def test_the_two_values_a_driver_may_write():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == TIME_SYNC_VALUES
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (AccessPointDriver, FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, TimeSyncMixin)
|
||||
|
||||
|
||||
def test_the_reading_is_declared_not_implemented():
|
||||
assert not hasattr(TimeSyncMixin, "get_time_sync")
|
||||
source = inspect.getsource(TimeSyncMixin)
|
||||
assert "def get_time_sync(self) -> TimeSyncDict" in source
|
||||
assert "time_sync_writes: ClassVar[frozenset[str]]" in source
|
||||
|
||||
|
||||
def test_the_template_takes_both_values_or_none_of_them():
|
||||
signature = inspect.signature(TimeSyncMixin.configure_time_sync)
|
||||
assert list(signature.parameters) == ["self", "ntp_servers", "timezone"]
|
||||
assert signature.parameters["ntp_servers"].default is None
|
||||
assert signature.parameters["timezone"].default is None
|
||||
|
||||
|
||||
class _Device(TimeSyncMixin):
|
||||
"""A device that keeps its values in memory and reads back what it was given."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
def __init__(self, *, keeps_zone_name: bool = True, fail: str = "") -> None:
|
||||
self.state: TimeSyncDict = {
|
||||
"ntp_servers": ["0.pool.ntp.org"],
|
||||
"timezone": "UTC",
|
||||
"synchronized": True,
|
||||
"daemon": None,
|
||||
}
|
||||
self.keeps_zone_name = keeps_zone_name
|
||||
self.fail = fail
|
||||
self.calls: list[tuple[str, object]] = []
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
state = dict(self.state)
|
||||
if not self.keeps_zone_name:
|
||||
state["timezone"] = None
|
||||
return state # type: ignore[return-value]
|
||||
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
if self.fail == "ntp_servers":
|
||||
return {"ok": False, "output": "refused"}
|
||||
self.state["ntp_servers"] = list(servers)
|
||||
return {"ok": True, "output": "written"}
|
||||
|
||||
def _write_time_zone(self, zone):
|
||||
self.calls.append(("timezone", zone))
|
||||
self.state["timezone"] = zone
|
||||
return {"ok": True, "output": f"zone {zone}"}
|
||||
|
||||
|
||||
class TestConfigureTimeSync:
|
||||
def test_both_values_then_a_check(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(["ntp1.example.org", "10.0.0.1"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
assert device.calls == [
|
||||
("ntp_servers", ["ntp1.example.org", "10.0.0.1"]),
|
||||
("timezone", "Europe/Berlin"),
|
||||
]
|
||||
|
||||
def test_none_leaves_a_value_alone(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert device.calls == [("timezone", "Europe/Berlin")]
|
||||
assert device.state["ntp_servers"] == ["0.pool.ntp.org"]
|
||||
|
||||
def test_servers_alone(self):
|
||||
device = _Device()
|
||||
assert device.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self):
|
||||
device = _Device()
|
||||
with pytest.raises(UnknownTimezone, match="Mars/Olympus_Mons"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_posix_rule_is_not_a_zone_on_the_wire(self):
|
||||
with pytest.raises(UnknownTimezone):
|
||||
_Device().configure_time_sync(timezone="CET-1CEST,M3.5.0,M10.5.0/3")
|
||||
|
||||
def test_a_value_the_driver_cannot_write_is_refused_before_anything_is_written(self):
|
||||
class Bmc(_Device):
|
||||
time_sync_writes = frozenset({"ntp_servers"})
|
||||
|
||||
device = Bmc()
|
||||
with pytest.raises(ValueError, match="timezone"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_driver_that_names_nothing_it_writes_writes_nothing(self):
|
||||
class Silent(TimeSyncMixin):
|
||||
"""Mixes the contract in and never says what it writes."""
|
||||
|
||||
with pytest.raises(ValueError, match="ntp_servers"):
|
||||
Silent().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
def test_nothing_asked_is_a_mistake(self):
|
||||
with pytest.raises(ValueError, match="nothing"):
|
||||
_Device().configure_time_sync()
|
||||
|
||||
def test_a_failed_step_stops_the_rest(self):
|
||||
device = _Device(fail="ntp_servers")
|
||||
result = device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers"]
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_the_check_reads_back_what_was_written(self):
|
||||
class Stubborn(_Device):
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
return {"ok": True, "output": "written"} # and nothing changed
|
||||
|
||||
result = Stubborn().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
verify = result["steps"][-1]
|
||||
assert verify["step"] == "verify"
|
||||
assert "0.pool.ntp.org" in verify["output"]
|
||||
|
||||
def test_the_check_names_the_zone_the_device_kept(self):
|
||||
class Elsewhere(_Device):
|
||||
def _write_time_zone(self, zone):
|
||||
return {"ok": True, "output": "set"} # and the device stayed on UTC
|
||||
|
||||
result = Elsewhere().configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["output"] == "the device is in the time zone UTC"
|
||||
|
||||
def test_servers_compare_without_case(self):
|
||||
class Lower(_Device):
|
||||
def get_time_sync(self):
|
||||
state = super().get_time_sync()
|
||||
state["ntp_servers"] = [s.lower() for s in state["ntp_servers"]]
|
||||
return state
|
||||
|
||||
assert Lower().configure_time_sync(["NTP1.Example.org"])["success"] is True
|
||||
|
||||
def test_servers_from_dhcp_do_not_fail_the_check(self):
|
||||
# DHCP option 42 counts: what the device got by DHCP stays and is no mistake.
|
||||
device = _Device()
|
||||
device.state["dhcp_ntp_servers"] = ["192.168.1.1"]
|
||||
result = device.configure_time_sync(["ntp1.example.org"])
|
||||
assert result["success"] is True, result["output"]
|
||||
|
||||
def test_a_device_without_a_zone_name_is_not_checked_for_one(self):
|
||||
# A switch keeps an offset and a summer-time rule, not a name: it reads None.
|
||||
result = _Device(keeps_zone_name=False).configure_time_sync(timezone="Europe/Berlin")
|
||||
assert result["success"] is True
|
||||
|
||||
|
||||
class TestCheckNtpServers:
|
||||
def test_in_order_stripped_each_once(self):
|
||||
assert check_ntp_servers([" ntp1.example.org", "10.0.0.1", "NTP1.example.org", "::1"]) == [
|
||||
"ntp1.example.org",
|
||||
"10.0.0.1",
|
||||
"::1",
|
||||
]
|
||||
|
||||
def test_none_at_all_is_refused(self):
|
||||
# A device without a source drifts; leaving the value alone is None, not [].
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([])
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([" "])
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"ntp1.example.org iburst",
|
||||
"-ntp.example.org",
|
||||
"ntp1.example.org\nserver evil",
|
||||
"ntp;reboot",
|
||||
"$(id)",
|
||||
"a" * 254,
|
||||
],
|
||||
)
|
||||
def test_only_a_host_name_or_an_address(self, bad):
|
||||
with pytest.raises(ValueError, match="not a host name or address"):
|
||||
check_ntp_servers([bad])
|
||||
|
||||
def test_a_string_is_not_a_list_of_servers(self):
|
||||
with pytest.raises(TypeError):
|
||||
check_ntp_servers("ntp1.example.org") # type: ignore[arg-type]
|
||||
@@ -0,0 +1,656 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
The reading is run by a real ``sh`` over a directory tree that stands in for the
|
||||
host, with ``timedatectl``, ``systemctl`` and ``chronyc`` replaced by scripts that
|
||||
keep a log -- so the shell itself is tested, not a hand-written copy of its output.
|
||||
``sed`` and ``install`` are the real ones, so the files the writes leave are what a
|
||||
host would get (NetOrk/netork#2361). Servers that DHCP brings are read apart and left
|
||||
alone by a write (NetOrk/netork#2380).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixTimeSyncMixin, UnknownTimezone, parse_time_sync
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.time_sync_posix import (
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
timesyncd_link_servers,
|
||||
timesyncd_servers,
|
||||
)
|
||||
|
||||
# What the tools print, as recorded.
|
||||
|
||||
TIMEDATECTL = """\
|
||||
Timezone=Europe/Berlin
|
||||
LocalRTC=no
|
||||
CanNTP=yes
|
||||
NTP=yes
|
||||
NTPSynchronized=yes
|
||||
TimeUSec=Wed 2026-11-11 14:23:16 CET
|
||||
RTCTimeUSec=Wed 2026-11-11 13:23:16 CET
|
||||
"""
|
||||
|
||||
#: Ubuntu 24.04 without any server configured: only the compiled-in fallback.
|
||||
TIMESYNC_FALLBACK = """\
|
||||
FallbackNTPServers=ntp.ubuntu.com
|
||||
ServerName=ntp.ubuntu.com
|
||||
ServerAddress=91.189.91.157
|
||||
RootDistanceMaxUSec=5s
|
||||
PollIntervalMinUSec=32s
|
||||
PollIntervalMaxUSec=34min 8s
|
||||
PollIntervalUSec=34min 8s
|
||||
NTPMessage={ Leap=0, Version=4, Mode=4, Stratum=2, Precision=-24, RootDelay=42.190ms, RootDispersion=1.190ms, Reference=84A36001, Ignored=no, PacketCount=119, Jitter=7.481ms }
|
||||
Frequency=455471
|
||||
"""
|
||||
|
||||
TIMESYNC_CONFIGURED = """\
|
||||
LinkNTPServers=192.168.1.1
|
||||
SystemNTPServers=ntp1.example.org ntp2.example.org
|
||||
FallbackNTPServers=0.debian.pool.ntp.org 1.debian.pool.ntp.org
|
||||
ServerName=ntp1.example.org
|
||||
"""
|
||||
|
||||
CHRONYC_SYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^- 162.159.200.123 3 6 377 41 +814us[ +814us] +/- 14ms
|
||||
^* 192.168.1.1 2 6 377 40 -45us[ -63us] +/- 10ms
|
||||
"""
|
||||
|
||||
CHRONYC_UNSYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^? 162.159.200.123 0 6 0 - +0ns[ +0ns] +/- 0ns
|
||||
"""
|
||||
|
||||
#: Debian 12's chrony.conf, comments shortened, its directories under the test's root.
|
||||
DEBIAN_CHRONY_CONF = """\
|
||||
# Welcome to the chrony configuration file. See chrony.conf(5) for more
|
||||
# information about usable directives.
|
||||
|
||||
# Include configuration files found in /etc/chrony/conf.d.
|
||||
confdir {root}/etc/chrony/conf.d
|
||||
|
||||
# Use Debian vendor zone.
|
||||
pool 2.debian.pool.ntp.org iburst
|
||||
|
||||
# Use time sources from DHCP.
|
||||
sourcedir {root}/run/chrony-dhcp
|
||||
|
||||
# Use NTP sources found in /etc/chrony/sources.d.
|
||||
sourcedir {root}/etc/chrony/sources.d
|
||||
|
||||
keyfile /etc/chrony/chrony.keys
|
||||
driftfile /var/lib/chrony/chrony.drift
|
||||
rtcsync
|
||||
makestep 1 3
|
||||
leapsectz right/UTC
|
||||
"""
|
||||
|
||||
TIMEDATECTL_TOOL = """#!/bin/sh
|
||||
echo "timedatectl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
show) cat "$FIX/timedatectl" ;;
|
||||
show-timesync) [ -f "$FIX/timesync" ] && cat "$FIX/timesync" ;;
|
||||
set-timezone)
|
||||
[ "$2" = "$REFUSE_ZONE" ] && { echo "Failed to set time zone: Invalid or not installed time zone '$2'" >&2; exit 1; }
|
||||
sed -i "s#^Timezone=.*#Timezone=$2#" "$FIX/timedatectl" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
SYSTEMCTL_TOOL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
is-active)
|
||||
u=${2%.service}
|
||||
case " $ACTIVE " in *" $u "*) echo active ;; *) echo inactive; exit 3 ;; esac ;;
|
||||
restart)
|
||||
[ -n "$FAIL_RESTART" ] && { echo "Job for $2 failed." >&2; exit 1; }
|
||||
if [ "$2" = systemd-timesyncd.service ]; then
|
||||
servers=$(sed -n 's/^NTP=\\(..*\\)$/\\1/p' "$DROPIN")
|
||||
printf 'SystemNTPServers=%s\\n' "$servers" > "$FIX/timesync"
|
||||
fi ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CHRONYC_TOOL = """#!/bin/sh
|
||||
echo "chronyc $*" >> "$LOG"
|
||||
case "$*" in
|
||||
"-n sources") [ -f "$FIX/chronyc" ] && cat "$FIX/chronyc" ;;
|
||||
"reload sources") exit "${RELOAD_EXIT:-0}" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixTimeSyncMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, active: str = "chrony") -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CHRONY_CONFIGS = (str(etc / "chrony/chrony.conf"), str(etc / "chrony.conf")) # type: ignore[misc]
|
||||
self.CHRONY_SOURCES_FILE = str(etc / "chrony/sources.d/netork.sources") # type: ignore[misc]
|
||||
self.CHRONY_INCLUDE_FILE = str(etc / "chrony.d/netork.conf") # type: ignore[misc]
|
||||
self.CHRONY_DHCP_DIRS = (str(root / "run/chrony-dhcp"),) # type: ignore[misc]
|
||||
self.TIMESYNCD_DROPIN = str(etc / "systemd/timesyncd.conf.d/netork.conf") # type: ignore[misc]
|
||||
self.NTPD_CONFIGS = (str(etc / "ntpsec/ntp.conf"), str(etc / "ntp.conf")) # type: ignore[misc]
|
||||
self.OPENNTPD_CONFIGS = (str(etc / "openntpd/ntpd.conf"),) # type: ignore[misc]
|
||||
self.TIME_ZONE_SH = ( # type: ignore[misc]
|
||||
f"cat {etc}/timezone 2>/dev/null || readlink {etc}/localtime 2>/dev/null"
|
||||
)
|
||||
self.env = {
|
||||
"PATH": f"{root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(root / "log"),
|
||||
"FIX": str(root / "fix"),
|
||||
"ACTIVE": active,
|
||||
"DROPIN": self.TIMESYNCD_DROPIN,
|
||||
}
|
||||
self.commands: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append((command, privileged))
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command],
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
env={**os.environ, **self.env},
|
||||
timeout=30,
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> list[str]:
|
||||
log = self.root / "log"
|
||||
return log.read_text().splitlines() if log.exists() else []
|
||||
|
||||
def privileged(self) -> list[str]:
|
||||
return [c for c, p in self.commands if p]
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
def _write(path: Path, text: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in ("bin", "fix", "etc"):
|
||||
(tmp_path / d).mkdir()
|
||||
(tmp_path / "fix/timedatectl").write_text(TIMEDATECTL)
|
||||
for name, body in (
|
||||
("timedatectl", TIMEDATECTL_TOOL),
|
||||
("systemctl", SYSTEMCTL_TOOL),
|
||||
("chronyc", CHRONYC_TOOL),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def debian(root: Path) -> Path:
|
||||
"""chrony as Debian 12 ships it, with a source in each place it reads."""
|
||||
_write(root / "etc/chrony/chrony.conf", DEBIAN_CHRONY_CONF.format(root=root))
|
||||
_write(root / "etc/chrony/conf.d/local.conf", "server ntp-conf.example.org iburst\n")
|
||||
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
|
||||
_write(root / "etc/chrony/sources.d/other.sources", "server ntp-other.example.org\n")
|
||||
(root / "fix/chronyc").write_text(CHRONYC_SYNCED)
|
||||
return root
|
||||
|
||||
|
||||
def _no_timedatectl(root: Path) -> None:
|
||||
"""A host without systemd: the command is not there (the build host's must not answer)."""
|
||||
_tool(root, "timedatectl", '#!/bin/sh\necho "sh: timedatectl: not found" >&2\nexit 127\n')
|
||||
|
||||
|
||||
def _text(path: Path) -> str:
|
||||
return path.read_text()
|
||||
|
||||
|
||||
class TestParsing:
|
||||
def test_key_values(self):
|
||||
values = parse_key_values(TIMEDATECTL)
|
||||
assert values["Timezone"] == "Europe/Berlin"
|
||||
assert values["NTPSynchronized"] == "yes"
|
||||
|
||||
def test_timesyncd_uses_the_fallback_only_when_nothing_else_is_known(self):
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_FALLBACK)) == ["ntp.ubuntu.com"]
|
||||
# runtime, then system: the configured ones; a link's servers are DHCP's.
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_CONFIGURED)) == [
|
||||
"ntp1.example.org",
|
||||
"ntp2.example.org",
|
||||
]
|
||||
assert timesyncd_servers({}) == []
|
||||
|
||||
def test_timesyncd_does_not_fall_back_while_a_link_brings_servers(self):
|
||||
values = {"LinkNTPServers": "192.168.1.1", "FallbackNTPServers": "ntp.ubuntu.com"}
|
||||
assert timesyncd_servers(values) == []
|
||||
assert timesyncd_link_servers(values) == ["192.168.1.1"]
|
||||
|
||||
def test_timesyncd_link_servers(self):
|
||||
assert timesyncd_link_servers(parse_key_values(TIMESYNC_CONFIGURED)) == ["192.168.1.1"]
|
||||
assert timesyncd_link_servers({"LinkNTPServers": "10.0.0.1 10.0.0.1 10.0.0.2"}) == [
|
||||
"10.0.0.1",
|
||||
"10.0.0.2",
|
||||
]
|
||||
assert timesyncd_link_servers(parse_key_values(TIMESYNC_FALLBACK)) == []
|
||||
|
||||
def test_chrony_is_synchronised_while_it_has_selected_a_source(self):
|
||||
assert chrony_synchronized(CHRONYC_SYNCED) is True
|
||||
assert chrony_synchronized(CHRONYC_UNSYNCED) is False
|
||||
assert chrony_synchronized("") is None
|
||||
|
||||
def test_chrony_without_a_main_configuration_has_no_sources(self):
|
||||
reading = parse_time_sync("@@ts active\nchrony chrony active\n@@ts end\n")
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], [])
|
||||
|
||||
def test_the_dhcp_directory_is_debian_s_and_red_hat_s_by_default(self):
|
||||
reading = parse_time_sync(
|
||||
"@@ts active\nchrony chrony active\n"
|
||||
"@@ts chrony 0 main /etc/chrony/chrony.conf\nsourcedir /run/chrony-dhcp\n"
|
||||
"@@ts chrony 1 sourcedir /run/chrony-dhcp/eth0.sources\nserver 10.0.0.1 iburst\n"
|
||||
"@@ts end\n"
|
||||
)
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], ["10.0.0.1"])
|
||||
|
||||
def test_a_reading_cut_short_raises(self):
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
parse_time_sync("@@ts timedatectl\nTimezone=UTC\n")
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_chrony_names_its_sources_in_the_order_it_reads_them(self, debian: Path):
|
||||
host = Host(debian)
|
||||
assert host.get_time_sync() == {
|
||||
"ntp_servers": [
|
||||
"ntp-conf.example.org",
|
||||
"2.debian.pool.ntp.org",
|
||||
"ntp-other.example.org",
|
||||
],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
"dhcp_ntp_servers": ["192.168.1.1"],
|
||||
}
|
||||
|
||||
def test_dhcp_servers_in_the_order_chrony_reads_them(self, debian: Path):
|
||||
_write(debian / "run/chrony-dhcp/wlan0.sources", "server 10.9.0.1 iburst\n")
|
||||
_write(debian / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1\npool p.example\n")
|
||||
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == [
|
||||
"192.168.1.1",
|
||||
"p.example",
|
||||
"10.9.0.1",
|
||||
]
|
||||
|
||||
def test_a_dhcp_directory_chrony_does_not_read_brings_nothing(self, debian: Path):
|
||||
# What 4.35.0 left behind: DHCP's sourcedir commented out.
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
conf.write_text(_text(conf).replace("\nsourcedir ", "\n#napalm-time-sync: sourcedir ", 1))
|
||||
reading = Host(debian).get_time_sync()
|
||||
assert reading["dhcp_ntp_servers"] == []
|
||||
assert "192.168.1.1" not in reading["ntp_servers"]
|
||||
|
||||
def test_without_a_dhcp_directory_dhcp_brings_nothing(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", "pool 2.rhel.pool.ntp.org iburst\n")
|
||||
assert Host(root, active="chronyd").get_time_sync()["dhcp_ntp_servers"] == []
|
||||
|
||||
def test_the_reading_is_one_round_trip_without_root(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.get_time_sync()
|
||||
assert [p for _, p in host.commands] == [False]
|
||||
|
||||
def test_chrony_says_whether_it_is_synchronised_not_the_kernel(self, debian: Path):
|
||||
(debian / "fix/chronyc").write_text(CHRONYC_UNSYNCED)
|
||||
assert Host(debian).get_time_sync()["synchronized"] is False
|
||||
|
||||
def test_without_an_answer_from_chronyc_the_kernel_says(self, debian: Path):
|
||||
(debian / "fix/chronyc").unlink()
|
||||
assert Host(debian).get_time_sync()["synchronized"] is True
|
||||
|
||||
def test_commented_and_other_directives_are_no_sources(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"#pool 2.rhel.pool.ntp.org iburst\n! server old.example.org\n"
|
||||
" server ntp1.example.org iburst\nrefclock PPS /dev/pps0\npeer peer.example.org\n",
|
||||
)
|
||||
reading = Host(root, active="chronyd").get_time_sync()
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "peer.example.org"]
|
||||
assert reading["daemon"] == "chrony"
|
||||
|
||||
def test_an_include_pattern_is_followed(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\nmakestep 1 3\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
assert Host(root).get_time_sync()["ntp_servers"] == ["suse.pool.ntp.org"]
|
||||
|
||||
def test_timesyncd(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
reading = Host(root, active="systemd-timesyncd").get_time_sync()
|
||||
assert reading["daemon"] == "systemd-timesyncd"
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "ntp2.example.org"]
|
||||
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
assert reading["synchronized"] is True
|
||||
|
||||
def test_ntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/ntpsec/ntp.conf",
|
||||
"driftfile /var/lib/ntpsec/ntp.drift\npool 0.debian.pool.ntp.org iburst\n"
|
||||
"server 127.127.1.0\nserver ntp1.example.org iburst\nrestrict default kod\n",
|
||||
)
|
||||
reading = Host(root, active="ntpsec").get_time_sync()
|
||||
assert reading["daemon"] == "ntpd"
|
||||
assert reading["ntp_servers"] == ["0.debian.pool.ntp.org", "ntp1.example.org"]
|
||||
# Debian's DHCP hook gives ntpd a configuration of its own: not read, so unknown.
|
||||
assert reading["dhcp_ntp_servers"] is None
|
||||
|
||||
def test_openntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/openntpd/ntpd.conf",
|
||||
'servers pool.ntp.org\nserver ntp1.example.org\nconstraints from "https://www.google.com"\n',
|
||||
)
|
||||
reading = Host(root, active="openntpd").get_time_sync()
|
||||
assert reading["daemon"] == "openntpd"
|
||||
assert reading["ntp_servers"] == ["pool.ntp.org", "ntp1.example.org"]
|
||||
assert reading["dhcp_ntp_servers"] is None
|
||||
|
||||
def test_no_client_runs(self, root: Path):
|
||||
(root / "fix/timedatectl").write_text(
|
||||
TIMEDATECTL.replace("NTPSynchronized=yes", "NTPSynchronized=no")
|
||||
)
|
||||
assert Host(root, active="").get_time_sync() == {
|
||||
"ntp_servers": [],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": False,
|
||||
"daemon": None,
|
||||
"dhcp_ntp_servers": [],
|
||||
}
|
||||
|
||||
def test_without_timedatectl_the_zone_comes_from_the_files(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/localtime").symlink_to("/usr/share/zoneinfo/America/Sao_Paulo")
|
||||
reading = Host(root, active="").get_time_sync()
|
||||
assert reading["timezone"] == "America/Sao_Paulo"
|
||||
assert reading["synchronized"] is None
|
||||
|
||||
def test_etc_timezone_wins_over_the_link(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/timezone").write_text("Australia/Lord_Howe\n")
|
||||
assert Host(root, active="").get_time_sync()["timezone"] == "Australia/Lord_Howe"
|
||||
|
||||
def test_a_copied_localtime_has_no_name(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
assert Host(root, active="").get_time_sync()["timezone"] is None
|
||||
|
||||
|
||||
class TestChronyServers:
|
||||
def test_exactly_these_servers(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp1.example.org iburst\nserver 10.0.0.1 iburst\npool 2.de.pool.ntp.org iburst\n"
|
||||
)
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
assert "\n#napalm-time-sync: pool 2.debian.pool.ntp.org iburst\n" in conf
|
||||
# DHCP option 42 counts: chrony keeps reading what DHCP brings.
|
||||
assert f"\nsourcedir {debian}/run/chrony-dhcp\n" in conf
|
||||
assert f"\nsourcedir {debian}/etc/chrony/sources.d\n" in conf
|
||||
assert f"\nconfdir {debian}/etc/chrony/conf.d\n" in conf
|
||||
assert _text(debian / "etc/chrony/conf.d/local.conf") == (
|
||||
"#napalm-time-sync: server ntp-conf.example.org iburst\n"
|
||||
)
|
||||
assert _text(debian / "etc/chrony/sources.d/other.sources") == (
|
||||
"#napalm-time-sync: server ntp-other.example.org\n"
|
||||
)
|
||||
# DHCP's own file is left alone: the DHCP client rewrites it on every lease.
|
||||
assert _text(debian / "run/chrony-dhcp/eth0.sources") == "server 192.168.1.1 iburst\n"
|
||||
# The main configuration changed: chronyd reads it only when it starts.
|
||||
assert "systemctl restart chrony.service" in host.log()
|
||||
reading = host.get_time_sync()
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"]
|
||||
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
|
||||
def test_what_4_35_0_took_from_dhcp_is_given_back(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
# A host 4.35.0 aligned: DHCP's sourcedir commented out behind the mark.
|
||||
dhcp = f"sourcedir {debian}/run/chrony-dhcp"
|
||||
conf.write_text(_text(conf).replace(f"\n{dhcp}\n", f"\n#napalm-time-sync: {dhcp}\n"))
|
||||
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == []
|
||||
(debian / "log").unlink()
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert f"\n{dhcp}\n" in _text(conf)
|
||||
assert "#napalm-time-sync: sourcedir" not in _text(conf)
|
||||
# The main configuration changed again: a restart, not a reload.
|
||||
assert "systemctl restart chrony.service" in host.log()
|
||||
assert host.get_time_sync()["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
|
||||
def test_a_foreign_sourcedir_4_35_0_took_out_stays_out(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
foreign = f"#napalm-time-sync: sourcedir {debian}/etc/chrony/extra.d"
|
||||
conf.write_text(_text(conf) + foreign + "\n")
|
||||
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
|
||||
|
||||
assert Host(debian).configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert f"\n{foreign}\n" in _text(conf)
|
||||
|
||||
def test_a_foreign_sourcedir_stays_and_its_sources_stop_counting(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
conf.write_text(_text(conf) + f"sourcedir {debian}/etc/chrony/extra.d\n")
|
||||
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert f"\nsourcedir {debian}/etc/chrony/extra.d\n" in _text(conf)
|
||||
assert _text(debian / "etc/chrony/extra.d/x.sources") == (
|
||||
"#napalm-time-sync: server ntp-extra.example.org\n"
|
||||
)
|
||||
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org"]
|
||||
|
||||
def test_dhcp_is_kept_where_the_servers_go_to_a_confdir(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony/chrony.conf",
|
||||
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n"
|
||||
f"sourcedir {root}/run/chrony-dhcp\n",
|
||||
)
|
||||
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert f"\nsourcedir {root}/run/chrony-dhcp\n" in _text(root / "etc/chrony/chrony.conf")
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
|
||||
["ntp1.example.org"],
|
||||
["192.168.1.1"],
|
||||
)
|
||||
|
||||
def test_a_second_run_only_reloads_the_sources(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
(debian / "log").unlink()
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp2.example.org", "ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/chrony.conf") == conf
|
||||
assert "chronyc reload sources" in host.log()
|
||||
assert not any(line.startswith("systemctl restart") for line in host.log())
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp2.example.org iburst\nserver ntp1.example.org iburst\n"
|
||||
)
|
||||
|
||||
def test_a_chronyd_that_cannot_reload_sources_is_restarted(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
host = Host(debian)
|
||||
host.env["RELOAD_EXIT"] = "1"
|
||||
assert host.configure_time_sync(["ntp2.example.org"])["success"] is True
|
||||
log = host.log()
|
||||
reload = log.index("chronyc reload sources")
|
||||
assert log[reload + 1] == "systemctl restart chrony.service"
|
||||
|
||||
def test_writes_are_root_s_and_root_reads_no_stdin(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.configure_time_sync(["ntp1.example.org"])
|
||||
staged = [c for c, p in host.commands if not p and "mktemp" in c]
|
||||
assert len(staged) == 1
|
||||
assert host.privileged()
|
||||
assert all("mktemp" not in c for c in host.privileged())
|
||||
|
||||
def test_rhel_without_a_directory_gets_an_include(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"pool 2.rhel.pool.ntp.org iburst\ndriftfile /var/lib/chrony/drift\n",
|
||||
)
|
||||
host = Host(root, active="chronyd")
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
include = root / "etc/chrony.d/netork.conf"
|
||||
assert _text(include) == "server ntp1.example.org iburst\n"
|
||||
assert _text(root / "etc/chrony.conf") == (
|
||||
"#napalm-time-sync: pool 2.rhel.pool.ntp.org iburst\n"
|
||||
"driftfile /var/lib/chrony/drift\n"
|
||||
f"# napalm-time-sync: the time sources\ninclude {include}\n"
|
||||
)
|
||||
assert "systemctl restart chronyd.service" in host.log()
|
||||
|
||||
def test_an_include_that_already_takes_the_file_is_used(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony.conf") == f"include {root}/etc/chrony.d/*.conf\n"
|
||||
assert (
|
||||
_text(root / "etc/chrony.d/pool.conf")
|
||||
== "#napalm-time-sync: pool suse.pool.ntp.org iburst\n"
|
||||
)
|
||||
assert _text(root / "etc/chrony.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
|
||||
def test_a_confdir_takes_the_file_where_sources_d_is_not_read(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony/chrony.conf",
|
||||
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n",
|
||||
)
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony/conf.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
assert not (root / "etc/chrony/sources.d/netork.sources").exists()
|
||||
|
||||
def test_a_failed_restart_fails_the_step(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.env["FAIL_RESTART"] = "1"
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Job for chrony.service failed." in result["steps"][0]["output"]
|
||||
|
||||
def test_a_file_that_cannot_be_staged_is_not_installed(self, debian: Path):
|
||||
_tool(debian, "mktemp", "#!/bin/sh\necho 'mktemp: failed' >&2\nexit 1\n")
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "cannot stage" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
def test_no_configuration_is_a_failed_step(self, root: Path):
|
||||
result = Host(root).configure_time_sync(["ntp1.example.org"])
|
||||
assert result["success"] is False
|
||||
assert "chrony" in result["steps"][0]["output"]
|
||||
|
||||
|
||||
class TestTimesyncdServers:
|
||||
def test_a_drop_in_with_exactly_these_servers(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_FALLBACK)
|
||||
host = Host(root, active="systemd-timesyncd")
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(root / "etc/systemd/timesyncd.conf.d/netork.conf").endswith(
|
||||
"[Time]\nNTP=\nNTP=ntp1.example.org 10.0.0.1\nFallbackNTP=\n"
|
||||
)
|
||||
assert "systemctl restart systemd-timesyncd.service" in host.log()
|
||||
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org", "10.0.0.1"]
|
||||
|
||||
def test_servers_from_dhcp_stay_and_count(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
tool = (root / "bin/systemctl").read_text()
|
||||
(root / "bin/systemctl").write_text(
|
||||
tool.replace(
|
||||
"printf 'SystemNTPServers", "printf 'LinkNTPServers=192.168.1.1\\nSystemNTPServers"
|
||||
)
|
||||
)
|
||||
host = Host(root, active="systemd-timesyncd")
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
|
||||
["ntp1.example.org"],
|
||||
["192.168.1.1"],
|
||||
)
|
||||
|
||||
|
||||
class TestOtherClients:
|
||||
@pytest.mark.parametrize("active", ["ntpsec", "openntpd", ""])
|
||||
def test_only_chrony_and_timesyncd_are_written(self, root: Path, active: str):
|
||||
host = Host(root, active=active)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "chrony and systemd-timesyncd" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
|
||||
class TestTimeZone:
|
||||
def test_timedatectl_sets_it(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
result = host.configure_time_sync(timezone="America/Sao_Paulo")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert host.privileged() == ["timedatectl set-timezone America/Sao_Paulo"]
|
||||
assert host.get_time_sync()["timezone"] == "America/Sao_Paulo"
|
||||
|
||||
def test_a_zone_the_host_does_not_know_fails_the_step(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
host.env["REFUSE_ZONE"] = "Europe/Kyiv"
|
||||
result = host.configure_time_sync(timezone="Europe/Kyiv")
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Invalid or not installed time zone" in result["steps"][0]["output"]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self, debian: Path):
|
||||
host = Host(debian)
|
||||
with pytest.raises(UnknownTimezone):
|
||||
host.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert host.commands == []
|
||||
|
||||
def test_both_at_once(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"], "UTC")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["timezone"]) == (["ntp1.example.org"], "UTC")
|
||||
|
||||
|
||||
def test_it_writes_both_values():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == PosixTimeSyncMixin.time_sync_writes
|
||||
@@ -0,0 +1,20 @@
|
||||
"""The contract of :class:`~napalm_device_types.updates.UpdateMixin`."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import napalm_device_types
|
||||
from napalm_device_types.updates import PackageIndexMissing
|
||||
|
||||
|
||||
def test_a_missing_index_is_its_own_failure():
|
||||
"""A reader can tell "no index to read from" from any other failure, and a
|
||||
caller that only knows failures still catches it as one."""
|
||||
exc = PackageIndexMissing("The package index is not loaded (run apk update first)")
|
||||
|
||||
assert isinstance(exc, RuntimeError)
|
||||
assert str(exc) == "The package index is not loaded (run apk update first)"
|
||||
|
||||
|
||||
def test_it_is_exported_from_the_package():
|
||||
assert napalm_device_types.PackageIndexMissing is PackageIndexMissing
|
||||
assert "PackageIndexMissing" in napalm_device_types.__all__
|
||||
Reference in new issue
Block a user