Compare commits

..
50 Commits
Author SHA1 Message Date
christianmanivong e729b935a4 Merge pull request 'feat(apt): read the release each source names, move a file to the host's (NetOrk/netork#2614)' (#127) from feature/apt-source-release into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 1m12s
driver / build (push) Successful in 28s
driver / security (push) Successful in 58s
driver / publish (push) Skipped
driver / types (push) Successful in 1m3s
driver / lowest (push) Successful in 37s
driver / standard (push) Successful in 18s
driver / test (3.12) (push) Successful in 1m21s
CI / driver (push) Successful in 2m28s
2026-11-17 12:44:12 +00:00
christianmanivong f654076560 feat(apt): read the release each source names, move a file to the host's (NetOrk/netork#2614)
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m6s
CI / driver (pull_request) Successful in 2m28s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 55s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m16s
driver / lowest (pull_request) Successful in 38s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 12s
A release upgrade leaves third-party apt sources on the old release, where they
still answer and their packages stop getting updates once the vendor stops
building for it: z2m-house, Ubuntu 24.04, kept Docker's source on focal and never
saw a Docker newer than 28.1.1.

AptReleaseMixin.get_apt_source_releases() reports the host's /etc/os-release and
every enabled source apt reads, one-line and deb822, with its file, line, types,
URI, suites and components. set_apt_source_release(path, release, codename) moves
every suite of release in one of apt's source files to codename, once each new
suite's InRelease or Release is fetched with apt-helper through apt's own proxy;
where the vendor has none, nothing is written. AptProxyMixin brings it, so every
apt driver has it.
2026-11-17 13:31:42 +01:00
christianmanivong 2dea517e89 Merge pull request 'feat(firmware): install the offered firmware through FirmwareUpgradeMixin (NetOrk/netork#2615)' (#126) from feature/openwrt-sysupgrade-2615 into main
driver / lint (push) Successful in 12s
driver / test (3.13) (push) Successful in 1m3s
driver / lowest (push) Successful in 39s
driver / build (push) Successful in 26s
driver / security (push) Successful in 54s
driver / publish (push) Successful in 28s
driver / types (push) Successful in 53s
driver / test (3.12) (push) Successful in 1m5s
driver / standard (push) Successful in 16s
CI / driver (push) Successful in 2m57s
2026-11-17 11:38:29 +00:00
christianmanivong 1a7885d22e refactor(firmware): FirmwareOfferDict lives with its contract
driver / build (pull_request) Successful in 24s
driver / security (pull_request) Successful in 53s
CI / driver (pull_request) Successful in 2m9s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 50s
driver / test (3.13) (pull_request) Successful in 59s
driver / test (3.12) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 35s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
models.py is over the file limit and may not grow; the offer's two new
fields moved it past its recorded length. It is still importable from
napalm_device_types.models.
2026-11-17 12:35:34 +01:00
christianmanivong 71588613fb chore(release): 4.41.0
driver / build (pull_request) Successful in 25s
driver / security (pull_request) Successful in 52s
CI / driver (pull_request) Failing after 2m10s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 53s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / lowest (pull_request) Successful in 40s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
2026-11-17 12:32:43 +01:00
christianmanivong 93ba335798 feat(firmware): install the offered firmware through FirmwareUpgradeMixin (NetOrk/netork#2615)
start_firmware_upgrade(version) starts an upgrade the device runs by itself and
restarts at the end of; get_update_progress follows it. The offer says whether
the driver can install it now (installable) and names a later release line it
never crosses to (latest_major_version).
2026-11-17 11:02:51 +01:00
christianmanivong eb1964c4b3 Merge pull request 'chore(release): 4.40.0' (#125) from chore/release-4.40.0 into main
driver / lint (push) Successful in 12s
driver / test (3.13) (push) Successful in 1m1s
driver / build (push) Successful in 33s
driver / security (push) Successful in 57s
driver / publish (push) Successful in 27s
driver / types (push) Successful in 57s
driver / lowest (push) Successful in 36s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m14s
CI / driver (push) Successful in 3m10s
2026-11-17 04:06:06 +00:00
christianmanivong 3622d0f825 chore(release): 4.40.0
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 57s
CI / driver (pull_request) Successful in 2m42s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 39s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
2026-11-17 05:00:46 +01:00
christianmanivong caf7ef4cc8 Merge pull request 'feat(fixes): ZabbixAgentMixin, a native Zabbix agent for a host without a container engine (NetOrk/netork#2571)' (#124) from feat/zabbix-agent-native into main
driver / lint (push) Successful in 14s
driver / lowest (push) Successful in 38s
driver / test (3.13) (push) Successful in 1m9s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m2s
driver / publish (push) Skipped
driver / types (push) Successful in 55s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m14s
CI / driver (push) Successful in 2m43s
2026-11-17 04:00:27 +00:00
christianmanivong 3422b7d030 Merge remote-tracking branch 'origin/main' into feat/zabbix-agent-native
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m38s
driver / standard (pull_request) Successful in 14s
driver / types (pull_request) Successful in 55s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / test (3.13) (pull_request) Successful in 59s
driver / lowest (pull_request) Successful in 38s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 12s
# Conflicts:
#	CHANGELOG.md
2026-11-17 04:55:56 +01:00
christianmanivong f1940aa918 Merge pull request 'feat(container-engine): reach rootful Podman like Docker (NetOrk/netork#818)' (#123) from feature/podman-engine-netork-818 into main
driver / lint (push) Successful in 12s
driver / test (3.13) (push) Successful in 58s
driver / build (push) Successful in 27s
driver / security (push) Successful in 55s
driver / publish (push) Skipped
driver / types (push) Successful in 49s
driver / lowest (push) Successful in 35s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m8s
CI / driver (push) Successful in 2m18s
2026-11-17 03:54:09 +00:00
christianmanivong 5217c6e333 feat(fixes): ZabbixAgentMixin, a native Zabbix agent for a host without a container engine (NetOrk/netork#2571)
driver / build (pull_request) Successful in 24s
driver / security (pull_request) Successful in 55s
CI / driver (pull_request) Successful in 2m20s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 49s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 34s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
configure_zabbix_agent(settings) loads the package index where the driver can,
installs ZABBIX_AGENT_PACKAGE from the host's own packages and, where those have
none and settings["release"] names the server's release line, from Zabbix's
repository (the driver's _add_zabbix_repository); then the driver writes the
configuration and opens the firewall, and the service is enabled and restarted.
Settings that do not hold raise ValueError before anything changes.

zabbix_agent.py holds what is the same on every host, without I/O: the settings
check, the agent's configuration with Server, ServerActive, ListenPort and
Hostname set and the rest kept, and the root command that adds Zabbix's
repository for Debian, Ubuntu, Raspberry Pi OS and the RHEL family, in the
layout before and since 7.2. LldpAgentMixin and ZabbixAgentMixin share the
start step (enable_and_restart).
2026-11-17 04:50:42 +01:00
christianmanivong 7051199589 feat(container-engine): reach rootful Podman like Docker (NetOrk/netork#818)
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m27s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 55s
driver / test (3.13) (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m13s
driver / lowest (pull_request) Successful in 37s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
container_engines() lists Podman with the API podman-libpod, and
open_container_engine("podman") reaches it with podman system dial-stdio
(Podman 4.0 and later). It is the host's rootful Podman: the engine of the
system's containers and of Quadlet's units, whose socket is root's. So the
connection runs every call as root (ContainerEngineConnection.privileged).

An engine is listed only when its CLI names itself on --version:
podman-docker's docker is podman, and listing it as Docker would read the
login user's rootless Podman a second time.
2026-11-16 23:26:55 +01:00
christianmanivong 7c0e4fffc5 Merge pull request 'feat(apt): send https sources through the cache in the form it stores (netork#2555)' (#122) from fix/apt-https-cache into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m14s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m3s
driver / publish (push) Skipped
driver / types (push) Successful in 58s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m21s
CI / driver (push) Successful in 2m33s
2026-11-15 19:23:46 +00:00
christianmanivong cb0f1cc05c feat(apt): send https sources through the cache in the form it stores (NetOrk/netork#2555)
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 2m29s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 1m1s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 35s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
apt-cacher-ng tunnels a source fetched over https and stores nothing of it. It
fetches http://HTTPS///<host>/<path> over https itself, with the certificate
verified, and stores it, where port 443 is allowed (AllowUserPorts=80 443).

- apt_sources: cacheable_https_hosts(report), and the root commands that rewrite
  the https sources of .list and deb822 .sources files to that form and back.
  A host apt has credentials for in auth.conf stays on https; the rewrite names
  it in the proxy file as Package-Proxy::Not-Cached, which the report reads.
- configure_package_proxy(url): a step "cache" before "https". It rewrites the
  sources, fetches one of them through url, and turns them all back to https
  where url cannot. The "fetch" step proves the proxy with a plain http source.
- configure_package_proxy(""): the sources go back to https before the proxy
  file goes; without the cache apt would look for a host called HTTPS.
2026-11-15 20:07:27 +01:00
christianmanivong b4929990f1 Merge pull request 'chore(release): 4.39.0' (#121) from chore/release-4.39.0 into main
driver / lint (push) Successful in 13s
driver / lowest (push) Successful in 36s
driver / test (3.13) (push) Successful in 1m2s
driver / build (push) Successful in 26s
driver / security (push) Successful in 1m20s
driver / publish (push) Successful in 29s
driver / types (push) Successful in 54s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m12s
CI / driver (push) Successful in 3m32s
2026-11-14 22:56:28 +00:00
christianmanivong 3db6c6385c Merge pull request 'feat(updates): PackageIndexMissing, for a host without a package index' (#120) from feat/package-index-missing into main
driver / lint (push) Successful in 12s
driver / test (3.13) (push) Successful in 1m1s
driver / lowest (push) Successful in 34s
driver / build (push) Successful in 28s
driver / security (push) Successful in 1m0s
driver / publish (push) Skipped
driver / types (push) Successful in 53s
driver / test (3.12) (push) Successful in 1m7s
driver / standard (push) Successful in 14s
CI / driver (push) Successful in 2m15s
2026-11-14 22:52:11 +00:00
christianmanivong c5154d954b chore(release): 4.39.0
driver / build (pull_request) Successful in 24s
driver / security (pull_request) Successful in 50s
CI / driver (pull_request) Successful in 2m19s
driver / standard (pull_request) Successful in 14s
driver / types (pull_request) Successful in 49s
driver / test (3.12) (pull_request) Successful in 1m3s
driver / lowest (pull_request) Successful in 33s
driver / test (3.13) (pull_request) Successful in 59s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 12s
2026-11-14 23:52:11 +01:00
christianmanivong 211d0c02eb feat(updates): PackageIndexMissing, for a host without a package index
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 52s
CI / driver (pull_request) Successful in 2m12s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / lowest (pull_request) Successful in 36s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
OpenWrt keeps its package index in RAM and loses it on every reboot.
get_available_updates() raises then, rightly, but only as a bare
RuntimeError, so a caller cannot tell it from a read that failed and
reports a communication error for a state a refresh cures.

PackageIndexMissing (a RuntimeError, so existing callers still catch it)
names that case; refresh_available_updates() is the way out.

Refs NetOrk/netork#2527
2026-11-14 23:45:38 +01:00
christianmanivong 74d2f42b0e Merge pull request 'feat(system-identity): read the system firmware with the identity' (#119) from feat/system-firmware into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m10s
driver / lowest (push) Successful in 43s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m6s
driver / publish (push) Skipped
driver / types (push) Successful in 56s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m13s
CI / driver (push) Successful in 2m52s
2026-11-13 22:11:26 +00:00
christianmanivong c25f39768f refactor(system-identity): keep SystemIdentityDict with its mixin
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 2m51s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 57s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / lowest (pull_request) Successful in 38s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
models.py is over its line limit and may not grow; the dict has no other
user than system_identity.py, as clusters.py and apt_proxy.py keep theirs.

NetOrk/netork#2518
2026-11-13 23:04:18 +01:00
christianmanivong 9ecc6823c8 feat(system-identity): read the system firmware with the identity
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 58s
CI / driver (pull_request) Failing after 2m17s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 52s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / lowest (pull_request) Successful in 39s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
get_system_identity() also reads bios_vendor, bios_version and bios_date
from /sys/class/dmi/id, in the same round trip as the SMBIOS identity.
The date is given as ISO 8601; anything that is no MM/DD/YYYY reads as
empty. Only the vendors' text defaults are placeholders for the firmware:
a version of "1" is a version, unlike a serial of one repeated character.

Output of the three identity fields alone still parses, with the firmware
empty.

NetOrk/netork#2518
2026-11-13 23:00:23 +01:00
christianmanivong 2080ccbfd7 Merge pull request 'chore(release): 4.38.0' (#118) from chore/release-4.38.0 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m1s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 28s
driver / security (push) Successful in 53s
driver / publish (push) Successful in 27s
driver / types (push) Successful in 56s
driver / test (3.12) (push) Successful in 1m10s
driver / standard (push) Successful in 16s
CI / driver (push) Successful in 3m1s
2026-11-13 21:56:00 +00:00
christianmanivong ce1fe18405 chore(release): 4.38.0
driver / build (pull_request) Successful in 31s
driver / security (pull_request) Successful in 1m11s
CI / driver (pull_request) Successful in 2m59s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / test (3.13) (pull_request) Successful in 1m9s
driver / lowest (pull_request) Successful in 46s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
2026-11-13 22:39:54 +01:00
christianmanivong e43926f1d5 Merge pull request 'feat(config-revisions): declare a device's own configuration history' (#117) from feat/config-revisions into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 1m6s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
driver / types (push) Successful in 1m11s
driver / lowest (push) Successful in 37s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m27s
CI / driver (push) Successful in 2m58s
2026-11-13 21:39:38 +00:00
christianmanivong 43e0210ff6 feat(config-revisions): declare a device's own configuration history
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 3m34s
CI / driver (pull_request) Successful in 5m2s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m3s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / lowest (pull_request) Successful in 47s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
Some devices keep a copy of their configuration for every save and can
go back to one. An application restores a configuration there by
reverting to the revision that holds it; on OPNsense that is the only
way, since its API takes no uploaded config.xml (NetOrk/netork#2511).

ConfigRevisionsMixin declares get_config_revisions() (newest first, as
ConfigRevisionDict) and get_config_revision(revision_id).
revert_config_revision(revision_id) is a template over the driver's
_revert_to_config_revision step: it refuses an id that is not in the
history (UnknownConfigRevision), reads the revision before the revert
can rotate it out, and afterwards checks that the newest revision holds
its configuration (ConfigRevertFailed otherwise), since a device may
report a revert it did not make. config_revision_content(text) is the
hook where a driver leaves out what its device stamps into every save,
so revisions with the same configuration compare equal.

Refs NetOrk/netork#2511
2026-11-13 22:28:30 +01:00
christianmanivong 241acef6c1 Merge pull request 'feat(accounts): add_service_account takes the keys the new one replaces' (#115) from fix/service-account-replaces into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 59s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 28s
driver / security (push) Successful in 57s
driver / publish (push) Skipped
driver / types (push) Successful in 55s
driver / test (3.12) (push) Successful in 1m10s
driver / standard (push) Successful in 16s
CI / driver (push) Successful in 2m35s
2026-11-13 08:55:08 +00:00
christianmanivong 4c0ba5b2c0 refactor(accounts): the account step of add_service_account in a helper
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 54s
CI / driver (pull_request) Successful in 2m32s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 52s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
replaces= made add_service_account 55 lines long, over the quality
ratchet's 50. Its account step -- none to take when the driver logs in
as the account already, else the create or reopen -- moves to
_accounts_service_account_step. Behaviour is unchanged; the tests of
add_service_account cover both branches.
2026-11-13 09:00:43 +01:00
christianmanivong e9da399568 Merge remote-tracking branch 'origin/main' into fix/service-account-replaces 2026-11-13 08:59:48 +01:00
christianmanivong 8673d1eda1 Merge pull request 'feat(log-forwarding): each forwarding rule names its template' (#116) from feat/forward-rule-template into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m13s
driver / lowest (push) Successful in 43s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m3s
driver / publish (push) Skipped
driver / types (push) Successful in 1m8s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m22s
CI / driver (push) Successful in 2m33s
2026-11-12 22:42:45 +00:00
christianmanivong 46d7125aa6 refactor(log-forwarding): a legacy rule's template is read on its own
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 2m57s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m21s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 40s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
The quality ratchet holds _Reader.legacy at complexity 10.
2026-11-12 23:38:28 +01:00
christianmanivong 8e43fe2ed9 feat(log-forwarding): each forwarding rule names its template
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Failing after 2m38s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m9s
driver / test (3.13) (pull_request) Successful in 1m15s
driver / test (3.12) (pull_request) Successful in 1m30s
driver / lowest (pull_request) Successful in 40s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
A rule's template decides the timestamp it sends: rsyslog's default, RFC 3164,
carries the host's local time without a zone, which a collector such as Graylog
files shifted by the host's UTC offset (NetOrk/netork#493). The rule now says
which template it uses: its own (`@host;Name`, `action(... template="...")`),
else the default set before it (`$ActionForwardDefaultTemplate`, or
`module(load="builtin:omfwd" template="...")` for omfwd actions), else None.
2026-11-12 23:36:38 +01:00
christianmanivong 679584766f feat(accounts): add_service_account takes the keys the new one replaces
driver / build (pull_request) Successful in 38s
driver / security (pull_request) Successful in 1m16s
CI / driver (pull_request) Failing after 2m45s
driver / standard (pull_request) Failing after 18s
driver / types (pull_request) Successful in 1m1s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / lowest (pull_request) Successful in 46s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
netOrk moves a host from its installation-wide key to its site's
(NetOrk/netork#2462): the old key has to go from its own account, which
remove_local_account_keys refuses (the login's own account, a system
account). With replaces, the old keys go once the new key is in -- never
the new key itself, and not while it could not be added.
2026-11-12 21:55:13 +01:00
christianmanivong 1b04818982 Merge pull request 'fix(dhcp): a refused subnet no longer stops the rest of the set' (#114) from fix/dhcp-subnet-apply-tolerance into main
driver / lint (push) Successful in 13s
driver / lowest (push) Successful in 43s
driver / test (3.13) (push) Successful in 1m6s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
driver / types (push) Successful in 56s
driver / test (3.12) (push) Successful in 1m12s
driver / standard (push) Successful in 14s
CI / driver (push) Successful in 2m44s
2026-11-12 19:06:13 +00:00
christianmanivong 65991d0930 fix(dhcp): a refused subnet no longer stops the rest of the set
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Successful in 3m15s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m19s
driver / test (3.12) (pull_request) Successful in 1m32s
driver / test (3.13) (pull_request) Successful in 1m16s
driver / lowest (pull_request) Successful in 41s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
apply_dhcp_subnetset called apply_dhcp_subnet for each add and update
and committed only at the end, so one refused write raised before the
commit: the subnets written ahead of it stayed saved but inactive, and
as the device reports them as configured, no later diff committed them.

It now behaves like apply_dhcp_reservationset since 4.35.1: a refused
subnet gets a [failed] line, the others are written, any non-empty diff
is committed, then DhcpApplyError is raised. DhcpApplyError takes an
optional kind ("reservation" by default) for its message.

Closes #113
2026-11-12 19:19:03 +01:00
christianmanivong 9dfb5596bc Merge pull request 'fix(dhcp): release an address before another reservation takes it' (#112) from fix/dhcp-reservation-apply-order into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m9s
driver / lowest (push) Successful in 40s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m9s
driver / publish (push) Skipped
driver / types (push) Successful in 1m1s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m17s
CI / driver (push) Successful in 2m59s
2026-11-12 18:14:11 +00:00
christianmanivong b4cdcfd347 fix(dhcp): release an address before another reservation takes it
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m10s
CI / driver (pull_request) Successful in 3m6s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m25s
driver / test (3.13) (pull_request) Successful in 1m13s
driver / lowest (pull_request) Successful in 45s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
apply_dhcp_reservationset wrote adds first, then updates in the order of
the desired set. Kea keeps addresses unique at each write, so when an
address moved from one reservation to another, the reservation taking it
was refused as a duplicate whenever it came first.

The changes are now ordered (_dhcp_order): an update that moves a
reservation off its address goes before the change that takes that
address, along a chain from its free end, and adds come after every
update. A swap or rotation of addresses has no such order; its changes
are still tried, so a server that checks only at the commit takes them,
and a refused one's [failed] line says it was a swap and with which
reservations. Nothing is moved to a temporary address: the contract
knows no free one, and a failed second write would leave it there.

The diff is computed against one read of the reservations, shared with
diff_dhcp_reservations. Refused rows still do not stop the others, the
rest is committed and DhcpApplyError follows, as since 4.35.1.

Closes #106
2026-11-12 19:07:31 +01:00
christianmanivong 5258a43dc9 Merge pull request 'feat(updates): the CVEs a host's installed advisories close (4.37.0)' (#111) from feature/installed-advisories into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m16s
driver / lowest (push) Successful in 39s
driver / build (push) Successful in 28s
driver / security (push) Successful in 1m6s
driver / publish (push) Successful in 28s
driver / types (push) Successful in 59s
driver / test (3.12) (push) Successful in 1m12s
driver / standard (push) Successful in 15s
CI / driver (push) Successful in 6m6s
2026-11-12 11:12:53 +00:00
christianmanivong f3cb3601a8 feat(updates): the CVEs a host's installed advisories close (4.37.0)
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m0s
CI / driver (pull_request) Successful in 2m29s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m2s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m24s
driver / lowest (pull_request) Successful in 40s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
UpdateMixin declares get_installed_advisory_cves(): per package, the CVEs
the installed security advisories close -- the distribution's word that a
backport is in although the upstream version did not move. The command
(DNF_INSTALLED_CVES_COMMAND, dnf 4, from the cache) and its parser
(parse_dnf_installed_cves) live here for every dnf driver, as the
security read's do. netOrk reads it where its knowledge base holds no
records of the host's release (NetOrk/netork#2394).
2026-11-12 12:09:37 +01:00
christianmanivong a8fedf299a Merge pull request 'fix(host-status): kernel releases compare the way rpm does (4.36.2)' (#110) from fix/rpm-kernel-version-order into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m13s
driver / lowest (push) Successful in 46s
driver / build (push) Successful in 40s
driver / security (push) Successful in 1m16s
driver / publish (push) Successful in 30s
driver / types (push) Successful in 1m4s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m20s
CI / driver (push) Successful in 3m36s
2026-11-12 10:08:48 +00:00
christianmanivong c736fb30d0 refactor(host-status): compare one version segment in a helper of its own
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 57s
CI / driver (pull_request) Successful in 2m30s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 56s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 40s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
The quality ratchet refused _version_cmp at complexity 12. Comparing one
segment moves to _segment_cmp, which also takes 01 and 1 as equal, as
rpm does.
2026-11-12 11:05:49 +01:00
christianmanivong cf555d006d fix(host-status): kernel releases compare the way rpm does (4.36.2)
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m3s
CI / driver (pull_request) Failing after 2m32s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 1m5s
driver / test (3.13) (pull_request) Successful in 1m14s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / lowest (pull_request) Successful in 42s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
_version_key compared the text between numbers as plain strings, so
'.el' outranked '.' and 6.12.0-208.112.5.el9uek counted as newer than
the running 6.12.0-208.112.5.1.el9uek: an Oracle Linux host asked to
reboot into an older kernel, and a reboot never cleared it. The
releases now compare like rpmvercmp: separators only separate, a
number is newer than letters in its place, and '~' comes before
anything. Checked against rpm.labelCompare on Oracle Linux 9.

Closes #109
2026-11-12 11:03:46 +01:00
christianmanivong 4e79cd1fc8 Merge pull request 'fix(host-status): an Oracle Linux host's UEK kernels are not an interrupted dnf (4.36.1)' (#108) from fix/dnf-installonly-defaults into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m12s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m3s
driver / publish (push) Successful in 30s
driver / types (push) Successful in 1m0s
driver / test (3.12) (push) Successful in 1m17s
driver / standard (push) Successful in 16s
CI / driver (push) Successful in 3m21s
2026-11-12 09:37:34 +00:00
christianmanivong 304f404567 fix(host-status): an Oracle Linux host's UEK kernels are not an interrupted dnf (4.36.1)
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 56s
CI / driver (pull_request) Successful in 2m26s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 59s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m17s
driver / lowest (pull_request) Successful in 41s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
The installonly packages were dnf's upstream defaults plus dnf.conf's.
Oracle builds kernel-uek and its parts into libdnf's defaults, and the
UEK packages provide nothing of upstream's list, so every host with more
than one UEK kernel was reported as left halfway. The defaults are now
also asked of the host's libdnf through its Python binding; upstream's
list stays the floor for dnf 5 and yum.

Closes #107
2026-11-12 10:30:48 +01:00
christianmanivong aaa9f0504a Merge pull request 'feat(time-sync): DHCP-provided NTP servers count and are read apart (4.36.0)' (#105) from feat/time-sync-dhcp-ntp-2380 into main
driver / lint (push) Successful in 15s
driver / lowest (push) Successful in 42s
driver / test (3.13) (push) Successful in 1m11s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m2s
driver / publish (push) Successful in 31s
driver / types (push) Successful in 59s
driver / standard (push) Successful in 18s
driver / test (3.12) (push) Successful in 1m16s
CI / driver (push) Successful in 3m17s
2026-11-11 21:25:27 +00:00
christianmanivong bc47220318 Merge remote-tracking branch 'origin/main' into feat/time-sync-dhcp-ntp-2380
driver / build (pull_request) Successful in 48s
driver / security (pull_request) Successful in 1m26s
CI / driver (pull_request) Successful in 3m17s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m34s
driver / test (3.13) (pull_request) Successful in 1m32s
driver / test (3.12) (pull_request) Successful in 1m52s
driver / lowest (pull_request) Successful in 1m3s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
2026-11-11 22:21:34 +01:00
christianmanivong 08a62838f7 Merge pull request 'fix(dhcp): one refused reservation no longer stops the apply' (#104) from fix/dhcp-reservations-partial-apply into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m9s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 33s
driver / security (push) Successful in 1m7s
driver / publish (push) Successful in 32s
driver / types (push) Successful in 1m4s
driver / test (3.12) (push) Successful in 1m18s
driver / standard (push) Successful in 18s
CI / driver (push) Successful in 3m29s
2026-11-11 21:12:31 +00:00
christianmanivong ac38c68feb feat(time-sync): DHCP-provided NTP servers count and are read apart
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 3m22s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m16s
driver / test (3.13) (pull_request) Successful in 1m29s
driver / test (3.12) (pull_request) Successful in 1m33s
driver / lowest (pull_request) Successful in 41s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 31s
A device that gets the site's time giver through DHCP option 42 is
aligned (NetOrk/netork#2380): most clients already learn it that way.
4.35.0 switched that off on every write -- chrony's DHCP sourcedir was
commented out, and systemd-timesyncd's link servers failed the verify
step -- and folded DHCP's servers into ntp_servers, so an application
could not tell configured from DHCP-provided servers.

TimeSyncDict gains dhcp_ntp_servers (not required): the servers the
device got by DHCP and uses, in its order; [] for none, absent or None
where the driver cannot tell. ntp_servers keeps only the configured ones.

PosixTimeSyncMixin reads chrony's sources from the DHCP client's
sourcedir (CHRONY_DHCP_DIRS, /run/chrony-dhcp by default) as chronyd
reads them, and systemd-timesyncd's LinkNTPServers; ntpd and OpenNTPD
keep DHCP's servers in a configuration of their own, so there it is
unknown. A chrony write now comments out only server, pool and peer
lines and leaves every sourcedir and the DHCP client's files alone. A
DHCP sourcedir that 4.35.0 commented out behind the mark is given back
on the next write, with a restart, so hosts it aligned hear DHCP again.

Version 4.36.0.
2026-11-11 22:08:52 +01:00
christianmanivong 8516f2a1ee refactor(dhcp): one reservation's write in a helper of its own
driver / build (pull_request) Successful in 37s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 3m0s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / test (3.13) (pull_request) Successful in 1m10s
driver / lowest (pull_request) Successful in 42s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
apply_dhcp_reservationset grew to 59 lines with the per-write handling;
the quality ratchet allows no new function that long.

Refs NetOrk/netork#2381
2026-11-11 22:08:50 +01:00
christianmanivong c74173523a fix(dhcp): one refused reservation no longer stops the apply
driver / build (pull_request) Successful in 38s
driver / security (pull_request) Successful in 1m11s
CI / driver (pull_request) Failing after 2m57s
driver / standard (pull_request) Failing after 22s
driver / types (pull_request) Successful in 1m22s
driver / test (3.12) (pull_request) Successful in 1m28s
driver / test (3.13) (pull_request) Successful in 1m26s
driver / lowest (pull_request) Successful in 38s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 18s
apply_dhcp_reservationset wrote the diff one reservation at a time and
committed only after the last. A write the device refused -- on
OPNsense, an address in a subnet Kea does not manage -- ended the run
before the commit: every later reservation went unwritten, and the
earlier ones stayed saved but inactive. The next diff read those back as
configured, reported no changes and never committed them.

A refused change now gets a "[failed] <mac> -> <ip>: <reason>" line and
the others are still written. Any non-empty diff commits, also one whose
every write was refused, so writes an interrupted earlier run saved take
effect. DhcpApplyError (a RuntimeError, exported) is raised after the
commit with how many of how many changes were refused.

Refs NetOrk/netork#2381
2026-11-11 22:03:52 +01:00
46 changed files with 4682 additions and 293 deletions

No files matched your search

+180
View File
@@ -7,6 +7,186 @@ recorded here; their history is the git log.
## [Unreleased]
### Added
- `AptReleaseMixin.get_apt_source_releases()`: the host's release from `/etc/os-release`
(`ID`, `ID_LIKE`, `VERSION_CODENAME`, `UBUNTU_CODENAME`) and every enabled source apt
reads, one-line and deb822, with its file, line, types, URI, suites and components
(NetOrk/netork#2614). `AptProxyMixin` brings it, so every apt driver has it.
- `AptReleaseMixin.set_apt_source_release(path, release, codename)`: every suite of
*release* in one of apt's source files moved to *codename* (`focal-security` →
`noble-security`), once each new suite's `InRelease` or `Release` is fetched with apt's
downloader; nothing is written where the vendor has none.
- `parse_apt_releases`, `rewrite_release`, `suite_release`, `APT_RELEASES_COMMAND`.
## [4.41.0] - 2026-11-17
### Added
- `FirmwareUpgradeMixin.start_firmware_upgrade(version)`: start installing the firmware a
driver offers; the device runs it by itself and restarts at its end, and
`UpdateMixin.get_update_progress` follows it until then (NetOrk/netork#2615, OpenWrt's
sysupgrade through owut).
- `FirmwareOfferDict.installable`: whether the driver can install `latest_version` itself
now, False while the device lacks what that takes; and
`FirmwareOfferDict.latest_major_version`, the newest of a later release line, shown and
never crossed to by the driver.
## [4.40.0] - 2026-11-17
### Added
- `ZabbixAgentMixin.configure_zabbix_agent(settings)`: a native Zabbix agent, for a host
without a container engine (NetOrk/netork#2571). Install (`ensure_package` with
`ZABBIX_AGENT_PACKAGE`, and where the host's own packages have none and
`settings["release"]` names the server's release line, the driver's
`_add_zabbix_repository(release)` and a second try), configure (the driver's
`_write_zabbix_agent_config`), firewall (the driver's `_allow_zabbix_agent_sources`),
start (`manage_service` enable and restart of `ZABBIX_AGENT_SERVICE`). Settings that do
not hold raise `ValueError` before anything changes.
- `ZabbixAgentSettingsDict`; `zabbix_agent_config(text, settings)`, the agent's
configuration with `Server`, `ServerActive`, `ListenPort` and `Hostname` set and the rest
kept; `zabbix_repository_command(os_release, release)`, the root command that adds
Zabbix's repository for Debian, Ubuntu, Raspberry Pi OS, RHEL, CentOS, Oracle Linux,
AlmaLinux and Rocky Linux, in the layout before and since 7.2;
`check_zabbix_agent_settings(settings)`.
- `fixes.enable_and_restart(manage_service, service)`, the start step `LldpAgentMixin` and
`ZabbixAgentMixin` share.
- Podman as a container engine (NetOrk/netork#818): `container_engines()` lists it with the
API `podman-libpod`, and `open_container_engine("podman")` reaches it with `podman system
dial-stdio` (Podman 4.0 and later). It is the host's rootful Podman, so the connection
runs every call as root (`ContainerEngineConnection.privileged`, `ROOT_ENGINES`).
- `apt_sources`: apt's https sources through the package cache, in the form apt-cacher-ng
fetches over https itself and stores (`http://HTTPS///<host>/…`, with `AllowUserPorts=80
443` on the cache). `cacheable_https_hosts(report)` names the hosts this applies to, and
`rewrite_sources_command` / `revert_sources_command` are the root commands, for `.list`
and deb822 `.sources` files. A host apt keeps credentials for in `auth.conf` stays on
https, because apt finds them by the host's name and the cache is shared. The rewrite
names such a host in the proxy file as `Package-Proxy::Not-Cached` (NetOrk/netork#2555).
- `get_package_proxy_report()` reports those hosts as `not_cached`.
### Changed
- `container_engines()` lists an engine only when its CLI names itself on `--version`:
podman-docker's `docker` is podman, and is no longer listed as Docker.
- `configure_package_proxy(url)` has a step `cache` before `https`. It rewrites the https
sources it can, fetches one of them through *url*, and turns them all back to https
where *url* cannot. The `fetch` step proves the proxy with a plain http source only.
- `configure_package_proxy("")` turns the sources back to https (step `sources`) before it
removes the proxy file. Without the cache, apt would look for a host called `HTTPS`.
## [4.39.0] - 2026-11-14
### Added
- `PackageIndexMissing` (a `RuntimeError`): what `get_available_updates()` raises when
the host holds no package index to read from -- OpenWrt's, which lives in RAM, after
every reboot. `refresh_available_updates()` loads it again, so a caller can tell this
from a failed read and refresh instead of reporting an error (NetOrk/netork#2527).
## [4.38.0] - 2026-11-13
### Added
- `get_system_identity()` also reads the host's system firmware (BIOS or UEFI) from the
same SMBIOS tables in the same round trip: `bios_vendor`, `bios_version` and
`bios_date` (ISO 8601, from sysfs's `MM/DD/YYYY`; anything else reads as empty). Only
the vendors' text defaults (`To be filled by O.E.M.`, `Default string`) are
placeholders there -- a version of `1` is a version. Output of the three identity
fields alone still parses, with the firmware empty (NetOrk/netork#2518).
- `ConfigRevisionsMixin`: a device's own configuration history (NetOrk/netork#2511).
`get_config_revisions()` (newest first, the first in place now, as
`ConfigRevisionDict`: id, UTC time, description, user, size) and
`get_config_revision(revision_id)` are declared. `revert_config_revision(revision_id)`
is a template over the driver's `_revert_to_config_revision`: it refuses an id that is
not in the history (`UnknownConfigRevision`), and after the revert checks that the
newest revision holds the revision's configuration, else `ConfigRevertFailed`; it
returns that revision. `config_revision_content(text)` leaves out what a device stamps
into every save, so an application can compare revisions with a configuration it has.
- `add_service_account(..., replaces=[...])`: netOrk's keys the new one takes the place of
go from its account once the new key is in it (step `old keys`), never the new key
itself; while the new key could not be added they stay, so netOrk keeps its way in.
netOrk calls it through a login with the new key, after trying that login, to move a
host from its installation-wide key to its site's (NetOrk/netork#2462).
- Each forwarding rule of `LogForwardingDict` names its `template`: the one the rule
gives (`@host;RSYSLOG_SyslogProtocol23Format`, `action(… template="…")`), else the
default set before it (`$ActionForwardDefaultTemplate`, or
`module(load="builtin:omfwd" template="…")` for omfwd actions), else None -- rsyslog's
own default, RFC 3164, which sends the host's local time without a time zone. A
reader can tell a rule whose timestamps a collector files shifted (NetOrk/netork#493).
### Fixed
- `apply_dhcp_reservationset` writes its changes so an address is released before it is
taken: an update that moves a reservation off its address goes before the change that
takes that address (along a chain, from its free end), and adds come after every
update. A server that keeps addresses unique at each write, such as Kea, no longer
refuses the second reservation as a duplicate when an address moves from one to
another. A swap -- reservations trading addresses, or a longer rotation -- has no such
order: its changes are still tried, and where the server refuses them the `[failed]`
line says it was a swap and with which reservations. Everything else is still written
and committed, and `DhcpApplyError` follows (#106).
- `apply_dhcp_subnetset` no longer stops at the first subnet the device refuses. As
`apply_dhcp_reservationset` has since 4.35.1, a refused subnet gets a `[failed]` line,
the others are still written, any non-empty diff is committed -- even one whose every
write was refused -- and `DhcpApplyError` follows. Before, the subnets written ahead of
the refused one stayed saved but inactive, and since the device reports them as
configured, no later apply committed them. `DhcpApplyError` takes an optional `kind`
(`"reservation"` by default, `"subnet"` here) that its message names (#113).
## [4.37.0] - 2026-11-12
### Added
- `UpdateMixin.get_installed_advisory_cves()` (declared): the CVEs a host's installed
security advisories close, per package. With `DNF_INSTALLED_CVES_COMMAND` (dnf 4,
read from the cache) and `parse_dnf_installed_cves()`, the parser every dnf driver
shares. netOrk reads it where its knowledge base holds no records of the host's
release, as the distribution's word that a backport is in (netOrk #2394).
## [4.36.2] - 2026-11-12
### Fixed
- `kernel_reboot_pending` compares kernel releases the way rpm does (`rpmvercmp`). A
number is newer than letters in the same place, so an Oracle Linux host running
`6.12.0-208.112.5.1.el9uek` no longer asks to reboot into the older
`6.12.0-208.112.5.el9uek`. A `~` (release candidate) comes before its release (#109).
## [4.36.1] - 2026-11-12
### Fixed
- The host status no longer reports an Oracle Linux host's UEK kernels as an interrupted
dnf transaction. The installonly packages are now also the ones the host's libdnf was
built with, read from its Python binding: Oracle adds `kernel-uek` and its parts there,
and their packages provide nothing of upstream's defaults (#107).
## [4.36.0] - 2026-11-11
### Added
- `TimeSyncDict.dhcp_ntp_servers` (not required): the NTP servers a device got by DHCP
(option 42) and uses, in its order; `[]` for none, absent or None where the driver cannot
tell. `ntp_servers` stays the servers configured on the device and no longer holds
DHCP's. A device that gets the site's time giver by DHCP is aligned, so an application
has to tell the two apart (NetOrk/netork#2380).
- `PosixTimeSyncMixin` reads them: chrony's sources from the DHCP client's `sourcedir`
(`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp` by default), systemd-timesyncd's
`LinkNTPServers`. Where ntpd or OpenNTPD runs they are unknown (None).
`timesyncd_link_servers` is public.
### Changed
- `configure_time_sync` leaves DHCP-provided NTP servers alone. chrony: only `server`,
`pool` and `peer` lines are commented out; no `sourcedir` line is any more, and the DHCP
client's files are left alone. systemd-timesyncd: a link's servers stay and no longer
fail the `verify` step.
- A write gives back a DHCP client's `sourcedir` that 4.35.0 commented out
(`#napalm-time-sync: sourcedir /run/chrony-dhcp`), so a host aligned by 4.35.0 hears DHCP
again; chronyd restarts for it. Another `sourcedir` 4.35.0 took out stays out.
- `timesyncd_servers` returns the configured servers (runtime, then system) without a
link's; the fallback only while there is no server at all.
## [4.35.1] - 2026-11-11
### Fixed
- `apply_dhcp_reservationset()` no longer stops at the first reservation the device refuses.
One refused write (on OPNsense, an address in a subnet Kea does not manage) ended the run
before the commit: every later reservation went unwritten, and the earlier ones stayed
saved but inactive, where the next diff found them configured and never committed them. A
refused change now gets a `[failed] <mac> -> <ip>: <reason>` line, the others are written,
any non-empty diff commits, and `DhcpApplyError` (a `RuntimeError`, exported) is raised
after the commit with how many of how many were refused (NetOrk/netork#2381).
## [4.35.0] - 2026-11-11
### Added
+55 -13
View File
@@ -124,7 +124,10 @@ application wants made on a device runs as fixed steps the driver implements, wi
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
interfaces, enable and restart; the driver writes only the interfaces),
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
`ZabbixAgentMixin.configure_zabbix_agent(settings)` (install from the host's packages or
else Zabbix's repository, configure, firewall, enable and restart; the driver writes the
configuration and opens the firewall), `EnsurePackageMixin.ensure_package(name)`,
`PackageProxyMixin.configure_package_proxy(url)`,
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
change in one call overrides the template method. A driver mixes in the fixes its device
@@ -172,6 +175,15 @@ wins (`get_package_proxy`, `get_package_proxy_report` with every line that sets
different proxy, reads the result back and fetches one http source's `InRelease` through it.
A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_file`.
`AptReleaseMixin` (`napalm_device_types.apt_releases`) comes with it: every source apt reads,
with its file, line, URI and suites, and the host's release from `/etc/os-release`
(`get_apt_source_releases`). A release upgrade leaves third-party sources on the old release,
where they stop getting updates once the vendor stops building for it; whether a suite names
another release than the host's is the caller's to judge (`suite_release` gives the word).
`set_apt_source_release(path, release, codename)` moves every suite of *release* in one
source file to *codename*, after fetching each new suite's `InRelease` with apt's downloader:
where the vendor has none, the file stays as it is.
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
@@ -286,6 +298,9 @@ O.E.M.`, `Default string`, all-zero or the common `03000200-…-000700080009` UU
empty: matching on them would join unrelated machines. `is_placeholder_identity` is the
same filter for a BMC driver's side. `product_uuid` and `product_serial` are root's, so a
driver supplies `_run_system_identity_command(command, privileged=)`.
The same round trip reads the host's system firmware (BIOS or UEFI): `bios_vendor`,
`bios_version` and `bios_date` (`YYYY-MM-DD`, from sysfs's `MM/DD/YYYY`). Only the vendors'
text defaults read as empty there; a short version such as `1` is a version.
`OpenSshServerMixin` (`harden_ssh_server`, `get_ssh_server_algorithms`) takes algorithms
out of what an OpenSSH server offers -- the ones ssh-audit flags. It reads the lists sshd
@@ -348,10 +363,13 @@ cron expressions, gives their next runs and writes them as `OnCalendar` lines.
`TimeSyncMixin` (`get_time_sync`, `configure_time_sync`) is how a device keeps its clock:
the NTP servers it is configured with, in its order, its time zone as an IANA name, whether
it is synchronised now and which client does it (`TimeSyncDict`). `get_time_sync` is
declared only; `configure_time_sync(ntp_servers=None, timezone=None)` is a fix template --
`None` leaves a value alone, the device ends up with exactly the servers given, in that
order, and a `verify` step reads it back. A zone that is not in the IANA database, a
it is synchronised now and which client does it (`TimeSyncDict`). The servers it got by
DHCP (option 42) and uses are apart, in `dhcp_ntp_servers` (`[]` for none, left out where
the driver cannot tell): DHCP counts, a device that gets the site's time giver that way is
aligned. `get_time_sync` is declared only; `configure_time_sync(ntp_servers=None,
timezone=None)` is a fix template -- `None` leaves a value alone, the device ends up
configured with exactly the servers given, in that order, its DHCP-provided servers left
alone, and a `verify` step reads it back. A zone that is not in the IANA database, a
malformed server, an empty list or a value outside the driver's `time_sync_writes` is
refused before anything is written. A driver states in `time_sync_writes` which of
`ntp_servers` and `timezone` it can write (a BMC that runs on UTC only: `ntp_servers`).
@@ -362,11 +380,31 @@ login user (`timedatectl show`, the active client: chrony's configuration and `c
sources`, `timedatectl show-timesync`, `ntp.conf`, OpenNTPD's `ntpd.conf`); the zone by
`timedatectl set-timezone`; chrony's servers in a file of their own --
`/etc/chrony/sources.d/netork.sources` where chrony reads that directory, else a `confdir`
or an included `/etc/chrony.d/netork.conf` -- with every other `server`, `pool`, `peer` and
foreign `sourcedir` line commented out behind `#napalm-time-sync: `, then `chronyc reload
sources` or a restart; systemd-timesyncd's in `/etc/systemd/timesyncd.conf.d/netork.conf`
(`NTP=` reset, `FallbackNTP=` emptied), then a restart. ntpd and OpenNTPD are read, not
written.
or an included `/etc/chrony.d/netork.conf` -- with every other `server`, `pool` and `peer`
line commented out behind `#napalm-time-sync: `, then `chronyc reload sources` or a
restart; systemd-timesyncd's in `/etc/systemd/timesyncd.conf.d/netork.conf` (`NTP=` reset,
`FallbackNTP=` emptied), then a restart. What DHCP brings is read apart and kept: chrony's
DHCP `sourcedir` (`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp`) is never commented out -- one
that 4.35.0 commented out comes back on the next write -- and systemd-timesyncd's
`LinkNTPServers` stay. ntpd and OpenNTPD are read, not written; what DHCP gives them is
unknown.
`ConfigRevisionsMixin` (`get_config_revisions`, `get_config_revision`,
`revert_config_revision`) is a device's own configuration history: a copy of the
configuration for every save, which the device can go back to. An application restores a
configuration there by reverting to the revision that holds it -- on OPNsense the only way,
since its API takes no uploaded config.xml (NetOrk/netork#2511). Listing (newest first, the
first in place now: id, UTC time, description, user, size as `ConfigRevisionDict`) and
reading one revision are declared only. `revert_config_revision(revision_id)` is a
template: it refuses an id that is not in the history (`UnknownConfigRevision`, a
`ValueError`) before anything changes, reads the revision -- before the revert, which may
rotate it out -- lets the driver revert (`_revert_to_config_revision`), and checks that the
newest revision afterwards holds the revision's configuration (`ConfigRevertFailed`
otherwise: a device may say it reverted when it did not). It returns that newest revision.
`config_revision_content(text)` is what of a revision is its configuration; a driver whose
device stamps every save (who, when) strips the stamp there, and an application that looks
for the revision holding a configuration compares through it too. A driver that can list
but not revert sets `revert_config_revision = None`.
### Access channels: why there is no container model here
@@ -377,10 +415,11 @@ written.
The mixin sits in `DeviceTypeDriver` and only declares, so `hasattr(driver, "run_command")` is true exactly where a driver implements it. For SSH drivers, `channel.run_on_transport` and `channel.open_stream_on_transport` are the implementation over a paramiko exec channel: no PTY, stderr kept apart, a real exit code. How a command gains root stays with the driver.
**`ContainerEngineMixin` is deliberately different from `SystemdServicesMixin`.** The systemd mixin owns its command and its parse; this one owns neither.
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}]`).
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}, {"engine": "podman", "api": "podman-libpod"}]`). An engine is listed when its CLI answers `--version` with its own name; podman-docker's `docker` says it is podman and is left out.
- `open_container_engine(engine)` returns a `ContainerEngineConnection`:
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`);
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`, `podman system dial-stdio`);
- `run_cli(args)` and `stream_cli(args)` run the engine's CLI with arguments the caller chooses. With `privileged=True` the driver runs it the way `run_command` gains root, for an engine that refuses the login user.
- **Podman is reached as root** (`ROOT_ENGINES`, `connection.privileged`): the host's rootful Podman, the engine of the system's containers and of Quadlet's units, whose API `podman.socket` serves. A login user's rootless containers are that user's own and invisible to others. `podman system dial-stdio` exists from Podman 4.0 on and needs the socket listening (`systemctl enable --now podman.socket`) (NetOrk/netork#818).
- The driver decides only *how* the engine is reached. Its hook `_container_engine_binary(engine)` returns, for QNAP, the Container Station path, and the caller never sees it.
**What runs on the engine, and what to do with it, is netOrk's** (NetOrk/netork#765): the container model, the Engine API requests, compose, updates. Above the connection everything is specific to the service, so this package abstracts the connection and nothing more.
@@ -487,7 +526,10 @@ The same split applies to `DhcpServerMixin`: `get_dhcp_reservations`/
`apply_dhcp_reservation`/`commit_dhcp_reservations` are abstract (Kea REST on
OPNsense, dnsmasq/odhcpd UCI on OpenWrt), while `diff_dhcp_reservations` and
`apply_dhcp_reservationset` are concrete — matching by normalised MAC and the
apply-then-commit orchestration are identical for every DHCP server.
apply-then-commit orchestration are identical for every DHCP server. The apply writes
its changes so an address is released before another reservation takes it (an update
that frees an address first, adds last); a swap of addresses has no such order and is
reported as one where the server refuses it.
A new driver (FortiGate, pfSense, …) gets `diff_firewall_rules`/
`apply_firewall_ruleset` for free the moment it implements the three abstract
+4 -1
View File
@@ -39,7 +39,7 @@ application SNMP polling, ping sweeps, discovery scoring, guest
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `_dhcp_order`, `lag` |
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
@@ -78,8 +78,10 @@ The same shape replaces every action in today's `run_device_action`:
|---|---|
| `fix_snmp` | `configure_snmp_agent(settings)` |
| `fix_apt_proxy` | `configure_package_proxy(url)` |
| an apt source on another release than the host (NetOrk/netork#2614) | `set_apt_source_release(path, release, codename)` |
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
| the Zabbix agent where no container engine runs (NetOrk/netork#2571) | `configure_zabbix_agent(settings)` |
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
driver that inherits a template it cannot run on its device, as a NAS driver built on the
@@ -158,6 +160,7 @@ the driver.
| `virtual_ips` | vip-manager, keepalived, awk |
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
| `package_updates` | apt and dnf output |
| `apt_releases` | apt's source files (one-line and deb822), `/etc/os-release`, `apt-helper` |
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
| `terminal` | ANSI escape codes in shell output |
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
+49 -4
View File
@@ -38,6 +38,7 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.channel.CommandChannelMixin`
* :class:`~napalm_device_types.config_lifecycle.ConfigLifecycleMixin`
* :class:`~napalm_device_types.config_revisions.ConfigRevisionsMixin`
* :class:`~napalm_device_types.config_sync.ConfigSyncMixin`
* :class:`~napalm_device_types.container_engine.ContainerEngineMixin`
* :class:`~napalm_device_types.dhcp.DhcpServerMixin`
@@ -85,6 +86,15 @@ from napalm_device_types.apt_proxy import (
parse_apt_proxy,
same_proxy_url,
)
from napalm_device_types.apt_releases import (
APT_RELEASES_COMMAND,
AptReleaseFixDict,
AptReleaseMixin,
AptReleasesDict,
AptSourceDict,
parse_apt_releases,
suite_release,
)
from napalm_device_types.arp import ARP_AGE_UNKNOWN
from napalm_device_types.backup_server import BackupServerDriver
from napalm_device_types.backups import (
@@ -121,22 +131,29 @@ from napalm_device_types.clusters import (
HAResourceDict,
)
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
from napalm_device_types.config_revisions import (
ConfigRevertFailed,
ConfigRevisionDict,
ConfigRevisionsMixin,
UnknownConfigRevision,
)
from napalm_device_types.config_sync import ConfigSyncMixin
from napalm_device_types.container_engine import (
ContainerEngineConnection,
ContainerEngineMixin,
ContainerEngineUnavailable,
)
from napalm_device_types.dhcp import DhcpServerMixin, normalize_cidr, normalize_mac
from napalm_device_types.dhcp import DhcpApplyError, DhcpServerMixin, normalize_cidr, normalize_mac
from napalm_device_types.dns_forward import DnsForwardMixin
from napalm_device_types.firewall import FirewallDriver
from napalm_device_types.firewall_rules import FirewallRuleMixin
from napalm_device_types.firmware import FirmwareOfferMixin
from napalm_device_types.firmware import FirmwareOfferDict, FirmwareOfferMixin, FirmwareUpgradeMixin
from napalm_device_types.fixes import (
EnsurePackageMixin,
LldpAgentMixin,
PackageProxyMixin,
SnmpAgentMixin,
ZabbixAgentMixin,
run_fix_steps,
)
from napalm_device_types.health_metrics import HealthMetricsMixin
@@ -197,7 +214,6 @@ from napalm_device_types.models import (
DnsForwardDict,
EventLogEntryDict,
FirmwareDict,
FirmwareOfferDict,
FixResultDict,
FixStepDict,
FixStepOutcomeDict,
@@ -217,8 +233,10 @@ from napalm_device_types.os import OSDriver
from napalm_device_types.ospf import OspfMixin
from napalm_device_types.package_updates import (
APT_UPGRADABLE_COMMAND,
DNF_INSTALLED_CVES_COMMAND,
DNF_SECURITY_COMMAND,
parse_apt_upgradable,
parse_dnf_installed_cves,
parse_dnf_security,
)
from napalm_device_types.packages import PackageManagementMixin
@@ -326,7 +344,7 @@ from napalm_device_types.time_sync import (
check_ntp_servers,
)
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
from napalm_device_types.updates import UpdateMixin
from napalm_device_types.updates import PackageIndexMissing, UpdateMixin
from napalm_device_types.virtual_ips import (
VIRTUAL_IPS_COMMAND,
VirtualIpsMixin,
@@ -334,6 +352,12 @@ from napalm_device_types.virtual_ips import (
)
from napalm_device_types.vm_migration import MigrationPreflightDict
from napalm_device_types.vm_stop_request import VmStopRequestMixin
from napalm_device_types.zabbix_agent import (
ZabbixAgentSettingsDict,
check_zabbix_agent_settings,
zabbix_agent_config,
zabbix_repository_command,
)
__all__ = [
"BACKUP_JOB_KINDS",
@@ -366,6 +390,7 @@ __all__ = [
"OspfMixin",
"UpdateProgressDict",
"FirmwareOfferMixin",
"FirmwareUpgradeMixin",
"LicenseDict",
"LicenseMixin",
"EnsurePackageMixin",
@@ -380,9 +405,21 @@ __all__ = [
"parse_apt_fetch",
"parse_apt_proxy",
"same_proxy_url",
"APT_RELEASES_COMMAND",
"AptReleaseFixDict",
"AptReleaseMixin",
"AptReleasesDict",
"AptSourceDict",
"parse_apt_releases",
"suite_release",
"SnmpAgentMixin",
"SnmpAgentSettingsDict",
"run_fix_steps",
"ZabbixAgentMixin",
"ZabbixAgentSettingsDict",
"check_zabbix_agent_settings",
"zabbix_agent_config",
"zabbix_repository_command",
"SnmpHealthProfile",
"SnmpInterfacesSpec",
"SnmpMemorySpec",
@@ -399,11 +436,16 @@ __all__ = [
"open_stream_on_transport",
"run_on_transport",
"ConfigLifecycleMixin",
"ConfigRevisionsMixin",
"ConfigRevisionDict",
"ConfigRevertFailed",
"UnknownConfigRevision",
"ConfigSyncDict",
"ConfigSyncMixin",
"DhcpHaDict",
"DhcpHaPeerDict",
"DeviceTypeDriver",
"DhcpApplyError",
"DhcpServerMixin",
"FingerprintRule",
"rules_max_weight",
@@ -431,8 +473,10 @@ __all__ = [
"OSDriver",
"PackageManagementMixin",
"APT_UPGRADABLE_COMMAND",
"DNF_INSTALLED_CVES_COMMAND",
"DNF_SECURITY_COMMAND",
"parse_apt_upgradable",
"parse_dnf_installed_cves",
"parse_dnf_security",
"parse_host_status",
"strip_terminal_codes",
@@ -489,6 +533,7 @@ __all__ = [
"SystemdServicesMixin",
"SystemdUnavailable",
"parse_systemd_services",
"PackageIndexMissing",
"UpdateMixin",
"VIRTUAL_IPS_COMMAND",
"VirtualIpsMixin",
+62 -30
View File
@@ -10,10 +10,17 @@ The sources go to one file of their own (:func:`plan_chrony` picks it):
4. else the include file, with an ``include`` line appended to the main configuration
(RHEL's ``/etc/chrony.conf``).
Every other source stops counting: a ``server``, ``pool`` or ``peer`` line in any
other file chrony reads is commented out behind :data:`COMMENT_MARK`, and so is a
``sourcedir`` line other than the one the sources file is in -- the DHCP client's
``/run/chrony-dhcp``, whose files are rewritten on every lease and so are left alone.
Every other configured source stops counting: a ``server``, ``pool`` or ``peer``
line in any other file chrony reads is commented out behind :data:`COMMENT_MARK`.
``sourcedir`` lines stay. The DHCP client's directory (``/run/chrony-dhcp``) and its
files are left alone: DHCP option 42 counts (NetOrk/netork#2380), and the client
rewrites those files on every lease anyway.
4.35.0 commented out every ``sourcedir`` other than the sources file's, the DHCP
client's among them, which left a host deaf to DHCP. A write gives a DHCP client's
``sourcedir`` back where it finds one behind :data:`COMMENT_MARK`; any other one it
took out stays out, its sources would be commented out now anyway.
chronyd reads its main configuration only when it starts: a change there means a
restart.
"""
@@ -26,7 +33,12 @@ import shlex
from dataclasses import dataclass, field
from fnmatch import fnmatchcase
from napalm_device_types._time_sync_reading import SOURCE_LINE, ConfigFile
from napalm_device_types._time_sync_reading import (
CHRONY_DHCP_DIRS,
SOURCE_LINE,
ConfigFile,
in_dhcp_dir,
)
#: What a source line taken out of chrony's configuration starts with.
COMMENT_MARK = "#napalm-time-sync: "
@@ -34,6 +46,8 @@ COMMENT_MARK = "#napalm-time-sync: "
INCLUDE_COMMENT = "# napalm-time-sync: the time sources"
_DIRECTORY_LINE = re.compile(r"\s*(sourcedir|confdir|include)\s+(.+?)\s*$")
#: A ``sourcedir`` line 4.35.0 commented out.
_MARKED_SOURCEDIR = re.compile(rf"{re.escape(COMMENT_MARK)}sourcedir\s+(.+?)\s*$")
@dataclass
@@ -43,6 +57,7 @@ class ChronyPlan:
target: str # the file the sources go to
content: str
comment_out: dict[str, list[int]] = field(default_factory=dict) # path: line numbers
restore: dict[str, list[int]] = field(default_factory=dict) # path: marked sourcedir lines
include_in: str | None = None # the main configuration an include of target is appended to
reload: bool = False # True: ``chronyc reload sources`` takes the change
@@ -80,27 +95,27 @@ def _target(main: ConfigFile, sources_file: str, include_file: str) -> tuple[str
return include_file, main.path, False
def _dropped_sourcedirs(main: ConfigFile, kept: str | None) -> set[int]:
"""The ``sourcedir`` lines of *main* that do not name the directory *kept*."""
return {
number
for number, kind, args in _directives(main)
if kind == "sourcedir" and (kept is None or kept not in args)
}
def _lines_to_comment(f: ConfigFile, dropped: set[int]) -> list[int]:
def _dhcp_sourcedirs_to_restore(main: ConfigFile, dhcp_dirs: tuple[str, ...]) -> list[int]:
"""The lines of *main* where 4.35.0 commented out a DHCP client's ``sourcedir``."""
dhcp = {posixpath.normpath(d) for d in dhcp_dirs}
return [
number
for number, line in enumerate(f.lines, start=1)
if SOURCE_LINE.match(line) or (f.kind == "main" and number in dropped)
for number, line in enumerate(main.lines, start=1)
if (marked := _MARKED_SOURCEDIR.match(line))
and all(posixpath.normpath(arg) in dhcp for arg in marked[1].split())
]
def plan_chrony(
files: list[ConfigFile], servers: list[str], *, sources_file: str, include_file: str
files: list[ConfigFile],
servers: list[str],
*,
sources_file: str,
include_file: str,
dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS,
) -> ChronyPlan:
"""Where *servers* go and what else is commented out.
"""Where *servers* go, what else is commented out, and which DHCP ``sourcedir``
(one of *dhcp_dirs*) comes back.
:raises RuntimeError: when the reading found no chrony configuration.
"""
@@ -108,14 +123,20 @@ def plan_chrony(
if main is None:
raise RuntimeError("chrony runs, but no chrony configuration was found to write to")
target, include_in, is_sources = _target(main, sources_file, include_file)
dropped = _dropped_sourcedirs(main, posixpath.dirname(target) if is_sources else None)
comment_out = _comment_out(files, target, dropped)
comment_out = _comment_out(files, target, dhcp_dirs)
restore = _dhcp_sourcedirs_to_restore(main, dhcp_dirs)
return ChronyPlan(
target=target,
content="".join(f"{source_line(s)}\n" for s in servers),
comment_out=comment_out,
restore={main.path: restore} if restore else {},
include_in=include_in,
reload=is_sources and include_in is None and _only_sources_files(files, comment_out),
reload=(
is_sources
and include_in is None
and not restore
and _only_sources_files(files, comment_out)
),
)
@@ -124,25 +145,36 @@ def _only_sources_files(files: list[ConfigFile], changed: dict[str, list[int]])
return all(f.kind == "sourcedir" for f in files if f.path in changed)
def _comment_out(files: list[ConfigFile], target: str, dropped: set[int]) -> dict[str, list[int]]:
"""Per file, the lines that take a source; a dropped ``sourcedir``'s files are not read."""
def _comment_out(
files: list[ConfigFile], target: str, dhcp_dirs: tuple[str, ...]
) -> dict[str, list[int]]:
"""Per file, the lines that take a source; the DHCP client's files are left alone."""
found: dict[str, list[int]] = {}
for f in files:
if f.path == target or (f.kind == "sourcedir" and f.at in dropped):
if f.path == target or in_dhcp_dir(f, dhcp_dirs):
continue
numbers = _lines_to_comment(f, dropped)
numbers = [n for n, line in enumerate(f.lines, start=1) if SOURCE_LINE.match(line)]
if numbers:
found[f.path] = numbers
return found
def comment_out_command(path: str, numbers: list[int]) -> str:
"""``sed`` that comments out those lines of *path* that still hold a source or a sourcedir."""
"""``sed`` that comments out those lines of *path* that still hold a source."""
edits = " ".join(
"-e "
+ shlex.quote(
f"{n}{{/^[[:space:]]*(server|pool|peer|sourcedir)[[:space:]]/s/^/{COMMENT_MARK}/;}}"
)
+ shlex.quote(f"{n}{{/^[[:space:]]*(server|pool|peer)[[:space:]]/s/^/{COMMENT_MARK}/;}}")
for n in numbers
)
return f"sed -i -E {edits} {shlex.quote(path)}"
def restore_command(path: str, numbers: list[int]) -> str:
"""``sed`` that takes :data:`COMMENT_MARK` off those lines of *path* that are still a
``sourcedir`` behind it. The mark holds nothing a regular expression or ``sed``'s
``s///`` would read as special."""
edits = " ".join(
"-e " + shlex.quote(f"{n}{{s/^{COMMENT_MARK}(sourcedir[[:space:]])/\\1/;}}")
for n in numbers
)
return f"sed -i -E {edits} {shlex.quote(path)}"
+116
View File
@@ -0,0 +1,116 @@
"""The order an applied reservation set is written in (#106).
A DHCP server such as Kea keeps every address unique at each write, not only at the
commit: a reservation that takes an address another one still holds is refused as a
duplicate. So the changes are ordered for an address to be released before it is taken:
* An update that moves a reservation off its address goes before the change that takes
that address -- along a chain, from its free end.
* Adds come after every update: they release nothing, and an update may free their
address.
A *swap* -- two reservations trading addresses, or a longer rotation -- has no such
order. Its changes are still tried, in the order of the desired set: a server that checks
only at the commit takes them. A server that refuses them refuses each with its own
message, and the progress line says it was a swap. Nothing is moved to a temporary
address: the contract knows no free one (pools, leases and reservations nobody manages
all hold addresses), and a second write that failed would leave the reservation on it.
Without I/O: a reconciler's part, see ``docs/STRUCTURE.md``.
"""
from __future__ import annotations
import ipaddress
from dataclasses import dataclass
from napalm_device_types.models import DhcpReservationDict, DhcpReservationDiffDict
@dataclass(frozen=True)
class ReservationChange:
"""One write of an applied set, in the order it is made."""
reservation: DhcpReservationDict
#: The live reservation's id for an update, None for an add.
uuid: str | None
#: The changed fields of an update (``"ip, hostname"``), empty for an add.
fields: str = ""
#: The other reservations' MACs of the swap this change is part of, if any.
swap: tuple[str, ...] = ()
def _address(ip: str | None) -> str:
"""*ip* in one spelling, so ``192.0.2.010``-style differences cannot hide a match."""
text = (ip or "").strip()
try:
return str(ipaddress.ip_address(text))
except ValueError:
return text.lower()
def _waits_on(updates: list[tuple[str, str]]) -> list[int | None]:
"""For each ``(old, new)`` address pair, the update whose release it waits for."""
released = {old: i for i, (old, new) in enumerate(updates) if old and old != new}
return [released.get(new) if new != old else None for old, new in updates]
def _cycles(waits_on: list[int | None]) -> dict[int, list[int]]:
"""Each update that is part of a cycle, with the cycle's members in order."""
members: dict[int, list[int]] = {}
for start in range(len(waits_on)):
path: list[int] = []
node: int | None = start
while node is not None and node not in path and node not in members:
path.append(node)
node = waits_on[node]
if node is not None and node in path:
cycle = sorted(path[path.index(node) :])
members.update(dict.fromkeys(cycle, cycle))
return members
def _update_order(waits_on: list[int | None], cycles: dict[int, list[int]]) -> list[int]:
"""Every update after the one it waits on; stable otherwise. Where only cycles are
left, the first of them goes as it is, and so on."""
order: list[int] = []
done: set[int] = set()
pending = list(range(len(waits_on)))
while pending:
ready = [i for i in pending if waits_on[i] is None or waits_on[i] in done]
if not ready:
ready = [next(i for i in pending if i in cycles)]
order += ready
done.update(ready)
pending = [i for i in pending if i not in done]
return order
def order_reservation_changes(
diff: DhcpReservationDiffDict, live: list[DhcpReservationDict]
) -> list[ReservationChange]:
"""The changes of *diff* in the order they are written; see the module.
:param diff: What ``diff_dhcp_reservations`` found.
:param live: The reservations it compared against, for the addresses updates release.
"""
held = {r.get("uuid"): _address(r.get("ip")) for r in live if r.get("uuid")}
updates = diff["update"]
waits_on = _waits_on(
[(held.get(e["uuid"], ""), _address(e["reservation"].get("ip"))) for e in updates]
)
cycles = _cycles(waits_on)
def swap(i: int) -> tuple[str, ...]:
return tuple(updates[j]["reservation"]["mac"] for j in cycles.get(i, []) if j != i)
ordered = [
ReservationChange(
updates[i]["reservation"],
updates[i]["uuid"],
", ".join(updates[i]["changed_fields"]),
swap(i),
)
for i in _update_order(waits_on, cycles)
]
return ordered + [ReservationChange(r, None) for r in diff["add"]]
+85 -17
View File
@@ -14,10 +14,15 @@ One command, as the login user, prints framed sections (``@@ts <name> ...``):
line of it reads;
* ``ntpd <path>``, ``openntpd <path>`` -- their configuration;
* ``end`` -- the reading is complete.
What DHCP brought is read apart from the configured servers: chrony's files in a
DHCP client's ``sourcedir`` (:data:`CHRONY_DHCP_DIRS`), systemd-timesyncd's
``LinkNTPServers``.
"""
from __future__ import annotations
import posixpath
import re
import shlex
from dataclasses import dataclass, field
@@ -37,6 +42,11 @@ TIME_SYNC_UNITS: tuple[tuple[str, str], ...] = (
("openntpd", "openntpd"),
)
#: The ``sourcedir`` a DHCP client writes chrony's sources to, one file per interface
#: and lease: the NetworkManager dispatcher and dhclient hook of Debian, Ubuntu and
#: Red Hat. What chrony reads from there is DHCP's (option 42), not configuration.
CHRONY_DHCP_DIRS: tuple[str, ...] = ("/run/chrony-dhcp", "/var/run/chrony-dhcp")
#: A source line of chrony's or ntpd's configuration: the directive and what follows.
SOURCE_LINE = re.compile(r"\s*(server|pool|peer)\s+(.*)")
_OPENNTPD_SOURCE = re.compile(r"\s*(servers?)\s+(\S+)")
@@ -197,19 +207,47 @@ def _source_name(rest: str) -> str | None:
return next((word for word in rest.split() if not word.startswith("-")), None)
def chrony_servers(files: list[ConfigFile]) -> list[str]:
"""chrony's sources in the order it reads them: each file in place of the line naming it."""
def in_dhcp_dir(f: ConfigFile, dhcp_dirs: tuple[str, ...]) -> bool:
"""Whether *f* is a sources file a DHCP client wrote to one of *dhcp_dirs*."""
if f.kind != "sourcedir":
return False
directory = posixpath.normpath(posixpath.dirname(f.path))
return directory in {posixpath.normpath(d) for d in dhcp_dirs}
def _named_on(files: list[ConfigFile], number: int) -> list[ConfigFile]:
"""The files the main configuration's line *number* reads."""
return [f for f in files if f.at == number and f.kind != "main"]
def _read_order(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[tuple[bool, str]]:
"""Every line chrony reads, in its order -- each file in place of the line naming
it -- and whether a DHCP client wrote it."""
main = next((f for f in files if f.kind == "main"), None)
if main is None:
return []
found: list[str] = []
found: list[tuple[bool, str]] = []
for number, line in enumerate(main.lines, start=1):
found.extend(_sources([line]))
for included in (f for f in files if f.at == number and f.kind != "main"):
found.extend(_sources(included.lines))
found.append((False, line))
for included in _named_on(files, number):
dhcp = in_dhcp_dir(included, dhcp_dirs)
found.extend((dhcp, text) for text in included.lines)
return found
def chrony_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...] = ()) -> list[str]:
"""chrony's configured sources in the order it reads them; not those in *dhcp_dirs*."""
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if not dhcp])
def chrony_dhcp_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[str]:
"""The sources chrony reads from the DHCP client's files in *dhcp_dirs*, in its order.
A ``sourcedir`` chrony does not read (commented out) prints no files: nothing then.
"""
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if dhcp])
def _sources(lines: list[str]) -> list[str]:
names = []
for line in lines:
@@ -229,14 +267,30 @@ def openntpd_servers(lines: list[str]) -> list[str]:
return [m[2] for line in lines if (m := _OPENNTPD_SOURCE.match(line))]
def timesyncd_servers(values: dict[str, str]) -> list[str]:
"""systemd-timesyncd's servers in the order it tries them; the fallback only alone."""
def _listed(values: dict[str, str], *keys: str) -> list[str]:
found: list[str] = []
for key in ("RuntimeNTPServers", "SystemNTPServers", "LinkNTPServers"):
for key in keys:
for server in values.get(key, "").split():
if server not in found:
found.append(server)
return found or values.get("FallbackNTPServers", "").split()
return found
def timesyncd_servers(values: dict[str, str]) -> list[str]:
"""systemd-timesyncd's configured servers, runtime then system; the fallback only
while it has no other server, a link's included. A link's servers are not among
them: :func:`timesyncd_link_servers`."""
configured = _listed(values, "RuntimeNTPServers", "SystemNTPServers")
if configured or timesyncd_link_servers(values):
return configured
return _listed(values, "FallbackNTPServers")
def timesyncd_link_servers(values: dict[str, str]) -> list[str]:
"""The servers network links gave systemd-timesyncd: systemd-networkd's, from a DHCP
lease (option 42) or a ``.network`` file's ``NTP=`` -- the reading cannot tell those
apart and counts them as DHCP's."""
return _listed(values, "LinkNTPServers")
def chrony_synchronized(sources: str) -> bool | None:
@@ -263,9 +317,9 @@ def _time_zone(reading: Reading) -> str | None:
return next((z for line in reading.zone_files if (z := _zone_name(line))), None)
def _servers(reading: Reading) -> list[str]:
def _servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str]:
if reading.daemon == "chrony":
return chrony_servers(reading.chrony)
return chrony_servers(reading.chrony, dhcp_dirs)
if reading.daemon == "systemd-timesyncd":
return timesyncd_servers(reading.timesync)
if reading.daemon == "ntpd" and reading.ntpd is not None:
@@ -275,6 +329,18 @@ def _servers(reading: Reading) -> list[str]:
return []
def _dhcp_servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str] | None:
"""What DHCP brought: chrony's and timesyncd's; None where ntpd or OpenNTPD runs,
whose DHCP hooks keep a configuration of their own that is not read."""
if reading.daemon == "chrony":
return chrony_dhcp_servers(reading.chrony, dhcp_dirs)
if reading.daemon == "systemd-timesyncd":
return timesyncd_link_servers(reading.timesync)
if reading.daemon is None:
return []
return None
def _synchronized(reading: Reading) -> bool | None:
if reading.daemon == "chrony":
chrony = chrony_synchronized(reading.chronyc)
@@ -283,19 +349,21 @@ def _synchronized(reading: Reading) -> bool | None:
return {"yes": True, "no": False}.get(reading.timedatectl.get("NTPSynchronized", ""))
def time_sync_of(reading: Reading) -> TimeSyncDict:
"""What a reading says, as ``get_time_sync`` returns it."""
def time_sync_of(reading: Reading, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
"""What a reading says, as ``get_time_sync`` returns it; *dhcp_dirs* are the DHCP
client's ``sourcedir`` of chrony."""
return {
"ntp_servers": _servers(reading),
"ntp_servers": _servers(reading, dhcp_dirs),
"timezone": _time_zone(reading),
"synchronized": _synchronized(reading),
"daemon": reading.daemon,
"dhcp_ntp_servers": _dhcp_servers(reading, dhcp_dirs),
}
def parse_time_sync(text: str) -> TimeSyncDict:
def parse_time_sync(text: str, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
"""The reading's output as ``get_time_sync`` returns it.
:raises ValueError: for a reading cut short.
"""
return time_sync_of(parse_reading(text))
return time_sync_of(parse_reading(text), dhcp_dirs)
+108 -18
View File
@@ -35,8 +35,12 @@ page is the right one, the caller says (``expect``).
out every other line that sets a different one, reads the result back from
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
last step is what proves it: apt reached a mirror through the proxy it is now set
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
to. The source chosen is one apt does not fetch ``DIRECT``. Then the https sources
go through the proxy in the form it stores (:mod:`~napalm_device_types.apt_sources`),
and one of them is fetched; where the proxy cannot, they go back to https. Last it
fetches one https source left the same way; where the proxy refuses it, https goes
``DIRECT``. Removing the proxy turns the sources back to https first: without the
proxy apt cannot fetch the stored form.
"""
from __future__ import annotations
@@ -49,6 +53,15 @@ from urllib.parse import urlsplit
from typing_extensions import TypedDict
from napalm_device_types.apt_releases import AptReleaseMixin
from napalm_device_types.apt_sources import (
NOT_CACHED_KEY,
VIA_CACHE,
cacheable_https_hosts,
is_host_name,
revert_sources_command,
rewrite_sources_command,
)
from napalm_device_types.fixes import PackageProxyMixin, run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
from napalm_device_types.terminal import strip_terminal_codes
@@ -87,6 +100,9 @@ class AptProxyReportDict(TypedDict):
#: The base URIs of the host's sources over http and over https.
http_sources: list[str]
https_sources: list[str]
#: The hosts the cache rewrite left on https, for their credentials
#: (:data:`~napalm_device_types.apt_sources.NOT_CACHED_KEY`).
not_cached: list[str]
class ProxyFetchDict(TypedDict):
@@ -123,7 +139,8 @@ PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
#: them early.
APT_PROXY_COMMAND = (
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
"echo '[dump]'; apt-config dump 2>/dev/null "
"| grep -i -E '^(Acquire::https?::Proxy|Package-Proxy::)'; "
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
@@ -133,6 +150,7 @@ APT_PROXY_COMMAND = (
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
)
_NOT_CACHED = re.compile(rf'^{re.escape(NOT_CACHED_KEY)}::\s+"([^"]+)"', re.I)
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
@@ -293,6 +311,7 @@ def parse_apt_proxy(output: str) -> AptProxyReportDict:
"https_settings": _settings(lines, _https_setting),
"http_sources": _of_scheme(sources, "http"),
"https_sources": _of_scheme(sources, "https"),
"not_cached": [m.group(1) for m in map(_NOT_CACHED.match, sections.get("dump", [])) if m],
}
@@ -409,7 +428,7 @@ def _described(s: AptProxySettingDict) -> str:
return f"{s['file']}:{s['line']} ({s['value']}) {what}"
class AptProxyMixin(PackageProxyMixin):
class AptProxyMixin(AptReleaseMixin, PackageProxyMixin):
""":class:`~napalm_device_types.fixes.PackageProxyMixin` for a Debian-family host.
The template form, like :class:`~napalm_device_types.host_status.HostStatusMixin`:
@@ -498,16 +517,20 @@ class AptProxyMixin(PackageProxyMixin):
def configure_package_proxy(self, url: str) -> FixResultDict:
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
The https sources go through *url* in the form it stores, where it can; the
rest goes through *url* too, unless *url* refuses it: then ``DIRECT``.
"""
if not url:
return run_fix_steps([("configure", self._remove_package_proxy)])
return run_fix_steps(
[("sources", self._revert_sources), ("configure", self._remove_package_proxy)]
)
return run_fix_steps(
[
("configure", lambda: self._write_package_proxy(url)),
("conflicts", lambda: self._comment_out_other_proxies(url)),
("verify", lambda: self._verify_package_proxy(url)),
("fetch", lambda: self._fetch_through_package_proxy(url)),
("cache", lambda: self._cache_https_sources(url)),
("https", lambda: self._https_through_package_proxy(url)),
]
)
@@ -515,6 +538,11 @@ class AptProxyMixin(PackageProxyMixin):
def _not_apt(self) -> FixStepOutcomeDict:
return {"ok": False, "output": "not an apt host"}
def _revert_sources(self) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
return self._run_apt_root_command(revert_sources_command(), "sources back on https")
def _remove_package_proxy(self) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
@@ -522,7 +550,9 @@ class AptProxyMixin(PackageProxyMixin):
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
)
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
def _write_package_proxy(
self, url: str, https_direct: bool = False, not_cached: list[str] | None = None
) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
if not PROXY_URL.match(url):
@@ -530,6 +560,10 @@ class AptProxyMixin(PackageProxyMixin):
content = f'Acquire::http::Proxy "{url}";\n'
if https_direct:
content += 'Acquire::https::Proxy "DIRECT";\n'
# Host names the report read from this file; one a line could not carry is left out.
content += "".join(
f'{NOT_CACHED_KEY}:: "{host}";\n' for host in not_cached or [] if is_host_name(host)
)
return self._write_apt_file(self.APT_PROXY_FILE, content)
def _report(self) -> AptProxyReportDict:
@@ -566,16 +600,7 @@ class AptProxyMixin(PackageProxyMixin):
}
def _fetch_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
report = self._report()
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
source = next(
(
s
for s in report["http_sources"]
if "/dists/" in s and (urlsplit(s).hostname or "").lower() not in direct
),
None,
)
source = _plain_http_source(self._report())
if source is None:
return {"ok": True, "output": "no http source to fetch through the proxy; not tried"}
target = source.rstrip("/") + "/InRelease"
@@ -584,6 +609,49 @@ class AptProxyMixin(PackageProxyMixin):
return {"ok": True, "output": f"fetched {target} through {url}"}
return {"ok": False, "output": f"{target}: {fetched['error']}"}
def _cache_https_sources(self, url: str) -> FixStepOutcomeDict:
"""The https sources through *url* in the form it stores; back to https where it cannot."""
hosts = cacheable_https_hosts(self._report())
if not hosts:
return {"ok": True, "output": "no https source to cache"}
rewritten = self._run_apt_root_command(
rewrite_sources_command(hosts, self.APT_PROXY_FILE), "sources rewritten"
)
if not rewritten["ok"]:
return rewritten
return self._fetch_cached(url, hosts, self._report())
def _fetch_cached(
self, url: str, hosts: list[str], report: AptProxyReportDict
) -> FixStepOutcomeDict:
"""One rewritten source through *url*: it stays where *url* fetched it."""
left = sorted(set(hosts) & set(report["not_cached"]))
named = f"; left on https, apt has credentials for them: {', '.join(left)}" if left else ""
target = next(
(s.rstrip("/") + "/InRelease" for s in report["http_sources"] if _cached(s)), None
)
if target is None:
return {"ok": True, "output": f"no https source to cache{named}"}
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
if fetched["reachable"] and not fetched["error"]:
cached = ", ".join(h for h in hosts if h not in left)
return {"ok": True, "output": f"https sources cached through {url}: {cached}{named}"}
return self._back_to_https(url, f"{target}: {fetched['error']}", named)
def _back_to_https(self, url: str, why: str, named: str) -> FixStepOutcomeDict:
"""*url* did not fetch the stored form: every source goes back to https."""
back = self._run_apt_root_command(revert_sources_command(), "sources back on https")
if not back["ok"]:
return {
"ok": False,
"output": f"{why}; turning the sources back to https failed: {back['output']}",
}
return {
"ok": True,
"output": f"{url} does not fetch https sources itself ({why}); "
f"they stay on https, not cached{named}",
}
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
@@ -609,7 +677,9 @@ class AptProxyMixin(PackageProxyMixin):
def _https_direct(self, url: str) -> FixStepOutcomeDict:
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
written = self._write_package_proxy(url, https_direct=True)
written = self._write_package_proxy(
url, https_direct=True, not_cached=self._report()["not_cached"]
)
if not written["ok"]:
return written
later = self._later_https_lines(self._report(), url)
@@ -635,6 +705,26 @@ class AptProxyMixin(PackageProxyMixin):
]
def _plain_http_source(report: AptProxyReportDict) -> str | None:
"""A suite's http source the proxy takes, not in the cache's https form: its proof."""
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
return next(
(
s
for s in report["http_sources"]
if "/dists/" in s
and not s.startswith(VIA_CACHE)
and (urlsplit(s).hostname or "").lower() not in direct
),
None,
)
def _cached(source: str) -> bool:
"""A source of a suite in the form the cache stores."""
return source.startswith(VIA_CACHE) and "/dists/" in source
def _https_source(
report: AptProxyReportDict, through: Callable[[str | None], bool]
) -> tuple[str, str] | None:
+486
View File
@@ -0,0 +1,486 @@
"""apt's sources and the release each names, and moving one file's sources to the host's.
A release upgrade moves the distribution's own sources to the new release and
leaves the third-party ones where they were -- or turns them off, and the person
who turns them back on keeps the old name. A source on the old release still
answers, so apt says nothing; its packages simply stop getting updates once the
vendor stops building for that release. Docker's last build for Ubuntu 20.04
(``focal``) is 28.1.1, and a noble host on ``focal`` never sees a newer one
(NetOrk/netork#2614).
**What is read** (:meth:`AptReleaseMixin.get_apt_source_releases`): the host's
``/etc/os-release`` (``ID``, ``ID_LIKE``, ``VERSION_CODENAME``,
``UBUNTU_CODENAME``) and every source apt reads -- ``sources.list`` and the
``*.list`` and ``*.sources`` files of ``sources.list.d``, wherever ``apt-config``
says they are -- with its file, line, types, URI, suites and components.
Commented-out lines and deb822 paragraphs with ``Enabled: no`` are not sources.
Whether a suite names another release than the host's is not said here: which
words are releases of which distribution is the caller's to know
(:func:`suite_release` gives the word).
**Moving a file** (:meth:`AptReleaseMixin.set_apt_source_release`): every suite of
*release* in one of apt's source files becomes the same suite of *codename*
(``focal-security`` → ``noble-security``), on enabled lines only. Before anything is
written, each new suite's ``InRelease`` (or ``Release``) is fetched from its URI with
apt's own downloader and the proxy apt is set to; when one of them is not there,
the vendor does not build for *codename* and the file stays as it is. The file is
written whole, as root, mode 644. Loading the package index afterwards is the
caller's (``refresh_available_updates``).
"""
from __future__ import annotations
import base64
import re
import shlex
from dataclasses import dataclass, field
from typing import TYPE_CHECKING
from typing_extensions import TypedDict
from napalm_device_types.fixes import run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
from napalm_device_types.terminal import strip_terminal_codes
_BEGIN = "ARELEASE_BEGIN"
_END = "ARELEASE_END"
#: A release's codename: one lower-case word (``noble``, ``bookworm``).
_CODENAME = re.compile(r"^[a-z][a-z0-9]*$")
#: A suite that names a release: its codename, then perhaps a suffix (``-security``).
_SUITE = re.compile(r"^([a-z][a-z0-9]*)(-[a-z0-9-]+)?$")
#: One source line of a ``.list`` file, its comment cut off: types, options, URI, suite.
_LIST_LINE = re.compile(r"^(\s*(deb|deb-src)\s+(?:\[[^\]]*\]\s+)?(\S+)\s+)(\S+)(.*)$")
#: A deb822 field's first line.
_FIELD = re.compile(r"^([A-Za-z][A-Za-z0-9-]*):(.*)$")
_DISABLED = frozenset({"no", "false", "0"})
class AptSourceDict(TypedDict):
"""One source apt reads (``get_apt_source_releases``)."""
#: The file it is in.
file: str
#: The line of a ``.list`` file's entry; of a deb822 paragraph's ``Suites:`` field.
line: int
#: ``deb``, ``deb-src``; a deb822 paragraph may name both.
types: list[str]
#: Its URI, as written; a deb822 paragraph with several URIs is one source per URI.
uri: str
#: Its suites; a flat repository's path (``./``) is one too.
suites: list[str]
components: list[str]
class AptReleasesDict(TypedDict):
"""The host's release and its sources (``get_apt_source_releases``)."""
#: ``ID`` and ``ID_LIKE`` of ``/etc/os-release``; "" and [] when it says none.
os_id: str
os_like: list[str]
#: ``VERSION_CODENAME``; a derivative's own (Linux Mint's ``virginia``).
codename: str
#: ``UBUNTU_CODENAME``: the Ubuntu release Ubuntu and its derivatives are built on.
ubuntu_codename: str
sources: list[AptSourceDict]
class AptReleaseFixDict(FixResultDict):
"""The result of ``set_apt_source_release``: the fix, and how many suites moved."""
rewritten: int
def apt_releases_command(
apt_config: str = "apt-config", os_release: str = "/etc/os-release"
) -> str:
"""The read-only command that prints the host's release and apt's source files.
One line, POSIX ``sh``, no privileges. Every line of a file is printed behind a
``|``, so an empty line -- which ends a deb822 paragraph -- comes through.
*apt_config* and *os_release* let a test point it elsewhere.
"""
return (
f"( export LC_ALL=C; printf '%s%s\\n' ARELEASE_ BEGIN; echo '@@os'; "
f'o={shlex.quote(os_release)}; [ -r "$o" ] || o=/usr/lib/os-release; '
'[ -r "$o" ] && sed \'s/^/|/\' "$o"; '
f'eval "$({apt_config} shell sl Dir::Etc::sourcelist/f sp Dir::Etc::sourceparts/d '
'2>/dev/null)"; sl="${sl:-/etc/apt/sources.list}"; sp="${sp:-/etc/apt/sources.list.d/}"; '
'for f in "$sl" "$sp"*.list "$sp"*.sources; do [ -f "$f" ] || continue; '
'if [ -r "$f" ]; then printf \'@@file %s\\n\' "$f"; sed \'s/^/|/\' "$f"; '
"else printf '@@unreadable %s\\n' \"$f\"; fi; done; "
"printf '%s%s\\n' ARELEASE_ END ) 2>/dev/null | cat"
)
#: The command as a driver runs it.
APT_RELEASES_COMMAND = apt_releases_command()
def suite_release(suite: str) -> str | None:
"""The word a suite starts with, ``noble`` of ``noble-security``; None for a path.
Whether that word is a release, and of which distribution, the caller knows:
``stable`` and ``nodistro`` have the form too.
"""
found = _SUITE.match(suite)
return found.group(1) if found else None
def _codename(name: str) -> str:
if not _CODENAME.match(name):
raise ValueError(f"not a release's codename: {name!r}")
return name
# ── Reading ──────────────────────────────────────────────────────────────────
def _blocks(output: str) -> list[tuple[str, list[str]]]:
"""``(header, lines)`` of each block between the frame markers, in order."""
lines = strip_terminal_codes(output).splitlines()
stripped = [line.strip() for line in lines]
try:
start = stripped.index(_BEGIN)
end = stripped.index(_END, start)
except ValueError:
raise ValueError("no intact apt release report in the output") from None
blocks: list[tuple[str, list[str]]] = []
for line in lines[start + 1 : end]:
if line.startswith("@@"):
blocks.append((line[2:].strip(), []))
elif line.startswith("|") and blocks:
blocks[-1][1].append(line[1:].rstrip("\r"))
return blocks
def _os_release(lines: list[str]) -> dict[str, str]:
values: dict[str, str] = {}
for line in lines:
key, sep, value = line.strip().partition("=")
if sep and key:
values[key] = value.strip().strip("'\"")
return values
def _code(line: str) -> str:
"""*line* without its comment."""
return line.split("#", 1)[0]
def _list_sources(path: str, lines: list[str]) -> list[AptSourceDict]:
found: list[AptSourceDict] = []
for number, line in enumerate(lines, start=1):
entry = _LIST_LINE.match(_code(line))
if entry:
_, kind, uri, suite, rest = entry.groups()
found.append(
{
"file": path,
"line": number,
"types": [kind],
"uri": uri,
"suites": [suite],
"components": rest.split(),
}
)
return found
@dataclass
class _Field:
first: int # index of its first line
lines: list[int] = field(default_factory=list) # indices of every line it spans
value: str = ""
@dataclass
class _Paragraph:
fields: dict[str, _Field] = field(default_factory=dict)
def value(self, name: str) -> str:
found = self.fields.get(name)
return found.value if found else ""
@property
def enabled(self) -> bool:
return self.value("enabled").strip().lower() not in _DISABLED
def _paragraphs(lines: list[str]) -> list[_Paragraph]:
"""The deb822 paragraphs of *lines*; comments are skipped, an empty line ends one."""
paragraphs: list[_Paragraph] = []
current: _Paragraph | None = None
last: _Field | None = None
for index, line in enumerate(lines):
if not line.strip():
current, last = None, None
continue
if line.lstrip().startswith("#"):
continue
if line[0] in " \t":
if last is not None:
last.lines.append(index)
last.value += " " + line.strip()
continue
found = _FIELD.match(line)
if not found:
continue
if current is None:
current = _Paragraph()
paragraphs.append(current)
last = _Field(first=index, lines=[index], value=found.group(2).strip())
current.fields[found.group(1).lower()] = last
return paragraphs
def _deb822_sources(path: str, lines: list[str]) -> list[AptSourceDict]:
found: list[AptSourceDict] = []
for paragraph in _paragraphs(lines):
suites = paragraph.fields.get("suites")
if not paragraph.enabled or suites is None:
continue
for uri in paragraph.value("uris").split():
found.append(
{
"file": path,
"line": suites.first + 1,
"types": paragraph.value("types").split(),
"uri": uri,
"suites": suites.value.split(),
"components": paragraph.value("components").split(),
}
)
return found
def _is_deb822(path: str) -> bool:
return path.endswith(".sources")
def parse_apt_releases(output: str) -> AptReleasesDict:
"""What :func:`apt_releases_command` printed, as :class:`AptReleasesDict`.
:raises ValueError: if the output carried no intact report.
"""
os_values: dict[str, str] = {}
sources: list[AptSourceDict] = []
for header, lines in _blocks(output):
kind, _, path = header.partition(" ")
if kind == "os":
os_values = _os_release(lines)
elif kind == "file" and path:
read = _deb822_sources if _is_deb822(path) else _list_sources
sources.extend(read(path, lines))
return {
"os_id": os_values.get("ID", "").lower(),
"os_like": os_values.get("ID_LIKE", "").lower().split(),
"codename": os_values.get("VERSION_CODENAME", "").lower(),
"ubuntu_codename": os_values.get("UBUNTU_CODENAME", "").lower(),
"sources": sources,
}
# ── Rewriting ────────────────────────────────────────────────────────────────
def _moved(suite: str, release: str, codename: str) -> str:
return codename + suite[len(release) :] if suite_release(suite) == release else suite
def _rewrite_list_line(line: str, release: str, codename: str) -> tuple[str, int]:
code = _code(line)
entry = _LIST_LINE.match(code)
if not entry:
return line, 0
suite = entry.group(4)
new = _moved(suite, release, codename)
if new == suite:
return line, 0
head = entry.group(1)
return head + new + line[len(head) + len(suite) :], 1
def _rewrite_tokens(text: str, release: str, codename: str) -> tuple[str, int]:
changed = 0
def one(token: re.Match[str]) -> str:
nonlocal changed
new = _moved(token.group(), release, codename)
changed += new != token.group()
return new
return re.sub(r"\S+", one, text), changed
def _rewrite_deb822(lines: list[str], release: str, codename: str) -> int:
changed = 0
for paragraph in _paragraphs(lines):
suites = paragraph.fields.get("suites")
if not paragraph.enabled or suites is None:
continue
for index in suites.lines:
line = lines[index]
cut = line.index(":") + 1 if index == suites.first else 0
tail, count = _rewrite_tokens(line[cut:], release, codename)
lines[index] = line[:cut] + tail
changed += count
return changed
def rewrite_release(text: str, release: str, codename: str, *, deb822: bool) -> tuple[str, int]:
"""*text*, a source file, with every suite of *release* on *codename*; and how many.
Only enabled sources: not a commented-out line, not a paragraph with
``Enabled: no``. A URI or comment that names *release* stays as it is.
:raises ValueError: if *release* or *codename* is not a codename.
"""
release, codename = _codename(release), _codename(codename)
lines = text.split("\n")
if deb822:
changed = _rewrite_deb822(lines, release, codename)
else:
changed = 0
for index, line in enumerate(lines):
lines[index], count = _rewrite_list_line(line, release, codename)
changed += count
return "\n".join(lines), changed
# ── The mixin ────────────────────────────────────────────────────────────────
def _release_urls(uri: str, suite: str) -> list[str]:
base = f"{uri.rstrip('/')}/dists/{suite}/"
return [base + "InRelease", base + "Release"]
class AptReleaseMixin:
"""Reads the release each apt source names; moves one file's sources to another.
The template form, like :class:`~napalm_device_types.apt_proxy.AptProxyMixin`,
which brings it to every driver that has it: a driver supplies
``_run_apt_command`` and ``_write_apt_file``, and ``_is_apt_host`` where it
can be something else.
"""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def _run_apt_command(self, command: str) -> str:
"""Run *command* on the host with ``sh``, without privileges; what it printed."""
...
def _write_apt_file(self, path: str, content: str) -> FixStepOutcomeDict:
"""Replace *path* with *content*, as a new file owned by root, mode 644."""
...
def _is_apt_host(self) -> bool:
"""Whether apt is the host's package manager; a driver that can tell overrides it."""
return True
def get_apt_source_releases(self) -> AptReleasesDict | None:
"""The host's release and every source apt reads; None on a host without apt.
:raises ValueError: if the host's output carried no intact report.
"""
if not self._is_apt_host():
return None
return parse_apt_releases(self._run_apt_command(APT_RELEASES_COMMAND))
def set_apt_source_release(self, path: str, release: str, codename: str) -> AptReleaseFixDict:
"""Every suite of *release* in *path* on *codename*, where the vendor serves it.
Three steps: ``read`` (*path* is one of apt's source files, and which of its
sources name *release*), ``check`` (each new suite's ``InRelease`` or
``Release`` is there) and ``rewrite``. Nothing is written when a step fails;
nothing is fetched or written when no source of *path* names *release*.
"""
moving: list[AptSourceDict] = []
rewritten = [0]
result = run_fix_steps(
[
("read", lambda: self._sources_on(path, release, codename, moving)),
("check", lambda: self._served(moving, release, codename)),
("rewrite", lambda: self._rewrite(path, release, codename, moving, rewritten)),
]
)
return {**result, "rewritten": rewritten[0]}
def _sources_on(
self, path: str, release: str, codename: str, moving: list[AptSourceDict]
) -> FixStepOutcomeDict:
_codename(release)
_codename(codename)
found = self.get_apt_source_releases()
if found is None:
return {"ok": False, "output": "not an apt host"}
in_file = [s for s in found["sources"] if s["file"] == path]
if not in_file:
return {"ok": False, "output": f"{path} is none of apt's source files"}
moving.extend(s for s in in_file if any(suite_release(x) == release for x in s["suites"]))
if not moving:
return {"ok": True, "output": f"no source in {path} is on {release} any more"}
named = "; ".join(f"{s['uri']} {' '.join(s['suites'])}" for s in moving)
return {"ok": True, "output": f"on {release} in {path}: {named}"}
def _served(
self, moving: list[AptSourceDict], release: str, codename: str
) -> FixStepOutcomeDict:
from napalm_device_types.apt_proxy import apt_fetch_command, parse_apt_fetch
wanted = sorted(
{
(s["uri"], _moved(suite, release, codename))
for s in moving
for suite in s["suites"]
if suite_release(suite) == release
}
)
said: list[str] = []
for uri, suite in wanted:
error = ""
for url in _release_urls(uri, suite):
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(url)))
error = fetched["error"]
if not error:
said.append(f"{uri} serves {suite}")
break
else:
return {"ok": False, "output": f"{uri} has no {suite}: {error}; nothing changed"}
return {"ok": True, "output": "; ".join(said) or "nothing to check"}
def _rewrite(
self,
path: str,
release: str,
codename: str,
moving: list[AptSourceDict],
rewritten: list[int],
) -> FixStepOutcomeDict:
if not moving:
return {"ok": True, "output": "nothing to rewrite"}
encoded = "".join(self._run_apt_command(f"base64 {shlex.quote(path)}").split())
text = base64.b64decode(encoded).decode()
new, count = rewrite_release(text, release, codename, deb822=_is_deb822(path))
if not count:
return {"ok": True, "output": f"no source in {path} is on {release} any more"}
written = self._write_apt_file(path, new)
if not written["ok"]:
return written
rewritten[0] = count
suites = "1 suite" if count == 1 else f"{count} suites"
return {"ok": True, "output": f"{path}: {suites} moved from {release} to {codename}"}
__all__ = [
"APT_RELEASES_COMMAND",
"AptReleaseFixDict",
"AptReleaseMixin",
"AptReleasesDict",
"AptSourceDict",
"apt_releases_command",
"parse_apt_releases",
"rewrite_release",
"suite_release",
]
+172
View File
@@ -0,0 +1,172 @@
"""apt's https sources through the package cache, so that the cache stores them.
A source fetched over https goes through apt-cacher-ng's tunnel, if at all: the
TLS runs end to end and nothing of it is stored. The cache fetches
``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>`` itself, with the
certificate verified, and stores it -- where it allows port 443
(``AllowUserPorts=80 443``) (NetOrk/netork#2555). apt fetches that form through
its http proxy, so it works only while the cache is that proxy: without one apt
looks for a host called ``HTTPS``.
**Which hosts** (:func:`cacheable_https_hosts`): the hosts of the https sources,
on port 443, while apt has an http proxy for every host and none of its own for
that one. Not a host with a login in the URL, nor one whose credentials apt keeps
in ``auth.conf``: apt finds those by the host's name, which the rewrite changes,
and paid content does not belong in a cache other hosts share. Only root reads
``auth.conf``, so the rewrite judges that, leaves such a host on https and names
it in the proxy file as ``Package-Proxy::Not-Cached``, which apt ignores and
``apt-config dump`` shows to the report.
**The rewrite** (:func:`rewrite_sources_command`) turns ``https://<host>`` into
the cache form on the lines that name sources: ``deb`` and ``deb-src`` in a
``.list`` file, ``URIs:`` in a deb822 ``.sources`` file. A file without the host
is not touched. :func:`revert_sources_command` turns every source back.
"""
from __future__ import annotations
import re
import shlex
from typing import TYPE_CHECKING
from urllib.parse import urlsplit
if TYPE_CHECKING: # pragma: no cover - the report's type, without an import cycle
from napalm_device_types.apt_proxy import AptProxyReportDict
#: The prefix apt-cacher-ng fetches over https.
VIA_CACHE = "http://HTTPS///"
#: The key in the proxy file that names the hosts left on https for their credentials.
NOT_CACHED_KEY = "Package-Proxy::Not-Cached"
_LABEL = r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?"
_HOST = re.compile(rf"^{_LABEL}(?:\.{_LABEL})*$")
_NO_PROXY = frozenset({"", "direct"})
#: Where apt's sources and credentials are, as shell variables; apt's defaults
#: when apt-config cannot say.
_LOCATE = (
'eval "$({apt_config} shell sl Dir::Etc::sourcelist/f sp Dir::Etc::sourceparts/d '
'nr Dir::Etc::netrc/f np Dir::Etc::netrcparts/d 2>/dev/null)"; '
'sl="${{sl:-/etc/apt/sources.list}}"; sp="${{sp:-/etc/apt/sources.list.d/}}"; '
'nr="${{nr:-/etc/apt/auth.conf}}"; np="${{np:-/etc/apt/auth.conf.d/}}"; '
)
#: The lines that name sources: ``deb``/``deb-src`` and deb822's ``URIs:``.
_SOURCE_LINES = "/^[[:space:]]*(deb|deb-src|[Uu][Rr][Ii][Ss]:)/"
#: The hosts apt has credentials for: each word after ``machine``, without its
#: scheme, port and path, in lower case.
_CREDENTIAL_HOSTS = (
'creds=$(cat "$nr" "$np"* 2>/dev/null | tr -s " \\t\\r" "\\n" '
"| awk 'p {{ print; p = 0; next }} $0 == \"machine\" {{ p = 1 }}' "
"| sed -e 's|^[A-Za-z+]*://||' -e 's|[:/].*||' | tr A-Z a-z); "
)
_REWRITE = (
"( " + _LOCATE + _CREDENTIAL_HOSTS + "for h in {hosts}; do "
'l=$(printf "%s" "$h" | tr A-Z a-z); '
'if printf "%s\\n" "$creds" | grep -qxF -- "$l"; then '
'grep -qF -- "\\"$h\\"" {proxy_file} 2>/dev/null || '
'printf \'{key}:: "%s";\\n\' "$h" >> {proxy_file} || exit 1; continue; fi; '
"e=$(printf '%s' \"$h\" | sed 's/[.]/[.]/g'); "
'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
'[ -f "$f" ] && grep -qF -- "https://$h" "$f" || continue; '
'sed -i -E "{lines} s#([[:space:]:])https://$e([/[:space:]]|\\$)#\\1{via}$h\\2#g" "$f" '
"|| exit 1; done; done )"
)
_REVERT = (
"( " + _LOCATE + 'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
'[ -f "$f" ] && grep -qF -- "{via}" "$f" || continue; '
"sed -i -E '{lines} s#([[:space:]:]){via}#\\1https://#g' \"$f\" || exit 1; done )"
)
def cache_form(url: str) -> str:
"""*url*, an https URL, in the form apt-cacher-ng fetches over https and stores.
:raises ValueError: if *url* is not an https URL.
"""
if not url.startswith("https://"):
raise ValueError(f"not an https URL: {url!r}")
return VIA_CACHE + url[len("https://") :]
def _per_host_direct(report: AptProxyReportDict) -> set[str]:
hosts = {**report["per_host"], **report["https_per_host"]}
return {h.lower() for h, v in hosts.items() if v.lower() in _NO_PROXY}
def _host_of(source: str) -> str | None:
"""The host of an https source the cache can fetch: port 443, no login in the URL."""
try:
parts = urlsplit(source)
port = parts.port
except ValueError:
return None
if parts.username is not None or port not in (None, 443) or not parts.hostname:
return None
return parts.hostname
def cacheable_https_hosts(report: AptProxyReportDict) -> list[str]:
"""The hosts of the https sources the rewrite would send through the cache, sorted.
None while apt has no http proxy for every host: the cache form goes through it.
Not a host apt fetches ``DIRECT``, nor one the rewrite left on https for its
credentials (``not_cached``), nor one on another port or with a login in the URL.
"""
if not report["proxy"]:
return []
skip = _per_host_direct(report) | {h.lower() for h in report.get("not_cached", [])}
hosts = {_host_of(s) for s in report["https_sources"]}
return sorted(h for h in hosts if h and h.lower() not in skip)
def is_host_name(host: str) -> bool:
"""Whether *host* is a DNS name: letters, digits and hyphens between dots."""
return bool(_HOST.match(host))
def _checked(hosts: list[str]) -> str:
for host in hosts:
if not is_host_name(host):
raise ValueError(f"not a host name: {host!r}")
return " ".join(hosts)
def rewrite_sources_command(
hosts: list[str], proxy_file: str, apt_config: str = "apt-config"
) -> str:
"""The root command that sends the https sources of *hosts* through the cache.
A host apt has credentials for stays on https and is named in *proxy_file*.
*apt_config* is how apt-config is called; a test points apt's paths elsewhere.
:raises ValueError: if one of *hosts* is not a host name.
"""
return _REWRITE.format(
apt_config=apt_config,
hosts=_checked(hosts),
proxy_file=shlex.quote(proxy_file),
key=NOT_CACHED_KEY,
lines=_SOURCE_LINES,
via=VIA_CACHE,
)
def revert_sources_command(apt_config: str = "apt-config") -> str:
"""The root command that turns every source in the cache form back to https."""
return _REVERT.format(apt_config=apt_config, lines=_SOURCE_LINES, via=VIA_CACHE)
__all__ = [
"NOT_CACHED_KEY",
"VIA_CACHE",
"cache_form",
"cacheable_https_hosts",
"is_host_name",
"revert_sources_command",
"rewrite_sources_command",
]
+140
View File
@@ -0,0 +1,140 @@
"""A device's own configuration history: list it, read a revision, revert to one.
Some devices keep their earlier configurations themselves: a copy of the
configuration for every save, which the device can go back to. An application
restores a configuration there by reverting to the revision that holds it -- on a
device whose API takes no uploaded configuration (OPNsense), the only way
(NetOrk/netork#2511). :class:`ConfigRevisionsMixin` is the contract:
* ``get_config_revisions()`` and ``get_config_revision(revision_id)`` -- declared
only, so ``callable(getattr(driver, "get_config_revisions", None))`` is the
truthful answer to "does this device keep a history";
* ``revert_config_revision(revision_id)`` -- a template: it refuses an id that is not
in the history, reads the revision, lets the driver revert
(``_revert_to_config_revision``), and reads the configuration in place afterwards
to check that it holds the revision's. A device may answer that it reverted when it
did not; the check does not take its word;
* ``config_revision_content(text)`` -- what of a revision is its configuration. Every
save may stamp a revision anew (OPNsense writes who saved it and when into
config.xml), so two revisions with the same configuration differ as text. A driver
whose device does that strips the stamp here; by default the whole text counts. An
application that looks for the revision holding a configuration compares through
it too.
A driver mixes it in when its device keeps a history; no role base carries it. One
that can list the history but not revert to it sets ``revert_config_revision`` to
``None``.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, TypedDict
class ConfigRevisionDict(TypedDict):
"""One revision of the device's configuration (``get_config_revisions``)."""
id: str # what get_config_revision and revert_config_revision take, as the device names it
time: str | None # when it was saved, UTC ISO 8601; None when the device does not say
description: str | None # what made the change, as the device records it; None: none
user: str | None # who saved it, as the device names them; None when it names nobody
size: int | None # its size in bytes; None when not given
class UnknownConfigRevision(ValueError):
"""The device's configuration history has no revision of that id."""
def __init__(self, revision_id: object) -> None:
super().__init__(f"no configuration revision {revision_id!r} in the device's history")
class ConfigRevertFailed(RuntimeError):
"""The device refused the revert, or its configuration afterwards is not the revision's."""
class ConfigRevisionsMixin:
"""List, read and revert to the device's configuration revisions; see the module."""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def get_config_revisions(self) -> list[ConfigRevisionDict]:
"""The device's configuration history, newest first.
The first revision is the configuration in place now.
Example::
[
{
"id": "config-1794603885.1239.xml",
"time": "2026-11-13T21:04:45+00:00",
"description": "/api/firewall/filter/apply made changes",
"user": "apiuser@192.0.2.10",
"size": 58213,
},
]
:raises Exception: when the history could not be read; never an empty
list for a failed read.
"""
...
def get_config_revision(self, revision_id: str) -> str:
"""The configuration of revision *revision_id*, as the device keeps it.
:raises UnknownConfigRevision: for an id that is not in the history.
:raises Exception: when the device could not be read.
"""
...
# The step a driver implements:
def _revert_to_config_revision(self, revision_id: str) -> None:
"""Make revision *revision_id* the device's configuration; return once it is
in place, as the newest revision of the history.
:raises UnknownConfigRevision: when the device no longer has it.
:raises ConfigRevertFailed: when the device refuses.
"""
...
def config_revision_content(self, text: str) -> str:
"""What of a revision's *text* is its configuration: the whole text by default.
A driver whose device stamps every save into the configuration (who, when)
returns the text without that stamp, so that two revisions with the same
configuration compare equal.
"""
return text
def revert_config_revision(self, revision_id: str) -> ConfigRevisionDict:
"""Make revision *revision_id* the device's configuration again.
Returns the revision in place afterwards: the newest of the history, which
holds the configuration of *revision_id* (on most devices a new revision the
revert saved). What the device does beyond that -- reload its services,
restart, nothing -- is the driver's to document.
:raises UnknownConfigRevision: for an id that is not in the history now;
nothing has changed then.
:raises ConfigRevertFailed: when the device refuses, or when the configuration
in place afterwards is not the revision's.
:raises Exception: when the device could not be reached.
"""
if revision_id not in {r["id"] for r in self.get_config_revisions()}:
raise UnknownConfigRevision(revision_id)
# Read before the revert: saving it may rotate the revision out of the history.
wanted = self.config_revision_content(self.get_config_revision(revision_id))
self._revert_to_config_revision(revision_id)
revisions = self.get_config_revisions()
if not revisions:
raise ConfigRevertFailed(
f"the device has no configuration after reverting to {revision_id!r}"
)
now = revisions[0]
if self.config_revision_content(self.get_config_revision(now["id"])) != wanted:
raise ConfigRevertFailed(
f"the configuration in place ({now['id']!r}) is not revision {revision_id!r}"
" after the revert"
)
return now
+59 -17
View File
@@ -23,6 +23,22 @@ Mixed in by a driver, not by a role base: a Windows host is an
:class:`~napalm_device_types.os.OSDriver` too, and without a shell channel it
has no ``dial-stdio`` to offer. ``hasattr(driver, "open_container_engine")``
stays truthful.
**Which engine a CLI is.** An engine is listed when its CLI answers
``--version`` with its own name: ``Docker version 29.0.1, build …``, ``podman
version 5.2.2``. podman-docker installs a ``docker`` that is podman; listed as
Docker it would be read twice, the second time as the login user's rootless
Podman (NetOrk/netork#818).
**Podman as root.** The Podman this reaches is the host's *rootful* one: the
engine of the system's containers, whose units Quadlet generates from
``/etc/containers/systemd`` and whose API ``podman.socket`` serves at
``/run/podman/podman.sock``. Rootless containers belong to the user who runs
them, in that user's own storage, and no other login sees them. So the
connection to Podman runs every call as root (``privileged``), the way the
driver gains root for any command: root itself, or sudo. ``podman system
dial-stdio`` exists from Podman 4.0 on; it reaches the API socket, which has
to be listening (``systemctl enable --now podman.socket``).
"""
from __future__ import annotations
@@ -34,8 +50,11 @@ from typing import TYPE_CHECKING, Any, cast
from napalm_device_types.channel import ByteStream, CommandResult
from napalm_device_types.models import ContainerEngineDict, FixResultDict
#: Engines this package knows how to reach, and the API each speaks.
ENGINES: dict[str, str] = {"docker": "docker-engine"}
#: Engines this package knows how to reach, and the API each speaks. Podman's
#: is its own service, which serves Docker's API next to libpod's.
ENGINES: dict[str, str] = {"docker": "docker-engine", "podman": "podman-libpod"}
#: Engines reached as root: rootful Podman's socket is root's (see above).
ROOT_ENGINES: frozenset[str] = frozenset({"podman"})
_PROBE_TIMEOUT = 15
_CLI_TIMEOUT = 120
@@ -52,17 +71,24 @@ class ContainerEngineConnection:
arguments; binary, quoting and the channel are the driver's.
"""
def __init__(self, driver: Any, engine: str, binary: str) -> None:
def __init__(self, driver: Any, engine: str, binary: str, *, privileged: bool = False) -> None:
self._driver = driver
self._binary = binary
self.engine = engine
#: Every call runs as root: the engine is root's (rootful Podman).
self.privileged = privileged
def _command(self, args: Sequence[str]) -> str:
return shlex.join([self._binary, *args])
def open_api(self) -> ByteStream:
"""A stream to the engine's API: HTTP/1.1 over ``system dial-stdio``."""
return cast(ByteStream, self._driver.open_stream(self._command(["system", "dial-stdio"])))
return cast(
ByteStream,
self._driver.open_stream(
self._command(["system", "dial-stdio"]), privileged=self.privileged
),
)
def run_cli(
self,
@@ -80,7 +106,10 @@ class ContainerEngineConnection:
return cast(
CommandResult,
self._driver.run_command(
self._command(args), privileged=privileged, timeout=timeout, stdin=stdin
self._command(args),
privileged=privileged or self.privileged,
timeout=timeout,
stdin=stdin,
),
)
@@ -96,7 +125,8 @@ class ContainerEngineConnection:
return cast(
ByteStream,
self._driver.open_stream(
f"{command} 2>&1" if merge_stderr else command, privileged=privileged
f"{command} 2>&1" if merge_stderr else command,
privileged=privileged or self.privileged,
),
)
@@ -134,20 +164,32 @@ class ContainerEngineMixin:
* engine (string) - e.g. "docker"
* api (string) - what its API is, e.g. "docker-engine"
An engine is listed when its CLI exists on the host. Whether the login
user may use it is decided by the first API call, which says
"permission denied" in the stream's stderr.
An engine is listed when its CLI exists on the host and names itself
when asked for its version: podman-docker's ``docker`` says it is podman,
and is left out. Whether the login user may use the engine is decided by
the first API call, which says "permission denied" in the stream's stderr.
"""
found: list[ContainerEngineDict] = []
for engine, api in ENGINES.items():
probe = shlex.join(["command", "-v", self._container_engine_binary(engine)])
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
if result.exit_code == 0 and result.stdout.strip():
found.append({"engine": engine, "api": api})
return found
return [
{"engine": engine, "api": api}
for engine, api in ENGINES.items()
if self._answers_as(engine)
]
def _answers_as(self, engine: str) -> bool:
"""Whether *engine*'s CLI is on the host and says it is *engine*."""
binary = shlex.quote(self._container_engine_binary(engine))
probe = f"command -v {binary} >/dev/null 2>&1 && {binary} --version"
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
words = result.stdout.split()
return result.exit_code == 0 and bool(words) and words[0].lower() == engine
def open_container_engine(self, engine: str) -> ContainerEngineConnection:
"""The connection to *engine*; :class:`ContainerEngineUnavailable` if unknown."""
if engine not in ENGINES:
raise ContainerEngineUnavailable(f"no way to reach container engine {engine!r}")
return ContainerEngineConnection(self, engine, self._container_engine_binary(engine))
return ContainerEngineConnection(
self,
engine,
self._container_engine_binary(engine),
privileged=engine in ROOT_ENGINES,
)
+156 -61
View File
@@ -26,6 +26,7 @@ import re
from collections.abc import Iterator
from typing import TYPE_CHECKING, Any
from napalm_device_types._dhcp_order import ReservationChange, order_reservation_changes
from napalm_device_types._matching import check_desired, index_live
from napalm_device_types.models import (
DhcpHaDict,
@@ -57,6 +58,27 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
_HEX_ONLY = re.compile(r"[^0-9a-f]")
#: Added to a refused change that is part of an address swap (#106).
_SWAP_NOTE = (
"an address swap with {}: no order frees an address first; "
"move one of them to a free address in a run of its own"
)
class DhcpApplyError(RuntimeError):
"""Some changes of an applied reservation or subnet set were refused by the device.
Raised after the rest were written and committed: ``failed`` of ``attempted``
changes did not go through, and each was named in a ``[failed]`` progress line.
``kind`` is what was applied: ``"reservation"`` or ``"subnet"``.
"""
def __init__(self, failed: int, attempted: int, kind: str = "reservation") -> None:
self.failed = failed
self.attempted = attempted
self.kind = kind
super().__init__(f"{failed} of {attempted} DHCP {kind} change(s) were refused")
def normalize_mac(mac: str | None) -> str:
"""Reduces a MAC address to lowercase colon-separated form.
@@ -106,6 +128,43 @@ def _subnet_field_differs(field: str, live: DhcpSubnetDict, desired: DhcpSubnetD
return any(live_options.get(option) != value for option, value in desired_options.items())
def _reservation_key(reservation: DhcpReservationDict) -> str:
return normalize_mac(reservation.get("mac"))
def _diff_reservations(
live: list[DhcpReservationDict], desired: list[DhcpReservationDict]
) -> DhcpReservationDiffDict:
"""``diff_dhcp_reservations`` against reservations already read."""
live_by_mac, shared = index_live(live, _reservation_key)
check_desired(desired, _reservation_key, shared, "MAC")
add: list[DhcpReservationDict] = []
update: list[DhcpReservationUpdateDict] = []
for desired_reservation in desired:
current = live_by_mac.get(_reservation_key(desired_reservation))
if current is None:
add.append(desired_reservation)
continue
changed_fields = [
field
for field in _DHCP_RESERVATION_COMPARE_FIELDS
if current.get(field) != desired_reservation.get(field)
]
if changed_fields:
update.append(
{
"uuid": current["uuid"],
"reservation": desired_reservation,
"changed_fields": changed_fields,
}
)
return {"add": add, "update": update}
class DhcpServerMixin:
"""Mixin providing DHCP reservation and subnet read/diff/apply.
@@ -258,36 +317,7 @@ class DhcpServerMixin:
:raises ValueError: When a desired reservation cannot name a single live one.
"""
def key(reservation: DhcpReservationDict) -> str:
return normalize_mac(reservation.get("mac"))
live_by_mac, shared = index_live(self.get_dhcp_reservations(), key)
check_desired(desired, key, shared, "MAC")
add: list[DhcpReservationDict] = []
update: list[DhcpReservationUpdateDict] = []
for desired_reservation in desired:
live = live_by_mac.get(key(desired_reservation))
if live is None:
add.append(desired_reservation)
continue
changed_fields = [
field
for field in _DHCP_RESERVATION_COMPARE_FIELDS
if live.get(field) != desired_reservation.get(field)
]
if changed_fields:
update.append(
{
"uuid": live["uuid"],
"reservation": desired_reservation,
"changed_fields": changed_fields,
}
)
return {"add": add, "update": update}
return _diff_reservations(self.get_dhcp_reservations(), desired)
def apply_dhcp_reservationset(self, desired: list[DhcpReservationDict]) -> Iterator[str]:
"""
@@ -298,29 +328,65 @@ class DhcpServerMixin:
committing reloads the DHCP daemon and drops in-flight requests, which
is too high a price for a no-op run.
The changes are written so an address is released before it is taken; a
swap of addresses has no such order and is reported as one (``_dhcp_order``).
A change the device refuses (say, an address in a subnet its DHCP server
does not manage) does not stop the others: it gets a ``[failed]`` line,
the rest are written and committed, and ``DhcpApplyError`` is raised at
the end. Any non-empty diff commits, even one whose every write was
refused, so that writes an earlier, interrupted run saved take effect.
:param desired: The complete desired reservation set.
:yields: Progress lines, one per applied add/update, plus a final
commit line.
:yields: Progress lines, one per applied, updated or refused change,
plus a final commit line.
:raises DhcpApplyError: after the commit, when a change was refused.
"""
diff = self.diff_dhcp_reservations(desired)
for reservation in diff["add"]:
self.apply_dhcp_reservation(reservation)
yield f"[add] {reservation['mac']} -> {reservation['ip']}"
for entry in diff["update"]:
self.apply_dhcp_reservation(entry["reservation"], uuid=entry["uuid"])
fields = ", ".join(entry["changed_fields"])
yield (
f"[update] {entry['reservation']['mac']} -> {entry['reservation']['ip']} ({fields})"
)
if not diff["add"] and not diff["update"]:
live = self.get_dhcp_reservations()
diff = _diff_reservations(live, desired)
attempted = len(diff["add"]) + len(diff["update"])
if not attempted:
yield "[commit] no changes"
return
added = updated = refused = 0
for change in order_reservation_changes(diff, live):
line, ok = self._write_change(change)
refused += not ok
added += ok and change.uuid is None
updated += ok and change.uuid is not None
yield line
# Committed even when every write was refused: a run that stopped before its
# commit left saved writes behind, which the device reports as configured, so
# no later diff would ever commit them.
self.commit_dhcp_reservations()
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
yield f"[commit] applied {added} add(s), {updated} update(s)"
if refused:
raise DhcpApplyError(refused, attempted)
def _write_change(self, change: ReservationChange) -> tuple[str, bool]:
"""Write one change of an ordered set; its progress line, and whether it went in."""
tag = "[add]" if change.uuid is None else "[update]"
line, ok = self._write_reservation(change.reservation, change.uuid, tag)
if ok and change.fields:
line = f"{line} ({change.fields})"
elif not ok and change.swap:
line = f"{line} -- {_SWAP_NOTE.format(', '.join(change.swap))}"
return line, ok
def _write_reservation(
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
) -> tuple[str, bool]:
"""Write one reservation; its progress line, and whether the device took it."""
target = f"{reservation['mac']} -> {reservation['ip']}"
try:
self.apply_dhcp_reservation(reservation, uuid=uuid)
except Exception as exc:
return f"[failed] {target}: {exc}", False
return f"{tag} {target}", True
def diff_dhcp_subnets(self, desired: list[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
"""
@@ -391,24 +457,53 @@ class DhcpServerMixin:
As with ``apply_dhcp_reservationset``, an empty diff does **not**
commit: reconfiguring the DHCP daemon is too expensive for a no-op.
A subnet the device refuses (say, one that overlaps a subnet it serves)
does not stop the others: it gets a ``[failed]`` line, the rest are
written and committed, and ``DhcpApplyError`` is raised at the end. Any
non-empty diff commits, even one whose every write was refused, so that
writes an earlier, interrupted run saved take effect (#113).
:param desired: The complete desired subnet set.
:yields: Progress lines, one per applied add/update, plus a final
commit line.
:yields: Progress lines, one per applied, updated or refused change,
plus a final commit line.
:raises DhcpApplyError: after the commit, when a change was refused.
"""
diff = self.diff_dhcp_subnets(desired)
for subnet in diff["add"]:
self.apply_dhcp_subnet(subnet)
yield f"[add] {subnet['subnet']}"
for entry in diff["update"]:
self.apply_dhcp_subnet(entry["subnet"], uuid=entry["uuid"])
fields = ", ".join(entry["changed_fields"])
yield f"[update] {entry['subnet']['subnet']} ({fields})"
if not diff["add"] and not diff["update"]:
attempted = len(diff["add"]) + len(diff["update"])
if not attempted:
yield "[commit] no changes"
return
changes: list[tuple[DhcpSubnetDict, str | None, str]] = [
(subnet, None, "") for subnet in diff["add"]
]
changes += [
(e["subnet"], e["uuid"], ", ".join(e["changed_fields"])) for e in diff["update"]
]
added = updated = refused = 0
for subnet, uuid, fields in changes:
line, ok = self._write_subnet(subnet, uuid, fields)
refused += not ok
added += ok and uuid is None
updated += ok and uuid is not None
yield line
# Committed even when every write was refused; see apply_dhcp_reservationset.
self.commit_dhcp_subnets()
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
yield f"[commit] applied {added} add(s), {updated} update(s)"
if refused:
raise DhcpApplyError(refused, attempted, "subnet")
def _write_subnet(
self, subnet: DhcpSubnetDict, uuid: str | None, fields: str
) -> tuple[str, bool]:
"""Write one subnet; its progress line, and whether the device took it."""
try:
self.apply_dhcp_subnet(subnet, uuid=uuid)
except Exception as exc:
return f"[failed] {subnet['subnet']}: {exc}", False
if uuid is None:
return f"[add] {subnet['subnet']}", True
return f"[update] {subnet['subnet']} ({fields})", True
+41 -5
View File
@@ -1,23 +1,39 @@
"""The firmware a vendor offers for a device.
"""The firmware a vendor offers for a device, and installing it.
A fact for the caller (docs/STRUCTURE.md): a driver may know the newer firmware
from the vendor's download catalogue or the device's own update check; whether
running an older one is a problem is decided above the driver. Comparing the
vendor's version strings is vendor knowledge, so the driver says itself whether
the offer is newer. Declared only; a driver that can tell implements it.
Installing it is a capability of its own (``FirmwareUpgradeMixin``): when and
whether is the caller's call, as with package updates; how is the driver's.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, NotRequired, TypedDict
from typing import TYPE_CHECKING
from napalm_device_types.models import FirmwareOfferDict
class FirmwareOfferDict(TypedDict):
"""The firmware a device runs and the newest its vendor offers
(``FirmwareOfferMixin.get_firmware_offer``). A fact, not a judgement."""
current_version: str
latest_version: str
newer: bool # the offer is newer than what runs, by the vendor's versioning
download_url: NotRequired[str]
# The driver can install latest_version itself (FirmwareUpgradeMixin) now; False
# while the device lacks what that takes (OpenWrt without owut). Left out where
# the driver never installs firmware on this device.
installable: NotRequired[bool]
# The newest of a later release line (OpenWrt 25.12 on a 24.10 device), which
# latest_version does not cross to: shown, never installed by the driver.
latest_major_version: NotRequired[str]
class FirmwareOfferMixin:
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def get_firmware_offer(self) -> FirmwareOfferDict | None:
def get_firmware_offer(self) -> "FirmwareOfferDict | None":
"""The firmware running and the newest the vendor offers.
None when the driver cannot tell for this device (an unknown model,
@@ -29,3 +45,23 @@ class FirmwareOfferMixin:
"newer": True, "download_url": "https://.../GS308E_1.0.2.12.zip"}
"""
...
class FirmwareUpgradeMixin:
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def start_firmware_upgrade(self, version: str) -> None:
"""Starts installing *version*, an offer's ``latest_version`` that
:meth:`FirmwareOfferMixin.get_firmware_offer` called ``installable``.
The device runs the upgrade by itself and restarts at its end, which
ends the session; the configuration is kept. Returns once it has
started. ``UpdateMixin.get_update_progress`` follows it until then:
``"failed"`` when it stopped before the device restarted, which leaves
the running firmware as it was.
:raises ValueError: *version* is not a version string the driver takes.
:raises RuntimeError: the device cannot start it (what it takes is
missing, an upgrade is already running).
"""
...
+126 -7
View File
@@ -16,6 +16,7 @@ They replace ``run_device_action`` (docs/STRUCTURE.md):
``fix_apt_proxy`` :meth:`PackageProxyMixin.configure_package_proxy`
``fix_docker_permissions`` ``ContainerEngineMixin.grant_container_engine_access``
``install_lldpd`` and co. :meth:`EnsurePackageMixin.ensure_package`
(new) :meth:`ZabbixAgentMixin.configure_zabbix_agent`
========================== ===================================================
"""
@@ -31,6 +32,7 @@ from napalm_device_types.models import (
PackageDict,
SnmpAgentSettingsDict,
)
from napalm_device_types.zabbix_agent import ZabbixAgentSettingsDict, check_zabbix_agent_settings
#: One step of a fix: its name and the call that does it.
FixStep = tuple[str, Callable[[], FixStepOutcomeDict]]
@@ -58,6 +60,23 @@ def run_fix_steps(steps: Sequence[FixStep]) -> FixResultDict:
}
def enable_and_restart(
manage_service: Callable[[str, str], dict[str, Any]], service: str
) -> FixStepOutcomeDict:
"""Enable *service* at boot, then restart it so it reads its configuration; not
restarted when it cannot be enabled."""
for action in ("enable", "restart"):
result = manage_service(service, action)
if not result.get("success"):
output = str(result.get("output", "")).strip()
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
return {"ok": True, "output": f"{service} enabled and restarted"}
def _joined(*parts: object) -> str:
return "\n".join(str(p).strip() for p in parts if p and str(p).strip())
class SnmpAgentMixin:
"""Run an SNMP agent on the device that answers the caller.
@@ -171,13 +190,7 @@ class LldpAgentMixin:
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
def _start_lldp_agent(self) -> FixStepOutcomeDict:
service = self.LLDP_AGENT_SERVICE
for action in ("enable", "restart"):
result = self.manage_service(service, action)
if not result.get("success"):
output = str(result.get("output", "")).strip()
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
return {"ok": True, "output": f"{service} enabled and restarted"}
return enable_and_restart(self.manage_service, self.LLDP_AGENT_SERVICE)
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
@@ -193,6 +206,110 @@ class LldpAgentMixin:
...
class ZabbixAgentMixin:
"""Run a native Zabbix agent that answers the caller's server.
For a host without a container engine, where the agent cannot run as a
container. The steps: load the package index where the driver can
(``refresh_available_updates``), install the agent from the host's own packages -- and
where those have none, from Zabbix's repository for ``settings["release"]``
(:func:`~napalm_device_types.zabbix_agent.zabbix_repository_command`) --, write
its configuration (:func:`~napalm_device_types.zabbix_agent.zabbix_agent_config`),
let the allowed sources through the host's firewall, enable and restart it.
Installing and starting are generic, over ``ensure_package`` and
``manage_service`` with ``ZABBIX_AGENT_PACKAGE`` and ``ZABBIX_AGENT_SERVICE``.
Whether Zabbix reaches it is the caller's to ask Zabbix.
The settings are checked before anything changes; ``ValueError`` names the one
that does not hold. A driver that cannot add a repository sets
``_add_zabbix_repository`` to ``None``: its hosts then take the distribution's
package or none.
"""
#: The package that brings the agent, as ``ensure_package`` takes it.
ZABBIX_AGENT_PACKAGE: ClassVar[str] = "zabbix-agent"
#: The agent's service, as ``manage_service`` takes it.
ZABBIX_AGENT_SERVICE: ClassVar[str] = "zabbix-agent"
def configure_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixResultDict:
"""Install, configure, open, enable and restart the agent with *settings*.
:raises ValueError: for settings the agent's configuration cannot take,
before anything changes.
"""
check_zabbix_agent_settings(settings)
return run_fix_steps(
[
("install", lambda: self._install_zabbix_agent(settings)),
("configure", lambda: self._write_zabbix_agent_config(settings)),
("firewall", lambda: self._allow_zabbix_agent_sources(settings)),
("start", self._start_zabbix_agent),
]
)
def _install_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixStepOutcomeDict:
index = self._load_package_index()
first = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
if first["success"]:
return {"ok": True, "output": _joined(index, first.get("output"))}
release = settings.get("release")
add = getattr(self, "_add_zabbix_repository", None)
if not release or not callable(add):
return {"ok": False, "output": _joined(index, first.get("output"))}
added = add(release)
if not added["ok"]:
return {"ok": False, "output": _joined(index, added.get("output"))}
second = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
return {
"ok": bool(second["success"]),
"output": _joined(index, added.get("output"), second.get("output")),
}
def _load_package_index(self) -> str:
"""Load the package index where the driver can (OpenWrt keeps it in RAM, gone
after a restart); what went wrong, or "". The install tries either way."""
refresh = getattr(self, "refresh_available_updates", None)
if not callable(refresh):
return ""
loaded = refresh()
if loaded.get("success"):
return ""
return f"package index not loaded: {str(loaded.get('output', '')).strip()}"
def _start_zabbix_agent(self) -> FixStepOutcomeDict:
return enable_and_restart(self.manage_service, self.ZABBIX_AGENT_SERVICE)
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def ensure_package(self, name: str) -> FixResultDict: ...
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
def refresh_available_updates(self) -> dict[str, Any]: ...
# The steps a driver implements:
def _add_zabbix_repository(self, release: str) -> FixStepOutcomeDict:
"""Add Zabbix's repository for *release* and refresh the package index
(``zabbix_repository_command`` builds the command for most hosts)."""
...
def _write_zabbix_agent_config(
self, settings: ZabbixAgentSettingsDict
) -> FixStepOutcomeDict:
"""Write *settings* into the agent's configuration file, keeping the rest of
it (``zabbix_agent_config``), and whatever else the device needs before the
agent may start."""
...
def _allow_zabbix_agent_sources(
self, settings: ZabbixAgentSettingsDict
) -> FixStepOutcomeDict:
"""Let ``settings["allowed_sources"]`` reach tcp/``listen_port``; ``ok`` with
nothing to do where the host has no firewall of its own, or for []."""
...
class PackageProxyMixin:
"""Point the device's package manager at a caching proxy."""
@@ -215,5 +332,7 @@ __all__ = [
"LldpAgentMixin",
"PackageProxyMixin",
"SnmpAgentMixin",
"ZabbixAgentMixin",
"enable_and_restart",
"run_fix_steps",
]
+37 -5
View File
@@ -60,6 +60,7 @@ too full is not decided here.
from __future__ import annotations
import re
from functools import cmp_to_key
from typing import TYPE_CHECKING
from napalm_device_types.models import (
@@ -171,9 +172,39 @@ def _sections(output: str) -> dict[str, list[str]]:
return sections
def _version_key(version: str) -> tuple[object, ...]:
"""Natural order: 6.8.0-142 after 6.8.0-87, 7.0.14 after 7.0.2."""
return tuple(int(part) if part.isdigit() else part for part in re.split(r"(\d+)", version))
#: A version's segments as rpm compares them: runs of digits, runs of letters, and
#: ``~``; whatever else stands between them only separates.
_VERSION_SEGMENT = re.compile(r"~|\d+|[A-Za-z]+")
def _segment_cmp(x: str, y: str) -> int:
"""One segment against another, as :func:`_version_cmp` takes them; 0 when equal."""
if "~" in (x, y):
return (y == "~") - (x == "~")
if x.isdigit() and y.isdigit():
return (int(x) > int(y)) - (int(x) < int(y))
if x.isdigit() or y.isdigit():
return 1 if x.isdigit() else -1
return (x > y) - (x < y)
def _version_cmp(a: str, b: str) -> int:
"""rpm's ``rpmvercmp``: below 0 when ``a`` is older, above when newer.
Numbers compare as numbers (6.8.0-142 after 6.8.0-87), letters as text, and a
number is newer than letters in its place: ``112.5.1.el9uek`` after
``112.5.el9uek`` (#109). A version that has more segments where the other
ends is the newer one, unless its next is ``~``, which comes before anything:
``6.13~rc5`` before ``6.13``, as dpkg sorts it too.
"""
left, right = _VERSION_SEGMENT.findall(a), _VERSION_SEGMENT.findall(b)
for x, y in zip(left, right, strict=False):
if order := _segment_cmp(x, y):
return order
if len(left) == len(right):
return 0
longer, sign = (left, 1) if len(left) > len(right) else (right, -1)
return -sign if longer[min(len(left), len(right))] == "~" else sign
def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
@@ -188,8 +219,9 @@ def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
same = [k for k in installed if _kernel_flavour(k)[1] == flavour]
if not same:
return None
newest = max(same, key=lambda k: _version_key(_kernel_flavour(k)[0]))
if _version_key(_kernel_flavour(newest)[0]) > _version_key(version):
order = cmp_to_key(_version_cmp)
newest = max(same, key=lambda k: order(_kernel_flavour(k)[0]))
if order(_kernel_flavour(newest)[0]) > order(version):
return newest
return None
+44 -16
View File
@@ -481,7 +481,12 @@ class LocalAccountsMixin:
return self._accounts_result(name, parts)
def add_service_account(
self, name: str, public_key: str, *, uid: int | None = None
self,
name: str,
public_key: str,
*,
uid: int | None = None,
replaces: Sequence[str] = (),
) -> LocalAccountChangeDict:
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
root without a password.
@@ -495,36 +500,59 @@ class LocalAccountsMixin:
once no other account has it; an account that is there keeps its own,
as a uid is never changed under files that carry it.
*replaces* are netOrk's keys *public_key* takes the place of: they go
from the account once *public_key* is in it, matched as in
:meth:`add_local_account`, and never *public_key* itself. Call it so
through a login with *public_key*, tried before: netOrk keeps its way
in (NetOrk/netork#2462).
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
``keys``. The new login itself is for the caller to try.
``keys``, and ``old keys`` with *replaces* when ``keys`` went. The new
login itself is for the caller to try.
An account that is there and cannot log in (``nologin``, ``false``) is
another service's -- a satellite's ``netork``, say -- and is left alone.
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
less, or another service's account.
:raises ValueError: for a name or key no host takes, a replaced key that
is none, root, a uid of 0 or less, or another service's account.
:raises RuntimeError: when the login user cannot be told.
"""
check_public_key(public_key)
old = [key for key in _checked_keys(replaces) if _key_blob(key) != _key_blob(public_key)]
login, existing = self._accounts_service_target(name, uid)
first = self._accounts_service_account_step(name, login, existing, uid)
parts = [first]
if first["success"]:
parts += self._accounts_service_keys(name, public_key, old)
return self._accounts_result(name, parts)
def _accounts_service_account_step(
self, name: str, login: str, existing: list[str] | None, uid: int | None
) -> FixResultDict:
"""The ``account`` step of netOrk's own account; nothing to do when the driver
logs in as *name* already."""
if name == login:
step: FixStepDict = {
"step": "account",
"ok": True,
"output": f"this driver logs in as {name} already",
}
first: FixResultDict = _joined([step])
else:
first = run_fix_steps(
[("account", lambda: self._accounts_service_account(name, existing, uid))]
)
parts = [first]
if first["success"]:
parts += [
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
return self._accounts_result(name, parts)
return _joined([step])
return run_fix_steps(
[("account", lambda: self._accounts_service_account(name, existing, uid))]
)
def _accounts_service_keys(
self, name: str, public_key: str, old: list[str]
) -> list[FixResultDict]:
"""``sudo`` and ``keys`` of netOrk's own account, then ``old keys`` once the key is in."""
parts = [
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
if old and parts[-1]["success"]:
parts.append(run_fix_steps([("old keys", lambda: self._accounts_drop_keys(name, old))]))
return parts
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
+39 -4
View File
@@ -35,6 +35,9 @@ class ForwardRuleDict(TypedDict):
selector: str # "*.*", "authpriv.*", or the condition around it
complete: bool # every message of its ruleset goes
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
#: The template its messages are sent in; None: rsyslog's default, RFC 3164,
#: which carries the sender's local time without a zone (NetOrk/netork#493).
template: str | None
class FileInputDict(TypedDict):
@@ -165,6 +168,10 @@ class _Reader:
self.pending: tuple[str, str | None] | None = None
self.legacy_ruleset: str | None = None
self.last_selector = "*.*"
#: ``$ActionForwardDefaultTemplate``, for the legacy rules after it.
self.legacy_template: str | None = None
#: ``module(load="builtin:omfwd" template=…)``, for every omfwd action.
self.module_template: str | None = None
def ruleset(self) -> str | None:
names = [value for kind, value in self.stack if kind == "ruleset"]
@@ -175,7 +182,13 @@ class _Reader:
return once or (conditions[-1] if conditions else None)
def forward(
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
self,
protocol: str,
target: str,
port: int | None,
selector: str,
once: str | None,
template: str | None,
) -> None:
condition = self.condition(once)
self.parsed["rules"].append(
@@ -186,6 +199,7 @@ class _Reader:
"selector": condition or selector,
"complete": condition is None and selector in _EVERYTHING,
"ruleset": self.ruleset(),
"template": template,
}
)
@@ -225,6 +239,8 @@ class _Reader:
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
if name == "module":
self.module(params.get("load", ""))
if params.get("load", "").removeprefix("builtin:") == "omfwd":
self.module_template = params.get("template") or self.module_template
elif name == "ruleset":
self.pending = ("ruleset", params.get("name"))
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
@@ -240,7 +256,10 @@ class _Reader:
kind = params.get("type", "")
if kind in _FORWARD_TYPES and params.get("target"):
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
template = params.get("template") or (self.module_template if kind == "omfwd" else None)
self.forward(
protocol, params["target"], _port(params.get("port")), selector, once, template
)
def module(self, name: str) -> None:
if name == "imjournal":
@@ -257,6 +276,8 @@ class _Reader:
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
elif name == "$ruleset":
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
elif name == "$actionforwarddefaulttemplate":
self.legacy_template = value or None
def legacy(self, text: str, once: str | None) -> None:
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
@@ -264,13 +285,27 @@ class _Reader:
return
selector = self.last_selector if match["sel"] == "&" else match["sel"]
self.last_selector = selector
forward = _FORWARD_RE.match(match["action"].strip())
action = match["action"].strip()
forward = _FORWARD_RE.match(action)
if forward:
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
self.forward(
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
protocol,
forward["v6"] or forward["host"],
_port(forward["port"]),
selector,
once,
self.legacy_template_of(action[forward.end() :], protocol),
)
def legacy_template_of(self, rest: str, protocol: str) -> str | None:
"""The template after a legacy rule's target (``;Name``), else the default
set before it; RELP takes no forwarding default."""
_, semicolon, named = rest.partition(";")
if semicolon and named.strip():
return named.strip()
return self.legacy_template if protocol != "relp" else None
def parse_rsyslog(text: str) -> RsyslogDict:
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
+2 -20
View File
@@ -1172,16 +1172,6 @@ class OspfDict(TypedDict):
interfaces: dict[str, OspfInterfaceDict] # by interface name
class FirmwareOfferDict(TypedDict):
"""The firmware a device runs and the newest its vendor offers
(``FirmwareOfferMixin.get_firmware_offer``). A fact, not a judgement."""
current_version: str
latest_version: str
newer: bool # the offer is newer than what runs, by the vendor's versioning
download_url: NotRequired[str]
class HypervisorHostDict(TypedDict):
"""One host of a hypervisor or cluster (``HypervisorDriver.get_hypervisor_hosts``)."""
@@ -1442,13 +1432,5 @@ class ManagementProtocolsDict(TypedDict):
virtual_media: ProtocolStateDict
class SystemIdentityDict(TypedDict):
"""A host's SMBIOS identity, what ties it to its BMC.
See :mod:`napalm_device_types.system_identity`.
Each value is empty when unknown or a vendor placeholder."""
system_uuid: str # lowercase
product_serial: str
board_serial: str
# Importable from here as before 4.41; it lives with its contract now.
from napalm_device_types.firmware import FirmwareOfferDict as FirmwareOfferDict # noqa: E402
+19 -6
View File
@@ -13,9 +13,13 @@ what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
per imported key. Also interrupted is a transaction that never ended: dnf 4's
history (libdnf's ``trans`` table) holds its row with ``state`` 0 until
``finish()`` writes 1 (done) or 2 (error), so the newest row at 0 is one cut off;
per imported key. The defaults are the ones the host's libdnf was built with, asked
of its Python binding, as a distribution may add its own: Oracle Linux adds
``kernel-uek`` and its parts, whose packages provide nothing of upstream's list.
Upstream's list is the floor where libdnf cannot be asked (dnf 5, yum). Also
interrupted is a transaction that never ended: dnf 4's history (libdnf's ``trans``
table) holds its row with ``state`` 0 until ``finish()`` writes 1 (done) or 2
(error), so the newest row at 0 is one cut off;
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
@@ -81,6 +85,13 @@ _DNF_HISTORY_PY = (
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
"r and print(*r)"
)
#: The ``installonlypkgs`` the host's libdnf (dnf 4) was built with, space-separated;
#: nothing where there is no libdnf. ``ConfigMain`` is held in a name: an option
#: taken from a temporary one outlives it, and reading it then crashes Python.
_LIBDNF_INSTALLONLY_PY = (
"import libdnf.conf; c = libdnf.conf.ConfigMain(); "
"print(c.installonlypkgs().getValueString().replace(',', ' '))"
)
_RUNNING_PART = (
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
@@ -90,7 +101,10 @@ _RUNNING_PART = (
_RPM_PART = (
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
"rpmpy=; for p in /usr/libexec/platform-python python3; do "
"if command -v $p >/dev/null 2>&1; then rpmpy=$p; break; fi; done; "
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
f'$([ -n "$rpmpy" ] && $rpmpy -c "{_LIBDNF_INSTALLONLY_PY}" 2>/dev/null) '
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
"| grep -v ' ' | sort -u; "
@@ -98,9 +112,8 @@ _RPM_PART = (
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
"| head -n 20; "
"echo '[dnf-history]'; for py in /usr/libexec/platform-python python3; do "
f'if command -v $py >/dev/null 2>&1; then $py -c "{_DNF_HISTORY_PY}" 2>/dev/null; '
"break; fi; done; fi; "
"echo '[dnf-history]'; "
f'[ -n "$rpmpy" ] && $rpmpy -c "{_DNF_HISTORY_PY}" 2>/dev/null; fi; '
)
_APK_PART = (
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
+31
View File
@@ -36,11 +36,24 @@ DNF_SECURITY_COMMAND = (
" echo __DNF_RC=$?; } | cat"
)
#: Read-only. Every advisory whose fix the installed packages carry, one line per
#: CVE it names (``CVE SEVERITY/Sec. NEVRA``): the distribution's own word that a
#: backport is in, where netOrk's knowledge base holds no records of the release
#: (netOrk #2394). dnf 4 only: dnf 5 reads ``--with-cve`` as a filter, and
#: Fedora's advisories reference bugs rather than CVEs. From the cache, as
#: :data:`DNF_SECURITY_COMMAND` reads it; the output is long (90 000 lines on an
#: Oracle Linux 9 host), so a caller reads it seldom.
DNF_INSTALLED_CVES_COMMAND = (
"{ LC_ALL=C dnf --cacheonly updateinfo list --installed --with-cve --quiet 2>/dev/null;"
" echo __DNF_RC=$?; } | cat"
)
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
_APT_LINE = re.compile(r"^(\S+)/(\S+)\s+(\S+)\s+\S+\s+\[upgradable from:\s+(\S+)\]")
_SECURITY_SUITE = "-security"
_APT_STATUS = re.compile(r"^__APT_RC=(\d+)\s*$", re.MULTILINE)
_DNF_STATUS = re.compile(r"^__DNF_RC=(\d+)\s*$", re.MULTILINE)
_CVE_ID = re.compile(r"^CVE-\d{4}-\d{4,}$")
def _joined_lines(output: str) -> list[str]:
@@ -118,3 +131,21 @@ def parse_dnf_security(output: str) -> set[str]:
if len(parts) >= 3 and parts[1].endswith("/Sec."):
names.add(nevra_name(parts[-1]))
return names
def parse_dnf_installed_cves(output: str) -> dict[str, list[str]]:
"""The CVEs the installed advisories close, per package name, each once and sorted.
Parses :data:`DNF_INSTALLED_CVES_COMMAND`'s ``CVE SEVERITY/Sec. NEVRA`` lines;
a line that names no CVE (a key prompt, an advisory id) is no statement.
:raises ValueError: when dnf failed or its exit status never arrived -- "could
not read" must not look like "no advisory installed".
"""
by_name: dict[str, set[str]] = {}
listing = _listing(output, _DNF_STATUS, "dnf updateinfo list --installed --with-cve")
for line in listing.splitlines():
parts = line.split()
if len(parts) >= 3 and _CVE_ID.match(parts[0]):
by_name.setdefault(nevra_name(parts[-1]), set()).add(parts[0])
return {name: sorted(cves) for name, cves in by_name.items()}
+66 -15
View File
@@ -1,6 +1,7 @@
"""The SMBIOS identity of a host: system UUID, product serial, board serial.
"""The SMBIOS identity of a host: system UUID, product serial, board serial -- and
its system firmware (BIOS or UEFI): vendor, version, release date.
What ties a server to its BMC. The BMC reports the server's serial number and
The identity is what ties a server to its BMC. The BMC reports the server's serial number and
system UUID; the host reads the same values from its firmware tables. Both sides
read them the same way on every Linux host, so the command, its parse and the
placeholder filter live here once, and a driver only carries the command across.
@@ -13,27 +14,54 @@ Matching on those would join unrelated machines, so they read as empty.
**Root.** ``product_uuid`` and ``product_serial`` are readable by root only; without
it they come back empty, which is "not known", not a mismatch.
**The firmware** comes from the same tables in the same round trip, so a driver that
reads the identity reads it too. It is a fact about the host, not part of its identity:
its placeholders are the vendors' text defaults, never a short value (a version of
``1`` is a version), and sysfs's ``MM/DD/YYYY`` release date is given as ISO 8601.
"""
from __future__ import annotations
import re
from typing import TYPE_CHECKING
from datetime import date
from typing import TYPE_CHECKING, TypedDict
from napalm_device_types.models import SystemIdentityDict
from napalm_device_types.terminal import strip_terminal_codes
class SystemIdentityDict(TypedDict):
"""A host's SMBIOS identity, what ties it to its BMC, and its system firmware.
Each value is empty when unknown or a vendor placeholder."""
system_uuid: str # lowercase
product_serial: str
board_serial: str
bios_vendor: str # the system firmware (BIOS or UEFI): its vendor,
bios_version: str # its version,
bios_date: str # its release date, YYYY-MM-DD
_BEGIN = "IDBEGIN"
_END = "IDEND"
_DMI = "/sys/class/dmi/id"
#: Reads the three files in one round trip, framed so a banner or prompt around the
#: output cannot shift a field.
SYSTEM_IDENTITY_COMMAND = (
f"printf '{_BEGIN}\\n%s\\n%s\\n%s\\n{_END}\\n' "
f'"$(cat {_DMI}/product_uuid 2>/dev/null)" '
f'"$(cat {_DMI}/product_serial 2>/dev/null)" '
f'"$(cat {_DMI}/board_serial 2>/dev/null)"'
#: The files read, in the order the output carries them.
_FILES = (
"product_uuid",
"product_serial",
"board_serial",
"bios_vendor",
"bios_version",
"bios_date",
)
#: Reads the files in one round trip, framed so a banner or prompt around the output
#: cannot shift a field.
_FIELDS = "%s\\n" * len(_FILES)
SYSTEM_IDENTITY_COMMAND = f"printf '{_BEGIN}\\n{_FIELDS}{_END}\\n' " + " ".join(
f'"$(cat {_DMI}/{name} 2>/dev/null)"' for name in _FILES
)
#: Lowercased defaults board vendors leave in place.
@@ -80,6 +108,22 @@ def _clean(value: str) -> str:
return "" if is_placeholder_identity(value) else value
def _clean_text(value: str) -> str:
"""A firmware text, empty for a vendor's text default; a short value stays."""
value = value.strip()
return "" if value.lower() in _PLACEHOLDERS else value
def _iso_date(value: str) -> str:
"""sysfs's ``MM/DD/YYYY`` as ``YYYY-MM-DD``; anything else is no date."""
month, _, rest = value.strip().partition("/")
day, _, year = rest.partition("/")
try:
return date(int(year), int(month), int(day)).isoformat()
except ValueError:
return ""
def parse_system_identity(output: str) -> SystemIdentityDict:
"""The identity from :data:`SYSTEM_IDENTITY_COMMAND`'s output, placeholders empty.
@@ -91,12 +135,16 @@ def parse_system_identity(output: str) -> SystemIdentityDict:
end = lines.index(_END, start)
except ValueError:
raise ValueError("system identity: no intact IDBEGIN/IDEND frame") from None
fields = (lines[start:end] + ["", "", ""])[:3]
uuid, product_serial, board_serial = (_clean(f) for f in fields)
fields = (lines[start:end] + [""] * len(_FILES))[: len(_FILES)]
uuid, product_serial, board_serial = (_clean(f) for f in fields[:3])
bios_vendor, bios_version, bios_date = fields[3:]
return {
"system_uuid": uuid.lower(),
"product_serial": product_serial,
"board_serial": board_serial,
"bios_vendor": _clean_text(bios_vendor),
"bios_version": _clean_text(bios_version),
"bios_date": _iso_date(bios_date),
}
@@ -119,12 +167,15 @@ class SystemIdentityMixin:
def get_system_identity(self) -> SystemIdentityDict:
"""
Returns the host's SMBIOS identity, each value empty when unknown or a vendor
placeholder:
Returns the host's SMBIOS identity and system firmware, each value empty when
unknown or a vendor placeholder:
* system_uuid (str) - lowercase
* product_serial (str)
* board_serial (str)
* bios_vendor (str) - the system firmware's vendor
* bios_version (str)
* bios_date (str) - its release date, ``YYYY-MM-DD``
:raises ValueError: if the output carried no intact reading.
"""
+25 -8
View File
@@ -7,7 +7,7 @@ device that differs. :class:`TimeSyncMixin` is the contract for that:
"get_time_sync", None))`` is the truthful answer to "can this driver tell";
* ``configure_time_sync(ntp_servers=None, timezone=None)`` -- a fix template
(:mod:`napalm_device_types.fixes`): ``None`` leaves that value as it is, and the
device ends up with exactly the servers given, in their order. A driver
device ends up configured with exactly the servers given, in their order. A driver
implements the steps; one whose device takes the change in a single call
overrides the method;
* ``time_sync_writes`` -- which of the two values the driver can write. A BMC that
@@ -17,16 +17,24 @@ device that differs. :class:`TimeSyncMixin` is the contract for that:
A driver that inherits the mixin but cannot run it sets ``get_time_sync`` (and
``configure_time_sync``) to ``None``.
**Servers from DHCP count** (NetOrk/netork#2380). A device that takes its time
servers from a DHCP lease (option 42) gets them from whoever serves the subnet --
usually the site's time giver. ``ntp_servers`` are only the servers configured on
the device; ``dhcp_ntp_servers`` are those the device got by DHCP and uses. A write
replaces the configured servers and leaves DHCP's alone: it never switches the use of
DHCP-provided servers off, and the ``verify`` step does not count them.
A zone is an IANA name on the wire. A device that wants a POSIX rule or an offset
with a summer-time rule gets it from :mod:`napalm_device_types.posix_tz`.
"""
from __future__ import annotations
# No ``from __future__ import annotations`` here: TypedDict tells a NotRequired key
# only from an evaluated annotation (``TimeSyncDict.__optional_keys__``).
import ipaddress
import re
from collections.abc import Sequence
from typing import TYPE_CHECKING, ClassVar, TypedDict
from typing import TYPE_CHECKING, ClassVar, NotRequired, TypedDict
from napalm_device_types.fixes import FixStep, run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
@@ -48,6 +56,9 @@ class TimeSyncDict(TypedDict):
timezone: str | None # IANA name ("Europe/Berlin"); None = not readable or no named zone
synchronized: bool | None # the clock is synchronised to a source right now; None = unknown
daemon: str | None # "chrony", "systemd-timesyncd", "ntpd", "openntpd", ...; None = own client
# The servers the device got by DHCP (option 42) and uses, in its order; never in
# ntp_servers. [] = none; absent or None = the driver cannot tell.
dhcp_ntp_servers: NotRequired[list[str] | None]
def _is_server(name: str) -> bool:
@@ -94,7 +105,9 @@ class TimeSyncMixin:
"""How the device keeps its clock now.
``ntp_servers`` are the sources its configuration names, in its order
(a pool by its name, not the addresses it resolved to). ``timezone``
(a pool by its name, not the addresses it resolved to);
``dhcp_ntp_servers`` those a DHCP lease gave it and it uses, apart from
them -- left out where the driver cannot tell. ``timezone``
is an IANA name, or None where the device keeps an offset and a rule
instead of a name. ``daemon`` names the client that keeps the clock on
a host that has a choice; None for a device's own.
@@ -106,6 +119,7 @@ class TimeSyncMixin:
"timezone": "Europe/Berlin",
"synchronized": True,
"daemon": "chrony",
"dhcp_ntp_servers": ["192.168.1.1"],
}
:raises Exception: when the device could not be read.
@@ -115,8 +129,9 @@ class TimeSyncMixin:
# The steps a driver implements:
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
"""Leave the device with exactly *servers*, in this order: sources it
had configured elsewhere stop counting (commented out, replaced)."""
"""Leave the device configured with exactly *servers*, in this order:
sources it had configured elsewhere stop counting (commented out,
replaced). Servers it gets by DHCP stay, and so does its use of them."""
...
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
@@ -128,6 +143,8 @@ class TimeSyncMixin:
) -> FixResultDict:
"""Give the device exactly *ntp_servers*, in order, and *timezone*; None leaves one alone.
What the device gets by DHCP is not touched: it keeps using those servers too.
The steps: ``ntp_servers``, ``timezone``, then ``verify``, which reads the
device again.
@@ -157,8 +174,8 @@ class TimeSyncMixin:
def _verify_time_sync(self, servers: list[str] | None, zone: str | None) -> FixStepOutcomeDict:
"""Read the device again: does it now have what was written?
Servers compare without case. A zone is compared where the device reads
one back by name.
Servers compare without case; the servers DHCP brought are not among
them. A zone is compared where the device reads one back by name.
"""
now = self.get_time_sync()
wrong: list[str] = []
+22 -6
View File
@@ -13,17 +13,22 @@ reads them, ``timedatectl show-timesync`` for systemd-timesyncd, ``ntp.conf``,
OpenNTPD's ``ntpd.conf``. Synchronised is chronyd's own word where it runs (a
selected source in ``chronyc -n sources``), else the kernel's (``NTPSynchronized``).
What DHCP brought (option 42) is read apart, as ``dhcp_ntp_servers``, and never as a
configured server: chrony's sources from the DHCP client's ``sourcedir``
(:attr:`~PosixTimeSyncMixin.CHRONY_DHCP_DIRS`, the files as chronyd reads them),
systemd-timesyncd's ``LinkNTPServers``. Where ntpd or OpenNTPD runs it is unknown.
**Writing** runs as root through ``run_command(privileged=True)``; root never reads
stdin -- a file is staged by the login user and installed by root.
* The zone: ``timedatectl set-timezone``.
* chrony: the servers go to a file of their own and every other source is commented
out (:mod:`napalm_device_types._chrony`), then ``chronyc reload sources``, or a
restart of chronyd where its main configuration changed or it cannot reload.
* chrony: the servers go to a file of their own and every other configured source is
commented out (:mod:`napalm_device_types._chrony`); the DHCP client's ``sourcedir``
stays, and one 4.35.0 commented out comes back. Then ``chronyc reload sources``, or
a restart of chronyd where its main configuration changed or it cannot reload.
* systemd-timesyncd: :attr:`~PosixTimeSyncMixin.TIMESYNCD_DROPIN` resets ``NTP=``,
sets the servers and empties ``FallbackNTP=``, then the service restarts. Servers a
network link brings (DHCP through systemd-networkd) are not timesyncd's to drop;
they stay, and the ``verify`` step names them.
network link brings (DHCP through systemd-networkd) stay and count.
* ntpd, OpenNTPD, or no client: the servers are read, not written -- the step fails
and says so.
"""
@@ -39,9 +44,11 @@ from napalm_device_types._chrony import (
comment_out_command,
include_command,
plan_chrony,
restore_command,
)
from napalm_device_types._scheduled_reading import TIME_ZONE_SH
from napalm_device_types._time_sync_reading import (
CHRONY_DHCP_DIRS,
TIME_SYNC_UNITS,
Reading,
ReadPaths,
@@ -51,6 +58,7 @@ from napalm_device_types._time_sync_reading import (
parse_time_sync,
reading_command,
time_sync_of,
timesyncd_link_servers,
timesyncd_servers,
)
from napalm_device_types.models import FixStepOutcomeDict
@@ -91,6 +99,9 @@ class PosixTimeSyncMixin(TimeSyncMixin):
CHRONY_SOURCES_FILE: ClassVar[str] = "/etc/chrony/sources.d/netork.sources"
#: The servers' file otherwise (its name also in a ``confdir``); included if need be.
CHRONY_INCLUDE_FILE: ClassVar[str] = "/etc/chrony.d/netork.conf"
#: The ``sourcedir`` a DHCP client writes chrony's sources to: read as
#: ``dhcp_ntp_servers``, never commented out.
CHRONY_DHCP_DIRS: ClassVar[tuple[str, ...]] = CHRONY_DHCP_DIRS
TIMESYNCD_DROPIN: ClassVar[str] = "/etc/systemd/timesyncd.conf.d/netork.conf"
NTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/ntpsec/ntp.conf", "/etc/ntp.conf")
OPENNTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/openntpd/ntpd.conf", "/etc/ntpd.conf")
@@ -125,7 +136,7 @@ class PosixTimeSyncMixin(TimeSyncMixin):
:raises ValueError: when the reading came back cut short.
"""
return time_sync_of(self._ts_read())
return time_sync_of(self._ts_read(), self.CHRONY_DHCP_DIRS)
# -- writing ---------------------------------------------------------------------------------
@@ -175,9 +186,13 @@ class PosixTimeSyncMixin(TimeSyncMixin):
servers,
sources_file=self.CHRONY_SOURCES_FILE,
include_file=self.CHRONY_INCLUDE_FILE,
dhcp_dirs=self.CHRONY_DHCP_DIRS,
)
said = [f"wrote {plan.target}"]
self._ts_install(plan.target, plan.content)
for path, numbers in plan.restore.items():
self._ts_checked(restore_command(path, numbers))
said.append(f"gave DHCP's sourcedir back in {path}:{','.join(map(str, numbers))}")
for path, numbers in plan.comment_out.items():
self._ts_checked(comment_out_command(path, numbers))
said.append(f"commented out {path}:{','.join(map(str, numbers))}")
@@ -199,5 +214,6 @@ __all__ = [
"parse_key_values",
"parse_time_sync",
"timesyncd_dropin",
"timesyncd_link_servers",
"timesyncd_servers",
]
+35 -1
View File
@@ -17,6 +17,15 @@ from typing import TYPE_CHECKING, Any
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict, UpdateProgressDict
class PackageIndexMissing(RuntimeError):
"""The host holds no package index to read pending updates from.
Not a fault: OpenWrt keeps its index in RAM and loses it on every reboot.
:meth:`UpdateMixin.refresh_available_updates` loads it again, so a caller
that sees this can refresh instead of reporting a failed read.
"""
class UpdateMixin:
if TYPE_CHECKING:
@@ -37,6 +46,8 @@ class UpdateMixin:
read -- no package index yet, an API that did not answer -- raises
instead: netOrk keeps "pending since" per package, and an empty
list for "don't know" would reset every one of those clocks.
Without a package index it raises :exc:`PackageIndexMissing`, which
:meth:`refresh_available_updates` cures.
Example::
@@ -50,6 +61,28 @@ class UpdateMixin:
"""
...
def get_installed_advisory_cves(self) -> dict[str, list[str]]:
"""
The CVEs the host's installed security advisories close, per package
name: the distribution's own word that a fix is in the installed build,
although its upstream version did not move (a backport).
netOrk asks for it only where its knowledge base holds no records of
the host's release (netOrk #2394), since the answer can be long. A
package manager whose advisories name no CVEs has nothing to say here.
**An empty dict means no installed advisory names a CVE.** A reader
that cannot read raises, as :meth:`get_available_updates` does.
``NotImplementedError`` where the host's package manager keeps no such
advisories (:data:`~napalm_device_types.package_updates.DNF_INSTALLED_CVES_COMMAND`
is dnf 4's).
Example::
{"glibc-common": ["CVE-2023-4911"], "curl": ["CVE-2023-38545"]}
"""
...
def refresh_available_updates(self) -> dict[str, Any]:
"""
Refreshes the host's package index, so that :meth:`get_available_updates`
@@ -64,7 +97,8 @@ class UpdateMixin:
"""
Where an update the device runs by itself stands, for a caller that
follows it: a firewall's firmware upgrade, which :meth:`apply_updates`
only starts. ``state`` is ``"running"`` until it ends in ``"done"``
only starts, or one that ``FirmwareUpgradeMixin.start_firmware_upgrade``
started. ``state`` is ``"running"`` until it ends in ``"done"``
or ``"failed"``; ``log`` is the device's whole log of it.
Declared only: a driver whose :meth:`apply_updates` returns when the
+210
View File
@@ -0,0 +1,210 @@
"""A native Zabbix agent: its configuration and Zabbix's package repository.
What :class:`~napalm_device_types.fixes.ZabbixAgentMixin` needs that is the same on
every host, without I/O: which settings an agent may be given
(:func:`check_zabbix_agent_settings`), the agent's configuration file with them
(:func:`zabbix_agent_config`), and the command that adds Zabbix's repository where
the host's own packages have no agent (:func:`zabbix_repository_command`). A driver
reads and writes the file and runs the command.
**Which lines.** ``Server`` (who may ask, for passive checks), ``ServerActive``
(where active checks go), ``ListenPort`` and, when the caller names one,
``Hostname`` -- the host's technical name in Zabbix, which an active agent sends.
Every other line stays as it was; a key the file does not set is added at its end.
**The repository.** Zabbix publishes a ``zabbix-release`` package per release line
and distribution that adds its repository and key. Since 7.2 it lives under
``<release>/release/<distribution>/``; up to 7.0 under ``<release>/<distribution>/``,
with the rpm in an ``x86_64`` directory. The release line is the server's: an agent
may be older than its server, not newer.
"""
from __future__ import annotations
import re
import shlex
from typing import NotRequired, TypedDict
class ZabbixAgentSettingsDict(TypedDict):
"""What the caller wants a native Zabbix agent to be (``ZabbixAgentMixin``)."""
server: str # Server: the addresses that may ask (passive checks), comma-separated
server_active: str # ServerActive: where active checks go, comma-separated; "" for none
listen_port: int # ListenPort
# addresses or CIDRs let through to the port; [] changes no firewall
allowed_sources: list[str]
# Hostname: the host's technical name in Zabbix; absent keeps the agent's
hostname: NotRequired[str]
# the server's release line ("7.4"): Zabbix's repository to fall back to where the
# host's own packages have no agent; absent: no fallback
release: NotRequired[str]
#: Addresses, comma-separated, as ``Server`` and ``ServerActive`` take them: names,
#: IPv4, ``[IPv6]`` and ports. Nothing a config line or a shell word breaks out of.
_ADDRESSES = re.compile(r"^[A-Za-z0-9.:\[\]-]+(,[A-Za-z0-9.:\[\]-]+)*$")
#: What Zabbix allows in a technical host name.
_HOSTNAME = re.compile(r"^[A-Za-z0-9._ -]+$")
#: A release line, as Zabbix's repository names it.
_RELEASE = re.compile(r"^(\d+)\.(\d+)$")
_KEYS = ("Server", "ServerActive", "ListenPort", "Hostname")
_LINE = re.compile(r"^(\s*)(" + "|".join(_KEYS) + r")\s*=\s*(.*?)\s*$")
REPOSITORY = "https://repo.zabbix.com/zabbix"
#: The distributions Zabbix publishes packages for, by ``/etc/os-release``'s ``ID``:
#: the name in the repository's path, and whether they take a deb.
_DISTRIBUTIONS: dict[str, tuple[str, bool]] = {
"ubuntu": ("ubuntu", True),
"debian": ("debian", True),
"raspbian": ("raspbian", True),
"rhel": ("rhel", False),
"centos": ("rhel", False),
"ol": ("oracle", False),
"almalinux": ("alma", False),
"rocky": ("rocky", False),
}
def check_zabbix_agent_settings(settings: ZabbixAgentSettingsDict) -> None:
"""Refuse what an agent's configuration or a shell line cannot take.
:raises ValueError: naming the setting.
"""
_check_addresses("Server", str(settings.get("server", "")), required=True)
_check_addresses("ServerActive", str(settings.get("server_active", "")), required=False)
hostname = settings.get("hostname")
if hostname is not None and not _HOSTNAME.match(hostname):
raise ValueError(f"not a Zabbix host name: {hostname!r}")
port = settings.get("listen_port")
if not isinstance(port, int) or isinstance(port, bool) or not 1 <= port <= 65535:
raise ValueError(f"not a port: {port!r}")
release = settings.get("release")
if release is not None and not _RELEASE.match(release):
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
def _check_addresses(key: str, value: str, *, required: bool) -> None:
if (value or required) and not _ADDRESSES.match(value):
raise ValueError(f"not addresses an agent's {key} takes: {value!r}")
def _wanted(settings: ZabbixAgentSettingsDict) -> dict[str, str]:
wanted = {
"Server": settings["server"],
"ServerActive": settings.get("server_active", ""),
"ListenPort": str(settings["listen_port"]),
}
if settings.get("hostname"):
wanted["Hostname"] = str(settings["hostname"])
return wanted
def zabbix_agent_config(text: str, settings: ZabbixAgentSettingsDict) -> tuple[str, list[str]]:
"""*text*, an agent's configuration file, with *settings*; and what changed, one
``"Key: old → new"`` each.
An empty ``server_active`` comments ``ServerActive`` out: the agent then runs
passive checks only.
"""
wanted = _wanted(settings)
lines: list[str] = []
changes: list[str] = []
seen: set[str] = set()
for line in text.splitlines(keepends=True):
match = _LINE.match(line.rstrip("\r\n"))
if not match or match.group(2) not in wanted:
lines.append(line)
continue
seen.add(match.group(2))
lines.append(_set_line(line, match, wanted, changes))
_add_missing(
lines, [key for key in _KEYS if wanted.get(key) and key not in seen], wanted, changes
)
return "".join(lines), changes
def _add_missing(
lines: list[str], keys: list[str], wanted: dict[str, str], changes: list[str]
) -> None:
"""*keys*, which the file does not set, at its end."""
if keys and lines and not lines[-1].endswith("\n"):
lines[-1] += "\n"
for key in keys:
lines.append(f"{key}={wanted[key]}\n")
changes.append(f"{key}: (not set) → {wanted[key]}")
def _set_line(line: str, match: re.Match[str], wanted: dict[str, str], changes: list[str]) -> str:
"""*line*, which sets a key the caller wants, as the caller wants it."""
indent, key, old = match.groups()
end = "\n" if line.endswith("\n") else ""
new = wanted[key]
if not new:
changes.append(f"{key}: {old} → (not set)")
return f"{indent}# {key}={old}{end}"
if old == new:
return line
changes.append(f"{key}: {old} → {new}")
return f"{indent}{key}={new}{end}"
def _os_release(text: str) -> dict[str, str]:
fields = {}
for line in text.splitlines():
key, sep, value = line.partition("=")
if sep:
fields[key.strip()] = value.strip().strip("'\"")
return fields
def _distribution(fields: dict[str, str]) -> tuple[str, bool]:
for candidate in [fields.get("ID", ""), *fields.get("ID_LIKE", "").split()]:
if candidate in _DISTRIBUTIONS:
return _DISTRIBUTIONS[candidate]
raise ValueError(f"Zabbix publishes no repository for {fields.get('ID') or 'this system'}")
def zabbix_repository_command(os_release: str, release: str) -> str:
"""The root command that adds Zabbix's *release* repository to the host whose
``/etc/os-release`` is *os_release*, and refreshes the package index.
:raises ValueError: for a release line that is not ``major.minor``, or a
distribution Zabbix publishes nothing for.
"""
match = _RELEASE.match(release)
if not match:
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
fields = _os_release(os_release)
name, deb = _distribution(fields)
version = fields.get("VERSION_ID", "")
new_layout = (int(match.group(1)), int(match.group(2))) >= (7, 2)
base = f"{REPOSITORY}/{release}/" + ("release/" if new_layout else "") + name
if deb:
# Raspberry Pi OS's packages are named after the Debian release they follow.
label = "debian" if name == "raspbian" else name
filename = f"zabbix-release_latest_{release}+{label}{version}_all.deb"
url = f"{base}/pool/main/z/zabbix-release/{filename}"
package = shlex.quote(url)
return (
"t=$(mktemp -d) && "
f'{{ curl -fsSL -o "$t/zabbix-release.deb" {package} '
f'|| wget -qO "$t/zabbix-release.deb" {package}; }} && '
'dpkg -i "$t/zabbix-release.deb" && apt-get update; '
# The download's directory goes either way; the status is the install's.
's=$?; rm -rf "$t"; [ "$s" -eq 0 ]'
)
major = version.split(".")[0]
arch = "noarch" if new_layout else "x86_64"
url = f"{base}/{major}/{arch}/zabbix-release-latest-{release}.el{major}.noarch.rpm"
return f"dnf -y install {shlex.quote(url)}"
__all__ = [
"REPOSITORY",
"ZabbixAgentSettingsDict",
"check_zabbix_agent_settings",
"zabbix_agent_config",
"zabbix_repository_command",
]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.35.0"
version = "4.41.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.12"
+1 -2
View File
@@ -20,7 +20,6 @@
},
"function_lines": {
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 119,
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_reservations": 54,
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_subnets": 60,
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
@@ -32,7 +31,7 @@
},
"mypy_relaxed": [],
"python_file_lines": {
"napalm_device_types/models.py": 1454
"napalm_device_types/models.py": 1436
},
"untyped_public": []
}
+195 -1
View File
@@ -25,6 +25,7 @@ from napalm_device_types.apt_proxy import (
parse_apt_proxy,
same_proxy_url,
)
from napalm_device_types.apt_sources import NOT_CACHED_KEY, VIA_CACHE
from napalm_device_types.fixes import PackageProxyMixin
@@ -302,6 +303,13 @@ _CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
E: Download Failed"""
#: A cache that does not fetch the cache form itself: ``AllowUserPorts`` is 80 alone
#: (apt-cacher-ng 3.7.5, NetOrk/netork#2555).
_PROHIBITED_PORT = """Err:1 http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease
403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
E: Failed to fetch http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease 403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
E: Download Failed"""
def _fetch(output, rc, match=None):
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
@@ -422,9 +430,21 @@ class _Host(AptProxyMixin):
"""
def __init__(
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
self,
files=None,
fetch_rc=0,
apt=True,
https_sources=(),
tunnel=True,
direct_rc=0,
cache_https=False,
credentials=(),
):
self.files = dict(files or {})
#: Whether the proxy fetches the cache form itself (AllowUserPorts 443), and
#: the hosts apt keeps credentials for.
self.cache_https = cache_https
self.credentials = set(credentials)
self.fetch_rc = fetch_rc
self.apt = apt
self.https_sources = list(https_sources)
@@ -459,6 +479,12 @@ class _Host(AptProxyMixin):
value, https = self._effective(), self._effective("https")
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
dump += [
line
for path, text in self.files.items()
for line in text.splitlines()
if line.startswith(NOT_CACHED_KEY)
]
return _report(
dump=dump,
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
@@ -466,6 +492,8 @@ class _Host(AptProxyMixin):
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
)
self.fetched.append(command)
if re.search(r"download-file '?http://HTTPS///", command):
return _fetch(_FETCHED, 0) if self.cache_https else _fetch(_PROHIBITED_PORT, 100)
if re.search(r"download-file '?https://", command):
return self._fetch_https(command)
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
@@ -482,6 +510,11 @@ class _Host(AptProxyMixin):
def _run_apt_root_command(self, command, done):
self.root.append(command)
if NOT_CACHED_KEY in command:
return self._rewrite(re.search(r"for h in ([^;]+); do", command).group(1).split(), done)
if "#\\1https://#g" in command:
self.https_sources = [s.replace(VIA_CACHE, "https://") for s in self.https_sources]
return {"ok": True, "output": done}
path = command.rsplit(" ", 1)[-1].strip("'")
for m in re.finditer(r"(\d+)\{/[^}]*\}", command):
n = int(m.group(1))
@@ -492,6 +525,19 @@ class _Host(AptProxyMixin):
self.files.pop(command.split()[-1].strip("'"), None)
return {"ok": True, "output": done}
def _rewrite(self, hosts, done):
for host in hosts:
if host in self.credentials:
proxy = self.files.get(f"{_D}00proxy", "")
self.files[f"{_D}00proxy"] = proxy + f'{NOT_CACHED_KEY}:: "{host}";\n'
continue
prefix = f"https://{host}/"
self.https_sources = [
VIA_CACHE + s[len("https://") :] if s.startswith(prefix) else s
for s in self.https_sources
]
return {"ok": True, "output": done}
class TestCheckHttpsSource:
def test_one_source_the_way_apt_fetches_it(self):
@@ -569,8 +615,10 @@ class TestConfigurePackageProxy:
"conflicts",
"verify",
"fetch",
"cache",
"https",
]
assert "no https source" in _step(result, "cache")["output"]
assert "no https source" in result["steps"][-1]["output"]
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
@@ -766,6 +814,144 @@ class TestConfigurePackageProxy:
assert host.files == {}
class TestCacheHttpsSources:
"""NetOrk/netork#2555: the cache stores an https source only in its own form."""
def test_a_cache_that_fetches_https_gets_them(self):
host = _Host(https_sources=[_DOCKER], cache_https=True)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.https_sources == [f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"]
assert _step(result, "cache")["output"] == (
"https sources cached through http://c:3142: download.docker.com"
)
assert any(VIA_CACHE in c for c in host.fetched)
assert "no https source goes through" in _step(result, "https")["output"]
def test_a_cache_that_does_not_turns_them_back(self):
host = _Host(https_sources=[_DOCKER])
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.https_sources == [_DOCKER]
output = _step(result, "cache")["output"]
assert "http://c:3142 does not fetch https sources itself" in output
assert "prohibited port (see AllowUserPorts)" in output
assert "they stay on https, not cached" in output
assert _step(result, "https")["output"] == (
f"fetched {_DOCKER}InRelease through http://c:3142"
)
def test_a_host_with_credentials_stays_on_https_and_is_named(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(
https_sources=[pve, _DOCKER],
cache_https=True,
credentials={"enterprise.proxmox.com"},
)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert pve in host.https_sources
output = _step(result, "cache")["output"]
assert "cached through http://c:3142: download.docker.com" in output
assert "left on https, apt has credentials for them: enterprise.proxmox.com" in output
assert host.get_package_proxy_report()["not_cached"] == ["enterprise.proxmox.com"]
def test_only_hosts_with_credentials(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(https_sources=[pve], cache_https=True, credentials={"enterprise.proxmox.com"})
result = host.configure_package_proxy("http://c:3142")
assert _step(result, "cache")["output"] == (
"no https source to cache; "
"left on https, apt has credentials for them: enterprise.proxmox.com"
)
def test_the_https_direct_fallback_keeps_the_hosts_left_on_https(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(https_sources=[pve], tunnel=False, credentials={"enterprise.proxmox.com"})
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.files[f"{_D}00proxy"] == (
'Acquire::http::Proxy "http://c:3142";\n'
'Acquire::https::Proxy "DIRECT";\n'
f'{NOT_CACHED_KEY}:: "enterprise.proxmox.com";\n'
)
def test_a_rewrite_that_fails_stops_the_fix(self):
host = _Host(https_sources=[_DOCKER], cache_https=True)
real = host._run_apt_root_command
def denied(command, done):
if NOT_CACHED_KEY in command:
return {"ok": False, "output": "sed: denied"}
return real(command, done)
host._run_apt_root_command = denied
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
assert result["steps"][-1] == {"step": "cache", "ok": False, "output": "sed: denied"}
def test_a_turn_back_that_fails_says_so(self):
host = _Host(https_sources=[_DOCKER])
real = host._run_apt_root_command
def denied(command, done):
if "#\\1https://#g" in command:
return {"ok": False, "output": "sed: denied"}
return real(command, done)
host._run_apt_root_command = denied
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
output = result["steps"][-1]["output"]
assert "turning the sources back to https failed: sed: denied" in output
def test_the_proof_of_the_proxy_is_a_plain_http_source(self):
"""A source already in the cache form says nothing about plain http."""
host = _Host(https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"])
host.configure_package_proxy("http://c:3142")
fetch = next(c for c in host.fetched if "InRelease" in c)
assert "deb.debian.org" in fetch
def test_removing_the_proxy_turns_the_sources_back_first(self):
host = _Host(
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"],
)
result = host.configure_package_proxy("")
assert result["success"] is True
assert [s["step"] for s in result["steps"]] == ["sources", "configure"]
assert host.https_sources == [_DOCKER]
assert host.files == {}
def test_a_turn_back_that_fails_keeps_the_proxy(self):
"""Without the cache apt looks for a host called HTTPS: the proxy stays."""
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
host._run_apt_root_command = lambda command, done: {"ok": False, "output": "denied"}
result = host.configure_package_proxy("")
assert result["success"] is False
assert f"{_D}00proxy" in host.files
class TestWhatTheParserSkips:
def test_a_line_that_is_no_grep_output(self):
assert (
@@ -824,3 +1010,11 @@ def test_the_comment_out_command_for_https():
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
)
def test_a_line_left_on_https_that_is_no_host_name_is_not_written_back():
host = _Host()
host._write_package_proxy("http://c:3142", not_cached=["ok.example", 'x"; Foo "1'])
assert host.files[f"{_D}00proxy"] == (
f'Acquire::http::Proxy "http://c:3142";\n{NOT_CACHED_KEY}:: "ok.example";\n'
)
+393
View File
@@ -0,0 +1,393 @@
"""apt's sources and the release they name, and moving one file's sources to the host's.
A release upgrade keeps third-party sources on the old release: z2m-house, Ubuntu
24.04 (noble), kept Docker's source on ``focal`` and never saw a Docker newer than
the last focal build (NetOrk/netork#2614). The read command runs here for real, on
files in a temporary directory that ``apt-config -o`` points apt's settings at.
"""
from __future__ import annotations
import base64
import shutil
import subprocess
import pytest
from napalm_device_types.apt_proxy import AptProxyMixin
from napalm_device_types.apt_releases import (
AptReleaseMixin,
apt_releases_command,
parse_apt_releases,
rewrite_release,
suite_release,
)
_APT_CONFIG = shutil.which("apt-config")
needs_apt = pytest.mark.skipif(_APT_CONFIG is None, reason="apt-config is not on this host")
_NOBLE = (
'PRETTY_NAME="Ubuntu 24.04.3 LTS"\nNAME="Ubuntu"\nVERSION_ID="24.04"\n'
"VERSION_CODENAME=noble\nID=ubuntu\nID_LIKE=debian\nUBUNTU_CODENAME=noble\n"
)
_TRIXIE = 'PRETTY_NAME="Debian GNU/Linux 13 (trixie)"\nVERSION_CODENAME=trixie\nID=debian\n'
_MINT = (
'NAME="Linux Mint"\nVERSION_CODENAME=virginia\nID=linuxmint\nID_LIKE="ubuntu debian"\n'
"UBUNTU_CODENAME=jammy\n"
)
#: z2m-house's Docker source, as found: through the cache, on focal (#2614).
_Z2M_DOCKER = "deb http://HTTPS///download.docker.com/linux/ubuntu focal stable\n"
_Z2M_DOCKER_FILE = "archive_uri-https_download_docker_com_linux_ubuntu-noble.list"
_UBUNTU_SOURCES = (
"Types: deb\n"
"URIs: http://archive.ubuntu.com/ubuntu/\n"
"Suites: noble noble-updates noble-backports\n"
"Components: main restricted universe multiverse\n"
"Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n"
"\n"
"Types: deb\n"
"URIs: http://security.ubuntu.com/ubuntu/\n"
"Suites: noble-security\n"
"Components: main restricted universe multiverse\n"
)
def _framed(os_release: str, files: dict[str, str]) -> str:
"""What :func:`apt_releases_command` prints for *os_release* and *files*."""
lines = ["ARELEASE_BEGIN", "@@os"]
lines += [f"|{line}" for line in os_release.splitlines()]
for path, text in files.items():
lines.append(f"@@file {path}")
lines += [f"|{line}" for line in text.splitlines()]
lines.append("ARELEASE_END")
return "\n".join(lines) + "\n"
class TestSuiteRelease:
@pytest.mark.parametrize(
("suite", "release"),
[
("noble", "noble"),
("noble-security", "noble"),
("noble-updates", "noble"),
("focal-backports", "focal"),
("bookworm-proposed-updates", "bookworm"),
("trixie-security", "trixie"),
],
)
def test_the_release_a_suite_names(self, suite, release):
assert suite_release(suite) == release
@pytest.mark.parametrize(
("suite", "word"),
[("stable", "stable"), ("nodistro", "nodistro"), ("pve-no-subscription", "pve")],
)
def test_a_word_the_caller_judges(self, suite, word):
# Which words are releases is the caller's to know: "stable" has the form too.
assert suite_release(suite) == word
@pytest.mark.parametrize("suite", ["/", "./", "focal/", "Focal", "$(ARCH)", ""])
def test_a_suite_of_another_form(self, suite):
# A flat repository's path, or anything not a lower-case word, names none.
assert suite_release(suite) is None
class TestParse:
def test_the_host_and_a_list_source(self):
found = parse_apt_releases(
_framed(_NOBLE, {f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}": _Z2M_DOCKER})
)
assert found["os_id"] == "ubuntu"
assert found["os_like"] == ["debian"]
assert found["codename"] == "noble"
assert found["ubuntu_codename"] == "noble"
assert found["sources"] == [
{
"file": f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}",
"line": 1,
"types": ["deb"],
"uri": "http://HTTPS///download.docker.com/linux/ubuntu",
"suites": ["focal"],
"components": ["stable"],
}
]
def test_options_comments_and_disabled_lines(self):
text = (
"# deb http://old.example/ubuntu bionic main\n"
"\n"
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
"https://download.docker.com/linux/ubuntu jammy stable # moved by hand\n"
"deb-src [ arch=amd64 ] http://archive.ubuntu.com/ubuntu noble main\n"
"deb https://repo.example/flat ./\n"
)
found = parse_apt_releases(_framed(_NOBLE, {"/etc/apt/sources.list": text}))
assert [
(s["line"], s["types"], s["uri"], s["suites"], s["components"])
for s in found["sources"]
] == [
(3, ["deb"], "https://download.docker.com/linux/ubuntu", ["jammy"], ["stable"]),
(4, ["deb-src"], "http://archive.ubuntu.com/ubuntu", ["noble"], ["main"]),
(5, ["deb"], "https://repo.example/flat", ["./"], []),
]
def test_deb822_paragraphs_one_per_uri(self):
text = (
"Types: deb deb-src\n"
"URIs: http://a.example/ubuntu http://b.example/ubuntu\n"
"Suites: focal\n"
" focal-updates\n"
"Components: main\n"
"\n"
"# a comment between paragraphs\n"
"Enabled: no\n"
"Types: deb\n"
"URIs: http://c.example/ubuntu\n"
"Suites: bionic\n"
"Components: main\n"
)
found = parse_apt_releases(_framed(_NOBLE, {"/etc/apt/sources.list.d/x.sources": text}))
assert [(s["line"], s["types"], s["uri"], s["suites"]) for s in found["sources"]] == [
(3, ["deb", "deb-src"], "http://a.example/ubuntu", ["focal", "focal-updates"]),
(3, ["deb", "deb-src"], "http://b.example/ubuntu", ["focal", "focal-updates"]),
]
def test_debian_and_a_derivative(self):
assert parse_apt_releases(_framed(_TRIXIE, {}))["codename"] == "trixie"
mint = parse_apt_releases(_framed(_MINT, {}))
assert (mint["os_id"], mint["os_like"], mint["codename"], mint["ubuntu_codename"]) == (
"linuxmint",
["ubuntu", "debian"],
"virginia",
"jammy",
)
def test_without_os_release(self):
found = parse_apt_releases("ARELEASE_BEGIN\n@@os\nARELEASE_END\n")
assert (found["os_id"], found["codename"], found["ubuntu_codename"]) == ("", "", "")
def test_an_unreadable_file_is_left_out(self):
out = _framed(_NOBLE, {}).replace(
"ARELEASE_END", "@@unreadable /etc/apt/sources.list\nARELEASE_END"
)
assert parse_apt_releases(out)["sources"] == []
def test_no_intact_report(self):
with pytest.raises(ValueError):
parse_apt_releases("ARELEASE_BEGIN\n@@os\n")
@needs_apt
class TestTheCommand:
def test_reads_list_and_sources_files_as_apt_names_them(self, tmp_path):
(tmp_path / "sources.list.d").mkdir()
(tmp_path / "sources.list").write_text("")
(tmp_path / "sources.list.d" / _Z2M_DOCKER_FILE).write_text(_Z2M_DOCKER)
(tmp_path / "sources.list.d" / "ubuntu.sources").write_text(_UBUNTU_SOURCES)
(tmp_path / "sources.list.d" / "old.list.distUpgrade").write_text(_Z2M_DOCKER)
(tmp_path / "os-release").write_text(_NOBLE)
command = apt_releases_command(
apt_config=(
f"apt-config -o Dir::Etc::sourcelist={tmp_path}/sources.list "
f"-o Dir::Etc::sourceparts={tmp_path}/sources.list.d/"
),
os_release=str(tmp_path / "os-release"),
)
done = subprocess.run(["sh", "-c", command], capture_output=True, text=True, check=False)
found = parse_apt_releases(done.stdout)
assert found["codename"] == "noble"
assert [(s["file"].rsplit("/", 1)[-1], s["suites"]) for s in found["sources"]] == [
(_Z2M_DOCKER_FILE, ["focal"]),
("ubuntu.sources", ["noble", "noble-updates", "noble-backports"]),
("ubuntu.sources", ["noble-security"]),
]
class TestRewriteRelease:
def test_a_list_line(self):
assert rewrite_release(_Z2M_DOCKER, "focal", "noble", deb822=False) == (
"deb http://HTTPS///download.docker.com/linux/ubuntu noble stable\n",
1,
)
def test_suffixes_options_and_a_uri_that_names_the_release(self):
text = (
"deb [signed-by=/k.asc] http://repo.example/focal/ focal-security main # focal\n"
"# deb http://repo.example/focal/ focal main\n"
"deb http://repo.example/other noble main\n"
"deb http://repo.example/flat ./\n"
)
assert rewrite_release(text, "focal", "noble", deb822=False) == (
"deb [signed-by=/k.asc] http://repo.example/focal/ noble-security main # focal\n"
"# deb http://repo.example/focal/ focal main\n"
"deb http://repo.example/other noble main\n"
"deb http://repo.example/flat ./\n",
1,
)
def test_a_release_that_is_only_a_prefix_is_not_it(self):
text = "deb http://repo.example/x focalfossa main\n"
assert rewrite_release(text, "focal", "noble", deb822=False) == (text, 0)
def test_deb822_suites_and_continuations_of_enabled_paragraphs(self):
text = (
"Types: deb\n"
"URIs: http://a.example/focal\n"
"Suites: focal focal-updates\n"
" focal-backports\n"
"Components: main\n"
"\n"
"Enabled: no\n"
"Types: deb\n"
"URIs: http://b.example/ubuntu\n"
"Suites: focal\n"
"Components: main\n"
)
new, changed = rewrite_release(text, "focal", "noble", deb822=True)
assert changed == 3
assert new == (
"Types: deb\n"
"URIs: http://a.example/focal\n"
"Suites: noble noble-updates\n"
" noble-backports\n"
"Components: main\n"
"\n"
"Enabled: no\n"
"Types: deb\n"
"URIs: http://b.example/ubuntu\n"
"Suites: focal\n"
"Components: main\n"
)
def test_the_text_without_a_final_newline_stays_so(self):
assert rewrite_release("deb http://a/ focal main", "focal", "noble", deb822=False) == (
"deb http://a/ noble main",
1,
)
@pytest.mark.parametrize("bad", ["", "Noble", "no ble", "noble;", "../x"])
def test_a_name_that_is_no_codename(self, bad):
with pytest.raises(ValueError):
rewrite_release(_Z2M_DOCKER, bad, "noble", deb822=False)
with pytest.raises(ValueError):
rewrite_release(_Z2M_DOCKER, "focal", bad, deb822=False)
_OK = "[fetch]\nGet:1 x\n[rc]\n0\n"
_NOT_FOUND = "[fetch]\nE: Failed to fetch {url} 404 Not Found [IP: 1.2.3.4 80]\n[rc]\n100\n"
class _Host(AptReleaseMixin):
"""An apt host whose files are a dict and whose fetches answer from a set."""
def __init__(self, files, os_release=_NOBLE, served=(), apt=True):
self.files = dict(files)
self.os_release = os_release
self.served = set(served)
self.apt = apt
self.fetched: list[str] = []
self.written: list[str] = []
def _is_apt_host(self):
return self.apt
def _run_apt_command(self, command):
if "ARELEASE_" in command:
return _framed(self.os_release, self.files)
if command.startswith("base64 "):
path = command.split(" ", 1)[1].strip("'")
return base64.b64encode(self.files[path].encode()).decode() + "\n"
url = command.split("download-file ", 1)[1].split()[0].strip("'")
self.fetched.append(url)
body = _OK if url in self.served else _NOT_FOUND.format(url=url)
return f"APROXY_BEGIN\n{body}APROXY_END\n"
def _write_apt_file(self, path, content):
self.written.append(path)
self.files[path] = content
return {"ok": True, "output": f"wrote {path}"}
_DOCKER_PATH = f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}"
_DOCKER_NOBLE = "http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease"
class TestTheMixin:
def test_reads_the_host(self):
found = _Host({_DOCKER_PATH: _Z2M_DOCKER}).get_apt_source_releases()
assert found is not None
assert found["codename"] == "noble"
assert found["sources"][0]["suites"] == ["focal"]
def test_none_without_apt(self):
assert _Host({}, apt=False).get_apt_source_releases() is None
def test_moves_the_file_once_the_vendor_serves_the_release(self):
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
assert result["success"], result["output"]
assert host.files[_DOCKER_PATH] == (
"deb http://HTTPS///download.docker.com/linux/ubuntu noble stable\n"
)
assert host.fetched == [_DOCKER_NOBLE]
assert result["rewritten"] == 1
assert [s["step"] for s in result["steps"]] == ["read", "check", "rewrite"]
def test_the_release_file_when_there_is_no_inrelease(self):
release = _DOCKER_NOBLE.replace("InRelease", "Release")
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={release})
assert host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")["success"]
assert host.fetched == [_DOCKER_NOBLE, release]
def test_refuses_where_the_vendor_has_no_such_release(self):
host = _Host({_DOCKER_PATH: _Z2M_DOCKER})
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
assert not result["success"]
assert host.written == []
assert "noble" in result["output"] and "404" in result["output"]
assert result["rewritten"] == 0
def test_one_suite_missing_leaves_the_whole_file(self):
text = "deb http://a.example/u focal main\ndeb http://a.example/u focal-security main\n"
served = {"http://a.example/u/dists/noble/InRelease"}
host = _Host({"/etc/apt/sources.list": text}, served=served)
result = host.set_apt_source_release("/etc/apt/sources.list", "focal", "noble")
assert not result["success"]
assert host.files["/etc/apt/sources.list"] == text
def test_a_file_apt_does_not_read_is_never_written(self):
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
result = host.set_apt_source_release("/etc/shadow", "focal", "noble")
assert not result["success"]
assert host.written == []
def test_nothing_left_on_that_release(self):
host = _Host({_DOCKER_PATH: _Z2M_DOCKER.replace("focal", "noble")})
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
assert result["success"]
assert result["rewritten"] == 0
assert host.written == [] and host.fetched == []
def test_a_deb822_file(self):
path = "/etc/apt/sources.list.d/docker.sources"
text = "Types: deb\nURIs: https://download.docker.com/linux/ubuntu\nSuites: focal\nComponents: stable\n"
served = {"https://download.docker.com/linux/ubuntu/dists/noble/InRelease"}
host = _Host({path: text}, served=served)
assert host.set_apt_source_release(path, "focal", "noble")["success"]
assert "Suites: noble\n" in host.files[path]
def test_not_an_apt_host(self):
result = _Host({}, apt=False).set_apt_source_release(_DOCKER_PATH, "focal", "noble")
assert not result["success"]
def test_a_name_that_is_no_codename_changes_nothing(self):
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble; rm -rf /")
assert not result["success"]
assert host.written == [] and host.fetched == []
def test_every_apt_proxy_host_reads_its_releases():
# Every driver with apt's proxy is an apt host: it reads its sources' releases too.
assert issubclass(AptProxyMixin, AptReleaseMixin)
+318
View File
@@ -0,0 +1,318 @@
"""https sources through the cache: which can be, and the root commands that turn them.
apt-cacher-ng fetches ``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>``
and stores it, where it allows port 443 (``AllowUserPorts=80 443``); a source
fetched over https itself goes through the cache's tunnel and is never stored
(NetOrk/netork#2555). The commands run here for real, on files in a temporary
directory that ``apt-config -o`` points apt's own settings at.
"""
from __future__ import annotations
import shutil
import subprocess
import pytest
from napalm_device_types.apt_proxy import parse_apt_proxy
from napalm_device_types.apt_sources import (
VIA_CACHE,
cache_form,
cacheable_https_hosts,
is_host_name,
revert_sources_command,
rewrite_sources_command,
)
def _report(**over):
report = {
"proxy": "http://c:3142",
"per_host": {},
"auto_detect": None,
"settings": [],
"https_proxy": "http://c:3142",
"https_per_host": {},
"https_settings": [],
"http_sources": ["http://deb.debian.org/debian/dists/trixie/"],
"https_sources": [],
"not_cached": [],
}
report.update(over)
return report
class TestCacheForm:
def test_an_https_url(self):
assert cache_form("https://download.docker.com/linux/ubuntu/dists/noble/") == (
"http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/"
)
def test_the_prefix(self):
assert VIA_CACHE == "http://HTTPS///"
def test_not_an_https_url(self):
with pytest.raises(ValueError):
cache_form("http://deb.debian.org/debian/")
class TestCacheableHttpsHosts:
def test_each_host_once_sorted(self):
report = _report(
https_sources=[
"https://packages.wazuh.com/4.x/apt/dists/stable/",
"https://download.docker.com/linux/ubuntu/dists/noble/",
"https://download.docker.com/linux/ubuntu/dists/noble-test/",
]
)
assert cacheable_https_hosts(report) == ["download.docker.com", "packages.wazuh.com"]
def test_none_without_an_http_proxy(self):
# The cache form goes through the http proxy: without one it goes nowhere.
report = _report(proxy=None, https_sources=["https://h.example/dists/x/"])
assert cacheable_https_hosts(report) == []
def test_a_host_apt_fetches_direct_stays(self):
report = _report(
per_host={"a.example": "DIRECT"},
https_per_host={"B.example": "direct"},
https_sources=[
"https://a.example/dists/x/",
"https://b.example/dists/x/",
"https://c.example/dists/x/",
],
)
assert cacheable_https_hosts(report) == ["c.example"]
def test_a_host_with_credentials_stays(self):
report = _report(
https_sources=["https://enterprise.proxmox.com/debian/pve/dists/trixie/"],
not_cached=["enterprise.proxmox.com"],
)
assert cacheable_https_hosts(report) == []
def test_another_port_or_a_login_in_the_url_stays(self):
# The cache allows port 443 alone, and a login in the URL is a credential too.
report = _report(
https_sources=[
"https://h.example:8443/dists/x/",
"https://u:p@i.example/dists/x/",
"https://j.example:443/dists/x/",
]
)
assert cacheable_https_hosts(report) == ["j.example"]
def test_a_report_from_before_not_cached(self):
report = _report(https_sources=["https://h.example/dists/x/"])
del report["not_cached"]
assert cacheable_https_hosts(report) == ["h.example"]
class TestNotCachedInTheReport:
def test_the_dump_names_them(self):
output = "\n".join(
[
"APROXY_BEGIN",
"[dump]",
'Acquire::http::Proxy "http://c:3142";',
'Package-Proxy::Not-Cached "";',
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";',
'Package-Proxy::Not-Cached:: "esm.ubuntu.com";',
"[sources]",
"APROXY_END",
]
)
assert parse_apt_proxy(output)["not_cached"] == [
"enterprise.proxmox.com",
"esm.ubuntu.com",
]
def test_none(self):
assert parse_apt_proxy("APROXY_BEGIN\n[dump]\nAPROXY_END")["not_cached"] == []
def test_the_hosts_are_checked():
with pytest.raises(ValueError):
rewrite_sources_command(["h.example; rm -rf /"], "/etc/apt/apt.conf.d/00proxy")
_APT_CONFIG = shutil.which("apt-config")
needs_apt = pytest.mark.skipif(_APT_CONFIG is None, reason="apt-config is not on this host")
@pytest.fixture
def etc(tmp_path):
"""apt's sources, its credentials and the proxy file, under *tmp_path*."""
(tmp_path / "sources.list.d").mkdir()
(tmp_path / "auth.conf.d").mkdir()
(tmp_path / "00proxy").write_text('Acquire::http::Proxy "http://c:3142";\n')
return tmp_path
def _apt_config(etc):
return (
f"apt-config -o Dir::Etc::sourcelist={etc}/sources.list "
f"-o Dir::Etc::sourceparts={etc}/sources.list.d/ "
f"-o Dir::Etc::netrc={etc}/auth.conf -o Dir::Etc::netrcparts={etc}/auth.conf.d/"
)
def _run(command):
done = subprocess.run(["sh", "-c", command], capture_output=True, text=True, check=False)
assert done.returncode == 0, done.stderr
return done.stdout
_DOCKER_LIST = (
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
"https://download.docker.com/linux/ubuntu noble stable\n"
)
_DOCKER_SOURCES = (
"Types: deb\n"
"URIs: https://download.docker.com/linux/ubuntu\n"
"Suites: noble\n"
"Components: stable\n"
"Signed-By: /etc/apt/keyrings/docker.asc\n"
)
@needs_apt
class TestRewrite:
def test_a_one_line_source(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list.d" / "docker.list").read_text() == (
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
"http://HTTPS///download.docker.com/linux/ubuntu noble stable\n"
)
def test_a_deb822_source(self, etc):
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list.d" / "docker.sources").read_text() == (
_DOCKER_SOURCES.replace("https://", "http://HTTPS///")
)
def test_sources_list_itself_and_only_the_hosts_named(self, etc):
(etc / "sources.list").write_text(
"deb http://deb.debian.org/debian trixie main\n"
"deb https://a.example/debian trixie main\n"
"deb-src https://a.example/debian trixie main\n"
"deb https://b.example/debian trixie main\n"
"# deb https://a.example/old trixie main\n"
)
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list").read_text() == (
"deb http://deb.debian.org/debian trixie main\n"
"deb http://HTTPS///a.example/debian trixie main\n"
"deb-src http://HTTPS///a.example/debian trixie main\n"
"deb https://b.example/debian trixie main\n"
"# deb https://a.example/old trixie main\n"
)
def test_a_longer_name_or_another_port_stays(self, etc):
text = (
"deb https://a.example.org/debian trixie main\n"
"deb https://a.example:8443/debian trixie main\n"
)
(etc / "sources.list").write_text(text)
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list").read_text() == text
def test_a_host_with_credentials_stays_and_is_named(self, etc):
(etc / "sources.list.d" / "pve.sources").write_text(
"Types: deb\nURIs: https://enterprise.proxmox.com/debian/pve\nSuites: trixie\n"
)
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
(etc / "auth.conf.d" / "pve.conf").write_text(
"machine https://Enterprise.Proxmox.com/debian/pve\nlogin key\npassword secret\n"
)
_run(
rewrite_sources_command(
["enterprise.proxmox.com", "download.docker.com"],
f"{etc}/00proxy",
_apt_config(etc),
)
)
assert "https://enterprise.proxmox.com" in (etc / "sources.list.d/pve.sources").read_text()
assert "HTTPS///download.docker.com" in (etc / "sources.list.d/docker.list").read_text()
assert (etc / "00proxy").read_text() == (
'Acquire::http::Proxy "http://c:3142";\n'
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";\n'
)
assert "secret" not in (etc / "00proxy").read_text()
def test_credentials_in_auth_conf_on_one_line(self, etc):
(etc / "sources.list").write_text("deb https://esm.ubuntu.com/apps/ubuntu noble main\n")
(etc / "auth.conf").write_text("machine esm.ubuntu.com/apps/ubuntu/ login a password b\n")
_run(rewrite_sources_command(["esm.ubuntu.com"], f"{etc}/00proxy", _apt_config(etc)))
assert "https://esm.ubuntu.com" in (etc / "sources.list").read_text()
assert "esm.ubuntu.com" in (etc / "00proxy").read_text()
def test_a_file_without_the_host_is_left_untouched(self, etc):
other = etc / "sources.list.d" / "other.list"
other.write_text("deb http://deb.debian.org/debian trixie main\n")
before = other.stat().st_ino # sed -i writes a new file; grep first leaves it be
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert other.stat().st_ino == before
def test_twice_is_once(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
command = rewrite_sources_command(
["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)
)
_run(command)
_run(command)
assert (etc / "sources.list.d" / "docker.list").read_text().count("HTTPS///") == 1
@needs_apt
class TestRevert:
def test_every_source_goes_back_to_https(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
_run(revert_sources_command(_apt_config(etc)))
assert (etc / "sources.list.d" / "docker.list").read_text() == _DOCKER_LIST
assert (etc / "sources.list.d" / "docker.sources").read_text() == _DOCKER_SOURCES
def test_nothing_to_revert(self, etc):
(etc / "sources.list").write_text("deb http://deb.debian.org/debian trixie main\n")
_run(revert_sources_command(_apt_config(etc)))
assert (
etc / "sources.list"
).read_text() == "deb http://deb.debian.org/debian trixie main\n"
def test_the_commands_are_valid_posix_sh():
for command in (
rewrite_sources_command(["a.example"], "/etc/apt/apt.conf.d/00proxy"),
revert_sources_command(),
):
assert subprocess.run(["sh", "-n", "-c", command], check=False).returncode == 0
def test_a_host_name():
assert is_host_name("download.docker.com")
assert not is_host_name('h"; Acquire::x "y')
assert not is_host_name("-h.example")
+172
View File
@@ -0,0 +1,172 @@
"""ConfigRevisionsMixin: a device's own configuration history (NetOrk/netork#2511).
Some devices keep their earlier configurations themselves. An application restores a
configuration there by reverting to the revision that holds it -- on OPNsense the only
way, since no API takes an uploaded config.xml. Listing and reading are declared only,
so ``callable(getattr(driver, "get_config_revisions", None))`` is the truthful answer;
``revert_config_revision`` is a template: it refuses an id that is not in the history,
lets the driver revert, and checks that the configuration in place now is the
revision's.
"""
from __future__ import annotations
import inspect
from typing import get_type_hints
import pytest
from napalm_device_types import (
ConfigRevertFailed,
ConfigRevisionDict,
ConfigRevisionsMixin,
DeviceTypeDriver,
FirewallDriver,
OSDriver,
SwitchDriver,
UnknownConfigRevision,
)
def test_the_shape():
assert get_type_hints(ConfigRevisionDict) == {
"id": str,
"time": str | None,
"description": str | None,
"user": str | None,
"size": int | None,
}
def test_no_role_demands_it():
for role in (FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
assert not issubclass(role, ConfigRevisionsMixin)
def test_listing_and_reading_are_declared_not_implemented():
assert not hasattr(ConfigRevisionsMixin, "get_config_revisions")
assert not hasattr(ConfigRevisionsMixin, "get_config_revision")
source = inspect.getsource(ConfigRevisionsMixin)
assert "def get_config_revisions(self) -> list[ConfigRevisionDict]" in source
assert "def get_config_revision(self, revision_id: str) -> str" in source
assert "def _revert_to_config_revision(self, revision_id: str) -> None" in source
def test_an_unknown_revision_is_a_value_error_that_names_it():
error = UnknownConfigRevision("r9")
assert isinstance(error, ValueError)
assert "'r9'" in str(error)
def test_a_failed_revert_is_a_runtime_error():
assert issubclass(ConfigRevertFailed, RuntimeError)
def _revision(revision_id: str, time: str) -> ConfigRevisionDict:
return {"id": revision_id, "time": time, "description": None, "user": None, "size": None}
class _Device(ConfigRevisionsMixin):
"""A device that keeps its history in memory, newest first, and saves a revert as a
new newest revision with a stamp of its own on the first line."""
def __init__(self) -> None:
self.history = [
(_revision("r3", "2026-11-13T21:00:00+00:00"), "#saved 3\nb\n"),
(_revision("r2", "2026-11-12T21:00:00+00:00"), "#saved 2\na\n"),
(_revision("r1", "2026-11-11T21:00:00+00:00"), "#saved 1\na\n"),
]
self.calls: list[str] = []
def get_config_revisions(self) -> list[ConfigRevisionDict]:
self.calls.append("list")
return [revision for revision, _ in self.history]
def get_config_revision(self, revision_id: str) -> str:
self.calls.append(f"read {revision_id}")
for revision, text in self.history:
if revision["id"] == revision_id:
return text
raise UnknownConfigRevision(revision_id)
def _revert_to_config_revision(self, revision_id: str) -> None:
self.calls.append(f"revert {revision_id}")
body = self.get_config_revision(revision_id).split("\n", 1)[1]
self.history.insert(0, (_revision("r4", "2026-11-14T08:00:00+00:00"), f"#saved 4\n{body}"))
def config_revision_content(self, text: str) -> str:
return text.split("\n", 1)[1]
class TestRevertConfigRevision:
def test_reverts_and_answers_with_the_revision_now_in_place(self):
device = _Device()
now = device.revert_config_revision("r1")
assert now["id"] == "r4"
assert device.calls == ["list", "read r1", "revert r1", "read r1", "list", "read r4"]
def test_the_revision_is_read_before_the_revert_which_may_rotate_it_out(self):
class Rotating(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
super()._revert_to_config_revision(revision_id)
self.history.pop() # the oldest, r1, drops out
assert Rotating().revert_config_revision("r1")["id"] == "r4"
def test_a_revision_not_in_the_history_is_refused_before_anything_changes(self):
device = _Device()
with pytest.raises(UnknownConfigRevision, match="r9"):
device.revert_config_revision("r9")
assert device.calls == ["list"]
def test_a_revert_the_device_did_not_take_fails(self):
# OPNsense answers "reverted" even when it fell back to the previous file.
class Deaf(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
self.calls.append(f"revert {revision_id}")
with pytest.raises(ConfigRevertFailed, match="r1"):
Deaf().revert_config_revision("r1")
def test_a_history_that_is_empty_afterwards_fails(self):
class Forgetful(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
self.history.clear()
with pytest.raises(ConfigRevertFailed):
Forgetful().revert_config_revision("r1")
def test_what_a_save_stamps_anew_does_not_count(self):
# r2 and r1 differ only in their stamp: reverting to r2 leaves r1's content.
device = _Device()
assert device.revert_config_revision("r2")["id"] == "r4"
def test_a_refusal_of_the_device_reaches_the_caller(self):
class Refusing(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
raise ConfigRevertFailed("read-only user")
with pytest.raises(ConfigRevertFailed, match="read-only"):
Refusing().revert_config_revision("r1")
class TestConfigRevisionContent:
def test_by_default_the_whole_text(self):
class Plain(ConfigRevisionsMixin):
pass
assert Plain().config_revision_content("a\nb\n") == "a\nb\n"
def test_without_a_stamp_every_save_differs(self):
class Plain(_Device):
def config_revision_content(self, text: str) -> str:
return ConfigRevisionsMixin.config_revision_content(self, text)
with pytest.raises(ConfigRevertFailed):
Plain().revert_config_revision("r1")
+73 -9
View File
@@ -19,30 +19,40 @@ from napalm_device_types.container_engine import (
ContainerEngineUnavailable,
)
#: What each engine's CLI says to ``--version``.
VERSIONS = {
"docker": "Docker version 29.0.1, build 1a2b3c4",
"podman": "podman version 5.2.2",
}
class FakeDriver(ContainerEngineMixin):
"""A driver with a channel: records what it is asked to run."""
"""A driver with a channel: records what it is asked to run.
def __init__(self, present=("docker",), binary=None):
*present* are the engines whose CLI is installed; *says* overrides what an
engine's CLI answers to ``--version`` (podman-docker's ``docker`` is podman).
"""
def __init__(self, present=("docker",), binary=None, says=None):
self.present = set(present)
self.binary = binary
self.says = dict(says or {})
self.commands = []
self.streams = []
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
self.commands.append((command, privileged, timeout, stdin))
found = any(command == f"command -v {b}" for b in self._binaries())
return CommandResult(
stdout="/usr/bin/x\n" if found else "", stderr="", exit_code=0 if found else 1
)
for engine in self.present:
b = self._container_engine_binary(engine)
if command == f"command -v {b} >/dev/null 2>&1 && {b} --version":
answer = self.says.get(engine, VERSIONS[engine])
return CommandResult(stdout=answer + "\n", stderr="", exit_code=0)
return CommandResult(stdout="", stderr="", exit_code=1)
def open_stream(self, command, *, privileged=False):
self.streams.append((command, privileged))
return object()
def _binaries(self):
return {self._container_engine_binary(e) for e in self.present}
def _container_engine_binary(self, engine):
return self.binary or super()._container_engine_binary(engine)
@@ -123,6 +133,60 @@ def test_an_unknown_engine_is_refused():
FakeDriver().open_container_engine("rkt")
class TestPodman:
"""Podman, reached the way Docker is (NetOrk/netork#818).
Its API is Podman's own service, which also speaks Docker's: netOrk reads the
host's *rootful* Podman, the engine of the system's containers (Quadlet units
in ``/etc/containers/systemd``, ``podman.socket``). A login user's rootless
containers are that user's own and invisible to anybody else, so the
connection to Podman runs as root, the way the driver gains root for any command.
"""
def test_a_host_with_podman_lists_it_with_its_api(self):
driver = FakeDriver(present=("podman",))
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
def test_a_host_with_both_lists_both(self):
driver = FakeDriver(present=("docker", "podman"))
assert driver.container_engines() == [
{"engine": "docker", "api": "docker-engine"},
{"engine": "podman", "api": "podman-libpod"},
]
def test_podman_docker_is_not_docker(self):
"""podman-docker installs a ``docker`` that runs podman: listing it as
Docker would read the login user's rootless Podman a second time."""
driver = FakeDriver(present=("docker", "podman"), says={"docker": VERSIONS["podman"]})
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
def test_the_api_stream_runs_as_root(self):
driver = FakeDriver(present=("podman",))
driver.open_container_engine("podman").open_api()
assert driver.streams == [("podman system dial-stdio", True)]
def test_the_cli_runs_as_root_too(self):
driver = FakeDriver(present=("podman",))
conn = driver.open_container_engine("podman")
conn.run_cli(["pull", "nginx"])
conn.stream_cli(["pull", "nginx"])
assert driver.commands[-1][:2] == ("podman pull nginx", True)
assert driver.streams == [("podman pull nginx", True)]
def test_the_connection_says_it_runs_as_root(self):
driver = FakeDriver(present=("docker", "podman"))
assert driver.open_container_engine("podman").privileged is True
assert driver.open_container_engine("docker").privileged is False
def test_the_connection_names_its_engine():
conn = FakeDriver().open_container_engine("docker")
+240 -1
View File
@@ -14,7 +14,7 @@ from typing import Any
import pytest
from napalm_device_types import DhcpServerMixin, FirewallDriver, ResidentialGatewayDriver
from napalm_device_types.dhcp import normalize_mac
from napalm_device_types.dhcp import DhcpApplyError, normalize_mac
from napalm_device_types.models import DhcpReservationDict
@@ -209,6 +209,245 @@ class TestApplyReservationSet:
assert lines == ["[commit] no changes"]
class _RefusingDhcpServer(_FakeDhcpServer):
"""A server that refuses the reservations for the IPs in *refused*, as Kea does
for one whose subnet it does not manage."""
def __init__(self, live: list[DhcpReservationDict] | None = None, *, refused: set[str]) -> None:
super().__init__(live)
self.refused = refused
def apply_dhcp_reservation(
self, reservation: DhcpReservationDict, *, uuid: str | None = None
) -> dict[str, Any]:
if reservation["ip"] in self.refused:
raise ValueError(f"No Kea-managed subnet contains {reservation['ip']}")
return super().apply_dhcp_reservation(reservation, uuid=uuid)
def _three() -> list[DhcpReservationDict]:
return [
_reservation(mac="aa:bb:cc:dd:ee:01", ip="10.10.20.50"),
_reservation(mac="aa:bb:cc:dd:ee:02", ip="10.10.99.5", subnet=""),
_reservation(mac="aa:bb:cc:dd:ee:03", ip="10.10.20.52"),
]
class TestApplyWithARefusedReservation:
"""One reservation the server refuses must not hold back the others, and what was
written must take effect: a server that saved a write and was never committed shows
it as configured, so the next apply finds nothing to do and never commits it."""
def test_the_others_are_still_written(self):
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
with pytest.raises(DhcpApplyError):
list(driver.apply_dhcp_reservationset(_three()))
assert [entry["reservation"]["ip"] for entry in driver.applied] == [
"10.10.20.50",
"10.10.20.52",
]
def test_what_was_written_is_committed(self):
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
with pytest.raises(DhcpApplyError):
list(driver.apply_dhcp_reservationset(_three()))
assert driver.commits == 1
def test_a_refused_set_still_commits(self):
# An earlier apply that stopped before its commit left saved writes behind; a
# run whose every write is refused commits too, so those take effect.
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
with pytest.raises(DhcpApplyError):
list(driver.apply_dhcp_reservationset([_three()[1]]))
assert driver.commits == 1
def test_progress_names_the_refused_one_and_why(self):
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
lines: list[str] = []
with pytest.raises(DhcpApplyError) as raised:
for line in driver.apply_dhcp_reservationset(_three()):
lines.append(line)
failed = [line for line in lines if line.startswith("[failed]")]
assert failed == [
"[failed] aa:bb:cc:dd:ee:02 -> 10.10.99.5: No Kea-managed subnet contains 10.10.99.5"
]
assert lines[-1] == "[commit] applied 2 add(s), 0 update(s)"
assert "1 of 3" in str(raised.value)
def test_a_refused_update_is_reported_like_an_add(self):
driver = _RefusingDhcpServer(
[_reservation(uuid="u1", ip="10.10.20.9")], refused={"10.10.20.50"}
)
with pytest.raises(DhcpApplyError, match="1 of 1"):
list(driver.apply_dhcp_reservationset([_reservation()]))
assert driver.applied == []
assert driver.commits == 1
def test_the_error_is_a_runtime_error(self):
# Callers that catch RuntimeError for a failed apply keep working.
assert issubclass(DhcpApplyError, RuntimeError)
#: What Kea answers through OPNsense's API for an address another reservation holds.
_KEA_DUPLICATE = (
"{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}"
)
class _KeaLikeServer(_FakeDhcpServer):
"""A server that keeps each address unique at every write, as Kea does: a write
taking an address another reservation still holds is refused."""
def apply_dhcp_reservation(
self, reservation: DhcpReservationDict, *, uuid: str | None = None
) -> dict[str, Any]:
if any(r["ip"] == reservation["ip"] and r["uuid"] != uuid for r in self.live):
raise RuntimeError(_KEA_DUPLICATE)
stored = {**reservation, "uuid": uuid or f"new-{len(self.live)}"}
self.live = [r for r in self.live if r["uuid"] != uuid] + [stored] # type: ignore[list-item]
return super().apply_dhcp_reservation(reservation, uuid=uuid)
def ip_of(self, mac: str) -> str:
return next(r["ip"] for r in self.live if r["mac"] == mac)
_A, _B, _C, _N = (f"02:00:00:00:00:0{i}" for i in "abcd")
def _at(mac: str, ip: str, uuid: str = "") -> DhcpReservationDict:
return _reservation(mac=mac, ip=ip, uuid=uuid, subnet="192.0.2.0/24")
class TestApplyOrderReleasesBeforeTaking:
"""An address is released before another change takes it (#106)."""
def test_an_update_that_frees_an_address_goes_first(self):
# The issue: B takes A's address, A moves on; desired lists B first.
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")])
lines = list(
driver.apply_dhcp_reservationset([_at(_B, "192.0.2.10"), _at(_A, "192.0.2.20")])
)
assert driver.ip_of(_A) == "192.0.2.20"
assert driver.ip_of(_B) == "192.0.2.10"
assert lines == [
f"[update] {_A} -> 192.0.2.20 (ip)",
f"[update] {_B} -> 192.0.2.10 (ip)",
"[commit] applied 0 add(s), 2 update(s)",
]
def test_a_chain_is_written_from_its_free_end(self):
live = [
_at(_A, "192.0.2.10", "ua"),
_at(_B, "192.0.2.11", "ub"),
_at(_C, "192.0.2.12", "uc"),
]
driver = _KeaLikeServer(live)
list(
driver.apply_dhcp_reservationset(
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.13")]
)
)
assert [e["reservation"]["mac"] for e in driver.applied] == [_C, _B, _A]
assert [driver.ip_of(m) for m in (_A, _B, _C)] == ["192.0.2.11", "192.0.2.12", "192.0.2.13"]
def test_an_add_comes_after_the_update_that_frees_its_address(self):
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
lines = list(
driver.apply_dhcp_reservationset([_at(_N, "192.0.2.10"), _at(_A, "192.0.2.20")])
)
assert driver.ip_of(_N) == "192.0.2.10"
assert driver.ip_of(_A) == "192.0.2.20"
assert lines[-1] == "[commit] applied 1 add(s), 1 update(s)"
def test_addresses_match_however_they_are_spelled(self):
driver = _FakeDhcpServer([_at(_A, "2001:DB8::10", "ua"), _at(_B, "2001:DB8::11", "ub")])
list(driver.apply_dhcp_reservationset([_at(_B, "2001:db8::10"), _at(_A, "2001:db8::20")]))
assert [e["reservation"]["mac"] for e in driver.applied] == [_A, _B]
def test_an_update_that_keeps_its_address_frees_nothing(self):
# A hostname change is no release: the add for A's address stays refused.
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
with pytest.raises(DhcpApplyError, match="1 of 2"):
list(
driver.apply_dhcp_reservationset(
[_at(_N, "192.0.2.10"), {**_at(_A, "192.0.2.10"), "hostname": "renamed"}]
)
)
assert driver.ip_of(_A) == "192.0.2.10"
class TestApplyOrderWithASwap:
"""Two reservations trading addresses: no order frees one first. Each is still
tried -- a server that checks only at the commit takes them -- and a refusal says it
was a swap. Nothing is moved to a temporary address: the contract has no way to know
a free one, and a second write that failed would leave the reservation there."""
def _swap(self) -> list[DhcpReservationDict]:
return [_at(_A, "192.0.2.11"), _at(_B, "192.0.2.10"), _at(_N, "192.0.2.30")]
def _live(self) -> list[DhcpReservationDict]:
return [_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")]
def test_a_refused_swap_is_reported_as_one_and_the_rest_is_written(self):
driver = _KeaLikeServer(self._live())
lines: list[str] = []
with pytest.raises(DhcpApplyError, match="2 of 3"):
for line in driver.apply_dhcp_reservationset(self._swap()):
lines.append(line)
failed = [line for line in lines if line.startswith("[failed]")]
assert len(failed) == 2
assert all("Duplicate entry exists" in line for line in failed)
assert f"swap with {_B}" in failed[0] and f"swap with {_A}" in failed[1]
assert driver.ip_of(_A) == "192.0.2.10" and driver.ip_of(_B) == "192.0.2.11"
assert driver.ip_of(_N) == "192.0.2.30"
assert driver.commits == 1
def test_a_server_that_takes_the_swap_gets_it(self):
driver = _FakeDhcpServer(self._live())
lines = list(driver.apply_dhcp_reservationset(self._swap()))
assert len(driver.applied) == 3
assert not any("swap" in line for line in lines)
def test_a_rotation_of_three_is_a_swap_too(self):
driver = _KeaLikeServer([*self._live(), _at(_C, "192.0.2.12", "uc")])
lines: list[str] = []
with pytest.raises(DhcpApplyError, match="3 of 3"):
for line in driver.apply_dhcp_reservationset(
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.10")]
):
lines.append(line)
by_mac = {line.split()[1]: line for line in lines if line.startswith("[failed]")}
assert f"swap with {_B}, {_C}" in by_mac[_A]
assert f"swap with {_A}, {_C}" in by_mac[_B]
assert f"swap with {_A}, {_B}" in by_mac[_C]
class TestKeysThatCannotNameOneReservation:
def test_a_live_reservation_without_mac_is_never_matched(self):
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
+90
View File
@@ -16,6 +16,7 @@ from typing import Any
import pytest
from napalm_device_types import DhcpServerMixin
from napalm_device_types.dhcp import DhcpApplyError
from napalm_device_types.models import DhcpSubnetDict
@@ -235,3 +236,92 @@ def test_calling_a_missing_device_method_fails_loudly() -> None:
def test_a_subnet_that_cannot_name_one_raises(desired):
with pytest.raises(ValueError):
_FakeDhcpServer([]).diff_dhcp_subnets(desired)
# ── apply: a refused subnet does not hold back the others (#113) ──────────────
class _RefusingDhcpServer(_FakeDhcpServer):
"""A server that refuses the subnets in *refused*, as Kea refuses one that
overlaps a subnet it already serves."""
def __init__(self, live: list[DhcpSubnetDict] | None = None, *, refused: set[str]) -> None:
super().__init__(live)
self.refused = refused
def apply_dhcp_subnet(
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
) -> dict[str, Any]:
if subnet["subnet"] in self.refused:
raise RuntimeError(f"{subnet['subnet']} overlaps a subnet the server has")
return super().apply_dhcp_subnet(subnet, uuid=uuid)
def _three_subnets() -> list[DhcpSubnetDict]:
return [
_subnet(subnet="192.0.2.0/26"),
_subnet(subnet="192.0.2.64/26"),
_subnet(subnet="192.0.2.128/26"),
]
def test_a_refused_subnet_does_not_stop_the_others() -> None:
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset(_three_subnets()))
assert [e["subnet"]["subnet"] for e in fake.applied] == ["192.0.2.0/26", "192.0.2.128/26"]
def test_what_was_written_is_committed() -> None:
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset(_three_subnets()))
assert fake.commits == 1
def test_a_set_whose_every_write_is_refused_still_commits() -> None:
# A run that stopped before its commit left saved writes behind, which the device
# reports as configured: no later diff would commit them.
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset([_three_subnets()[1]]))
assert fake.commits == 1
def test_progress_names_the_refused_subnet_and_why() -> None:
fake = _RefusingDhcpServer(
[_subnet(uuid="dev-1", subnet="192.0.2.128/26")], refused={"192.0.2.64/26"}
)
lines: list[str] = []
with pytest.raises(DhcpApplyError) as raised:
for line in fake.apply_dhcp_subnetset(
[*_three_subnets()[:2], _subnet(subnet="192.0.2.128/26", description="Office")]
):
lines.append(line)
assert lines == [
"[add] 192.0.2.0/26",
"[failed] 192.0.2.64/26: 192.0.2.64/26 overlaps a subnet the server has",
"[update] 192.0.2.128/26 (description)",
"[commit] applied 1 add(s), 1 update(s)",
]
assert (raised.value.failed, raised.value.attempted) == (1, 3)
assert str(raised.value) == "1 of 3 DHCP subnet change(s) were refused"
def test_a_refused_update_is_counted_like_an_add() -> None:
live = _subnet(uuid="dev-1", subnet="192.0.2.0/24")
fake = _RefusingDhcpServer([live], refused={"192.0.2.0/24"})
with pytest.raises(DhcpApplyError, match="1 of 1"):
list(fake.apply_dhcp_subnetset([_subnet(subnet="192.0.2.0/24", description="Office")]))
assert fake.applied == []
assert fake.commits == 1
+19 -1
View File
@@ -56,8 +56,26 @@ def test_a_firmware_offer_says_what_runs_and_what_is_offered():
from napalm_device_types import FirmwareOfferDict, FirmwareOfferMixin
hints = get_type_hints(FirmwareOfferDict)
assert set(hints) == {"current_version", "latest_version", "newer", "download_url"}
assert set(hints) == {
"current_version",
"latest_version",
"newer",
"download_url",
"installable",
"latest_major_version",
}
assert FirmwareOfferDict.__required_keys__ == frozenset(
{"current_version", "latest_version", "newer"}
)
assert not hasattr(FirmwareOfferMixin, "get_firmware_offer")
def test_a_firmware_upgrade_is_started_and_followed_like_a_device_run_update():
"""A driver that can install the firmware it offers starts it and the device
runs it by itself, restarting at its end (OpenWrt's sysupgrade, NetOrk/netork#2615);
the caller follows it through ``get_update_progress``. Declared only, so
``hasattr`` tells a caller whether the driver offers it."""
from napalm_device_types import FirmwareUpgradeMixin, UpdateMixin
assert not hasattr(FirmwareUpgradeMixin, "start_firmware_upgrade")
assert not hasattr(UpdateMixin, "get_update_progress")
+1
View File
@@ -259,6 +259,7 @@ class TestLldpAgent:
"ensure_package",
"configure_package_proxy",
"grant_container_engine_access",
"configure_zabbix_agent",
],
)
def test_no_role_base_offers_a_fix(base, method):
+60 -2
View File
@@ -154,6 +154,24 @@ class TestKernelRebootPending:
"5.14.0-503.11.1.el9_5.x86_64",
),
("6.9.7-200.fc40.x86_64", ["6.9.7-200.fc40.x86_64"], None),
# Oracle UEK: 112.5.1 is newer than 112.5, as rpm counts -- a number after
# 5 beats the letters ``el`` (#109, build-arm-01).
(
"6.12.0-208.112.5.1.el9uek.aarch64",
[
"6.12.0-207.111.5.6.el9uek.aarch64",
"6.12.0-208.112.5.el9uek.aarch64",
"6.12.0-208.112.5.1.el9uek.aarch64",
],
None,
),
(
"6.12.0-208.112.5.el9uek.aarch64",
["6.12.0-208.112.5.el9uek.aarch64", "6.12.0-208.112.5.1.el9uek.aarch64"],
"6.12.0-208.112.5.1.el9uek.aarch64",
),
# A release candidate (``~``) comes before its release, in rpm and in dpkg.
("6.13~rc5-1-amd64", ["6.13~rc5-1-amd64", "6.13-1-amd64"], "6.13-1-amd64"),
],
)
def test_the_newer_kernel_of_the_running_flavour(self, running, installed, newer):
@@ -598,8 +616,8 @@ class TestInterruptedRpm:
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
1 or ERROR 2), yum's ``transaction-all.*`` journal that
``yum-complete-transaction`` reads -- and still have to be checked against a
real Oracle Linux 9 host.
``yum-complete-transaction`` reads. Checked against Oracle Linux 9 (build-arm-01,
2026-11-12): its UEK kernels are installonly only by libdnf's built-in defaults.
"""
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
@@ -804,6 +822,29 @@ case "$*" in
esac
"""
#: An rpm on an Oracle Linux host: its UEK kernel ``vendor-kernel`` provides nothing
#: of dnf's upstream defaults and sits in the database three times, like glibc twice.
_RPM_VENDOR_KERNEL_STUB = """#!/bin/sh
case "$*" in
*--whatprovides*) for a; do [ "$a" = vendor-kernel ] && echo vendor-kernel; done
echo 'no package provides kernel-PAE' ;;
-qa*) printf '%s\\n' 'vendor-kernel aarch64' 'vendor-kernel aarch64' 'vendor-kernel aarch64' \\
'glibc aarch64' 'glibc aarch64' ;;
esac
"""
#: libdnf's Python binding, as far as the command asks it: the installonlypkgs a
#: distribution built into its libdnf -- here one more than upstream's.
_LIBDNF_CONF = """class _Option:
def getValueString(self):
return "kernel, installonlypkg(kernel), vendor-kernel"
class ConfigMain:
def installonlypkgs(self):
return _Option()
"""
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
_APK_DB = """C:Q1abc=
P:alpine-baselayout
@@ -886,6 +927,23 @@ class TestThePackageManagerCommandOnARealShell:
assert status["package_manager"] == "dnf"
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_the_installonly_packages_libdnf_was_built_with(self, tmp_path, shell):
"""Oracle builds kernel-uek into libdnf's defaults; its UEK packages provide
nothing of upstream's, so only libdnf itself tells (#107)."""
_skip_busybox_rpm(shell)
env = _stubs(tmp_path, "rpm", "dnf")
(tmp_path / "bin" / "rpm").write_text(_RPM_VENDOR_KERNEL_STUB)
libdnf = tmp_path / "py" / "libdnf"
libdnf.mkdir(parents=True)
(libdnf / "__init__.py").write_text("")
(libdnf / "conf.py").write_text(_LIBDNF_CONF)
env["PYTHONPATH"] = str(tmp_path / "py")
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
status = parse_host_status(out)
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
_skip_busybox_rpm(shell)
history = tmp_path / "history.sqlite"
+31
View File
@@ -37,9 +37,40 @@ class TestRules:
"selector": "*.*",
"complete": True,
"ruleset": None,
"template": "RSYSLOG_SyslogProtocol23Format",
},
]
def test_a_rule_without_a_template_sends_rsyslogs_default(self):
"""None: RFC 3164, the sender's local time without a zone (NetOrk/netork#493)."""
(rule,) = _rules("*.* @graylog.example.net:514\n")
assert rule["template"] is None
def test_an_actions_template_parameter(self):
(rule,) = _rules(
'action(type="omfwd" target="10.0.0.7" Template="RSYSLOG_ForwardFormat")\n'
)
assert rule["template"] == "RSYSLOG_ForwardFormat"
def test_the_legacy_default_template_holds_for_the_rules_after_it(self):
rules = _rules(
"*.* @a\n$ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format\n"
"*.* @@b:514\n*.* @c;RSYSLOG_TraditionalForwardFormat\n"
)
assert [(r["target"], r["template"]) for r in rules] == [
("a", None),
("b", "RSYSLOG_SyslogProtocol23Format"),
("c", "RSYSLOG_TraditionalForwardFormat"),
]
def test_the_forward_modules_template_holds_for_its_actions(self):
text = (
'module(load="builtin:omfwd" template="RSYSLOG_SyslogProtocol23Format")\n'
'action(type="omfwd" target="10.0.0.7")\n'
)
(rule,) = _rules(text)
assert rule["template"] == "RSYSLOG_SyslogProtocol23Format"
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
(rule,) = _rules("*.* @@logs.lan\n")
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
+74
View File
@@ -18,9 +18,11 @@ import pytest
from napalm_device_types.package_updates import (
APT_UPGRADABLE_COMMAND,
DNF_INSTALLED_CVES_COMMAND,
DNF_SECURITY_COMMAND,
nevra_name,
parse_apt_upgradable,
parse_dnf_installed_cves,
parse_dnf_security,
)
@@ -198,3 +200,75 @@ __DNF_RC=0
)
def test_the_name_of_a_nevra(self, nevra, name):
assert nevra_name(nevra) == name
class TestDnfInstalledCves:
"""The CVEs the host's installed advisories close, per package (netOrk #2394).
Where netOrk's knowledge base holds no records of a release, the host's own
advisories are the distribution's word that a backport is in. Lines from real
hosts: Oracle Linux 9.8 (dnf 4.14, aarch64) and Amazon Linux 2023 (dnf 4.14).
"""
ORACLE = """\
CVE-2024-6501 Low/Sec. NetworkManager-1:1.48.10-2.0.1.el9_5.aarch64
CVE-2024-3661 Moderate/Sec. NetworkManager-1:1.48.10-5.0.1.el9_5.aarch64
CVE-2023-38545 Important/Sec. curl-7.76.1-23.el9_2.4.aarch64
CVE-2023-38545 Important/Sec. curl-7.76.1-26.el9_3.2.aarch64
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9.aarch64
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9_2.7.aarch64
__DNF_RC=0
"""
AMAZON = """\
CVE-2022-3715 Medium/Sec. bash-5.2.15-1.amzn2023.0.2.x86_64
CVE-2022-23491 Important/Sec. ca-certificates-2021.2.50-1.0.amzn2023.0.6.noarch
CVE-2023-32803 Important/Sec. ca-certificates-2023.2.60-1.0.amzn2023.0.2.noarch
__DNF_RC=0
"""
def test_cves_per_package_once_each_and_sorted(self):
assert parse_dnf_installed_cves(self.ORACLE) == {
"NetworkManager": ["CVE-2024-3661", "CVE-2024-6501"],
"curl": ["CVE-2023-38545"],
"glibc-common": ["CVE-2023-4911"],
}
def test_amazon_linux(self):
assert parse_dnf_installed_cves(self.AMAZON) == {
"bash": ["CVE-2022-3715"],
"ca-certificates": ["CVE-2022-23491", "CVE-2023-32803"],
}
def test_lines_that_name_no_cve_are_left_out(self):
"""A GPG prompt a repository without its key prints, an advisory id."""
out = (
"Importing GPG key 0x6E93CD0C:\n"
' Userid : "https://packagecloud.io/crowdsec/crowdsec"\n'
"ELSA-2023-5453 Important/Sec. glibc-2.34-60.0.3.el9_2.7.aarch64\n"
"__DNF_RC=0\n"
)
assert parse_dnf_installed_cves(out) == {}
def test_a_failed_dnf_raises_instead_of_reading_as_no_advisories(self):
with pytest.raises(ValueError, match="exit status 1"):
parse_dnf_installed_cves("__DNF_RC=1\n")
def test_output_cut_short_raises(self):
with pytest.raises(ValueError, match="no exit status"):
parse_dnf_installed_cves(self.ORACLE.replace("__DNF_RC=0\n", ""))
def test_the_command_lists_installed_advisories_with_their_cves_from_the_cache(self, tmp_path):
args = tmp_path / "args"
dnf = tmp_path / "dnf"
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
dnf.chmod(0o755)
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
out = subprocess.run(
["sh", "-c", DNF_INSTALLED_CVES_COMMAND], capture_output=True, text=True, env=env
).stdout
assert {"--cacheonly", "updateinfo", "list", "--installed", "--with-cve"} <= set(
args.read_text().split()
)
assert out == "__DNF_RC=0\n"
+49
View File
@@ -299,3 +299,52 @@ def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
driver.add_service_account("netork", NETORK_KEY)
assert not _root_commands(host)
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOldNetork netork@central"
def test_a_replaced_key_goes_once_the_new_one_is_in(host: _ServiceHost, driver: _Driver) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{OWN_KEY}\n{OLD_KEY}\n"
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
def test_the_new_key_is_never_one_that_goes(host: _ServiceHost, driver: _Driver) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{NETORK_KEY}\n"
result = driver.add_service_account(
"netork", NETORK_KEY, replaces=[f"{NETORK_KEY.split()[0]} {NETORK_KEY.split()[1]}"]
)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{NETORK_KEY}\n"
def test_a_replaced_key_stays_while_the_new_one_is_not_in(
host: _ServiceHost, driver: _Driver
) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{OLD_KEY}\n"
host.failing.add("as install")
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
assert not result["success"]
assert "old keys" not in [s["step"] for s in result["steps"]]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OLD_KEY}\n"
def test_replaced_keys_are_a_list_not_one_string(driver: _Driver) -> None:
with pytest.raises(ValueError, match="list"):
driver.add_service_account("netork", NETORK_KEY, replaces=OLD_KEY)
+71 -4
View File
@@ -14,8 +14,19 @@ from napalm_device_types import (
UUID = "31303043-534D-C40C-7A72-6F1300000000"
def _out(uuid: str, product_serial: str, board_serial: str) -> str:
return f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\nIDEND\n"
def _out(
uuid: str,
product_serial: str,
board_serial: str,
bios: tuple[str, str, str] = ("", "", ""),
) -> str:
vendor, version, date = bios
return (
f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\n{vendor}\n{version}\n{date}\nIDEND\n"
)
_NO_FIRMWARE = {"bios_vendor": "", "bios_version": "", "bios_date": ""}
# ── parsing ───────────────────────────────────────────────────────────────────
@@ -26,6 +37,7 @@ def test_reads_uuid_and_both_serials():
"system_uuid": "31303043-534d-c40c-7a72-6f1300000000",
"product_serial": "S123",
"board_serial": "ZM155S026022",
**_NO_FIRMWARE,
}
@@ -81,9 +93,57 @@ def test_unreadable_files_read_as_empty():
"system_uuid": "",
"product_serial": "",
"board_serial": "BS1",
**_NO_FIRMWARE,
}
# ── the system firmware ───────────────────────────────────────────────────────
def test_reads_the_system_firmware():
# pve-dual, a Supermicro X10DRL-i
out = _out(UUID, "", "ZM155S026022", ("American Megatrends Inc.", "3.4", "11/22/2021"))
identity = parse_system_identity(out)
assert identity["bios_vendor"] == "American Megatrends Inc."
assert identity["bios_version"] == "3.4"
assert identity["bios_date"] == "2021-11-22"
def test_a_virtual_machines_firmware_is_read_like_any_other():
out = _out(UUID, "", "", ("EDK II", "4.2023.08-4", "02/15/2024"))
assert parse_system_identity(out)["bios_version"] == "4.2023.08-4"
@pytest.mark.parametrize("version", ["1", "0", "1.0", "A"])
def test_a_short_firmware_version_is_no_placeholder(version):
# Serials of one repeated character are placeholders, versions are not.
out = _out(UUID, "", "", ("Vendor", version, ""))
assert parse_system_identity(out)["bios_version"] == version
@pytest.mark.parametrize("value", ["", "To be filled by O.E.M.", "Default string", "Not Specified"])
def test_a_placeholder_firmware_version_reads_as_empty(value):
out = _out(UUID, "", "", (value, value, ""))
identity = parse_system_identity(out)
assert identity["bios_vendor"] == ""
assert identity["bios_version"] == ""
@pytest.mark.parametrize("date", ["", "not a date", "13/45/2021", "2021-11-22"])
def test_a_firmware_date_that_is_no_smbios_date_reads_as_empty(date):
# sysfs writes the SMBIOS date as MM/DD/YYYY; anything else is no reading of it.
out = _out(UUID, "", "", ("Vendor", "1.2", date))
assert parse_system_identity(out)["bios_date"] == ""
def test_output_of_the_identity_alone_has_no_firmware():
# What a host answered before the firmware was read with it: three fields.
old = f"IDBEGIN\n{UUID}\nS1\nB1\nIDEND\n"
identity = parse_system_identity(old)
assert identity["board_serial"] == "B1"
assert {k: identity[k] for k in _NO_FIRMWARE} == _NO_FIRMWARE
def test_output_without_its_frame_is_not_a_reading():
with pytest.raises(ValueError):
parse_system_identity("sudo: a password is required\n")
@@ -115,6 +175,13 @@ def test_the_driver_only_carries_the_command_across():
assert privileged is True
def test_the_command_reads_the_three_dmi_files():
for name in ("product_uuid", "product_serial", "board_serial"):
def test_the_command_reads_the_dmi_files():
for name in (
"product_uuid",
"product_serial",
"board_serial",
"bios_vendor",
"bios_version",
"bios_date",
):
assert f"/sys/class/dmi/id/{name}" in SYSTEM_IDENTITY_COMMAND
+14
View File
@@ -34,9 +34,16 @@ def test_the_shape():
"timezone": str | None,
"synchronized": bool | None,
"daemon": str | None,
"dhcp_ntp_servers": list[str] | None,
}
def test_the_dhcp_servers_are_optional():
# A driver that cannot tell what DHCP brought leaves the key out (NetOrk/netork#2380).
assert TimeSyncDict.__required_keys__ == {"ntp_servers", "timezone", "synchronized", "daemon"}
assert TimeSyncDict.__optional_keys__ == {"dhcp_ntp_servers"}
def test_the_two_values_a_driver_may_write():
assert frozenset({"ntp_servers", "timezone"}) == TIME_SYNC_VALUES
@@ -190,6 +197,13 @@ class TestConfigureTimeSync:
assert Lower().configure_time_sync(["NTP1.Example.org"])["success"] is True
def test_servers_from_dhcp_do_not_fail_the_check(self):
# DHCP option 42 counts: what the device got by DHCP stays and is no mistake.
device = _Device()
device.state["dhcp_ntp_servers"] = ["192.168.1.1"]
result = device.configure_time_sync(["ntp1.example.org"])
assert result["success"] is True, result["output"]
def test_a_device_without_a_zone_name_is_not_checked_for_one(self):
# A switch keeps an offset and a summer-time rule, not a name: it reads None.
result = _Device(keeps_zone_name=False).configure_time_sync(timezone="Europe/Berlin")
+131 -18
View File
@@ -4,7 +4,8 @@ The reading is run by a real ``sh`` over a directory tree that stands in for the
host, with ``timedatectl``, ``systemctl`` and ``chronyc`` replaced by scripts that
keep a log -- so the shell itself is tested, not a hand-written copy of its output.
``sed`` and ``install`` are the real ones, so the files the writes leave are what a
host would get (NetOrk/netork#2361).
host would get (NetOrk/netork#2361). Servers that DHCP brings are read apart and left
alone by a write (NetOrk/netork#2380).
"""
from __future__ import annotations
@@ -20,6 +21,7 @@ from napalm_device_types.channel import CommandResult
from napalm_device_types.time_sync_posix import (
chrony_synchronized,
parse_key_values,
timesyncd_link_servers,
timesyncd_servers,
)
@@ -136,6 +138,7 @@ class Host(PosixTimeSyncMixin):
self.CHRONY_CONFIGS = (str(etc / "chrony/chrony.conf"), str(etc / "chrony.conf")) # type: ignore[misc]
self.CHRONY_SOURCES_FILE = str(etc / "chrony/sources.d/netork.sources") # type: ignore[misc]
self.CHRONY_INCLUDE_FILE = str(etc / "chrony.d/netork.conf") # type: ignore[misc]
self.CHRONY_DHCP_DIRS = (str(root / "run/chrony-dhcp"),) # type: ignore[misc]
self.TIMESYNCD_DROPIN = str(etc / "systemd/timesyncd.conf.d/netork.conf") # type: ignore[misc]
self.NTPD_CONFIGS = (str(etc / "ntpsec/ntp.conf"), str(etc / "ntp.conf")) # type: ignore[misc]
self.OPENNTPD_CONFIGS = (str(etc / "openntpd/ntpd.conf"),) # type: ignore[misc]
@@ -223,21 +226,43 @@ class TestParsing:
def test_timesyncd_uses_the_fallback_only_when_nothing_else_is_known(self):
assert timesyncd_servers(parse_key_values(TIMESYNC_FALLBACK)) == ["ntp.ubuntu.com"]
# runtime, then system, then link: the order timesyncd tries them in.
# runtime, then system: the configured ones; a link's servers are DHCP's.
assert timesyncd_servers(parse_key_values(TIMESYNC_CONFIGURED)) == [
"ntp1.example.org",
"ntp2.example.org",
"192.168.1.1",
]
assert timesyncd_servers({}) == []
def test_timesyncd_does_not_fall_back_while_a_link_brings_servers(self):
values = {"LinkNTPServers": "192.168.1.1", "FallbackNTPServers": "ntp.ubuntu.com"}
assert timesyncd_servers(values) == []
assert timesyncd_link_servers(values) == ["192.168.1.1"]
def test_timesyncd_link_servers(self):
assert timesyncd_link_servers(parse_key_values(TIMESYNC_CONFIGURED)) == ["192.168.1.1"]
assert timesyncd_link_servers({"LinkNTPServers": "10.0.0.1 10.0.0.1 10.0.0.2"}) == [
"10.0.0.1",
"10.0.0.2",
]
assert timesyncd_link_servers(parse_key_values(TIMESYNC_FALLBACK)) == []
def test_chrony_is_synchronised_while_it_has_selected_a_source(self):
assert chrony_synchronized(CHRONYC_SYNCED) is True
assert chrony_synchronized(CHRONYC_UNSYNCED) is False
assert chrony_synchronized("") is None
def test_chrony_without_a_main_configuration_has_no_sources(self):
assert parse_time_sync("@@ts active\nchrony chrony active\n@@ts end\n")["ntp_servers"] == []
reading = parse_time_sync("@@ts active\nchrony chrony active\n@@ts end\n")
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], [])
def test_the_dhcp_directory_is_debian_s_and_red_hat_s_by_default(self):
reading = parse_time_sync(
"@@ts active\nchrony chrony active\n"
"@@ts chrony 0 main /etc/chrony/chrony.conf\nsourcedir /run/chrony-dhcp\n"
"@@ts chrony 1 sourcedir /run/chrony-dhcp/eth0.sources\nserver 10.0.0.1 iburst\n"
"@@ts end\n"
)
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], ["10.0.0.1"])
def test_a_reading_cut_short_raises(self):
with pytest.raises(ValueError, match="cut short"):
@@ -251,14 +276,35 @@ class TestReading:
"ntp_servers": [
"ntp-conf.example.org",
"2.debian.pool.ntp.org",
"192.168.1.1",
"ntp-other.example.org",
],
"timezone": "Europe/Berlin",
"synchronized": True,
"daemon": "chrony",
"dhcp_ntp_servers": ["192.168.1.1"],
}
def test_dhcp_servers_in_the_order_chrony_reads_them(self, debian: Path):
_write(debian / "run/chrony-dhcp/wlan0.sources", "server 10.9.0.1 iburst\n")
_write(debian / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1\npool p.example\n")
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == [
"192.168.1.1",
"p.example",
"10.9.0.1",
]
def test_a_dhcp_directory_chrony_does_not_read_brings_nothing(self, debian: Path):
# What 4.35.0 left behind: DHCP's sourcedir commented out.
conf = debian / "etc/chrony/chrony.conf"
conf.write_text(_text(conf).replace("\nsourcedir ", "\n#napalm-time-sync: sourcedir ", 1))
reading = Host(debian).get_time_sync()
assert reading["dhcp_ntp_servers"] == []
assert "192.168.1.1" not in reading["ntp_servers"]
def test_without_a_dhcp_directory_dhcp_brings_nothing(self, root: Path):
_write(root / "etc/chrony.conf", "pool 2.rhel.pool.ntp.org iburst\n")
assert Host(root, active="chronyd").get_time_sync()["dhcp_ntp_servers"] == []
def test_the_reading_is_one_round_trip_without_root(self, debian: Path):
host = Host(debian)
host.get_time_sync()
@@ -291,7 +337,8 @@ class TestReading:
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
reading = Host(root, active="systemd-timesyncd").get_time_sync()
assert reading["daemon"] == "systemd-timesyncd"
assert reading["ntp_servers"] == ["ntp1.example.org", "ntp2.example.org", "192.168.1.1"]
assert reading["ntp_servers"] == ["ntp1.example.org", "ntp2.example.org"]
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
assert reading["synchronized"] is True
def test_ntpd(self, root: Path):
@@ -303,6 +350,8 @@ class TestReading:
reading = Host(root, active="ntpsec").get_time_sync()
assert reading["daemon"] == "ntpd"
assert reading["ntp_servers"] == ["0.debian.pool.ntp.org", "ntp1.example.org"]
# Debian's DHCP hook gives ntpd a configuration of its own: not read, so unknown.
assert reading["dhcp_ntp_servers"] is None
def test_openntpd(self, root: Path):
_write(
@@ -312,6 +361,7 @@ class TestReading:
reading = Host(root, active="openntpd").get_time_sync()
assert reading["daemon"] == "openntpd"
assert reading["ntp_servers"] == ["pool.ntp.org", "ntp1.example.org"]
assert reading["dhcp_ntp_servers"] is None
def test_no_client_runs(self, root: Path):
(root / "fix/timedatectl").write_text(
@@ -322,6 +372,7 @@ class TestReading:
"timezone": "Europe/Berlin",
"synchronized": False,
"daemon": None,
"dhcp_ntp_servers": [],
}
def test_without_timedatectl_the_zone_comes_from_the_files(self, root: Path):
@@ -352,7 +403,8 @@ class TestChronyServers:
)
conf = _text(debian / "etc/chrony/chrony.conf")
assert "\n#napalm-time-sync: pool 2.debian.pool.ntp.org iburst\n" in conf
assert f"\n#napalm-time-sync: sourcedir {debian}/run/chrony-dhcp\n" in conf
# DHCP option 42 counts: chrony keeps reading what DHCP brings.
assert f"\nsourcedir {debian}/run/chrony-dhcp\n" in conf
assert f"\nsourcedir {debian}/etc/chrony/sources.d\n" in conf
assert f"\nconfdir {debian}/etc/chrony/conf.d\n" in conf
assert _text(debian / "etc/chrony/conf.d/local.conf") == (
@@ -361,15 +413,72 @@ class TestChronyServers:
assert _text(debian / "etc/chrony/sources.d/other.sources") == (
"#napalm-time-sync: server ntp-other.example.org\n"
)
# DHCP's own file is left alone: it is not read any more.
# DHCP's own file is left alone: the DHCP client rewrites it on every lease.
assert _text(debian / "run/chrony-dhcp/eth0.sources") == "server 192.168.1.1 iburst\n"
# The main configuration changed: chronyd reads it only when it starts.
assert "systemctl restart chrony.service" in host.log()
assert host.get_time_sync()["ntp_servers"] == [
"ntp1.example.org",
"10.0.0.1",
"2.de.pool.ntp.org",
]
reading = host.get_time_sync()
assert reading["ntp_servers"] == ["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"]
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
def test_what_4_35_0_took_from_dhcp_is_given_back(self, debian: Path):
conf = debian / "etc/chrony/chrony.conf"
Host(debian).configure_time_sync(["ntp1.example.org"])
# A host 4.35.0 aligned: DHCP's sourcedir commented out behind the mark.
dhcp = f"sourcedir {debian}/run/chrony-dhcp"
conf.write_text(_text(conf).replace(f"\n{dhcp}\n", f"\n#napalm-time-sync: {dhcp}\n"))
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == []
(debian / "log").unlink()
host = Host(debian)
result = host.configure_time_sync(["ntp1.example.org"])
assert result["success"] is True, result["output"]
assert f"\n{dhcp}\n" in _text(conf)
assert "#napalm-time-sync: sourcedir" not in _text(conf)
# The main configuration changed again: a restart, not a reload.
assert "systemctl restart chrony.service" in host.log()
assert host.get_time_sync()["dhcp_ntp_servers"] == ["192.168.1.1"]
def test_a_foreign_sourcedir_4_35_0_took_out_stays_out(self, debian: Path):
conf = debian / "etc/chrony/chrony.conf"
foreign = f"#napalm-time-sync: sourcedir {debian}/etc/chrony/extra.d"
conf.write_text(_text(conf) + foreign + "\n")
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
assert Host(debian).configure_time_sync(["ntp1.example.org"])["success"] is True
assert f"\n{foreign}\n" in _text(conf)
def test_a_foreign_sourcedir_stays_and_its_sources_stop_counting(self, debian: Path):
conf = debian / "etc/chrony/chrony.conf"
conf.write_text(_text(conf) + f"sourcedir {debian}/etc/chrony/extra.d\n")
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
host = Host(debian)
result = host.configure_time_sync(["ntp1.example.org"])
assert result["success"] is True, result["output"]
assert f"\nsourcedir {debian}/etc/chrony/extra.d\n" in _text(conf)
assert _text(debian / "etc/chrony/extra.d/x.sources") == (
"#napalm-time-sync: server ntp-extra.example.org\n"
)
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org"]
def test_dhcp_is_kept_where_the_servers_go_to_a_confdir(self, root: Path):
_write(
root / "etc/chrony/chrony.conf",
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n"
f"sourcedir {root}/run/chrony-dhcp\n",
)
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
host = Host(root)
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
assert f"\nsourcedir {root}/run/chrony-dhcp\n" in _text(root / "etc/chrony/chrony.conf")
reading = host.get_time_sync()
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
["ntp1.example.org"],
["192.168.1.1"],
)
def test_a_second_run_only_reloads_the_sources(self, debian: Path):
Host(debian).configure_time_sync(["ntp1.example.org"])
@@ -480,7 +589,7 @@ class TestTimesyncdServers:
assert "systemctl restart systemd-timesyncd.service" in host.log()
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org", "10.0.0.1"]
def test_servers_from_dhcp_stay_and_the_check_says_so(self, root: Path):
def test_servers_from_dhcp_stay_and_count(self, root: Path):
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
tool = (root / "bin/systemctl").read_text()
(root / "bin/systemctl").write_text(
@@ -488,11 +597,15 @@ class TestTimesyncdServers:
"printf 'SystemNTPServers", "printf 'LinkNTPServers=192.168.1.1\\nSystemNTPServers"
)
)
result = Host(root, active="systemd-timesyncd").configure_time_sync(["ntp1.example.org"])
host = Host(root, active="systemd-timesyncd")
result = host.configure_time_sync(["ntp1.example.org"])
assert result["success"] is False
assert result["steps"][-1]["step"] == "verify"
assert "192.168.1.1" in result["steps"][-1]["output"]
assert result["success"] is True, result["output"]
reading = host.get_time_sync()
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
["ntp1.example.org"],
["192.168.1.1"],
)
class TestOtherClients:
+20
View File
@@ -0,0 +1,20 @@
"""The contract of :class:`~napalm_device_types.updates.UpdateMixin`."""
from __future__ import annotations
import napalm_device_types
from napalm_device_types.updates import PackageIndexMissing
def test_a_missing_index_is_its_own_failure():
"""A reader can tell "no index to read from" from any other failure, and a
caller that only knows failures still catches it as one."""
exc = PackageIndexMissing("The package index is not loaded (run apk update first)")
assert isinstance(exc, RuntimeError)
assert str(exc) == "The package index is not loaded (run apk update first)"
def test_it_is_exported_from_the_package():
assert napalm_device_types.PackageIndexMissing is PackageIndexMissing
assert "PackageIndexMissing" in napalm_device_types.__all__
+360
View File
@@ -0,0 +1,360 @@
"""A native Zabbix agent: its configuration, its repository and the fix template
(``ZabbixAgentMixin``, NetOrk/netork#2571).
Where a host has no container engine the application installs the agent from the
host's own packages, and from Zabbix's repository only where those have none.
"""
from __future__ import annotations
import pytest
from napalm_device_types import (
ZabbixAgentMixin,
check_zabbix_agent_settings,
zabbix_agent_config,
zabbix_repository_command,
)
SETTINGS = {
"server": "zabbix.example.com",
"server_active": "zabbix.example.com",
"listen_port": 10050,
"allowed_sources": ["10.40.100.254"],
"hostname": "ap-1og",
"release": "7.4",
}
SAMPLE = """# This is a configuration file for Zabbix agent daemon (Unix)
LogFile=/tmp/zabbix_agentd.log
### Option: Server
# Server=
Server=127.0.0.1
ServerActive=127.0.0.1
Hostname=Zabbix server
# Include=/etc/zabbix_agentd.conf.d/*.conf
"""
class TestConfig:
def test_the_lines_that_say_who_may_ask_get_the_caller_s_values(self):
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
assert "\nServer=zabbix.example.com\n" in text
assert "\nServerActive=zabbix.example.com\n" in text
assert "\nHostname=ap-1og\n" in text
assert "Server: 127.0.0.1 → zabbix.example.com" in changes
assert "Hostname: Zabbix server → ap-1og" in changes
def test_comments_and_other_lines_stay_as_they_were(self):
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
assert text.startswith("# This is a configuration file")
assert "LogFile=/tmp/zabbix_agentd.log\n" in text
assert "# Server=\n" in text
assert text.endswith("# Include=/etc/zabbix_agentd.conf.d/*.conf\n" + "ListenPort=10050\n")
def test_a_key_the_file_does_not_set_is_added_at_the_end(self):
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
assert text.endswith("ListenPort=10050\n")
assert "ListenPort: (not set) → 10050" in changes
def test_a_file_that_already_says_it_changes_nothing(self):
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
again, changes = zabbix_agent_config(text, SETTINGS)
assert again == text
assert changes == []
def test_without_a_hostname_the_agent_keeps_its_own(self):
settings = {k: v for k, v in SETTINGS.items() if k != "hostname"}
text, changes = zabbix_agent_config(SAMPLE, settings)
assert "\nHostname=Zabbix server\n" in text
assert not any(c.startswith("Hostname") for c in changes)
def test_no_active_server_comments_the_line_out(self):
text, changes = zabbix_agent_config(SAMPLE, {**SETTINGS, "server_active": ""})
assert "\n# ServerActive=127.0.0.1\n" in text
assert "\nServerActive=" not in text
assert "ServerActive: 127.0.0.1 → (not set)" in changes
def test_a_file_without_a_final_newline_gets_one_before_what_is_added(self):
text, _ = zabbix_agent_config("Server=127.0.0.1", SETTINGS)
assert text.startswith("Server=zabbix.example.com\n")
assert text.endswith("\n")
def test_every_line_of_a_key_set_twice_says_the_same(self):
text, _ = zabbix_agent_config("Server=a\nServer=b\n", SETTINGS)
assert text.count("Server=zabbix.example.com\n") == 2
class TestSettings:
@pytest.mark.parametrize(
"server",
["", "zabbix.example.com\nUserParameter=x,id", "a b", "a;rm -rf /", "'a'"],
)
def test_a_server_that_is_not_addresses_is_refused(self, server):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "server": server})
def test_several_servers_and_ports_pass(self):
check_zabbix_agent_settings(
{**SETTINGS, "server": "10.0.0.1,zabbix.example.com", "server_active": "[::1]:10051"}
)
def test_a_hostname_with_a_newline_is_refused(self):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "hostname": "ap\nServer=evil"})
@pytest.mark.parametrize("port", [0, 65536, "10050"])
def test_a_listen_port_out_of_range_is_refused(self, port):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "listen_port": port})
@pytest.mark.parametrize("release", ["7", "7.4.1", "latest", "7.4; id"])
def test_a_release_that_is_not_major_dot_minor_is_refused(self, release):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "release": release})
def _os_release(**fields: str) -> str:
return "".join(f'{key}="{value}"\n' for key, value in fields.items())
class TestRepository:
BASE = "https://repo.zabbix.com/zabbix"
@pytest.mark.parametrize(
("os_release", "url"),
[
(
_os_release(ID="ubuntu", VERSION_ID="24.04", ID_LIKE="debian"),
"7.4/release/ubuntu/pool/main/z/zabbix-release/"
"zabbix-release_latest_7.4+ubuntu24.04_all.deb",
),
(
_os_release(ID="debian", VERSION_ID="12"),
"7.4/release/debian/pool/main/z/zabbix-release/"
"zabbix-release_latest_7.4+debian12_all.deb",
),
(
_os_release(ID="ol", VERSION_ID="9.4", ID_LIKE="fedora"),
"7.4/release/oracle/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
(
_os_release(ID="rhel", VERSION_ID="10.0", ID_LIKE="fedora"),
"7.4/release/rhel/10/noarch/zabbix-release-latest-7.4.el10.noarch.rpm",
),
(
_os_release(ID="almalinux", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
"7.4/release/alma/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
(
_os_release(ID="rocky", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
"7.4/release/rocky/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
],
)
def test_the_release_package_of_the_host_s_distribution(self, os_release, url):
command = zabbix_repository_command(os_release, "7.4")
assert f"{self.BASE}/{url}" in command
def test_a_release_before_7_2_lives_in_the_old_layout(self):
deb = zabbix_repository_command(_os_release(ID="ubuntu", VERSION_ID="24.04"), "7.0")
rpm = zabbix_repository_command(_os_release(ID="ol", VERSION_ID="9.4"), "7.0")
assert f"{self.BASE}/7.0/ubuntu/pool/main/z/zabbix-release/" in deb
assert f"{self.BASE}/7.0/oracle/9/x86_64/zabbix-release-latest-7.0.el9.noarch.rpm" in rpm
def test_a_deb_is_installed_with_dpkg_and_the_index_refreshed(self):
command = zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4")
assert "dpkg -i" in command
assert "apt-get update" in command
def test_an_rpm_is_installed_with_the_package_manager(self):
command = zabbix_repository_command(_os_release(ID="rocky", VERSION_ID="9.5"), "7.4")
assert "dnf -y install" in command
def test_a_derivative_falls_back_to_what_it_is_like(self):
command = zabbix_repository_command(
_os_release(ID="centos", VERSION_ID="9", ID_LIKE="rhel fedora"), "7.4"
)
assert "/7.4/release/rhel/9/noarch/" in command
def test_a_distribution_zabbix_has_no_repository_for_is_refused(self):
with pytest.raises(ValueError, match="openwrt"):
zabbix_repository_command(_os_release(ID="openwrt", VERSION_ID="25.12.2"), "7.4")
def test_a_release_that_is_not_major_dot_minor_is_refused(self):
with pytest.raises(ValueError):
zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4;id")
class _Agent(ZabbixAgentMixin):
"""The generic steps over ensure_package and manage_service; the driver's others."""
def __init__(self, refuse: frozenset[str] = frozenset(), repository: bool = True):
self.calls: list = []
self.refuse = refuse
self.installs = 0
if not repository:
self._add_zabbix_repository = None # type: ignore[assignment]
def refresh_available_updates(self):
self.calls.append(("refresh",))
ok = "refresh" not in self.refuse
return {"success": ok, "output": "" if ok else "Temporary failure resolving"}
def ensure_package(self, name):
self.calls.append(("ensure_package", name))
self.installs += 1
# The distribution has no package; after the repository is added it has.
ok = "distribution" not in self.refuse or self.installs > 1
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
def _add_zabbix_repository(self, release):
self.calls.append(("repository", release))
ok = "repository" not in self.refuse
return {"ok": ok, "output": f"Zabbix {release} repository added" if ok else "404"}
def _write_zabbix_agent_config(self, settings):
self.calls.append(("configure", settings["server"]))
return {"ok": "configure" not in self.refuse, "output": "written"}
def _allow_zabbix_agent_sources(self, settings):
self.calls.append(("firewall", settings["allowed_sources"]))
return {"ok": "firewall" not in self.refuse, "output": "open"}
def manage_service(self, name, action):
self.calls.append((action, name))
ok = action not in self.refuse
return {"success": ok, "output": "" if ok else f"cannot {action}"}
class TestZabbixAgentMixin:
def test_install_configure_open_enable_and_restart(self):
agent = _Agent()
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls == [
("refresh",),
("ensure_package", "zabbix-agent"),
("configure", "zabbix.example.com"),
("firewall", ["10.40.100.254"]),
("enable", "zabbix-agent"),
("restart", "zabbix-agent"),
]
assert [s["step"] for s in result["steps"]] == ["install", "configure", "firewall", "start"]
def test_without_a_package_in_the_distribution_zabbix_s_repository_is_added(self):
agent = _Agent(refuse=frozenset({"distribution"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls[:4] == [
("refresh",),
("ensure_package", "zabbix-agent"),
("repository", "7.4"),
("ensure_package", "zabbix-agent"),
]
assert "Zabbix 7.4 repository added" in result["output"]
def test_without_a_release_there_is_no_fallback(self):
settings = {k: v for k, v in SETTINGS.items() if k != "release"}
agent = _Agent(refuse=frozenset({"distribution"}))
result = agent.configure_zabbix_agent(settings)
assert result["success"] is False
assert ("repository", "7.4") not in agent.calls
assert "no candidate" in result["output"]
def test_a_driver_that_cannot_add_the_repository_fails_with_the_distribution_s_answer(self):
agent = _Agent(refuse=frozenset({"distribution"}), repository=False)
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert "no candidate" in result["output"]
def test_a_repository_that_cannot_be_added_configures_nothing(self):
agent = _Agent(refuse=frozenset({"distribution", "repository"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert [c[0] for c in agent.calls] == ["refresh", "ensure_package", "repository"]
assert "404" in result["output"]
def test_the_package_index_is_loaded_first(self):
agent = _Agent()
agent.configure_zabbix_agent(SETTINGS)
assert agent.calls[0] == ("refresh",)
def test_an_index_that_will_not_load_still_lets_the_install_try(self):
agent = _Agent(refuse=frozenset({"refresh"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert ("ensure_package", "zabbix-agent") in agent.calls
assert "Temporary failure resolving" in result["steps"][0]["output"]
def test_a_driver_without_an_index_to_load_installs_straight_away(self):
agent = _Agent()
agent.refresh_available_updates = None # type: ignore[assignment]
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls[0] == ("ensure_package", "zabbix-agent")
def test_a_closed_firewall_starts_nothing(self):
agent = _Agent(refuse=frozenset({"firewall"}))
assert agent.configure_zabbix_agent(SETTINGS)["success"] is False
assert ("restart", "zabbix-agent") not in agent.calls
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
agent = _Agent(refuse=frozenset({"enable"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert ("restart", "zabbix-agent") not in agent.calls
assert "cannot enable" in result["output"]
def test_the_package_and_service_names_are_the_driver_s(self):
agent = _Agent()
agent.ZABBIX_AGENT_PACKAGE = "zabbix-agentd"
agent.ZABBIX_AGENT_SERVICE = "zabbix_agentd"
agent.configure_zabbix_agent(SETTINGS)
assert ("ensure_package", "zabbix-agentd") in agent.calls
assert ("restart", "zabbix_agentd") in agent.calls
def test_settings_that_do_not_hold_are_refused_before_anything_changes(self):
agent = _Agent()
with pytest.raises(ValueError):
agent.configure_zabbix_agent({**SETTINGS, "server": "a\nb"})
assert agent.calls == []