Compare commits
36
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f654076560 | ||
|
|
2dea517e89 | ||
|
|
1a7885d22e | ||
|
|
71588613fb | ||
|
|
93ba335798 | ||
|
|
eb1964c4b3 | ||
|
|
3622d0f825 | ||
|
|
caf7ef4cc8 | ||
|
|
3422b7d030 | ||
|
|
f1940aa918 | ||
|
|
5217c6e333 | ||
|
|
7051199589 | ||
|
|
7c0e4fffc5 | ||
|
|
cb0f1cc05c | ||
|
|
b4929990f1 | ||
|
|
3db6c6385c | ||
|
|
c5154d954b | ||
|
|
211d0c02eb | ||
|
|
74d2f42b0e | ||
|
|
c25f39768f | ||
|
|
9ecc6823c8 | ||
|
|
2080ccbfd7 | ||
|
|
ce1fe18405 | ||
|
|
e43926f1d5 | ||
|
|
43e0210ff6 | ||
|
|
241acef6c1 | ||
|
|
4c0ba5b2c0 | ||
|
|
e9da399568 | ||
|
|
8673d1eda1 | ||
|
|
46d7125aa6 | ||
|
|
8e43fe2ed9 | ||
|
|
679584766f | ||
|
|
1b04818982 | ||
|
|
65991d0930 | ||
|
|
9dfb5596bc | ||
|
|
b4cdcfd347 |
No files matched your search
+120
@@ -7,6 +7,126 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- `AptReleaseMixin.get_apt_source_releases()`: the host's release from `/etc/os-release`
|
||||
(`ID`, `ID_LIKE`, `VERSION_CODENAME`, `UBUNTU_CODENAME`) and every enabled source apt
|
||||
reads, one-line and deb822, with its file, line, types, URI, suites and components
|
||||
(NetOrk/netork#2614). `AptProxyMixin` brings it, so every apt driver has it.
|
||||
- `AptReleaseMixin.set_apt_source_release(path, release, codename)`: every suite of
|
||||
*release* in one of apt's source files moved to *codename* (`focal-security` →
|
||||
`noble-security`), once each new suite's `InRelease` or `Release` is fetched with apt's
|
||||
downloader; nothing is written where the vendor has none.
|
||||
- `parse_apt_releases`, `rewrite_release`, `suite_release`, `APT_RELEASES_COMMAND`.
|
||||
|
||||
## [4.41.0] - 2026-11-17
|
||||
|
||||
### Added
|
||||
- `FirmwareUpgradeMixin.start_firmware_upgrade(version)`: start installing the firmware a
|
||||
driver offers; the device runs it by itself and restarts at its end, and
|
||||
`UpdateMixin.get_update_progress` follows it until then (NetOrk/netork#2615, OpenWrt's
|
||||
sysupgrade through owut).
|
||||
- `FirmwareOfferDict.installable`: whether the driver can install `latest_version` itself
|
||||
now, False while the device lacks what that takes; and
|
||||
`FirmwareOfferDict.latest_major_version`, the newest of a later release line, shown and
|
||||
never crossed to by the driver.
|
||||
|
||||
## [4.40.0] - 2026-11-17
|
||||
|
||||
### Added
|
||||
- `ZabbixAgentMixin.configure_zabbix_agent(settings)`: a native Zabbix agent, for a host
|
||||
without a container engine (NetOrk/netork#2571). Install (`ensure_package` with
|
||||
`ZABBIX_AGENT_PACKAGE`, and where the host's own packages have none and
|
||||
`settings["release"]` names the server's release line, the driver's
|
||||
`_add_zabbix_repository(release)` and a second try), configure (the driver's
|
||||
`_write_zabbix_agent_config`), firewall (the driver's `_allow_zabbix_agent_sources`),
|
||||
start (`manage_service` enable and restart of `ZABBIX_AGENT_SERVICE`). Settings that do
|
||||
not hold raise `ValueError` before anything changes.
|
||||
- `ZabbixAgentSettingsDict`; `zabbix_agent_config(text, settings)`, the agent's
|
||||
configuration with `Server`, `ServerActive`, `ListenPort` and `Hostname` set and the rest
|
||||
kept; `zabbix_repository_command(os_release, release)`, the root command that adds
|
||||
Zabbix's repository for Debian, Ubuntu, Raspberry Pi OS, RHEL, CentOS, Oracle Linux,
|
||||
AlmaLinux and Rocky Linux, in the layout before and since 7.2;
|
||||
`check_zabbix_agent_settings(settings)`.
|
||||
- `fixes.enable_and_restart(manage_service, service)`, the start step `LldpAgentMixin` and
|
||||
`ZabbixAgentMixin` share.
|
||||
- Podman as a container engine (NetOrk/netork#818): `container_engines()` lists it with the
|
||||
API `podman-libpod`, and `open_container_engine("podman")` reaches it with `podman system
|
||||
dial-stdio` (Podman 4.0 and later). It is the host's rootful Podman, so the connection
|
||||
runs every call as root (`ContainerEngineConnection.privileged`, `ROOT_ENGINES`).
|
||||
- `apt_sources`: apt's https sources through the package cache, in the form apt-cacher-ng
|
||||
fetches over https itself and stores (`http://HTTPS///<host>/…`, with `AllowUserPorts=80
|
||||
443` on the cache). `cacheable_https_hosts(report)` names the hosts this applies to, and
|
||||
`rewrite_sources_command` / `revert_sources_command` are the root commands, for `.list`
|
||||
and deb822 `.sources` files. A host apt keeps credentials for in `auth.conf` stays on
|
||||
https, because apt finds them by the host's name and the cache is shared. The rewrite
|
||||
names such a host in the proxy file as `Package-Proxy::Not-Cached` (NetOrk/netork#2555).
|
||||
- `get_package_proxy_report()` reports those hosts as `not_cached`.
|
||||
|
||||
### Changed
|
||||
- `container_engines()` lists an engine only when its CLI names itself on `--version`:
|
||||
podman-docker's `docker` is podman, and is no longer listed as Docker.
|
||||
- `configure_package_proxy(url)` has a step `cache` before `https`. It rewrites the https
|
||||
sources it can, fetches one of them through *url*, and turns them all back to https
|
||||
where *url* cannot. The `fetch` step proves the proxy with a plain http source only.
|
||||
- `configure_package_proxy("")` turns the sources back to https (step `sources`) before it
|
||||
removes the proxy file. Without the cache, apt would look for a host called `HTTPS`.
|
||||
|
||||
## [4.39.0] - 2026-11-14
|
||||
|
||||
### Added
|
||||
- `PackageIndexMissing` (a `RuntimeError`): what `get_available_updates()` raises when
|
||||
the host holds no package index to read from -- OpenWrt's, which lives in RAM, after
|
||||
every reboot. `refresh_available_updates()` loads it again, so a caller can tell this
|
||||
from a failed read and refresh instead of reporting an error (NetOrk/netork#2527).
|
||||
|
||||
## [4.38.0] - 2026-11-13
|
||||
|
||||
### Added
|
||||
- `get_system_identity()` also reads the host's system firmware (BIOS or UEFI) from the
|
||||
same SMBIOS tables in the same round trip: `bios_vendor`, `bios_version` and
|
||||
`bios_date` (ISO 8601, from sysfs's `MM/DD/YYYY`; anything else reads as empty). Only
|
||||
the vendors' text defaults (`To be filled by O.E.M.`, `Default string`) are
|
||||
placeholders there -- a version of `1` is a version. Output of the three identity
|
||||
fields alone still parses, with the firmware empty (NetOrk/netork#2518).
|
||||
- `ConfigRevisionsMixin`: a device's own configuration history (NetOrk/netork#2511).
|
||||
`get_config_revisions()` (newest first, the first in place now, as
|
||||
`ConfigRevisionDict`: id, UTC time, description, user, size) and
|
||||
`get_config_revision(revision_id)` are declared. `revert_config_revision(revision_id)`
|
||||
is a template over the driver's `_revert_to_config_revision`: it refuses an id that is
|
||||
not in the history (`UnknownConfigRevision`), and after the revert checks that the
|
||||
newest revision holds the revision's configuration, else `ConfigRevertFailed`; it
|
||||
returns that revision. `config_revision_content(text)` leaves out what a device stamps
|
||||
into every save, so an application can compare revisions with a configuration it has.
|
||||
- `add_service_account(..., replaces=[...])`: netOrk's keys the new one takes the place of
|
||||
go from its account once the new key is in it (step `old keys`), never the new key
|
||||
itself; while the new key could not be added they stay, so netOrk keeps its way in.
|
||||
netOrk calls it through a login with the new key, after trying that login, to move a
|
||||
host from its installation-wide key to its site's (NetOrk/netork#2462).
|
||||
- Each forwarding rule of `LogForwardingDict` names its `template`: the one the rule
|
||||
gives (`@host;RSYSLOG_SyslogProtocol23Format`, `action(… template="…")`), else the
|
||||
default set before it (`$ActionForwardDefaultTemplate`, or
|
||||
`module(load="builtin:omfwd" template="…")` for omfwd actions), else None -- rsyslog's
|
||||
own default, RFC 3164, which sends the host's local time without a time zone. A
|
||||
reader can tell a rule whose timestamps a collector files shifted (NetOrk/netork#493).
|
||||
|
||||
### Fixed
|
||||
- `apply_dhcp_reservationset` writes its changes so an address is released before it is
|
||||
taken: an update that moves a reservation off its address goes before the change that
|
||||
takes that address (along a chain, from its free end), and adds come after every
|
||||
update. A server that keeps addresses unique at each write, such as Kea, no longer
|
||||
refuses the second reservation as a duplicate when an address moves from one to
|
||||
another. A swap -- reservations trading addresses, or a longer rotation -- has no such
|
||||
order: its changes are still tried, and where the server refuses them the `[failed]`
|
||||
line says it was a swap and with which reservations. Everything else is still written
|
||||
and committed, and `DhcpApplyError` follows (#106).
|
||||
- `apply_dhcp_subnetset` no longer stops at the first subnet the device refuses. As
|
||||
`apply_dhcp_reservationset` has since 4.35.1, a refused subnet gets a `[failed]` line,
|
||||
the others are still written, any non-empty diff is committed -- even one whose every
|
||||
write was refused -- and `DhcpApplyError` follows. Before, the subnets written ahead of
|
||||
the refused one stayed saved but inactive, and since the device reports them as
|
||||
configured, no later apply committed them. `DhcpApplyError` takes an optional `kind`
|
||||
(`"reservation"` by default, `"subnet"` here) that its message names (#113).
|
||||
|
||||
## [4.37.0] - 2026-11-12
|
||||
|
||||
### Added
|
||||
|
||||
@@ -124,7 +124,10 @@ application wants made on a device runs as fixed steps the driver implements, wi
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
|
||||
`ZabbixAgentMixin.configure_zabbix_agent(settings)` (install from the host's packages or
|
||||
else Zabbix's repository, configure, firewall, enable and restart; the driver writes the
|
||||
configuration and opens the firewall), `EnsurePackageMixin.ensure_package(name)`,
|
||||
`PackageProxyMixin.configure_package_proxy(url)`,
|
||||
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
@@ -172,6 +175,15 @@ wins (`get_package_proxy`, `get_package_proxy_report` with every line that sets
|
||||
different proxy, reads the result back and fetches one http source's `InRelease` through it.
|
||||
A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_file`.
|
||||
|
||||
`AptReleaseMixin` (`napalm_device_types.apt_releases`) comes with it: every source apt reads,
|
||||
with its file, line, URI and suites, and the host's release from `/etc/os-release`
|
||||
(`get_apt_source_releases`). A release upgrade leaves third-party sources on the old release,
|
||||
where they stop getting updates once the vendor stops building for it; whether a suite names
|
||||
another release than the host's is the caller's to judge (`suite_release` gives the word).
|
||||
`set_apt_source_release(path, release, codename)` moves every suite of *release* in one
|
||||
source file to *codename*, after fetching each new suite's `InRelease` with apt's downloader:
|
||||
where the vendor has none, the file stays as it is.
|
||||
|
||||
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
|
||||
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
|
||||
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
|
||||
@@ -286,6 +298,9 @@ O.E.M.`, `Default string`, all-zero or the common `03000200-…-000700080009` UU
|
||||
empty: matching on them would join unrelated machines. `is_placeholder_identity` is the
|
||||
same filter for a BMC driver's side. `product_uuid` and `product_serial` are root's, so a
|
||||
driver supplies `_run_system_identity_command(command, privileged=)`.
|
||||
The same round trip reads the host's system firmware (BIOS or UEFI): `bios_vendor`,
|
||||
`bios_version` and `bios_date` (`YYYY-MM-DD`, from sysfs's `MM/DD/YYYY`). Only the vendors'
|
||||
text defaults read as empty there; a short version such as `1` is a version.
|
||||
|
||||
`OpenSshServerMixin` (`harden_ssh_server`, `get_ssh_server_algorithms`) takes algorithms
|
||||
out of what an OpenSSH server offers -- the ones ssh-audit flags. It reads the lists sshd
|
||||
@@ -374,6 +389,23 @@ that 4.35.0 commented out comes back on the next write -- and systemd-timesyncd'
|
||||
`LinkNTPServers` stay. ntpd and OpenNTPD are read, not written; what DHCP gives them is
|
||||
unknown.
|
||||
|
||||
`ConfigRevisionsMixin` (`get_config_revisions`, `get_config_revision`,
|
||||
`revert_config_revision`) is a device's own configuration history: a copy of the
|
||||
configuration for every save, which the device can go back to. An application restores a
|
||||
configuration there by reverting to the revision that holds it -- on OPNsense the only way,
|
||||
since its API takes no uploaded config.xml (NetOrk/netork#2511). Listing (newest first, the
|
||||
first in place now: id, UTC time, description, user, size as `ConfigRevisionDict`) and
|
||||
reading one revision are declared only. `revert_config_revision(revision_id)` is a
|
||||
template: it refuses an id that is not in the history (`UnknownConfigRevision`, a
|
||||
`ValueError`) before anything changes, reads the revision -- before the revert, which may
|
||||
rotate it out -- lets the driver revert (`_revert_to_config_revision`), and checks that the
|
||||
newest revision afterwards holds the revision's configuration (`ConfigRevertFailed`
|
||||
otherwise: a device may say it reverted when it did not). It returns that newest revision.
|
||||
`config_revision_content(text)` is what of a revision is its configuration; a driver whose
|
||||
device stamps every save (who, when) strips the stamp there, and an application that looks
|
||||
for the revision holding a configuration compares through it too. A driver that can list
|
||||
but not revert sets `revert_config_revision = None`.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
@@ -383,10 +415,11 @@ unknown.
|
||||
The mixin sits in `DeviceTypeDriver` and only declares, so `hasattr(driver, "run_command")` is true exactly where a driver implements it. For SSH drivers, `channel.run_on_transport` and `channel.open_stream_on_transport` are the implementation over a paramiko exec channel: no PTY, stderr kept apart, a real exit code. How a command gains root stays with the driver.
|
||||
|
||||
**`ContainerEngineMixin` is deliberately different from `SystemdServicesMixin`.** The systemd mixin owns its command and its parse; this one owns neither.
|
||||
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}]`).
|
||||
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}, {"engine": "podman", "api": "podman-libpod"}]`). An engine is listed when its CLI answers `--version` with its own name; podman-docker's `docker` says it is podman and is left out.
|
||||
- `open_container_engine(engine)` returns a `ContainerEngineConnection`:
|
||||
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`);
|
||||
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`, `podman system dial-stdio`);
|
||||
- `run_cli(args)` and `stream_cli(args)` run the engine's CLI with arguments the caller chooses. With `privileged=True` the driver runs it the way `run_command` gains root, for an engine that refuses the login user.
|
||||
- **Podman is reached as root** (`ROOT_ENGINES`, `connection.privileged`): the host's rootful Podman, the engine of the system's containers and of Quadlet's units, whose API `podman.socket` serves. A login user's rootless containers are that user's own and invisible to others. `podman system dial-stdio` exists from Podman 4.0 on and needs the socket listening (`systemctl enable --now podman.socket`) (NetOrk/netork#818).
|
||||
- The driver decides only *how* the engine is reached. Its hook `_container_engine_binary(engine)` returns, for QNAP, the Container Station path, and the caller never sees it.
|
||||
|
||||
**What runs on the engine, and what to do with it, is netOrk's** (NetOrk/netork#765): the container model, the Engine API requests, compose, updates. Above the connection everything is specific to the service, so this package abstracts the connection and nothing more.
|
||||
@@ -493,7 +526,10 @@ The same split applies to `DhcpServerMixin`: `get_dhcp_reservations`/
|
||||
`apply_dhcp_reservation`/`commit_dhcp_reservations` are abstract (Kea REST on
|
||||
OPNsense, dnsmasq/odhcpd UCI on OpenWrt), while `diff_dhcp_reservations` and
|
||||
`apply_dhcp_reservationset` are concrete — matching by normalised MAC and the
|
||||
apply-then-commit orchestration are identical for every DHCP server.
|
||||
apply-then-commit orchestration are identical for every DHCP server. The apply writes
|
||||
its changes so an address is released before another reservation takes it (an update
|
||||
that frees an address first, adds last); a swap of addresses has no such order and is
|
||||
reported as one where the server refuses it.
|
||||
|
||||
A new driver (FortiGate, pfSense, …) gets `diff_firewall_rules`/
|
||||
`apply_firewall_ruleset` for free the moment it implements the three abstract
|
||||
|
||||
+4
-1
@@ -39,7 +39,7 @@ application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `_dhcp_order`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
@@ -78,8 +78,10 @@ The same shape replaces every action in today's `run_device_action`:
|
||||
|---|---|
|
||||
| `fix_snmp` | `configure_snmp_agent(settings)` |
|
||||
| `fix_apt_proxy` | `configure_package_proxy(url)` |
|
||||
| an apt source on another release than the host (NetOrk/netork#2614) | `set_apt_source_release(path, release, codename)` |
|
||||
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
|
||||
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
|
||||
| the Zabbix agent where no container engine runs (NetOrk/netork#2571) | `configure_zabbix_agent(settings)` |
|
||||
|
||||
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
|
||||
driver that inherits a template it cannot run on its device, as a NAS driver built on the
|
||||
@@ -158,6 +160,7 @@ the driver.
|
||||
| `virtual_ips` | vip-manager, keepalived, awk |
|
||||
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
|
||||
| `package_updates` | apt and dnf output |
|
||||
| `apt_releases` | apt's source files (one-line and deb822), `/etc/os-release`, `apt-helper` |
|
||||
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
|
||||
| `terminal` | ANSI escape codes in shell output |
|
||||
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
|
||||
|
||||
@@ -38,6 +38,7 @@ instead of being restated on every role that happens to need it:
|
||||
|
||||
* :class:`~napalm_device_types.channel.CommandChannelMixin`
|
||||
* :class:`~napalm_device_types.config_lifecycle.ConfigLifecycleMixin`
|
||||
* :class:`~napalm_device_types.config_revisions.ConfigRevisionsMixin`
|
||||
* :class:`~napalm_device_types.config_sync.ConfigSyncMixin`
|
||||
* :class:`~napalm_device_types.container_engine.ContainerEngineMixin`
|
||||
* :class:`~napalm_device_types.dhcp.DhcpServerMixin`
|
||||
@@ -85,6 +86,15 @@ from napalm_device_types.apt_proxy import (
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.apt_releases import (
|
||||
APT_RELEASES_COMMAND,
|
||||
AptReleaseFixDict,
|
||||
AptReleaseMixin,
|
||||
AptReleasesDict,
|
||||
AptSourceDict,
|
||||
parse_apt_releases,
|
||||
suite_release,
|
||||
)
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.backup_server import BackupServerDriver
|
||||
from napalm_device_types.backups import (
|
||||
@@ -121,6 +131,12 @@ from napalm_device_types.clusters import (
|
||||
HAResourceDict,
|
||||
)
|
||||
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
|
||||
from napalm_device_types.config_revisions import (
|
||||
ConfigRevertFailed,
|
||||
ConfigRevisionDict,
|
||||
ConfigRevisionsMixin,
|
||||
UnknownConfigRevision,
|
||||
)
|
||||
from napalm_device_types.config_sync import ConfigSyncMixin
|
||||
from napalm_device_types.container_engine import (
|
||||
ContainerEngineConnection,
|
||||
@@ -131,12 +147,13 @@ from napalm_device_types.dhcp import DhcpApplyError, DhcpServerMixin, normalize_
|
||||
from napalm_device_types.dns_forward import DnsForwardMixin
|
||||
from napalm_device_types.firewall import FirewallDriver
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
from napalm_device_types.firmware import FirmwareOfferMixin
|
||||
from napalm_device_types.firmware import FirmwareOfferDict, FirmwareOfferMixin, FirmwareUpgradeMixin
|
||||
from napalm_device_types.fixes import (
|
||||
EnsurePackageMixin,
|
||||
LldpAgentMixin,
|
||||
PackageProxyMixin,
|
||||
SnmpAgentMixin,
|
||||
ZabbixAgentMixin,
|
||||
run_fix_steps,
|
||||
)
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
@@ -197,7 +214,6 @@ from napalm_device_types.models import (
|
||||
DnsForwardDict,
|
||||
EventLogEntryDict,
|
||||
FirmwareDict,
|
||||
FirmwareOfferDict,
|
||||
FixResultDict,
|
||||
FixStepDict,
|
||||
FixStepOutcomeDict,
|
||||
@@ -328,7 +344,7 @@ from napalm_device_types.time_sync import (
|
||||
check_ntp_servers,
|
||||
)
|
||||
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.updates import PackageIndexMissing, UpdateMixin
|
||||
from napalm_device_types.virtual_ips import (
|
||||
VIRTUAL_IPS_COMMAND,
|
||||
VirtualIpsMixin,
|
||||
@@ -336,6 +352,12 @@ from napalm_device_types.virtual_ips import (
|
||||
)
|
||||
from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
from napalm_device_types.vm_stop_request import VmStopRequestMixin
|
||||
from napalm_device_types.zabbix_agent import (
|
||||
ZabbixAgentSettingsDict,
|
||||
check_zabbix_agent_settings,
|
||||
zabbix_agent_config,
|
||||
zabbix_repository_command,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"BACKUP_JOB_KINDS",
|
||||
@@ -368,6 +390,7 @@ __all__ = [
|
||||
"OspfMixin",
|
||||
"UpdateProgressDict",
|
||||
"FirmwareOfferMixin",
|
||||
"FirmwareUpgradeMixin",
|
||||
"LicenseDict",
|
||||
"LicenseMixin",
|
||||
"EnsurePackageMixin",
|
||||
@@ -382,9 +405,21 @@ __all__ = [
|
||||
"parse_apt_fetch",
|
||||
"parse_apt_proxy",
|
||||
"same_proxy_url",
|
||||
"APT_RELEASES_COMMAND",
|
||||
"AptReleaseFixDict",
|
||||
"AptReleaseMixin",
|
||||
"AptReleasesDict",
|
||||
"AptSourceDict",
|
||||
"parse_apt_releases",
|
||||
"suite_release",
|
||||
"SnmpAgentMixin",
|
||||
"SnmpAgentSettingsDict",
|
||||
"run_fix_steps",
|
||||
"ZabbixAgentMixin",
|
||||
"ZabbixAgentSettingsDict",
|
||||
"check_zabbix_agent_settings",
|
||||
"zabbix_agent_config",
|
||||
"zabbix_repository_command",
|
||||
"SnmpHealthProfile",
|
||||
"SnmpInterfacesSpec",
|
||||
"SnmpMemorySpec",
|
||||
@@ -401,6 +436,10 @@ __all__ = [
|
||||
"open_stream_on_transport",
|
||||
"run_on_transport",
|
||||
"ConfigLifecycleMixin",
|
||||
"ConfigRevisionsMixin",
|
||||
"ConfigRevisionDict",
|
||||
"ConfigRevertFailed",
|
||||
"UnknownConfigRevision",
|
||||
"ConfigSyncDict",
|
||||
"ConfigSyncMixin",
|
||||
"DhcpHaDict",
|
||||
@@ -494,6 +533,7 @@ __all__ = [
|
||||
"SystemdServicesMixin",
|
||||
"SystemdUnavailable",
|
||||
"parse_systemd_services",
|
||||
"PackageIndexMissing",
|
||||
"UpdateMixin",
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""The order an applied reservation set is written in (#106).
|
||||
|
||||
A DHCP server such as Kea keeps every address unique at each write, not only at the
|
||||
commit: a reservation that takes an address another one still holds is refused as a
|
||||
duplicate. So the changes are ordered for an address to be released before it is taken:
|
||||
|
||||
* An update that moves a reservation off its address goes before the change that takes
|
||||
that address -- along a chain, from its free end.
|
||||
* Adds come after every update: they release nothing, and an update may free their
|
||||
address.
|
||||
|
||||
A *swap* -- two reservations trading addresses, or a longer rotation -- has no such
|
||||
order. Its changes are still tried, in the order of the desired set: a server that checks
|
||||
only at the commit takes them. A server that refuses them refuses each with its own
|
||||
message, and the progress line says it was a swap. Nothing is moved to a temporary
|
||||
address: the contract knows no free one (pools, leases and reservations nobody manages
|
||||
all hold addresses), and a second write that failed would leave the reservation on it.
|
||||
|
||||
Without I/O: a reconciler's part, see ``docs/STRUCTURE.md``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
from dataclasses import dataclass
|
||||
|
||||
from napalm_device_types.models import DhcpReservationDict, DhcpReservationDiffDict
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReservationChange:
|
||||
"""One write of an applied set, in the order it is made."""
|
||||
|
||||
reservation: DhcpReservationDict
|
||||
#: The live reservation's id for an update, None for an add.
|
||||
uuid: str | None
|
||||
#: The changed fields of an update (``"ip, hostname"``), empty for an add.
|
||||
fields: str = ""
|
||||
#: The other reservations' MACs of the swap this change is part of, if any.
|
||||
swap: tuple[str, ...] = ()
|
||||
|
||||
|
||||
def _address(ip: str | None) -> str:
|
||||
"""*ip* in one spelling, so ``192.0.2.010``-style differences cannot hide a match."""
|
||||
text = (ip or "").strip()
|
||||
try:
|
||||
return str(ipaddress.ip_address(text))
|
||||
except ValueError:
|
||||
return text.lower()
|
||||
|
||||
|
||||
def _waits_on(updates: list[tuple[str, str]]) -> list[int | None]:
|
||||
"""For each ``(old, new)`` address pair, the update whose release it waits for."""
|
||||
released = {old: i for i, (old, new) in enumerate(updates) if old and old != new}
|
||||
return [released.get(new) if new != old else None for old, new in updates]
|
||||
|
||||
|
||||
def _cycles(waits_on: list[int | None]) -> dict[int, list[int]]:
|
||||
"""Each update that is part of a cycle, with the cycle's members in order."""
|
||||
members: dict[int, list[int]] = {}
|
||||
for start in range(len(waits_on)):
|
||||
path: list[int] = []
|
||||
node: int | None = start
|
||||
while node is not None and node not in path and node not in members:
|
||||
path.append(node)
|
||||
node = waits_on[node]
|
||||
if node is not None and node in path:
|
||||
cycle = sorted(path[path.index(node) :])
|
||||
members.update(dict.fromkeys(cycle, cycle))
|
||||
return members
|
||||
|
||||
|
||||
def _update_order(waits_on: list[int | None], cycles: dict[int, list[int]]) -> list[int]:
|
||||
"""Every update after the one it waits on; stable otherwise. Where only cycles are
|
||||
left, the first of them goes as it is, and so on."""
|
||||
order: list[int] = []
|
||||
done: set[int] = set()
|
||||
pending = list(range(len(waits_on)))
|
||||
while pending:
|
||||
ready = [i for i in pending if waits_on[i] is None or waits_on[i] in done]
|
||||
if not ready:
|
||||
ready = [next(i for i in pending if i in cycles)]
|
||||
order += ready
|
||||
done.update(ready)
|
||||
pending = [i for i in pending if i not in done]
|
||||
return order
|
||||
|
||||
|
||||
def order_reservation_changes(
|
||||
diff: DhcpReservationDiffDict, live: list[DhcpReservationDict]
|
||||
) -> list[ReservationChange]:
|
||||
"""The changes of *diff* in the order they are written; see the module.
|
||||
|
||||
:param diff: What ``diff_dhcp_reservations`` found.
|
||||
:param live: The reservations it compared against, for the addresses updates release.
|
||||
"""
|
||||
held = {r.get("uuid"): _address(r.get("ip")) for r in live if r.get("uuid")}
|
||||
updates = diff["update"]
|
||||
waits_on = _waits_on(
|
||||
[(held.get(e["uuid"], ""), _address(e["reservation"].get("ip"))) for e in updates]
|
||||
)
|
||||
cycles = _cycles(waits_on)
|
||||
|
||||
def swap(i: int) -> tuple[str, ...]:
|
||||
return tuple(updates[j]["reservation"]["mac"] for j in cycles.get(i, []) if j != i)
|
||||
|
||||
ordered = [
|
||||
ReservationChange(
|
||||
updates[i]["reservation"],
|
||||
updates[i]["uuid"],
|
||||
", ".join(updates[i]["changed_fields"]),
|
||||
swap(i),
|
||||
)
|
||||
for i in _update_order(waits_on, cycles)
|
||||
]
|
||||
return ordered + [ReservationChange(r, None) for r in diff["add"]]
|
||||
@@ -35,8 +35,12 @@ page is the right one, the caller says (``expect``).
|
||||
out every other line that sets a different one, reads the result back from
|
||||
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
|
||||
last step is what proves it: apt reached a mirror through the proxy it is now set
|
||||
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
|
||||
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
|
||||
to. The source chosen is one apt does not fetch ``DIRECT``. Then the https sources
|
||||
go through the proxy in the form it stores (:mod:`~napalm_device_types.apt_sources`),
|
||||
and one of them is fetched; where the proxy cannot, they go back to https. Last it
|
||||
fetches one https source left the same way; where the proxy refuses it, https goes
|
||||
``DIRECT``. Removing the proxy turns the sources back to https first: without the
|
||||
proxy apt cannot fetch the stored form.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -49,6 +53,15 @@ from urllib.parse import urlsplit
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.apt_releases import AptReleaseMixin
|
||||
from napalm_device_types.apt_sources import (
|
||||
NOT_CACHED_KEY,
|
||||
VIA_CACHE,
|
||||
cacheable_https_hosts,
|
||||
is_host_name,
|
||||
revert_sources_command,
|
||||
rewrite_sources_command,
|
||||
)
|
||||
from napalm_device_types.fixes import PackageProxyMixin, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
@@ -87,6 +100,9 @@ class AptProxyReportDict(TypedDict):
|
||||
#: The base URIs of the host's sources over http and over https.
|
||||
http_sources: list[str]
|
||||
https_sources: list[str]
|
||||
#: The hosts the cache rewrite left on https, for their credentials
|
||||
#: (:data:`~napalm_device_types.apt_sources.NOT_CACHED_KEY`).
|
||||
not_cached: list[str]
|
||||
|
||||
|
||||
class ProxyFetchDict(TypedDict):
|
||||
@@ -123,7 +139,8 @@ PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
|
||||
#: them early.
|
||||
APT_PROXY_COMMAND = (
|
||||
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
|
||||
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
|
||||
"echo '[dump]'; apt-config dump 2>/dev/null "
|
||||
"| grep -i -E '^(Acquire::https?::Proxy|Package-Proxy::)'; "
|
||||
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
|
||||
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
|
||||
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
|
||||
@@ -133,6 +150,7 @@ APT_PROXY_COMMAND = (
|
||||
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
|
||||
)
|
||||
|
||||
_NOT_CACHED = re.compile(rf'^{re.escape(NOT_CACHED_KEY)}::\s+"([^"]+)"', re.I)
|
||||
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
|
||||
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
|
||||
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
|
||||
@@ -293,6 +311,7 @@ def parse_apt_proxy(output: str) -> AptProxyReportDict:
|
||||
"https_settings": _settings(lines, _https_setting),
|
||||
"http_sources": _of_scheme(sources, "http"),
|
||||
"https_sources": _of_scheme(sources, "https"),
|
||||
"not_cached": [m.group(1) for m in map(_NOT_CACHED.match, sections.get("dump", [])) if m],
|
||||
}
|
||||
|
||||
|
||||
@@ -409,7 +428,7 @@ def _described(s: AptProxySettingDict) -> str:
|
||||
return f"{s['file']}:{s['line']} ({s['value']}) {what}"
|
||||
|
||||
|
||||
class AptProxyMixin(PackageProxyMixin):
|
||||
class AptProxyMixin(AptReleaseMixin, PackageProxyMixin):
|
||||
""":class:`~napalm_device_types.fixes.PackageProxyMixin` for a Debian-family host.
|
||||
|
||||
The template form, like :class:`~napalm_device_types.host_status.HostStatusMixin`:
|
||||
@@ -498,16 +517,20 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
def configure_package_proxy(self, url: str) -> FixResultDict:
|
||||
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
|
||||
|
||||
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
|
||||
The https sources go through *url* in the form it stores, where it can; the
|
||||
rest goes through *url* too, unless *url* refuses it: then ``DIRECT``.
|
||||
"""
|
||||
if not url:
|
||||
return run_fix_steps([("configure", self._remove_package_proxy)])
|
||||
return run_fix_steps(
|
||||
[("sources", self._revert_sources), ("configure", self._remove_package_proxy)]
|
||||
)
|
||||
return run_fix_steps(
|
||||
[
|
||||
("configure", lambda: self._write_package_proxy(url)),
|
||||
("conflicts", lambda: self._comment_out_other_proxies(url)),
|
||||
("verify", lambda: self._verify_package_proxy(url)),
|
||||
("fetch", lambda: self._fetch_through_package_proxy(url)),
|
||||
("cache", lambda: self._cache_https_sources(url)),
|
||||
("https", lambda: self._https_through_package_proxy(url)),
|
||||
]
|
||||
)
|
||||
@@ -515,6 +538,11 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
def _not_apt(self) -> FixStepOutcomeDict:
|
||||
return {"ok": False, "output": "not an apt host"}
|
||||
|
||||
def _revert_sources(self) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
return self._run_apt_root_command(revert_sources_command(), "sources back on https")
|
||||
|
||||
def _remove_package_proxy(self) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
@@ -522,7 +550,9 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
|
||||
)
|
||||
|
||||
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
|
||||
def _write_package_proxy(
|
||||
self, url: str, https_direct: bool = False, not_cached: list[str] | None = None
|
||||
) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
if not PROXY_URL.match(url):
|
||||
@@ -530,6 +560,10 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
content = f'Acquire::http::Proxy "{url}";\n'
|
||||
if https_direct:
|
||||
content += 'Acquire::https::Proxy "DIRECT";\n'
|
||||
# Host names the report read from this file; one a line could not carry is left out.
|
||||
content += "".join(
|
||||
f'{NOT_CACHED_KEY}:: "{host}";\n' for host in not_cached or [] if is_host_name(host)
|
||||
)
|
||||
return self._write_apt_file(self.APT_PROXY_FILE, content)
|
||||
|
||||
def _report(self) -> AptProxyReportDict:
|
||||
@@ -566,16 +600,7 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
}
|
||||
|
||||
def _fetch_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
report = self._report()
|
||||
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
|
||||
source = next(
|
||||
(
|
||||
s
|
||||
for s in report["http_sources"]
|
||||
if "/dists/" in s and (urlsplit(s).hostname or "").lower() not in direct
|
||||
),
|
||||
None,
|
||||
)
|
||||
source = _plain_http_source(self._report())
|
||||
if source is None:
|
||||
return {"ok": True, "output": "no http source to fetch through the proxy; not tried"}
|
||||
target = source.rstrip("/") + "/InRelease"
|
||||
@@ -584,6 +609,49 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
return {"ok": True, "output": f"fetched {target} through {url}"}
|
||||
return {"ok": False, "output": f"{target}: {fetched['error']}"}
|
||||
|
||||
def _cache_https_sources(self, url: str) -> FixStepOutcomeDict:
|
||||
"""The https sources through *url* in the form it stores; back to https where it cannot."""
|
||||
hosts = cacheable_https_hosts(self._report())
|
||||
if not hosts:
|
||||
return {"ok": True, "output": "no https source to cache"}
|
||||
rewritten = self._run_apt_root_command(
|
||||
rewrite_sources_command(hosts, self.APT_PROXY_FILE), "sources rewritten"
|
||||
)
|
||||
if not rewritten["ok"]:
|
||||
return rewritten
|
||||
return self._fetch_cached(url, hosts, self._report())
|
||||
|
||||
def _fetch_cached(
|
||||
self, url: str, hosts: list[str], report: AptProxyReportDict
|
||||
) -> FixStepOutcomeDict:
|
||||
"""One rewritten source through *url*: it stays where *url* fetched it."""
|
||||
left = sorted(set(hosts) & set(report["not_cached"]))
|
||||
named = f"; left on https, apt has credentials for them: {', '.join(left)}" if left else ""
|
||||
target = next(
|
||||
(s.rstrip("/") + "/InRelease" for s in report["http_sources"] if _cached(s)), None
|
||||
)
|
||||
if target is None:
|
||||
return {"ok": True, "output": f"no https source to cache{named}"}
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
if fetched["reachable"] and not fetched["error"]:
|
||||
cached = ", ".join(h for h in hosts if h not in left)
|
||||
return {"ok": True, "output": f"https sources cached through {url}: {cached}{named}"}
|
||||
return self._back_to_https(url, f"{target}: {fetched['error']}", named)
|
||||
|
||||
def _back_to_https(self, url: str, why: str, named: str) -> FixStepOutcomeDict:
|
||||
"""*url* did not fetch the stored form: every source goes back to https."""
|
||||
back = self._run_apt_root_command(revert_sources_command(), "sources back on https")
|
||||
if not back["ok"]:
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"{why}; turning the sources back to https failed: {back['output']}",
|
||||
}
|
||||
return {
|
||||
"ok": True,
|
||||
"output": f"{url} does not fetch https sources itself ({why}); "
|
||||
f"they stay on https, not cached{named}",
|
||||
}
|
||||
|
||||
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
|
||||
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
|
||||
@@ -609,7 +677,9 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
|
||||
def _https_direct(self, url: str) -> FixStepOutcomeDict:
|
||||
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
|
||||
written = self._write_package_proxy(url, https_direct=True)
|
||||
written = self._write_package_proxy(
|
||||
url, https_direct=True, not_cached=self._report()["not_cached"]
|
||||
)
|
||||
if not written["ok"]:
|
||||
return written
|
||||
later = self._later_https_lines(self._report(), url)
|
||||
@@ -635,6 +705,26 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
]
|
||||
|
||||
|
||||
def _plain_http_source(report: AptProxyReportDict) -> str | None:
|
||||
"""A suite's http source the proxy takes, not in the cache's https form: its proof."""
|
||||
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
|
||||
return next(
|
||||
(
|
||||
s
|
||||
for s in report["http_sources"]
|
||||
if "/dists/" in s
|
||||
and not s.startswith(VIA_CACHE)
|
||||
and (urlsplit(s).hostname or "").lower() not in direct
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _cached(source: str) -> bool:
|
||||
"""A source of a suite in the form the cache stores."""
|
||||
return source.startswith(VIA_CACHE) and "/dists/" in source
|
||||
|
||||
|
||||
def _https_source(
|
||||
report: AptProxyReportDict, through: Callable[[str | None], bool]
|
||||
) -> tuple[str, str] | None:
|
||||
|
||||
@@ -0,0 +1,486 @@
|
||||
"""apt's sources and the release each names, and moving one file's sources to the host's.
|
||||
|
||||
A release upgrade moves the distribution's own sources to the new release and
|
||||
leaves the third-party ones where they were -- or turns them off, and the person
|
||||
who turns them back on keeps the old name. A source on the old release still
|
||||
answers, so apt says nothing; its packages simply stop getting updates once the
|
||||
vendor stops building for that release. Docker's last build for Ubuntu 20.04
|
||||
(``focal``) is 28.1.1, and a noble host on ``focal`` never sees a newer one
|
||||
(NetOrk/netork#2614).
|
||||
|
||||
**What is read** (:meth:`AptReleaseMixin.get_apt_source_releases`): the host's
|
||||
``/etc/os-release`` (``ID``, ``ID_LIKE``, ``VERSION_CODENAME``,
|
||||
``UBUNTU_CODENAME``) and every source apt reads -- ``sources.list`` and the
|
||||
``*.list`` and ``*.sources`` files of ``sources.list.d``, wherever ``apt-config``
|
||||
says they are -- with its file, line, types, URI, suites and components.
|
||||
Commented-out lines and deb822 paragraphs with ``Enabled: no`` are not sources.
|
||||
Whether a suite names another release than the host's is not said here: which
|
||||
words are releases of which distribution is the caller's to know
|
||||
(:func:`suite_release` gives the word).
|
||||
|
||||
**Moving a file** (:meth:`AptReleaseMixin.set_apt_source_release`): every suite of
|
||||
*release* in one of apt's source files becomes the same suite of *codename*
|
||||
(``focal-security`` → ``noble-security``), on enabled lines only. Before anything is
|
||||
written, each new suite's ``InRelease`` (or ``Release``) is fetched from its URI with
|
||||
apt's own downloader and the proxy apt is set to; when one of them is not there,
|
||||
the vendor does not build for *codename* and the file stays as it is. The file is
|
||||
written whole, as root, mode 644. Loading the package index afterwards is the
|
||||
caller's (``refresh_available_updates``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "ARELEASE_BEGIN"
|
||||
_END = "ARELEASE_END"
|
||||
|
||||
#: A release's codename: one lower-case word (``noble``, ``bookworm``).
|
||||
_CODENAME = re.compile(r"^[a-z][a-z0-9]*$")
|
||||
#: A suite that names a release: its codename, then perhaps a suffix (``-security``).
|
||||
_SUITE = re.compile(r"^([a-z][a-z0-9]*)(-[a-z0-9-]+)?$")
|
||||
#: One source line of a ``.list`` file, its comment cut off: types, options, URI, suite.
|
||||
_LIST_LINE = re.compile(r"^(\s*(deb|deb-src)\s+(?:\[[^\]]*\]\s+)?(\S+)\s+)(\S+)(.*)$")
|
||||
#: A deb822 field's first line.
|
||||
_FIELD = re.compile(r"^([A-Za-z][A-Za-z0-9-]*):(.*)$")
|
||||
_DISABLED = frozenset({"no", "false", "0"})
|
||||
|
||||
|
||||
class AptSourceDict(TypedDict):
|
||||
"""One source apt reads (``get_apt_source_releases``)."""
|
||||
|
||||
#: The file it is in.
|
||||
file: str
|
||||
#: The line of a ``.list`` file's entry; of a deb822 paragraph's ``Suites:`` field.
|
||||
line: int
|
||||
#: ``deb``, ``deb-src``; a deb822 paragraph may name both.
|
||||
types: list[str]
|
||||
#: Its URI, as written; a deb822 paragraph with several URIs is one source per URI.
|
||||
uri: str
|
||||
#: Its suites; a flat repository's path (``./``) is one too.
|
||||
suites: list[str]
|
||||
components: list[str]
|
||||
|
||||
|
||||
class AptReleasesDict(TypedDict):
|
||||
"""The host's release and its sources (``get_apt_source_releases``)."""
|
||||
|
||||
#: ``ID`` and ``ID_LIKE`` of ``/etc/os-release``; "" and [] when it says none.
|
||||
os_id: str
|
||||
os_like: list[str]
|
||||
#: ``VERSION_CODENAME``; a derivative's own (Linux Mint's ``virginia``).
|
||||
codename: str
|
||||
#: ``UBUNTU_CODENAME``: the Ubuntu release Ubuntu and its derivatives are built on.
|
||||
ubuntu_codename: str
|
||||
sources: list[AptSourceDict]
|
||||
|
||||
|
||||
class AptReleaseFixDict(FixResultDict):
|
||||
"""The result of ``set_apt_source_release``: the fix, and how many suites moved."""
|
||||
|
||||
rewritten: int
|
||||
|
||||
|
||||
def apt_releases_command(
|
||||
apt_config: str = "apt-config", os_release: str = "/etc/os-release"
|
||||
) -> str:
|
||||
"""The read-only command that prints the host's release and apt's source files.
|
||||
|
||||
One line, POSIX ``sh``, no privileges. Every line of a file is printed behind a
|
||||
``|``, so an empty line -- which ends a deb822 paragraph -- comes through.
|
||||
*apt_config* and *os_release* let a test point it elsewhere.
|
||||
"""
|
||||
return (
|
||||
f"( export LC_ALL=C; printf '%s%s\\n' ARELEASE_ BEGIN; echo '@@os'; "
|
||||
f'o={shlex.quote(os_release)}; [ -r "$o" ] || o=/usr/lib/os-release; '
|
||||
'[ -r "$o" ] && sed \'s/^/|/\' "$o"; '
|
||||
f'eval "$({apt_config} shell sl Dir::Etc::sourcelist/f sp Dir::Etc::sourceparts/d '
|
||||
'2>/dev/null)"; sl="${sl:-/etc/apt/sources.list}"; sp="${sp:-/etc/apt/sources.list.d/}"; '
|
||||
'for f in "$sl" "$sp"*.list "$sp"*.sources; do [ -f "$f" ] || continue; '
|
||||
'if [ -r "$f" ]; then printf \'@@file %s\\n\' "$f"; sed \'s/^/|/\' "$f"; '
|
||||
"else printf '@@unreadable %s\\n' \"$f\"; fi; done; "
|
||||
"printf '%s%s\\n' ARELEASE_ END ) 2>/dev/null | cat"
|
||||
)
|
||||
|
||||
|
||||
#: The command as a driver runs it.
|
||||
APT_RELEASES_COMMAND = apt_releases_command()
|
||||
|
||||
|
||||
def suite_release(suite: str) -> str | None:
|
||||
"""The word a suite starts with, ``noble`` of ``noble-security``; None for a path.
|
||||
|
||||
Whether that word is a release, and of which distribution, the caller knows:
|
||||
``stable`` and ``nodistro`` have the form too.
|
||||
"""
|
||||
found = _SUITE.match(suite)
|
||||
return found.group(1) if found else None
|
||||
|
||||
|
||||
def _codename(name: str) -> str:
|
||||
if not _CODENAME.match(name):
|
||||
raise ValueError(f"not a release's codename: {name!r}")
|
||||
return name
|
||||
|
||||
|
||||
# ── Reading ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _blocks(output: str) -> list[tuple[str, list[str]]]:
|
||||
"""``(header, lines)`` of each block between the frame markers, in order."""
|
||||
lines = strip_terminal_codes(output).splitlines()
|
||||
stripped = [line.strip() for line in lines]
|
||||
try:
|
||||
start = stripped.index(_BEGIN)
|
||||
end = stripped.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact apt release report in the output") from None
|
||||
blocks: list[tuple[str, list[str]]] = []
|
||||
for line in lines[start + 1 : end]:
|
||||
if line.startswith("@@"):
|
||||
blocks.append((line[2:].strip(), []))
|
||||
elif line.startswith("|") and blocks:
|
||||
blocks[-1][1].append(line[1:].rstrip("\r"))
|
||||
return blocks
|
||||
|
||||
|
||||
def _os_release(lines: list[str]) -> dict[str, str]:
|
||||
values: dict[str, str] = {}
|
||||
for line in lines:
|
||||
key, sep, value = line.strip().partition("=")
|
||||
if sep and key:
|
||||
values[key] = value.strip().strip("'\"")
|
||||
return values
|
||||
|
||||
|
||||
def _code(line: str) -> str:
|
||||
"""*line* without its comment."""
|
||||
return line.split("#", 1)[0]
|
||||
|
||||
|
||||
def _list_sources(path: str, lines: list[str]) -> list[AptSourceDict]:
|
||||
found: list[AptSourceDict] = []
|
||||
for number, line in enumerate(lines, start=1):
|
||||
entry = _LIST_LINE.match(_code(line))
|
||||
if entry:
|
||||
_, kind, uri, suite, rest = entry.groups()
|
||||
found.append(
|
||||
{
|
||||
"file": path,
|
||||
"line": number,
|
||||
"types": [kind],
|
||||
"uri": uri,
|
||||
"suites": [suite],
|
||||
"components": rest.split(),
|
||||
}
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Field:
|
||||
first: int # index of its first line
|
||||
lines: list[int] = field(default_factory=list) # indices of every line it spans
|
||||
value: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Paragraph:
|
||||
fields: dict[str, _Field] = field(default_factory=dict)
|
||||
|
||||
def value(self, name: str) -> str:
|
||||
found = self.fields.get(name)
|
||||
return found.value if found else ""
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self.value("enabled").strip().lower() not in _DISABLED
|
||||
|
||||
|
||||
def _paragraphs(lines: list[str]) -> list[_Paragraph]:
|
||||
"""The deb822 paragraphs of *lines*; comments are skipped, an empty line ends one."""
|
||||
paragraphs: list[_Paragraph] = []
|
||||
current: _Paragraph | None = None
|
||||
last: _Field | None = None
|
||||
for index, line in enumerate(lines):
|
||||
if not line.strip():
|
||||
current, last = None, None
|
||||
continue
|
||||
if line.lstrip().startswith("#"):
|
||||
continue
|
||||
if line[0] in " \t":
|
||||
if last is not None:
|
||||
last.lines.append(index)
|
||||
last.value += " " + line.strip()
|
||||
continue
|
||||
found = _FIELD.match(line)
|
||||
if not found:
|
||||
continue
|
||||
if current is None:
|
||||
current = _Paragraph()
|
||||
paragraphs.append(current)
|
||||
last = _Field(first=index, lines=[index], value=found.group(2).strip())
|
||||
current.fields[found.group(1).lower()] = last
|
||||
return paragraphs
|
||||
|
||||
|
||||
def _deb822_sources(path: str, lines: list[str]) -> list[AptSourceDict]:
|
||||
found: list[AptSourceDict] = []
|
||||
for paragraph in _paragraphs(lines):
|
||||
suites = paragraph.fields.get("suites")
|
||||
if not paragraph.enabled or suites is None:
|
||||
continue
|
||||
for uri in paragraph.value("uris").split():
|
||||
found.append(
|
||||
{
|
||||
"file": path,
|
||||
"line": suites.first + 1,
|
||||
"types": paragraph.value("types").split(),
|
||||
"uri": uri,
|
||||
"suites": suites.value.split(),
|
||||
"components": paragraph.value("components").split(),
|
||||
}
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
def _is_deb822(path: str) -> bool:
|
||||
return path.endswith(".sources")
|
||||
|
||||
|
||||
def parse_apt_releases(output: str) -> AptReleasesDict:
|
||||
"""What :func:`apt_releases_command` printed, as :class:`AptReleasesDict`.
|
||||
|
||||
:raises ValueError: if the output carried no intact report.
|
||||
"""
|
||||
os_values: dict[str, str] = {}
|
||||
sources: list[AptSourceDict] = []
|
||||
for header, lines in _blocks(output):
|
||||
kind, _, path = header.partition(" ")
|
||||
if kind == "os":
|
||||
os_values = _os_release(lines)
|
||||
elif kind == "file" and path:
|
||||
read = _deb822_sources if _is_deb822(path) else _list_sources
|
||||
sources.extend(read(path, lines))
|
||||
return {
|
||||
"os_id": os_values.get("ID", "").lower(),
|
||||
"os_like": os_values.get("ID_LIKE", "").lower().split(),
|
||||
"codename": os_values.get("VERSION_CODENAME", "").lower(),
|
||||
"ubuntu_codename": os_values.get("UBUNTU_CODENAME", "").lower(),
|
||||
"sources": sources,
|
||||
}
|
||||
|
||||
|
||||
# ── Rewriting ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _moved(suite: str, release: str, codename: str) -> str:
|
||||
return codename + suite[len(release) :] if suite_release(suite) == release else suite
|
||||
|
||||
|
||||
def _rewrite_list_line(line: str, release: str, codename: str) -> tuple[str, int]:
|
||||
code = _code(line)
|
||||
entry = _LIST_LINE.match(code)
|
||||
if not entry:
|
||||
return line, 0
|
||||
suite = entry.group(4)
|
||||
new = _moved(suite, release, codename)
|
||||
if new == suite:
|
||||
return line, 0
|
||||
head = entry.group(1)
|
||||
return head + new + line[len(head) + len(suite) :], 1
|
||||
|
||||
|
||||
def _rewrite_tokens(text: str, release: str, codename: str) -> tuple[str, int]:
|
||||
changed = 0
|
||||
|
||||
def one(token: re.Match[str]) -> str:
|
||||
nonlocal changed
|
||||
new = _moved(token.group(), release, codename)
|
||||
changed += new != token.group()
|
||||
return new
|
||||
|
||||
return re.sub(r"\S+", one, text), changed
|
||||
|
||||
|
||||
def _rewrite_deb822(lines: list[str], release: str, codename: str) -> int:
|
||||
changed = 0
|
||||
for paragraph in _paragraphs(lines):
|
||||
suites = paragraph.fields.get("suites")
|
||||
if not paragraph.enabled or suites is None:
|
||||
continue
|
||||
for index in suites.lines:
|
||||
line = lines[index]
|
||||
cut = line.index(":") + 1 if index == suites.first else 0
|
||||
tail, count = _rewrite_tokens(line[cut:], release, codename)
|
||||
lines[index] = line[:cut] + tail
|
||||
changed += count
|
||||
return changed
|
||||
|
||||
|
||||
def rewrite_release(text: str, release: str, codename: str, *, deb822: bool) -> tuple[str, int]:
|
||||
"""*text*, a source file, with every suite of *release* on *codename*; and how many.
|
||||
|
||||
Only enabled sources: not a commented-out line, not a paragraph with
|
||||
``Enabled: no``. A URI or comment that names *release* stays as it is.
|
||||
|
||||
:raises ValueError: if *release* or *codename* is not a codename.
|
||||
"""
|
||||
release, codename = _codename(release), _codename(codename)
|
||||
lines = text.split("\n")
|
||||
if deb822:
|
||||
changed = _rewrite_deb822(lines, release, codename)
|
||||
else:
|
||||
changed = 0
|
||||
for index, line in enumerate(lines):
|
||||
lines[index], count = _rewrite_list_line(line, release, codename)
|
||||
changed += count
|
||||
return "\n".join(lines), changed
|
||||
|
||||
|
||||
# ── The mixin ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _release_urls(uri: str, suite: str) -> list[str]:
|
||||
base = f"{uri.rstrip('/')}/dists/{suite}/"
|
||||
return [base + "InRelease", base + "Release"]
|
||||
|
||||
|
||||
class AptReleaseMixin:
|
||||
"""Reads the release each apt source names; moves one file's sources to another.
|
||||
|
||||
The template form, like :class:`~napalm_device_types.apt_proxy.AptProxyMixin`,
|
||||
which brings it to every driver that has it: a driver supplies
|
||||
``_run_apt_command`` and ``_write_apt_file``, and ``_is_apt_host`` where it
|
||||
can be something else.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_apt_command(self, command: str) -> str:
|
||||
"""Run *command* on the host with ``sh``, without privileges; what it printed."""
|
||||
...
|
||||
|
||||
def _write_apt_file(self, path: str, content: str) -> FixStepOutcomeDict:
|
||||
"""Replace *path* with *content*, as a new file owned by root, mode 644."""
|
||||
...
|
||||
|
||||
def _is_apt_host(self) -> bool:
|
||||
"""Whether apt is the host's package manager; a driver that can tell overrides it."""
|
||||
return True
|
||||
|
||||
def get_apt_source_releases(self) -> AptReleasesDict | None:
|
||||
"""The host's release and every source apt reads; None on a host without apt.
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
if not self._is_apt_host():
|
||||
return None
|
||||
return parse_apt_releases(self._run_apt_command(APT_RELEASES_COMMAND))
|
||||
|
||||
def set_apt_source_release(self, path: str, release: str, codename: str) -> AptReleaseFixDict:
|
||||
"""Every suite of *release* in *path* on *codename*, where the vendor serves it.
|
||||
|
||||
Three steps: ``read`` (*path* is one of apt's source files, and which of its
|
||||
sources name *release*), ``check`` (each new suite's ``InRelease`` or
|
||||
``Release`` is there) and ``rewrite``. Nothing is written when a step fails;
|
||||
nothing is fetched or written when no source of *path* names *release*.
|
||||
"""
|
||||
moving: list[AptSourceDict] = []
|
||||
rewritten = [0]
|
||||
result = run_fix_steps(
|
||||
[
|
||||
("read", lambda: self._sources_on(path, release, codename, moving)),
|
||||
("check", lambda: self._served(moving, release, codename)),
|
||||
("rewrite", lambda: self._rewrite(path, release, codename, moving, rewritten)),
|
||||
]
|
||||
)
|
||||
return {**result, "rewritten": rewritten[0]}
|
||||
|
||||
def _sources_on(
|
||||
self, path: str, release: str, codename: str, moving: list[AptSourceDict]
|
||||
) -> FixStepOutcomeDict:
|
||||
_codename(release)
|
||||
_codename(codename)
|
||||
found = self.get_apt_source_releases()
|
||||
if found is None:
|
||||
return {"ok": False, "output": "not an apt host"}
|
||||
in_file = [s for s in found["sources"] if s["file"] == path]
|
||||
if not in_file:
|
||||
return {"ok": False, "output": f"{path} is none of apt's source files"}
|
||||
moving.extend(s for s in in_file if any(suite_release(x) == release for x in s["suites"]))
|
||||
if not moving:
|
||||
return {"ok": True, "output": f"no source in {path} is on {release} any more"}
|
||||
named = "; ".join(f"{s['uri']} {' '.join(s['suites'])}" for s in moving)
|
||||
return {"ok": True, "output": f"on {release} in {path}: {named}"}
|
||||
|
||||
def _served(
|
||||
self, moving: list[AptSourceDict], release: str, codename: str
|
||||
) -> FixStepOutcomeDict:
|
||||
from napalm_device_types.apt_proxy import apt_fetch_command, parse_apt_fetch
|
||||
|
||||
wanted = sorted(
|
||||
{
|
||||
(s["uri"], _moved(suite, release, codename))
|
||||
for s in moving
|
||||
for suite in s["suites"]
|
||||
if suite_release(suite) == release
|
||||
}
|
||||
)
|
||||
said: list[str] = []
|
||||
for uri, suite in wanted:
|
||||
error = ""
|
||||
for url in _release_urls(uri, suite):
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(url)))
|
||||
error = fetched["error"]
|
||||
if not error:
|
||||
said.append(f"{uri} serves {suite}")
|
||||
break
|
||||
else:
|
||||
return {"ok": False, "output": f"{uri} has no {suite}: {error}; nothing changed"}
|
||||
return {"ok": True, "output": "; ".join(said) or "nothing to check"}
|
||||
|
||||
def _rewrite(
|
||||
self,
|
||||
path: str,
|
||||
release: str,
|
||||
codename: str,
|
||||
moving: list[AptSourceDict],
|
||||
rewritten: list[int],
|
||||
) -> FixStepOutcomeDict:
|
||||
if not moving:
|
||||
return {"ok": True, "output": "nothing to rewrite"}
|
||||
encoded = "".join(self._run_apt_command(f"base64 {shlex.quote(path)}").split())
|
||||
text = base64.b64decode(encoded).decode()
|
||||
new, count = rewrite_release(text, release, codename, deb822=_is_deb822(path))
|
||||
if not count:
|
||||
return {"ok": True, "output": f"no source in {path} is on {release} any more"}
|
||||
written = self._write_apt_file(path, new)
|
||||
if not written["ok"]:
|
||||
return written
|
||||
rewritten[0] = count
|
||||
suites = "1 suite" if count == 1 else f"{count} suites"
|
||||
return {"ok": True, "output": f"{path}: {suites} moved from {release} to {codename}"}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"APT_RELEASES_COMMAND",
|
||||
"AptReleaseFixDict",
|
||||
"AptReleaseMixin",
|
||||
"AptReleasesDict",
|
||||
"AptSourceDict",
|
||||
"apt_releases_command",
|
||||
"parse_apt_releases",
|
||||
"rewrite_release",
|
||||
"suite_release",
|
||||
]
|
||||
@@ -0,0 +1,172 @@
|
||||
"""apt's https sources through the package cache, so that the cache stores them.
|
||||
|
||||
A source fetched over https goes through apt-cacher-ng's tunnel, if at all: the
|
||||
TLS runs end to end and nothing of it is stored. The cache fetches
|
||||
``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>`` itself, with the
|
||||
certificate verified, and stores it -- where it allows port 443
|
||||
(``AllowUserPorts=80 443``) (NetOrk/netork#2555). apt fetches that form through
|
||||
its http proxy, so it works only while the cache is that proxy: without one apt
|
||||
looks for a host called ``HTTPS``.
|
||||
|
||||
**Which hosts** (:func:`cacheable_https_hosts`): the hosts of the https sources,
|
||||
on port 443, while apt has an http proxy for every host and none of its own for
|
||||
that one. Not a host with a login in the URL, nor one whose credentials apt keeps
|
||||
in ``auth.conf``: apt finds those by the host's name, which the rewrite changes,
|
||||
and paid content does not belong in a cache other hosts share. Only root reads
|
||||
``auth.conf``, so the rewrite judges that, leaves such a host on https and names
|
||||
it in the proxy file as ``Package-Proxy::Not-Cached``, which apt ignores and
|
||||
``apt-config dump`` shows to the report.
|
||||
|
||||
**The rewrite** (:func:`rewrite_sources_command`) turns ``https://<host>`` into
|
||||
the cache form on the lines that name sources: ``deb`` and ``deb-src`` in a
|
||||
``.list`` file, ``URIs:`` in a deb822 ``.sources`` file. A file without the host
|
||||
is not touched. :func:`revert_sources_command` turns every source back.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - the report's type, without an import cycle
|
||||
from napalm_device_types.apt_proxy import AptProxyReportDict
|
||||
|
||||
#: The prefix apt-cacher-ng fetches over https.
|
||||
VIA_CACHE = "http://HTTPS///"
|
||||
|
||||
#: The key in the proxy file that names the hosts left on https for their credentials.
|
||||
NOT_CACHED_KEY = "Package-Proxy::Not-Cached"
|
||||
|
||||
_LABEL = r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?"
|
||||
_HOST = re.compile(rf"^{_LABEL}(?:\.{_LABEL})*$")
|
||||
_NO_PROXY = frozenset({"", "direct"})
|
||||
|
||||
#: Where apt's sources and credentials are, as shell variables; apt's defaults
|
||||
#: when apt-config cannot say.
|
||||
_LOCATE = (
|
||||
'eval "$({apt_config} shell sl Dir::Etc::sourcelist/f sp Dir::Etc::sourceparts/d '
|
||||
'nr Dir::Etc::netrc/f np Dir::Etc::netrcparts/d 2>/dev/null)"; '
|
||||
'sl="${{sl:-/etc/apt/sources.list}}"; sp="${{sp:-/etc/apt/sources.list.d/}}"; '
|
||||
'nr="${{nr:-/etc/apt/auth.conf}}"; np="${{np:-/etc/apt/auth.conf.d/}}"; '
|
||||
)
|
||||
|
||||
#: The lines that name sources: ``deb``/``deb-src`` and deb822's ``URIs:``.
|
||||
_SOURCE_LINES = "/^[[:space:]]*(deb|deb-src|[Uu][Rr][Ii][Ss]:)/"
|
||||
|
||||
#: The hosts apt has credentials for: each word after ``machine``, without its
|
||||
#: scheme, port and path, in lower case.
|
||||
_CREDENTIAL_HOSTS = (
|
||||
'creds=$(cat "$nr" "$np"* 2>/dev/null | tr -s " \\t\\r" "\\n" '
|
||||
"| awk 'p {{ print; p = 0; next }} $0 == \"machine\" {{ p = 1 }}' "
|
||||
"| sed -e 's|^[A-Za-z+]*://||' -e 's|[:/].*||' | tr A-Z a-z); "
|
||||
)
|
||||
|
||||
_REWRITE = (
|
||||
"( " + _LOCATE + _CREDENTIAL_HOSTS + "for h in {hosts}; do "
|
||||
'l=$(printf "%s" "$h" | tr A-Z a-z); '
|
||||
'if printf "%s\\n" "$creds" | grep -qxF -- "$l"; then '
|
||||
'grep -qF -- "\\"$h\\"" {proxy_file} 2>/dev/null || '
|
||||
'printf \'{key}:: "%s";\\n\' "$h" >> {proxy_file} || exit 1; continue; fi; '
|
||||
"e=$(printf '%s' \"$h\" | sed 's/[.]/[.]/g'); "
|
||||
'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
|
||||
'[ -f "$f" ] && grep -qF -- "https://$h" "$f" || continue; '
|
||||
'sed -i -E "{lines} s#([[:space:]:])https://$e([/[:space:]]|\\$)#\\1{via}$h\\2#g" "$f" '
|
||||
"|| exit 1; done; done )"
|
||||
)
|
||||
|
||||
_REVERT = (
|
||||
"( " + _LOCATE + 'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
|
||||
'[ -f "$f" ] && grep -qF -- "{via}" "$f" || continue; '
|
||||
"sed -i -E '{lines} s#([[:space:]:]){via}#\\1https://#g' \"$f\" || exit 1; done )"
|
||||
)
|
||||
|
||||
|
||||
def cache_form(url: str) -> str:
|
||||
"""*url*, an https URL, in the form apt-cacher-ng fetches over https and stores.
|
||||
|
||||
:raises ValueError: if *url* is not an https URL.
|
||||
"""
|
||||
if not url.startswith("https://"):
|
||||
raise ValueError(f"not an https URL: {url!r}")
|
||||
return VIA_CACHE + url[len("https://") :]
|
||||
|
||||
|
||||
def _per_host_direct(report: AptProxyReportDict) -> set[str]:
|
||||
hosts = {**report["per_host"], **report["https_per_host"]}
|
||||
return {h.lower() for h, v in hosts.items() if v.lower() in _NO_PROXY}
|
||||
|
||||
|
||||
def _host_of(source: str) -> str | None:
|
||||
"""The host of an https source the cache can fetch: port 443, no login in the URL."""
|
||||
try:
|
||||
parts = urlsplit(source)
|
||||
port = parts.port
|
||||
except ValueError:
|
||||
return None
|
||||
if parts.username is not None or port not in (None, 443) or not parts.hostname:
|
||||
return None
|
||||
return parts.hostname
|
||||
|
||||
|
||||
def cacheable_https_hosts(report: AptProxyReportDict) -> list[str]:
|
||||
"""The hosts of the https sources the rewrite would send through the cache, sorted.
|
||||
|
||||
None while apt has no http proxy for every host: the cache form goes through it.
|
||||
Not a host apt fetches ``DIRECT``, nor one the rewrite left on https for its
|
||||
credentials (``not_cached``), nor one on another port or with a login in the URL.
|
||||
"""
|
||||
if not report["proxy"]:
|
||||
return []
|
||||
skip = _per_host_direct(report) | {h.lower() for h in report.get("not_cached", [])}
|
||||
hosts = {_host_of(s) for s in report["https_sources"]}
|
||||
return sorted(h for h in hosts if h and h.lower() not in skip)
|
||||
|
||||
|
||||
def is_host_name(host: str) -> bool:
|
||||
"""Whether *host* is a DNS name: letters, digits and hyphens between dots."""
|
||||
return bool(_HOST.match(host))
|
||||
|
||||
|
||||
def _checked(hosts: list[str]) -> str:
|
||||
for host in hosts:
|
||||
if not is_host_name(host):
|
||||
raise ValueError(f"not a host name: {host!r}")
|
||||
return " ".join(hosts)
|
||||
|
||||
|
||||
def rewrite_sources_command(
|
||||
hosts: list[str], proxy_file: str, apt_config: str = "apt-config"
|
||||
) -> str:
|
||||
"""The root command that sends the https sources of *hosts* through the cache.
|
||||
|
||||
A host apt has credentials for stays on https and is named in *proxy_file*.
|
||||
*apt_config* is how apt-config is called; a test points apt's paths elsewhere.
|
||||
|
||||
:raises ValueError: if one of *hosts* is not a host name.
|
||||
"""
|
||||
return _REWRITE.format(
|
||||
apt_config=apt_config,
|
||||
hosts=_checked(hosts),
|
||||
proxy_file=shlex.quote(proxy_file),
|
||||
key=NOT_CACHED_KEY,
|
||||
lines=_SOURCE_LINES,
|
||||
via=VIA_CACHE,
|
||||
)
|
||||
|
||||
|
||||
def revert_sources_command(apt_config: str = "apt-config") -> str:
|
||||
"""The root command that turns every source in the cache form back to https."""
|
||||
return _REVERT.format(apt_config=apt_config, lines=_SOURCE_LINES, via=VIA_CACHE)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"NOT_CACHED_KEY",
|
||||
"VIA_CACHE",
|
||||
"cache_form",
|
||||
"cacheable_https_hosts",
|
||||
"is_host_name",
|
||||
"revert_sources_command",
|
||||
"rewrite_sources_command",
|
||||
]
|
||||
@@ -0,0 +1,140 @@
|
||||
"""A device's own configuration history: list it, read a revision, revert to one.
|
||||
|
||||
Some devices keep their earlier configurations themselves: a copy of the
|
||||
configuration for every save, which the device can go back to. An application
|
||||
restores a configuration there by reverting to the revision that holds it -- on a
|
||||
device whose API takes no uploaded configuration (OPNsense), the only way
|
||||
(NetOrk/netork#2511). :class:`ConfigRevisionsMixin` is the contract:
|
||||
|
||||
* ``get_config_revisions()`` and ``get_config_revision(revision_id)`` -- declared
|
||||
only, so ``callable(getattr(driver, "get_config_revisions", None))`` is the
|
||||
truthful answer to "does this device keep a history";
|
||||
* ``revert_config_revision(revision_id)`` -- a template: it refuses an id that is not
|
||||
in the history, reads the revision, lets the driver revert
|
||||
(``_revert_to_config_revision``), and reads the configuration in place afterwards
|
||||
to check that it holds the revision's. A device may answer that it reverted when it
|
||||
did not; the check does not take its word;
|
||||
* ``config_revision_content(text)`` -- what of a revision is its configuration. Every
|
||||
save may stamp a revision anew (OPNsense writes who saved it and when into
|
||||
config.xml), so two revisions with the same configuration differ as text. A driver
|
||||
whose device does that strips the stamp here; by default the whole text counts. An
|
||||
application that looks for the revision holding a configuration compares through
|
||||
it too.
|
||||
|
||||
A driver mixes it in when its device keeps a history; no role base carries it. One
|
||||
that can list the history but not revert to it sets ``revert_config_revision`` to
|
||||
``None``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
|
||||
class ConfigRevisionDict(TypedDict):
|
||||
"""One revision of the device's configuration (``get_config_revisions``)."""
|
||||
|
||||
id: str # what get_config_revision and revert_config_revision take, as the device names it
|
||||
time: str | None # when it was saved, UTC ISO 8601; None when the device does not say
|
||||
description: str | None # what made the change, as the device records it; None: none
|
||||
user: str | None # who saved it, as the device names them; None when it names nobody
|
||||
size: int | None # its size in bytes; None when not given
|
||||
|
||||
|
||||
class UnknownConfigRevision(ValueError):
|
||||
"""The device's configuration history has no revision of that id."""
|
||||
|
||||
def __init__(self, revision_id: object) -> None:
|
||||
super().__init__(f"no configuration revision {revision_id!r} in the device's history")
|
||||
|
||||
|
||||
class ConfigRevertFailed(RuntimeError):
|
||||
"""The device refused the revert, or its configuration afterwards is not the revision's."""
|
||||
|
||||
|
||||
class ConfigRevisionsMixin:
|
||||
"""List, read and revert to the device's configuration revisions; see the module."""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_config_revisions(self) -> list[ConfigRevisionDict]:
|
||||
"""The device's configuration history, newest first.
|
||||
|
||||
The first revision is the configuration in place now.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "config-1794603885.1239.xml",
|
||||
"time": "2026-11-13T21:04:45+00:00",
|
||||
"description": "/api/firewall/filter/apply made changes",
|
||||
"user": "apiuser@192.0.2.10",
|
||||
"size": 58213,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the history could not be read; never an empty
|
||||
list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_config_revision(self, revision_id: str) -> str:
|
||||
"""The configuration of revision *revision_id*, as the device keeps it.
|
||||
|
||||
:raises UnknownConfigRevision: for an id that is not in the history.
|
||||
:raises Exception: when the device could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# The step a driver implements:
|
||||
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
"""Make revision *revision_id* the device's configuration; return once it is
|
||||
in place, as the newest revision of the history.
|
||||
|
||||
:raises UnknownConfigRevision: when the device no longer has it.
|
||||
:raises ConfigRevertFailed: when the device refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
"""What of a revision's *text* is its configuration: the whole text by default.
|
||||
|
||||
A driver whose device stamps every save into the configuration (who, when)
|
||||
returns the text without that stamp, so that two revisions with the same
|
||||
configuration compare equal.
|
||||
"""
|
||||
return text
|
||||
|
||||
def revert_config_revision(self, revision_id: str) -> ConfigRevisionDict:
|
||||
"""Make revision *revision_id* the device's configuration again.
|
||||
|
||||
Returns the revision in place afterwards: the newest of the history, which
|
||||
holds the configuration of *revision_id* (on most devices a new revision the
|
||||
revert saved). What the device does beyond that -- reload its services,
|
||||
restart, nothing -- is the driver's to document.
|
||||
|
||||
:raises UnknownConfigRevision: for an id that is not in the history now;
|
||||
nothing has changed then.
|
||||
:raises ConfigRevertFailed: when the device refuses, or when the configuration
|
||||
in place afterwards is not the revision's.
|
||||
:raises Exception: when the device could not be reached.
|
||||
"""
|
||||
if revision_id not in {r["id"] for r in self.get_config_revisions()}:
|
||||
raise UnknownConfigRevision(revision_id)
|
||||
# Read before the revert: saving it may rotate the revision out of the history.
|
||||
wanted = self.config_revision_content(self.get_config_revision(revision_id))
|
||||
self._revert_to_config_revision(revision_id)
|
||||
revisions = self.get_config_revisions()
|
||||
if not revisions:
|
||||
raise ConfigRevertFailed(
|
||||
f"the device has no configuration after reverting to {revision_id!r}"
|
||||
)
|
||||
now = revisions[0]
|
||||
if self.config_revision_content(self.get_config_revision(now["id"])) != wanted:
|
||||
raise ConfigRevertFailed(
|
||||
f"the configuration in place ({now['id']!r}) is not revision {revision_id!r}"
|
||||
" after the revert"
|
||||
)
|
||||
return now
|
||||
@@ -23,6 +23,22 @@ Mixed in by a driver, not by a role base: a Windows host is an
|
||||
:class:`~napalm_device_types.os.OSDriver` too, and without a shell channel it
|
||||
has no ``dial-stdio`` to offer. ``hasattr(driver, "open_container_engine")``
|
||||
stays truthful.
|
||||
|
||||
**Which engine a CLI is.** An engine is listed when its CLI answers
|
||||
``--version`` with its own name: ``Docker version 29.0.1, build …``, ``podman
|
||||
version 5.2.2``. podman-docker installs a ``docker`` that is podman; listed as
|
||||
Docker it would be read twice, the second time as the login user's rootless
|
||||
Podman (NetOrk/netork#818).
|
||||
|
||||
**Podman as root.** The Podman this reaches is the host's *rootful* one: the
|
||||
engine of the system's containers, whose units Quadlet generates from
|
||||
``/etc/containers/systemd`` and whose API ``podman.socket`` serves at
|
||||
``/run/podman/podman.sock``. Rootless containers belong to the user who runs
|
||||
them, in that user's own storage, and no other login sees them. So the
|
||||
connection to Podman runs every call as root (``privileged``), the way the
|
||||
driver gains root for any command: root itself, or sudo. ``podman system
|
||||
dial-stdio`` exists from Podman 4.0 on; it reaches the API socket, which has
|
||||
to be listening (``systemctl enable --now podman.socket``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -34,8 +50,11 @@ from typing import TYPE_CHECKING, Any, cast
|
||||
from napalm_device_types.channel import ByteStream, CommandResult
|
||||
from napalm_device_types.models import ContainerEngineDict, FixResultDict
|
||||
|
||||
#: Engines this package knows how to reach, and the API each speaks.
|
||||
ENGINES: dict[str, str] = {"docker": "docker-engine"}
|
||||
#: Engines this package knows how to reach, and the API each speaks. Podman's
|
||||
#: is its own service, which serves Docker's API next to libpod's.
|
||||
ENGINES: dict[str, str] = {"docker": "docker-engine", "podman": "podman-libpod"}
|
||||
#: Engines reached as root: rootful Podman's socket is root's (see above).
|
||||
ROOT_ENGINES: frozenset[str] = frozenset({"podman"})
|
||||
|
||||
_PROBE_TIMEOUT = 15
|
||||
_CLI_TIMEOUT = 120
|
||||
@@ -52,17 +71,24 @@ class ContainerEngineConnection:
|
||||
arguments; binary, quoting and the channel are the driver's.
|
||||
"""
|
||||
|
||||
def __init__(self, driver: Any, engine: str, binary: str) -> None:
|
||||
def __init__(self, driver: Any, engine: str, binary: str, *, privileged: bool = False) -> None:
|
||||
self._driver = driver
|
||||
self._binary = binary
|
||||
self.engine = engine
|
||||
#: Every call runs as root: the engine is root's (rootful Podman).
|
||||
self.privileged = privileged
|
||||
|
||||
def _command(self, args: Sequence[str]) -> str:
|
||||
return shlex.join([self._binary, *args])
|
||||
|
||||
def open_api(self) -> ByteStream:
|
||||
"""A stream to the engine's API: HTTP/1.1 over ``system dial-stdio``."""
|
||||
return cast(ByteStream, self._driver.open_stream(self._command(["system", "dial-stdio"])))
|
||||
return cast(
|
||||
ByteStream,
|
||||
self._driver.open_stream(
|
||||
self._command(["system", "dial-stdio"]), privileged=self.privileged
|
||||
),
|
||||
)
|
||||
|
||||
def run_cli(
|
||||
self,
|
||||
@@ -80,7 +106,10 @@ class ContainerEngineConnection:
|
||||
return cast(
|
||||
CommandResult,
|
||||
self._driver.run_command(
|
||||
self._command(args), privileged=privileged, timeout=timeout, stdin=stdin
|
||||
self._command(args),
|
||||
privileged=privileged or self.privileged,
|
||||
timeout=timeout,
|
||||
stdin=stdin,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -96,7 +125,8 @@ class ContainerEngineConnection:
|
||||
return cast(
|
||||
ByteStream,
|
||||
self._driver.open_stream(
|
||||
f"{command} 2>&1" if merge_stderr else command, privileged=privileged
|
||||
f"{command} 2>&1" if merge_stderr else command,
|
||||
privileged=privileged or self.privileged,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -134,20 +164,32 @@ class ContainerEngineMixin:
|
||||
* engine (string) - e.g. "docker"
|
||||
* api (string) - what its API is, e.g. "docker-engine"
|
||||
|
||||
An engine is listed when its CLI exists on the host. Whether the login
|
||||
user may use it is decided by the first API call, which says
|
||||
"permission denied" in the stream's stderr.
|
||||
An engine is listed when its CLI exists on the host and names itself
|
||||
when asked for its version: podman-docker's ``docker`` says it is podman,
|
||||
and is left out. Whether the login user may use the engine is decided by
|
||||
the first API call, which says "permission denied" in the stream's stderr.
|
||||
"""
|
||||
found: list[ContainerEngineDict] = []
|
||||
for engine, api in ENGINES.items():
|
||||
probe = shlex.join(["command", "-v", self._container_engine_binary(engine)])
|
||||
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
|
||||
if result.exit_code == 0 and result.stdout.strip():
|
||||
found.append({"engine": engine, "api": api})
|
||||
return found
|
||||
return [
|
||||
{"engine": engine, "api": api}
|
||||
for engine, api in ENGINES.items()
|
||||
if self._answers_as(engine)
|
||||
]
|
||||
|
||||
def _answers_as(self, engine: str) -> bool:
|
||||
"""Whether *engine*'s CLI is on the host and says it is *engine*."""
|
||||
binary = shlex.quote(self._container_engine_binary(engine))
|
||||
probe = f"command -v {binary} >/dev/null 2>&1 && {binary} --version"
|
||||
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
|
||||
words = result.stdout.split()
|
||||
return result.exit_code == 0 and bool(words) and words[0].lower() == engine
|
||||
|
||||
def open_container_engine(self, engine: str) -> ContainerEngineConnection:
|
||||
"""The connection to *engine*; :class:`ContainerEngineUnavailable` if unknown."""
|
||||
if engine not in ENGINES:
|
||||
raise ContainerEngineUnavailable(f"no way to reach container engine {engine!r}")
|
||||
return ContainerEngineConnection(self, engine, self._container_engine_binary(engine))
|
||||
return ContainerEngineConnection(
|
||||
self,
|
||||
engine,
|
||||
self._container_engine_binary(engine),
|
||||
privileged=engine in ROOT_ENGINES,
|
||||
)
|
||||
+113
-57
@@ -26,6 +26,7 @@ import re
|
||||
from collections.abc import Iterator
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types._dhcp_order import ReservationChange, order_reservation_changes
|
||||
from napalm_device_types._matching import check_desired, index_live
|
||||
from napalm_device_types.models import (
|
||||
DhcpHaDict,
|
||||
@@ -57,18 +58,26 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
|
||||
|
||||
_HEX_ONLY = re.compile(r"[^0-9a-f]")
|
||||
|
||||
#: Added to a refused change that is part of an address swap (#106).
|
||||
_SWAP_NOTE = (
|
||||
"an address swap with {}: no order frees an address first; "
|
||||
"move one of them to a free address in a run of its own"
|
||||
)
|
||||
|
||||
|
||||
class DhcpApplyError(RuntimeError):
|
||||
"""Some reservations of an applied set were refused by the device.
|
||||
"""Some changes of an applied reservation or subnet set were refused by the device.
|
||||
|
||||
Raised after the rest were written and committed: ``failed`` of ``attempted``
|
||||
changes did not go through, and each was named in a ``[failed]`` progress line.
|
||||
``kind`` is what was applied: ``"reservation"`` or ``"subnet"``.
|
||||
"""
|
||||
|
||||
def __init__(self, failed: int, attempted: int) -> None:
|
||||
def __init__(self, failed: int, attempted: int, kind: str = "reservation") -> None:
|
||||
self.failed = failed
|
||||
self.attempted = attempted
|
||||
super().__init__(f"{failed} of {attempted} DHCP reservation change(s) were refused")
|
||||
self.kind = kind
|
||||
super().__init__(f"{failed} of {attempted} DHCP {kind} change(s) were refused")
|
||||
|
||||
|
||||
def normalize_mac(mac: str | None) -> str:
|
||||
@@ -119,6 +128,43 @@ def _subnet_field_differs(field: str, live: DhcpSubnetDict, desired: DhcpSubnetD
|
||||
return any(live_options.get(option) != value for option, value in desired_options.items())
|
||||
|
||||
|
||||
def _reservation_key(reservation: DhcpReservationDict) -> str:
|
||||
return normalize_mac(reservation.get("mac"))
|
||||
|
||||
|
||||
def _diff_reservations(
|
||||
live: list[DhcpReservationDict], desired: list[DhcpReservationDict]
|
||||
) -> DhcpReservationDiffDict:
|
||||
"""``diff_dhcp_reservations`` against reservations already read."""
|
||||
live_by_mac, shared = index_live(live, _reservation_key)
|
||||
check_desired(desired, _reservation_key, shared, "MAC")
|
||||
|
||||
add: list[DhcpReservationDict] = []
|
||||
update: list[DhcpReservationUpdateDict] = []
|
||||
|
||||
for desired_reservation in desired:
|
||||
current = live_by_mac.get(_reservation_key(desired_reservation))
|
||||
if current is None:
|
||||
add.append(desired_reservation)
|
||||
continue
|
||||
|
||||
changed_fields = [
|
||||
field
|
||||
for field in _DHCP_RESERVATION_COMPARE_FIELDS
|
||||
if current.get(field) != desired_reservation.get(field)
|
||||
]
|
||||
if changed_fields:
|
||||
update.append(
|
||||
{
|
||||
"uuid": current["uuid"],
|
||||
"reservation": desired_reservation,
|
||||
"changed_fields": changed_fields,
|
||||
}
|
||||
)
|
||||
|
||||
return {"add": add, "update": update}
|
||||
|
||||
|
||||
class DhcpServerMixin:
|
||||
"""Mixin providing DHCP reservation and subnet read/diff/apply.
|
||||
|
||||
@@ -271,36 +317,7 @@ class DhcpServerMixin:
|
||||
:raises ValueError: When a desired reservation cannot name a single live one.
|
||||
"""
|
||||
|
||||
def key(reservation: DhcpReservationDict) -> str:
|
||||
return normalize_mac(reservation.get("mac"))
|
||||
|
||||
live_by_mac, shared = index_live(self.get_dhcp_reservations(), key)
|
||||
check_desired(desired, key, shared, "MAC")
|
||||
|
||||
add: list[DhcpReservationDict] = []
|
||||
update: list[DhcpReservationUpdateDict] = []
|
||||
|
||||
for desired_reservation in desired:
|
||||
live = live_by_mac.get(key(desired_reservation))
|
||||
if live is None:
|
||||
add.append(desired_reservation)
|
||||
continue
|
||||
|
||||
changed_fields = [
|
||||
field
|
||||
for field in _DHCP_RESERVATION_COMPARE_FIELDS
|
||||
if live.get(field) != desired_reservation.get(field)
|
||||
]
|
||||
if changed_fields:
|
||||
update.append(
|
||||
{
|
||||
"uuid": live["uuid"],
|
||||
"reservation": desired_reservation,
|
||||
"changed_fields": changed_fields,
|
||||
}
|
||||
)
|
||||
|
||||
return {"add": add, "update": update}
|
||||
return _diff_reservations(self.get_dhcp_reservations(), desired)
|
||||
|
||||
def apply_dhcp_reservationset(self, desired: list[DhcpReservationDict]) -> Iterator[str]:
|
||||
"""
|
||||
@@ -311,6 +328,9 @@ class DhcpServerMixin:
|
||||
committing reloads the DHCP daemon and drops in-flight requests, which
|
||||
is too high a price for a no-op run.
|
||||
|
||||
The changes are written so an address is released before it is taken; a
|
||||
swap of addresses has no such order and is reported as one (``_dhcp_order``).
|
||||
|
||||
A change the device refuses (say, an address in a subnet its DHCP server
|
||||
does not manage) does not stop the others: it gets a ``[failed]`` line,
|
||||
the rest are written and committed, and ``DhcpApplyError`` is raised at
|
||||
@@ -322,7 +342,8 @@ class DhcpServerMixin:
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_reservations(desired)
|
||||
live = self.get_dhcp_reservations()
|
||||
diff = _diff_reservations(live, desired)
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
@@ -330,17 +351,13 @@ class DhcpServerMixin:
|
||||
|
||||
added = updated = refused = 0
|
||||
|
||||
for reservation in diff["add"]:
|
||||
line, ok = self._write_reservation(reservation, None, "[add]")
|
||||
added, refused = added + ok, refused + (not ok)
|
||||
for change in order_reservation_changes(diff, live):
|
||||
line, ok = self._write_change(change)
|
||||
refused += not ok
|
||||
added += ok and change.uuid is None
|
||||
updated += ok and change.uuid is not None
|
||||
yield line
|
||||
|
||||
for entry in diff["update"]:
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
line, ok = self._write_reservation(entry["reservation"], entry["uuid"], "[update]")
|
||||
updated, refused = updated + ok, refused + (not ok)
|
||||
yield f"{line} ({fields})" if ok else line
|
||||
|
||||
# Committed even when every write was refused: a run that stopped before its
|
||||
# commit left saved writes behind, which the device reports as configured, so
|
||||
# no later diff would ever commit them.
|
||||
@@ -350,6 +367,16 @@ class DhcpServerMixin:
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted)
|
||||
|
||||
def _write_change(self, change: ReservationChange) -> tuple[str, bool]:
|
||||
"""Write one change of an ordered set; its progress line, and whether it went in."""
|
||||
tag = "[add]" if change.uuid is None else "[update]"
|
||||
line, ok = self._write_reservation(change.reservation, change.uuid, tag)
|
||||
if ok and change.fields:
|
||||
line = f"{line} ({change.fields})"
|
||||
elif not ok and change.swap:
|
||||
line = f"{line} -- {_SWAP_NOTE.format(', '.join(change.swap))}"
|
||||
return line, ok
|
||||
|
||||
def _write_reservation(
|
||||
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
|
||||
) -> tuple[str, bool]:
|
||||
@@ -430,24 +457,53 @@ class DhcpServerMixin:
|
||||
As with ``apply_dhcp_reservationset``, an empty diff does **not**
|
||||
commit: reconfiguring the DHCP daemon is too expensive for a no-op.
|
||||
|
||||
A subnet the device refuses (say, one that overlaps a subnet it serves)
|
||||
does not stop the others: it gets a ``[failed]`` line, the rest are
|
||||
written and committed, and ``DhcpApplyError`` is raised at the end. Any
|
||||
non-empty diff commits, even one whose every write was refused, so that
|
||||
writes an earlier, interrupted run saved take effect (#113).
|
||||
|
||||
:param desired: The complete desired subnet set.
|
||||
:yields: Progress lines, one per applied add/update, plus a final
|
||||
commit line.
|
||||
:yields: Progress lines, one per applied, updated or refused change,
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_subnets(desired)
|
||||
|
||||
for subnet in diff["add"]:
|
||||
self.apply_dhcp_subnet(subnet)
|
||||
yield f"[add] {subnet['subnet']}"
|
||||
|
||||
for entry in diff["update"]:
|
||||
self.apply_dhcp_subnet(entry["subnet"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield f"[update] {entry['subnet']['subnet']} ({fields})"
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
return
|
||||
|
||||
changes: list[tuple[DhcpSubnetDict, str | None, str]] = [
|
||||
(subnet, None, "") for subnet in diff["add"]
|
||||
]
|
||||
changes += [
|
||||
(e["subnet"], e["uuid"], ", ".join(e["changed_fields"])) for e in diff["update"]
|
||||
]
|
||||
added = updated = refused = 0
|
||||
|
||||
for subnet, uuid, fields in changes:
|
||||
line, ok = self._write_subnet(subnet, uuid, fields)
|
||||
refused += not ok
|
||||
added += ok and uuid is None
|
||||
updated += ok and uuid is not None
|
||||
yield line
|
||||
|
||||
# Committed even when every write was refused; see apply_dhcp_reservationset.
|
||||
self.commit_dhcp_subnets()
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
yield f"[commit] applied {added} add(s), {updated} update(s)"
|
||||
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted, "subnet")
|
||||
|
||||
def _write_subnet(
|
||||
self, subnet: DhcpSubnetDict, uuid: str | None, fields: str
|
||||
) -> tuple[str, bool]:
|
||||
"""Write one subnet; its progress line, and whether the device took it."""
|
||||
try:
|
||||
self.apply_dhcp_subnet(subnet, uuid=uuid)
|
||||
except Exception as exc:
|
||||
return f"[failed] {subnet['subnet']}: {exc}", False
|
||||
if uuid is None:
|
||||
return f"[add] {subnet['subnet']}", True
|
||||
return f"[update] {subnet['subnet']} ({fields})", True
|
||||
@@ -1,23 +1,39 @@
|
||||
"""The firmware a vendor offers for a device.
|
||||
"""The firmware a vendor offers for a device, and installing it.
|
||||
|
||||
A fact for the caller (docs/STRUCTURE.md): a driver may know the newer firmware
|
||||
from the vendor's download catalogue or the device's own update check; whether
|
||||
running an older one is a problem is decided above the driver. Comparing the
|
||||
vendor's version strings is vendor knowledge, so the driver says itself whether
|
||||
the offer is newer. Declared only; a driver that can tell implements it.
|
||||
|
||||
Installing it is a capability of its own (``FirmwareUpgradeMixin``): when and
|
||||
whether is the caller's call, as with package updates; how is the driver's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
from typing import TYPE_CHECKING, NotRequired, TypedDict
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import FirmwareOfferDict
|
||||
class FirmwareOfferDict(TypedDict):
|
||||
"""The firmware a device runs and the newest its vendor offers
|
||||
(``FirmwareOfferMixin.get_firmware_offer``). A fact, not a judgement."""
|
||||
|
||||
current_version: str
|
||||
latest_version: str
|
||||
newer: bool # the offer is newer than what runs, by the vendor's versioning
|
||||
download_url: NotRequired[str]
|
||||
# The driver can install latest_version itself (FirmwareUpgradeMixin) now; False
|
||||
# while the device lacks what that takes (OpenWrt without owut). Left out where
|
||||
# the driver never installs firmware on this device.
|
||||
installable: NotRequired[bool]
|
||||
# The newest of a later release line (OpenWrt 25.12 on a 24.10 device), which
|
||||
# latest_version does not cross to: shown, never installed by the driver.
|
||||
latest_major_version: NotRequired[str]
|
||||
|
||||
|
||||
class FirmwareOfferMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_firmware_offer(self) -> FirmwareOfferDict | None:
|
||||
def get_firmware_offer(self) -> "FirmwareOfferDict | None":
|
||||
"""The firmware running and the newest the vendor offers.
|
||||
|
||||
None when the driver cannot tell for this device (an unknown model,
|
||||
@@ -29,3 +45,23 @@ class FirmwareOfferMixin:
|
||||
"newer": True, "download_url": "https://.../GS308E_1.0.2.12.zip"}
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class FirmwareUpgradeMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def start_firmware_upgrade(self, version: str) -> None:
|
||||
"""Starts installing *version*, an offer's ``latest_version`` that
|
||||
:meth:`FirmwareOfferMixin.get_firmware_offer` called ``installable``.
|
||||
|
||||
The device runs the upgrade by itself and restarts at its end, which
|
||||
ends the session; the configuration is kept. Returns once it has
|
||||
started. ``UpdateMixin.get_update_progress`` follows it until then:
|
||||
``"failed"`` when it stopped before the device restarted, which leaves
|
||||
the running firmware as it was.
|
||||
|
||||
:raises ValueError: *version* is not a version string the driver takes.
|
||||
:raises RuntimeError: the device cannot start it (what it takes is
|
||||
missing, an upgrade is already running).
|
||||
"""
|
||||
...
|
||||
@@ -16,6 +16,7 @@ They replace ``run_device_action`` (docs/STRUCTURE.md):
|
||||
``fix_apt_proxy`` :meth:`PackageProxyMixin.configure_package_proxy`
|
||||
``fix_docker_permissions`` ``ContainerEngineMixin.grant_container_engine_access``
|
||||
``install_lldpd`` and co. :meth:`EnsurePackageMixin.ensure_package`
|
||||
(new) :meth:`ZabbixAgentMixin.configure_zabbix_agent`
|
||||
========================== ===================================================
|
||||
"""
|
||||
|
||||
@@ -31,6 +32,7 @@ from napalm_device_types.models import (
|
||||
PackageDict,
|
||||
SnmpAgentSettingsDict,
|
||||
)
|
||||
from napalm_device_types.zabbix_agent import ZabbixAgentSettingsDict, check_zabbix_agent_settings
|
||||
|
||||
#: One step of a fix: its name and the call that does it.
|
||||
FixStep = tuple[str, Callable[[], FixStepOutcomeDict]]
|
||||
@@ -58,6 +60,23 @@ def run_fix_steps(steps: Sequence[FixStep]) -> FixResultDict:
|
||||
}
|
||||
|
||||
|
||||
def enable_and_restart(
|
||||
manage_service: Callable[[str, str], dict[str, Any]], service: str
|
||||
) -> FixStepOutcomeDict:
|
||||
"""Enable *service* at boot, then restart it so it reads its configuration; not
|
||||
restarted when it cannot be enabled."""
|
||||
for action in ("enable", "restart"):
|
||||
result = manage_service(service, action)
|
||||
if not result.get("success"):
|
||||
output = str(result.get("output", "")).strip()
|
||||
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
|
||||
return {"ok": True, "output": f"{service} enabled and restarted"}
|
||||
|
||||
|
||||
def _joined(*parts: object) -> str:
|
||||
return "\n".join(str(p).strip() for p in parts if p and str(p).strip())
|
||||
|
||||
|
||||
class SnmpAgentMixin:
|
||||
"""Run an SNMP agent on the device that answers the caller.
|
||||
|
||||
@@ -171,13 +190,7 @@ class LldpAgentMixin:
|
||||
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
|
||||
|
||||
def _start_lldp_agent(self) -> FixStepOutcomeDict:
|
||||
service = self.LLDP_AGENT_SERVICE
|
||||
for action in ("enable", "restart"):
|
||||
result = self.manage_service(service, action)
|
||||
if not result.get("success"):
|
||||
output = str(result.get("output", "")).strip()
|
||||
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
|
||||
return {"ok": True, "output": f"{service} enabled and restarted"}
|
||||
return enable_and_restart(self.manage_service, self.LLDP_AGENT_SERVICE)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
@@ -193,6 +206,110 @@ class LldpAgentMixin:
|
||||
...
|
||||
|
||||
|
||||
class ZabbixAgentMixin:
|
||||
"""Run a native Zabbix agent that answers the caller's server.
|
||||
|
||||
For a host without a container engine, where the agent cannot run as a
|
||||
container. The steps: load the package index where the driver can
|
||||
(``refresh_available_updates``), install the agent from the host's own packages -- and
|
||||
where those have none, from Zabbix's repository for ``settings["release"]``
|
||||
(:func:`~napalm_device_types.zabbix_agent.zabbix_repository_command`) --, write
|
||||
its configuration (:func:`~napalm_device_types.zabbix_agent.zabbix_agent_config`),
|
||||
let the allowed sources through the host's firewall, enable and restart it.
|
||||
Installing and starting are generic, over ``ensure_package`` and
|
||||
``manage_service`` with ``ZABBIX_AGENT_PACKAGE`` and ``ZABBIX_AGENT_SERVICE``.
|
||||
Whether Zabbix reaches it is the caller's to ask Zabbix.
|
||||
|
||||
The settings are checked before anything changes; ``ValueError`` names the one
|
||||
that does not hold. A driver that cannot add a repository sets
|
||||
``_add_zabbix_repository`` to ``None``: its hosts then take the distribution's
|
||||
package or none.
|
||||
"""
|
||||
|
||||
#: The package that brings the agent, as ``ensure_package`` takes it.
|
||||
ZABBIX_AGENT_PACKAGE: ClassVar[str] = "zabbix-agent"
|
||||
#: The agent's service, as ``manage_service`` takes it.
|
||||
ZABBIX_AGENT_SERVICE: ClassVar[str] = "zabbix-agent"
|
||||
|
||||
def configure_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixResultDict:
|
||||
"""Install, configure, open, enable and restart the agent with *settings*.
|
||||
|
||||
:raises ValueError: for settings the agent's configuration cannot take,
|
||||
before anything changes.
|
||||
"""
|
||||
check_zabbix_agent_settings(settings)
|
||||
return run_fix_steps(
|
||||
[
|
||||
("install", lambda: self._install_zabbix_agent(settings)),
|
||||
("configure", lambda: self._write_zabbix_agent_config(settings)),
|
||||
("firewall", lambda: self._allow_zabbix_agent_sources(settings)),
|
||||
("start", self._start_zabbix_agent),
|
||||
]
|
||||
)
|
||||
|
||||
def _install_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixStepOutcomeDict:
|
||||
index = self._load_package_index()
|
||||
first = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
|
||||
if first["success"]:
|
||||
return {"ok": True, "output": _joined(index, first.get("output"))}
|
||||
release = settings.get("release")
|
||||
add = getattr(self, "_add_zabbix_repository", None)
|
||||
if not release or not callable(add):
|
||||
return {"ok": False, "output": _joined(index, first.get("output"))}
|
||||
added = add(release)
|
||||
if not added["ok"]:
|
||||
return {"ok": False, "output": _joined(index, added.get("output"))}
|
||||
second = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
|
||||
return {
|
||||
"ok": bool(second["success"]),
|
||||
"output": _joined(index, added.get("output"), second.get("output")),
|
||||
}
|
||||
|
||||
def _load_package_index(self) -> str:
|
||||
"""Load the package index where the driver can (OpenWrt keeps it in RAM, gone
|
||||
after a restart); what went wrong, or "". The install tries either way."""
|
||||
refresh = getattr(self, "refresh_available_updates", None)
|
||||
if not callable(refresh):
|
||||
return ""
|
||||
loaded = refresh()
|
||||
if loaded.get("success"):
|
||||
return ""
|
||||
return f"package index not loaded: {str(loaded.get('output', '')).strip()}"
|
||||
|
||||
def _start_zabbix_agent(self) -> FixStepOutcomeDict:
|
||||
return enable_and_restart(self.manage_service, self.ZABBIX_AGENT_SERVICE)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def ensure_package(self, name: str) -> FixResultDict: ...
|
||||
|
||||
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
|
||||
|
||||
def refresh_available_updates(self) -> dict[str, Any]: ...
|
||||
|
||||
# The steps a driver implements:
|
||||
|
||||
def _add_zabbix_repository(self, release: str) -> FixStepOutcomeDict:
|
||||
"""Add Zabbix's repository for *release* and refresh the package index
|
||||
(``zabbix_repository_command`` builds the command for most hosts)."""
|
||||
...
|
||||
|
||||
def _write_zabbix_agent_config(
|
||||
self, settings: ZabbixAgentSettingsDict
|
||||
) -> FixStepOutcomeDict:
|
||||
"""Write *settings* into the agent's configuration file, keeping the rest of
|
||||
it (``zabbix_agent_config``), and whatever else the device needs before the
|
||||
agent may start."""
|
||||
...
|
||||
|
||||
def _allow_zabbix_agent_sources(
|
||||
self, settings: ZabbixAgentSettingsDict
|
||||
) -> FixStepOutcomeDict:
|
||||
"""Let ``settings["allowed_sources"]`` reach tcp/``listen_port``; ``ok`` with
|
||||
nothing to do where the host has no firewall of its own, or for []."""
|
||||
...
|
||||
|
||||
|
||||
class PackageProxyMixin:
|
||||
"""Point the device's package manager at a caching proxy."""
|
||||
|
||||
@@ -215,5 +332,7 @@ __all__ = [
|
||||
"LldpAgentMixin",
|
||||
"PackageProxyMixin",
|
||||
"SnmpAgentMixin",
|
||||
"ZabbixAgentMixin",
|
||||
"enable_and_restart",
|
||||
"run_fix_steps",
|
||||
]
|
||||
@@ -481,7 +481,12 @@ class LocalAccountsMixin:
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def add_service_account(
|
||||
self, name: str, public_key: str, *, uid: int | None = None
|
||||
self,
|
||||
name: str,
|
||||
public_key: str,
|
||||
*,
|
||||
uid: int | None = None,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
|
||||
root without a password.
|
||||
@@ -495,36 +500,59 @@ class LocalAccountsMixin:
|
||||
once no other account has it; an account that is there keeps its own,
|
||||
as a uid is never changed under files that carry it.
|
||||
|
||||
*replaces* are netOrk's keys *public_key* takes the place of: they go
|
||||
from the account once *public_key* is in it, matched as in
|
||||
:meth:`add_local_account`, and never *public_key* itself. Call it so
|
||||
through a login with *public_key*, tried before: netOrk keeps its way
|
||||
in (NetOrk/netork#2462).
|
||||
|
||||
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
|
||||
``keys``. The new login itself is for the caller to try.
|
||||
``keys``, and ``old keys`` with *replaces* when ``keys`` went. The new
|
||||
login itself is for the caller to try.
|
||||
|
||||
An account that is there and cannot log in (``nologin``, ``false``) is
|
||||
another service's -- a satellite's ``netork``, say -- and is left alone.
|
||||
|
||||
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
|
||||
less, or another service's account.
|
||||
:raises ValueError: for a name or key no host takes, a replaced key that
|
||||
is none, root, a uid of 0 or less, or another service's account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
check_public_key(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != _key_blob(public_key)]
|
||||
login, existing = self._accounts_service_target(name, uid)
|
||||
first = self._accounts_service_account_step(name, login, existing, uid)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += self._accounts_service_keys(name, public_key, old)
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_service_account_step(
|
||||
self, name: str, login: str, existing: list[str] | None, uid: int | None
|
||||
) -> FixResultDict:
|
||||
"""The ``account`` step of netOrk's own account; nothing to do when the driver
|
||||
logs in as *name* already."""
|
||||
if name == login:
|
||||
step: FixStepDict = {
|
||||
"step": "account",
|
||||
"ok": True,
|
||||
"output": f"this driver logs in as {name} already",
|
||||
}
|
||||
first: FixResultDict = _joined([step])
|
||||
else:
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
return _joined([step])
|
||||
return run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
|
||||
def _accounts_service_keys(
|
||||
self, name: str, public_key: str, old: list[str]
|
||||
) -> list[FixResultDict]:
|
||||
"""``sudo`` and ``keys`` of netOrk's own account, then ``old keys`` once the key is in."""
|
||||
parts = [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
if old and parts[-1]["success"]:
|
||||
parts.append(run_fix_steps([("old keys", lambda: self._accounts_drop_keys(name, old))]))
|
||||
return parts
|
||||
|
||||
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
|
||||
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
|
||||
|
||||
@@ -35,6 +35,9 @@ class ForwardRuleDict(TypedDict):
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
#: The template its messages are sent in; None: rsyslog's default, RFC 3164,
|
||||
#: which carries the sender's local time without a zone (NetOrk/netork#493).
|
||||
template: str | None
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
@@ -165,6 +168,10 @@ class _Reader:
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
#: ``$ActionForwardDefaultTemplate``, for the legacy rules after it.
|
||||
self.legacy_template: str | None = None
|
||||
#: ``module(load="builtin:omfwd" template=…)``, for every omfwd action.
|
||||
self.module_template: str | None = None
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
@@ -175,7 +182,13 @@ class _Reader:
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
|
||||
self,
|
||||
protocol: str,
|
||||
target: str,
|
||||
port: int | None,
|
||||
selector: str,
|
||||
once: str | None,
|
||||
template: str | None,
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
@@ -186,6 +199,7 @@ class _Reader:
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
"template": template,
|
||||
}
|
||||
)
|
||||
|
||||
@@ -225,6 +239,8 @@ class _Reader:
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
if params.get("load", "").removeprefix("builtin:") == "omfwd":
|
||||
self.module_template = params.get("template") or self.module_template
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
@@ -240,7 +256,10 @@ class _Reader:
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
|
||||
template = params.get("template") or (self.module_template if kind == "omfwd" else None)
|
||||
self.forward(
|
||||
protocol, params["target"], _port(params.get("port")), selector, once, template
|
||||
)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
@@ -257,6 +276,8 @@ class _Reader:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
elif name == "$actionforwarddefaulttemplate":
|
||||
self.legacy_template = value or None
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
@@ -264,13 +285,27 @@ class _Reader:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
forward = _FORWARD_RE.match(match["action"].strip())
|
||||
action = match["action"].strip()
|
||||
forward = _FORWARD_RE.match(action)
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
|
||||
protocol,
|
||||
forward["v6"] or forward["host"],
|
||||
_port(forward["port"]),
|
||||
selector,
|
||||
once,
|
||||
self.legacy_template_of(action[forward.end() :], protocol),
|
||||
)
|
||||
|
||||
def legacy_template_of(self, rest: str, protocol: str) -> str | None:
|
||||
"""The template after a legacy rule's target (``;Name``), else the default
|
||||
set before it; RELP takes no forwarding default."""
|
||||
_, semicolon, named = rest.partition(";")
|
||||
if semicolon and named.strip():
|
||||
return named.strip()
|
||||
return self.legacy_template if protocol != "relp" else None
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
|
||||
@@ -1172,16 +1172,6 @@ class OspfDict(TypedDict):
|
||||
interfaces: dict[str, OspfInterfaceDict] # by interface name
|
||||
|
||||
|
||||
class FirmwareOfferDict(TypedDict):
|
||||
"""The firmware a device runs and the newest its vendor offers
|
||||
(``FirmwareOfferMixin.get_firmware_offer``). A fact, not a judgement."""
|
||||
|
||||
current_version: str
|
||||
latest_version: str
|
||||
newer: bool # the offer is newer than what runs, by the vendor's versioning
|
||||
download_url: NotRequired[str]
|
||||
|
||||
|
||||
class HypervisorHostDict(TypedDict):
|
||||
"""One host of a hypervisor or cluster (``HypervisorDriver.get_hypervisor_hosts``)."""
|
||||
|
||||
@@ -1442,13 +1432,5 @@ class ManagementProtocolsDict(TypedDict):
|
||||
virtual_media: ProtocolStateDict
|
||||
|
||||
|
||||
class SystemIdentityDict(TypedDict):
|
||||
"""A host's SMBIOS identity, what ties it to its BMC.
|
||||
|
||||
See :mod:`napalm_device_types.system_identity`.
|
||||
|
||||
Each value is empty when unknown or a vendor placeholder."""
|
||||
|
||||
system_uuid: str # lowercase
|
||||
product_serial: str
|
||||
board_serial: str
|
||||
# Importable from here as before 4.41; it lives with its contract now.
|
||||
from napalm_device_types.firmware import FirmwareOfferDict as FirmwareOfferDict # noqa: E402
|
||||
@@ -1,6 +1,7 @@
|
||||
"""The SMBIOS identity of a host: system UUID, product serial, board serial.
|
||||
"""The SMBIOS identity of a host: system UUID, product serial, board serial -- and
|
||||
its system firmware (BIOS or UEFI): vendor, version, release date.
|
||||
|
||||
What ties a server to its BMC. The BMC reports the server's serial number and
|
||||
The identity is what ties a server to its BMC. The BMC reports the server's serial number and
|
||||
system UUID; the host reads the same values from its firmware tables. Both sides
|
||||
read them the same way on every Linux host, so the command, its parse and the
|
||||
placeholder filter live here once, and a driver only carries the command across.
|
||||
@@ -13,27 +14,54 @@ Matching on those would join unrelated machines, so they read as empty.
|
||||
|
||||
**Root.** ``product_uuid`` and ``product_serial`` are readable by root only; without
|
||||
it they come back empty, which is "not known", not a mismatch.
|
||||
|
||||
**The firmware** comes from the same tables in the same round trip, so a driver that
|
||||
reads the identity reads it too. It is a fact about the host, not part of its identity:
|
||||
its placeholders are the vendors' text defaults, never a short value (a version of
|
||||
``1`` is a version), and sysfs's ``MM/DD/YYYY`` release date is given as ISO 8601.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
from datetime import date
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.models import SystemIdentityDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class SystemIdentityDict(TypedDict):
|
||||
"""A host's SMBIOS identity, what ties it to its BMC, and its system firmware.
|
||||
|
||||
Each value is empty when unknown or a vendor placeholder."""
|
||||
|
||||
system_uuid: str # lowercase
|
||||
product_serial: str
|
||||
board_serial: str
|
||||
bios_vendor: str # the system firmware (BIOS or UEFI): its vendor,
|
||||
bios_version: str # its version,
|
||||
bios_date: str # its release date, YYYY-MM-DD
|
||||
|
||||
|
||||
_BEGIN = "IDBEGIN"
|
||||
_END = "IDEND"
|
||||
_DMI = "/sys/class/dmi/id"
|
||||
|
||||
#: Reads the three files in one round trip, framed so a banner or prompt around the
|
||||
#: output cannot shift a field.
|
||||
SYSTEM_IDENTITY_COMMAND = (
|
||||
f"printf '{_BEGIN}\\n%s\\n%s\\n%s\\n{_END}\\n' "
|
||||
f'"$(cat {_DMI}/product_uuid 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/product_serial 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/board_serial 2>/dev/null)"'
|
||||
#: The files read, in the order the output carries them.
|
||||
_FILES = (
|
||||
"product_uuid",
|
||||
"product_serial",
|
||||
"board_serial",
|
||||
"bios_vendor",
|
||||
"bios_version",
|
||||
"bios_date",
|
||||
)
|
||||
|
||||
#: Reads the files in one round trip, framed so a banner or prompt around the output
|
||||
#: cannot shift a field.
|
||||
_FIELDS = "%s\\n" * len(_FILES)
|
||||
SYSTEM_IDENTITY_COMMAND = f"printf '{_BEGIN}\\n{_FIELDS}{_END}\\n' " + " ".join(
|
||||
f'"$(cat {_DMI}/{name} 2>/dev/null)"' for name in _FILES
|
||||
)
|
||||
|
||||
#: Lowercased defaults board vendors leave in place.
|
||||
@@ -80,6 +108,22 @@ def _clean(value: str) -> str:
|
||||
return "" if is_placeholder_identity(value) else value
|
||||
|
||||
|
||||
def _clean_text(value: str) -> str:
|
||||
"""A firmware text, empty for a vendor's text default; a short value stays."""
|
||||
value = value.strip()
|
||||
return "" if value.lower() in _PLACEHOLDERS else value
|
||||
|
||||
|
||||
def _iso_date(value: str) -> str:
|
||||
"""sysfs's ``MM/DD/YYYY`` as ``YYYY-MM-DD``; anything else is no date."""
|
||||
month, _, rest = value.strip().partition("/")
|
||||
day, _, year = rest.partition("/")
|
||||
try:
|
||||
return date(int(year), int(month), int(day)).isoformat()
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
|
||||
def parse_system_identity(output: str) -> SystemIdentityDict:
|
||||
"""The identity from :data:`SYSTEM_IDENTITY_COMMAND`'s output, placeholders empty.
|
||||
|
||||
@@ -91,12 +135,16 @@ def parse_system_identity(output: str) -> SystemIdentityDict:
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("system identity: no intact IDBEGIN/IDEND frame") from None
|
||||
fields = (lines[start:end] + ["", "", ""])[:3]
|
||||
uuid, product_serial, board_serial = (_clean(f) for f in fields)
|
||||
fields = (lines[start:end] + [""] * len(_FILES))[: len(_FILES)]
|
||||
uuid, product_serial, board_serial = (_clean(f) for f in fields[:3])
|
||||
bios_vendor, bios_version, bios_date = fields[3:]
|
||||
return {
|
||||
"system_uuid": uuid.lower(),
|
||||
"product_serial": product_serial,
|
||||
"board_serial": board_serial,
|
||||
"bios_vendor": _clean_text(bios_vendor),
|
||||
"bios_version": _clean_text(bios_version),
|
||||
"bios_date": _iso_date(bios_date),
|
||||
}
|
||||
|
||||
|
||||
@@ -119,12 +167,15 @@ class SystemIdentityMixin:
|
||||
|
||||
def get_system_identity(self) -> SystemIdentityDict:
|
||||
"""
|
||||
Returns the host's SMBIOS identity, each value empty when unknown or a vendor
|
||||
placeholder:
|
||||
Returns the host's SMBIOS identity and system firmware, each value empty when
|
||||
unknown or a vendor placeholder:
|
||||
|
||||
* system_uuid (str) - lowercase
|
||||
* product_serial (str)
|
||||
* board_serial (str)
|
||||
* bios_vendor (str) - the system firmware's vendor
|
||||
* bios_version (str)
|
||||
* bios_date (str) - its release date, ``YYYY-MM-DD``
|
||||
|
||||
:raises ValueError: if the output carried no intact reading.
|
||||
"""
|
||||
|
||||
@@ -17,6 +17,15 @@ from typing import TYPE_CHECKING, Any
|
||||
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict, UpdateProgressDict
|
||||
|
||||
|
||||
class PackageIndexMissing(RuntimeError):
|
||||
"""The host holds no package index to read pending updates from.
|
||||
|
||||
Not a fault: OpenWrt keeps its index in RAM and loses it on every reboot.
|
||||
:meth:`UpdateMixin.refresh_available_updates` loads it again, so a caller
|
||||
that sees this can refresh instead of reporting a failed read.
|
||||
"""
|
||||
|
||||
|
||||
class UpdateMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
@@ -37,6 +46,8 @@ class UpdateMixin:
|
||||
read -- no package index yet, an API that did not answer -- raises
|
||||
instead: netOrk keeps "pending since" per package, and an empty
|
||||
list for "don't know" would reset every one of those clocks.
|
||||
Without a package index it raises :exc:`PackageIndexMissing`, which
|
||||
:meth:`refresh_available_updates` cures.
|
||||
|
||||
Example::
|
||||
|
||||
@@ -86,7 +97,8 @@ class UpdateMixin:
|
||||
"""
|
||||
Where an update the device runs by itself stands, for a caller that
|
||||
follows it: a firewall's firmware upgrade, which :meth:`apply_updates`
|
||||
only starts. ``state`` is ``"running"`` until it ends in ``"done"``
|
||||
only starts, or one that ``FirmwareUpgradeMixin.start_firmware_upgrade``
|
||||
started. ``state`` is ``"running"`` until it ends in ``"done"``
|
||||
or ``"failed"``; ``log`` is the device's whole log of it.
|
||||
|
||||
Declared only: a driver whose :meth:`apply_updates` returns when the
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
"""A native Zabbix agent: its configuration and Zabbix's package repository.
|
||||
|
||||
What :class:`~napalm_device_types.fixes.ZabbixAgentMixin` needs that is the same on
|
||||
every host, without I/O: which settings an agent may be given
|
||||
(:func:`check_zabbix_agent_settings`), the agent's configuration file with them
|
||||
(:func:`zabbix_agent_config`), and the command that adds Zabbix's repository where
|
||||
the host's own packages have no agent (:func:`zabbix_repository_command`). A driver
|
||||
reads and writes the file and runs the command.
|
||||
|
||||
**Which lines.** ``Server`` (who may ask, for passive checks), ``ServerActive``
|
||||
(where active checks go), ``ListenPort`` and, when the caller names one,
|
||||
``Hostname`` -- the host's technical name in Zabbix, which an active agent sends.
|
||||
Every other line stays as it was; a key the file does not set is added at its end.
|
||||
|
||||
**The repository.** Zabbix publishes a ``zabbix-release`` package per release line
|
||||
and distribution that adds its repository and key. Since 7.2 it lives under
|
||||
``<release>/release/<distribution>/``; up to 7.0 under ``<release>/<distribution>/``,
|
||||
with the rpm in an ``x86_64`` directory. The release line is the server's: an agent
|
||||
may be older than its server, not newer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
|
||||
class ZabbixAgentSettingsDict(TypedDict):
|
||||
"""What the caller wants a native Zabbix agent to be (``ZabbixAgentMixin``)."""
|
||||
|
||||
server: str # Server: the addresses that may ask (passive checks), comma-separated
|
||||
server_active: str # ServerActive: where active checks go, comma-separated; "" for none
|
||||
listen_port: int # ListenPort
|
||||
# addresses or CIDRs let through to the port; [] changes no firewall
|
||||
allowed_sources: list[str]
|
||||
# Hostname: the host's technical name in Zabbix; absent keeps the agent's
|
||||
hostname: NotRequired[str]
|
||||
# the server's release line ("7.4"): Zabbix's repository to fall back to where the
|
||||
# host's own packages have no agent; absent: no fallback
|
||||
release: NotRequired[str]
|
||||
|
||||
|
||||
#: Addresses, comma-separated, as ``Server`` and ``ServerActive`` take them: names,
|
||||
#: IPv4, ``[IPv6]`` and ports. Nothing a config line or a shell word breaks out of.
|
||||
_ADDRESSES = re.compile(r"^[A-Za-z0-9.:\[\]-]+(,[A-Za-z0-9.:\[\]-]+)*$")
|
||||
#: What Zabbix allows in a technical host name.
|
||||
_HOSTNAME = re.compile(r"^[A-Za-z0-9._ -]+$")
|
||||
#: A release line, as Zabbix's repository names it.
|
||||
_RELEASE = re.compile(r"^(\d+)\.(\d+)$")
|
||||
|
||||
_KEYS = ("Server", "ServerActive", "ListenPort", "Hostname")
|
||||
_LINE = re.compile(r"^(\s*)(" + "|".join(_KEYS) + r")\s*=\s*(.*?)\s*$")
|
||||
|
||||
REPOSITORY = "https://repo.zabbix.com/zabbix"
|
||||
#: The distributions Zabbix publishes packages for, by ``/etc/os-release``'s ``ID``:
|
||||
#: the name in the repository's path, and whether they take a deb.
|
||||
_DISTRIBUTIONS: dict[str, tuple[str, bool]] = {
|
||||
"ubuntu": ("ubuntu", True),
|
||||
"debian": ("debian", True),
|
||||
"raspbian": ("raspbian", True),
|
||||
"rhel": ("rhel", False),
|
||||
"centos": ("rhel", False),
|
||||
"ol": ("oracle", False),
|
||||
"almalinux": ("alma", False),
|
||||
"rocky": ("rocky", False),
|
||||
}
|
||||
|
||||
|
||||
def check_zabbix_agent_settings(settings: ZabbixAgentSettingsDict) -> None:
|
||||
"""Refuse what an agent's configuration or a shell line cannot take.
|
||||
|
||||
:raises ValueError: naming the setting.
|
||||
"""
|
||||
_check_addresses("Server", str(settings.get("server", "")), required=True)
|
||||
_check_addresses("ServerActive", str(settings.get("server_active", "")), required=False)
|
||||
hostname = settings.get("hostname")
|
||||
if hostname is not None and not _HOSTNAME.match(hostname):
|
||||
raise ValueError(f"not a Zabbix host name: {hostname!r}")
|
||||
port = settings.get("listen_port")
|
||||
if not isinstance(port, int) or isinstance(port, bool) or not 1 <= port <= 65535:
|
||||
raise ValueError(f"not a port: {port!r}")
|
||||
release = settings.get("release")
|
||||
if release is not None and not _RELEASE.match(release):
|
||||
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
|
||||
|
||||
|
||||
def _check_addresses(key: str, value: str, *, required: bool) -> None:
|
||||
if (value or required) and not _ADDRESSES.match(value):
|
||||
raise ValueError(f"not addresses an agent's {key} takes: {value!r}")
|
||||
|
||||
|
||||
def _wanted(settings: ZabbixAgentSettingsDict) -> dict[str, str]:
|
||||
wanted = {
|
||||
"Server": settings["server"],
|
||||
"ServerActive": settings.get("server_active", ""),
|
||||
"ListenPort": str(settings["listen_port"]),
|
||||
}
|
||||
if settings.get("hostname"):
|
||||
wanted["Hostname"] = str(settings["hostname"])
|
||||
return wanted
|
||||
|
||||
|
||||
def zabbix_agent_config(text: str, settings: ZabbixAgentSettingsDict) -> tuple[str, list[str]]:
|
||||
"""*text*, an agent's configuration file, with *settings*; and what changed, one
|
||||
``"Key: old → new"`` each.
|
||||
|
||||
An empty ``server_active`` comments ``ServerActive`` out: the agent then runs
|
||||
passive checks only.
|
||||
"""
|
||||
wanted = _wanted(settings)
|
||||
lines: list[str] = []
|
||||
changes: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for line in text.splitlines(keepends=True):
|
||||
match = _LINE.match(line.rstrip("\r\n"))
|
||||
if not match or match.group(2) not in wanted:
|
||||
lines.append(line)
|
||||
continue
|
||||
seen.add(match.group(2))
|
||||
lines.append(_set_line(line, match, wanted, changes))
|
||||
_add_missing(
|
||||
lines, [key for key in _KEYS if wanted.get(key) and key not in seen], wanted, changes
|
||||
)
|
||||
return "".join(lines), changes
|
||||
|
||||
|
||||
def _add_missing(
|
||||
lines: list[str], keys: list[str], wanted: dict[str, str], changes: list[str]
|
||||
) -> None:
|
||||
"""*keys*, which the file does not set, at its end."""
|
||||
if keys and lines and not lines[-1].endswith("\n"):
|
||||
lines[-1] += "\n"
|
||||
for key in keys:
|
||||
lines.append(f"{key}={wanted[key]}\n")
|
||||
changes.append(f"{key}: (not set) → {wanted[key]}")
|
||||
|
||||
|
||||
def _set_line(line: str, match: re.Match[str], wanted: dict[str, str], changes: list[str]) -> str:
|
||||
"""*line*, which sets a key the caller wants, as the caller wants it."""
|
||||
indent, key, old = match.groups()
|
||||
end = "\n" if line.endswith("\n") else ""
|
||||
new = wanted[key]
|
||||
if not new:
|
||||
changes.append(f"{key}: {old} → (not set)")
|
||||
return f"{indent}# {key}={old}{end}"
|
||||
if old == new:
|
||||
return line
|
||||
changes.append(f"{key}: {old} → {new}")
|
||||
return f"{indent}{key}={new}{end}"
|
||||
|
||||
|
||||
def _os_release(text: str) -> dict[str, str]:
|
||||
fields = {}
|
||||
for line in text.splitlines():
|
||||
key, sep, value = line.partition("=")
|
||||
if sep:
|
||||
fields[key.strip()] = value.strip().strip("'\"")
|
||||
return fields
|
||||
|
||||
|
||||
def _distribution(fields: dict[str, str]) -> tuple[str, bool]:
|
||||
for candidate in [fields.get("ID", ""), *fields.get("ID_LIKE", "").split()]:
|
||||
if candidate in _DISTRIBUTIONS:
|
||||
return _DISTRIBUTIONS[candidate]
|
||||
raise ValueError(f"Zabbix publishes no repository for {fields.get('ID') or 'this system'}")
|
||||
|
||||
|
||||
def zabbix_repository_command(os_release: str, release: str) -> str:
|
||||
"""The root command that adds Zabbix's *release* repository to the host whose
|
||||
``/etc/os-release`` is *os_release*, and refreshes the package index.
|
||||
|
||||
:raises ValueError: for a release line that is not ``major.minor``, or a
|
||||
distribution Zabbix publishes nothing for.
|
||||
"""
|
||||
match = _RELEASE.match(release)
|
||||
if not match:
|
||||
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
|
||||
fields = _os_release(os_release)
|
||||
name, deb = _distribution(fields)
|
||||
version = fields.get("VERSION_ID", "")
|
||||
new_layout = (int(match.group(1)), int(match.group(2))) >= (7, 2)
|
||||
base = f"{REPOSITORY}/{release}/" + ("release/" if new_layout else "") + name
|
||||
if deb:
|
||||
# Raspberry Pi OS's packages are named after the Debian release they follow.
|
||||
label = "debian" if name == "raspbian" else name
|
||||
filename = f"zabbix-release_latest_{release}+{label}{version}_all.deb"
|
||||
url = f"{base}/pool/main/z/zabbix-release/{filename}"
|
||||
package = shlex.quote(url)
|
||||
return (
|
||||
"t=$(mktemp -d) && "
|
||||
f'{{ curl -fsSL -o "$t/zabbix-release.deb" {package} '
|
||||
f'|| wget -qO "$t/zabbix-release.deb" {package}; }} && '
|
||||
'dpkg -i "$t/zabbix-release.deb" && apt-get update; '
|
||||
# The download's directory goes either way; the status is the install's.
|
||||
's=$?; rm -rf "$t"; [ "$s" -eq 0 ]'
|
||||
)
|
||||
major = version.split(".")[0]
|
||||
arch = "noarch" if new_layout else "x86_64"
|
||||
url = f"{base}/{major}/{arch}/zabbix-release-latest-{release}.el{major}.noarch.rpm"
|
||||
return f"dnf -y install {shlex.quote(url)}"
|
||||
|
||||
|
||||
__all__ = [
|
||||
"REPOSITORY",
|
||||
"ZabbixAgentSettingsDict",
|
||||
"check_zabbix_agent_settings",
|
||||
"zabbix_agent_config",
|
||||
"zabbix_repository_command",
|
||||
]
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.37.0"
|
||||
version = "4.41.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
},
|
||||
"function_lines": {
|
||||
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 119,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_reservations": 54,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_subnets": 60,
|
||||
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
|
||||
@@ -32,7 +31,7 @@
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1454
|
||||
"napalm_device_types/models.py": 1436
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
+195
-1
@@ -25,6 +25,7 @@ from napalm_device_types.apt_proxy import (
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.apt_sources import NOT_CACHED_KEY, VIA_CACHE
|
||||
from napalm_device_types.fixes import PackageProxyMixin
|
||||
|
||||
|
||||
@@ -302,6 +303,13 @@ _CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/
|
||||
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
|
||||
E: Download Failed"""
|
||||
|
||||
#: A cache that does not fetch the cache form itself: ``AllowUserPorts`` is 80 alone
|
||||
#: (apt-cacher-ng 3.7.5, NetOrk/netork#2555).
|
||||
_PROHIBITED_PORT = """Err:1 http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease
|
||||
403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
|
||||
E: Failed to fetch http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease 403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
|
||||
E: Download Failed"""
|
||||
|
||||
|
||||
def _fetch(output, rc, match=None):
|
||||
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
|
||||
@@ -422,9 +430,21 @@ class _Host(AptProxyMixin):
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
|
||||
self,
|
||||
files=None,
|
||||
fetch_rc=0,
|
||||
apt=True,
|
||||
https_sources=(),
|
||||
tunnel=True,
|
||||
direct_rc=0,
|
||||
cache_https=False,
|
||||
credentials=(),
|
||||
):
|
||||
self.files = dict(files or {})
|
||||
#: Whether the proxy fetches the cache form itself (AllowUserPorts 443), and
|
||||
#: the hosts apt keeps credentials for.
|
||||
self.cache_https = cache_https
|
||||
self.credentials = set(credentials)
|
||||
self.fetch_rc = fetch_rc
|
||||
self.apt = apt
|
||||
self.https_sources = list(https_sources)
|
||||
@@ -459,6 +479,12 @@ class _Host(AptProxyMixin):
|
||||
value, https = self._effective(), self._effective("https")
|
||||
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
|
||||
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
|
||||
dump += [
|
||||
line
|
||||
for path, text in self.files.items()
|
||||
for line in text.splitlines()
|
||||
if line.startswith(NOT_CACHED_KEY)
|
||||
]
|
||||
return _report(
|
||||
dump=dump,
|
||||
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
|
||||
@@ -466,6 +492,8 @@ class _Host(AptProxyMixin):
|
||||
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
|
||||
)
|
||||
self.fetched.append(command)
|
||||
if re.search(r"download-file '?http://HTTPS///", command):
|
||||
return _fetch(_FETCHED, 0) if self.cache_https else _fetch(_PROHIBITED_PORT, 100)
|
||||
if re.search(r"download-file '?https://", command):
|
||||
return self._fetch_https(command)
|
||||
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
|
||||
@@ -482,6 +510,11 @@ class _Host(AptProxyMixin):
|
||||
|
||||
def _run_apt_root_command(self, command, done):
|
||||
self.root.append(command)
|
||||
if NOT_CACHED_KEY in command:
|
||||
return self._rewrite(re.search(r"for h in ([^;]+); do", command).group(1).split(), done)
|
||||
if "#\\1https://#g" in command:
|
||||
self.https_sources = [s.replace(VIA_CACHE, "https://") for s in self.https_sources]
|
||||
return {"ok": True, "output": done}
|
||||
path = command.rsplit(" ", 1)[-1].strip("'")
|
||||
for m in re.finditer(r"(\d+)\{/[^}]*\}", command):
|
||||
n = int(m.group(1))
|
||||
@@ -492,6 +525,19 @@ class _Host(AptProxyMixin):
|
||||
self.files.pop(command.split()[-1].strip("'"), None)
|
||||
return {"ok": True, "output": done}
|
||||
|
||||
def _rewrite(self, hosts, done):
|
||||
for host in hosts:
|
||||
if host in self.credentials:
|
||||
proxy = self.files.get(f"{_D}00proxy", "")
|
||||
self.files[f"{_D}00proxy"] = proxy + f'{NOT_CACHED_KEY}:: "{host}";\n'
|
||||
continue
|
||||
prefix = f"https://{host}/"
|
||||
self.https_sources = [
|
||||
VIA_CACHE + s[len("https://") :] if s.startswith(prefix) else s
|
||||
for s in self.https_sources
|
||||
]
|
||||
return {"ok": True, "output": done}
|
||||
|
||||
|
||||
class TestCheckHttpsSource:
|
||||
def test_one_source_the_way_apt_fetches_it(self):
|
||||
@@ -569,8 +615,10 @@ class TestConfigurePackageProxy:
|
||||
"conflicts",
|
||||
"verify",
|
||||
"fetch",
|
||||
"cache",
|
||||
"https",
|
||||
]
|
||||
assert "no https source" in _step(result, "cache")["output"]
|
||||
assert "no https source" in result["steps"][-1]["output"]
|
||||
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
|
||||
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
|
||||
@@ -766,6 +814,144 @@ class TestConfigurePackageProxy:
|
||||
assert host.files == {}
|
||||
|
||||
|
||||
class TestCacheHttpsSources:
|
||||
"""NetOrk/netork#2555: the cache stores an https source only in its own form."""
|
||||
|
||||
def test_a_cache_that_fetches_https_gets_them(self):
|
||||
host = _Host(https_sources=[_DOCKER], cache_https=True)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.https_sources == [f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"]
|
||||
assert _step(result, "cache")["output"] == (
|
||||
"https sources cached through http://c:3142: download.docker.com"
|
||||
)
|
||||
assert any(VIA_CACHE in c for c in host.fetched)
|
||||
assert "no https source goes through" in _step(result, "https")["output"]
|
||||
|
||||
def test_a_cache_that_does_not_turns_them_back(self):
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.https_sources == [_DOCKER]
|
||||
output = _step(result, "cache")["output"]
|
||||
assert "http://c:3142 does not fetch https sources itself" in output
|
||||
assert "prohibited port (see AllowUserPorts)" in output
|
||||
assert "they stay on https, not cached" in output
|
||||
assert _step(result, "https")["output"] == (
|
||||
f"fetched {_DOCKER}InRelease through http://c:3142"
|
||||
)
|
||||
|
||||
def test_a_host_with_credentials_stays_on_https_and_is_named(self):
|
||||
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
|
||||
host = _Host(
|
||||
https_sources=[pve, _DOCKER],
|
||||
cache_https=True,
|
||||
credentials={"enterprise.proxmox.com"},
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert pve in host.https_sources
|
||||
output = _step(result, "cache")["output"]
|
||||
assert "cached through http://c:3142: download.docker.com" in output
|
||||
assert "left on https, apt has credentials for them: enterprise.proxmox.com" in output
|
||||
assert host.get_package_proxy_report()["not_cached"] == ["enterprise.proxmox.com"]
|
||||
|
||||
def test_only_hosts_with_credentials(self):
|
||||
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
|
||||
host = _Host(https_sources=[pve], cache_https=True, credentials={"enterprise.proxmox.com"})
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert _step(result, "cache")["output"] == (
|
||||
"no https source to cache; "
|
||||
"left on https, apt has credentials for them: enterprise.proxmox.com"
|
||||
)
|
||||
|
||||
def test_the_https_direct_fallback_keeps_the_hosts_left_on_https(self):
|
||||
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
|
||||
host = _Host(https_sources=[pve], tunnel=False, credentials={"enterprise.proxmox.com"})
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}00proxy"] == (
|
||||
'Acquire::http::Proxy "http://c:3142";\n'
|
||||
'Acquire::https::Proxy "DIRECT";\n'
|
||||
f'{NOT_CACHED_KEY}:: "enterprise.proxmox.com";\n'
|
||||
)
|
||||
|
||||
def test_a_rewrite_that_fails_stops_the_fix(self):
|
||||
host = _Host(https_sources=[_DOCKER], cache_https=True)
|
||||
real = host._run_apt_root_command
|
||||
|
||||
def denied(command, done):
|
||||
if NOT_CACHED_KEY in command:
|
||||
return {"ok": False, "output": "sed: denied"}
|
||||
return real(command, done)
|
||||
|
||||
host._run_apt_root_command = denied
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1] == {"step": "cache", "ok": False, "output": "sed: denied"}
|
||||
|
||||
def test_a_turn_back_that_fails_says_so(self):
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
real = host._run_apt_root_command
|
||||
|
||||
def denied(command, done):
|
||||
if "#\\1https://#g" in command:
|
||||
return {"ok": False, "output": "sed: denied"}
|
||||
return real(command, done)
|
||||
|
||||
host._run_apt_root_command = denied
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
output = result["steps"][-1]["output"]
|
||||
assert "turning the sources back to https failed: sed: denied" in output
|
||||
|
||||
def test_the_proof_of_the_proxy_is_a_plain_http_source(self):
|
||||
"""A source already in the cache form says nothing about plain http."""
|
||||
host = _Host(https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"])
|
||||
|
||||
host.configure_package_proxy("http://c:3142")
|
||||
|
||||
fetch = next(c for c in host.fetched if "InRelease" in c)
|
||||
assert "deb.debian.org" in fetch
|
||||
|
||||
def test_removing_the_proxy_turns_the_sources_back_first(self):
|
||||
host = _Host(
|
||||
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
|
||||
https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"],
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == ["sources", "configure"]
|
||||
assert host.https_sources == [_DOCKER]
|
||||
assert host.files == {}
|
||||
|
||||
def test_a_turn_back_that_fails_keeps_the_proxy(self):
|
||||
"""Without the cache apt looks for a host called HTTPS: the proxy stays."""
|
||||
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
|
||||
host._run_apt_root_command = lambda command, done: {"ok": False, "output": "denied"}
|
||||
|
||||
result = host.configure_package_proxy("")
|
||||
|
||||
assert result["success"] is False
|
||||
assert f"{_D}00proxy" in host.files
|
||||
|
||||
|
||||
class TestWhatTheParserSkips:
|
||||
def test_a_line_that_is_no_grep_output(self):
|
||||
assert (
|
||||
@@ -824,3 +1010,11 @@ def test_the_comment_out_command_for_https():
|
||||
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
|
||||
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
|
||||
)
|
||||
|
||||
|
||||
def test_a_line_left_on_https_that_is_no_host_name_is_not_written_back():
|
||||
host = _Host()
|
||||
host._write_package_proxy("http://c:3142", not_cached=["ok.example", 'x"; Foo "1'])
|
||||
assert host.files[f"{_D}00proxy"] == (
|
||||
f'Acquire::http::Proxy "http://c:3142";\n{NOT_CACHED_KEY}:: "ok.example";\n'
|
||||
)
|
||||
@@ -0,0 +1,393 @@
|
||||
"""apt's sources and the release they name, and moving one file's sources to the host's.
|
||||
|
||||
A release upgrade keeps third-party sources on the old release: z2m-house, Ubuntu
|
||||
24.04 (noble), kept Docker's source on ``focal`` and never saw a Docker newer than
|
||||
the last focal build (NetOrk/netork#2614). The read command runs here for real, on
|
||||
files in a temporary directory that ``apt-config -o`` points apt's settings at.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.apt_proxy import AptProxyMixin
|
||||
from napalm_device_types.apt_releases import (
|
||||
AptReleaseMixin,
|
||||
apt_releases_command,
|
||||
parse_apt_releases,
|
||||
rewrite_release,
|
||||
suite_release,
|
||||
)
|
||||
|
||||
_APT_CONFIG = shutil.which("apt-config")
|
||||
needs_apt = pytest.mark.skipif(_APT_CONFIG is None, reason="apt-config is not on this host")
|
||||
|
||||
_NOBLE = (
|
||||
'PRETTY_NAME="Ubuntu 24.04.3 LTS"\nNAME="Ubuntu"\nVERSION_ID="24.04"\n'
|
||||
"VERSION_CODENAME=noble\nID=ubuntu\nID_LIKE=debian\nUBUNTU_CODENAME=noble\n"
|
||||
)
|
||||
_TRIXIE = 'PRETTY_NAME="Debian GNU/Linux 13 (trixie)"\nVERSION_CODENAME=trixie\nID=debian\n'
|
||||
_MINT = (
|
||||
'NAME="Linux Mint"\nVERSION_CODENAME=virginia\nID=linuxmint\nID_LIKE="ubuntu debian"\n'
|
||||
"UBUNTU_CODENAME=jammy\n"
|
||||
)
|
||||
|
||||
#: z2m-house's Docker source, as found: through the cache, on focal (#2614).
|
||||
_Z2M_DOCKER = "deb http://HTTPS///download.docker.com/linux/ubuntu focal stable\n"
|
||||
_Z2M_DOCKER_FILE = "archive_uri-https_download_docker_com_linux_ubuntu-noble.list"
|
||||
_UBUNTU_SOURCES = (
|
||||
"Types: deb\n"
|
||||
"URIs: http://archive.ubuntu.com/ubuntu/\n"
|
||||
"Suites: noble noble-updates noble-backports\n"
|
||||
"Components: main restricted universe multiverse\n"
|
||||
"Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n"
|
||||
"\n"
|
||||
"Types: deb\n"
|
||||
"URIs: http://security.ubuntu.com/ubuntu/\n"
|
||||
"Suites: noble-security\n"
|
||||
"Components: main restricted universe multiverse\n"
|
||||
)
|
||||
|
||||
|
||||
def _framed(os_release: str, files: dict[str, str]) -> str:
|
||||
"""What :func:`apt_releases_command` prints for *os_release* and *files*."""
|
||||
lines = ["ARELEASE_BEGIN", "@@os"]
|
||||
lines += [f"|{line}" for line in os_release.splitlines()]
|
||||
for path, text in files.items():
|
||||
lines.append(f"@@file {path}")
|
||||
lines += [f"|{line}" for line in text.splitlines()]
|
||||
lines.append("ARELEASE_END")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
class TestSuiteRelease:
|
||||
@pytest.mark.parametrize(
|
||||
("suite", "release"),
|
||||
[
|
||||
("noble", "noble"),
|
||||
("noble-security", "noble"),
|
||||
("noble-updates", "noble"),
|
||||
("focal-backports", "focal"),
|
||||
("bookworm-proposed-updates", "bookworm"),
|
||||
("trixie-security", "trixie"),
|
||||
],
|
||||
)
|
||||
def test_the_release_a_suite_names(self, suite, release):
|
||||
assert suite_release(suite) == release
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("suite", "word"),
|
||||
[("stable", "stable"), ("nodistro", "nodistro"), ("pve-no-subscription", "pve")],
|
||||
)
|
||||
def test_a_word_the_caller_judges(self, suite, word):
|
||||
# Which words are releases is the caller's to know: "stable" has the form too.
|
||||
assert suite_release(suite) == word
|
||||
|
||||
@pytest.mark.parametrize("suite", ["/", "./", "focal/", "Focal", "$(ARCH)", ""])
|
||||
def test_a_suite_of_another_form(self, suite):
|
||||
# A flat repository's path, or anything not a lower-case word, names none.
|
||||
assert suite_release(suite) is None
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_the_host_and_a_list_source(self):
|
||||
found = parse_apt_releases(
|
||||
_framed(_NOBLE, {f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}": _Z2M_DOCKER})
|
||||
)
|
||||
assert found["os_id"] == "ubuntu"
|
||||
assert found["os_like"] == ["debian"]
|
||||
assert found["codename"] == "noble"
|
||||
assert found["ubuntu_codename"] == "noble"
|
||||
assert found["sources"] == [
|
||||
{
|
||||
"file": f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}",
|
||||
"line": 1,
|
||||
"types": ["deb"],
|
||||
"uri": "http://HTTPS///download.docker.com/linux/ubuntu",
|
||||
"suites": ["focal"],
|
||||
"components": ["stable"],
|
||||
}
|
||||
]
|
||||
|
||||
def test_options_comments_and_disabled_lines(self):
|
||||
text = (
|
||||
"# deb http://old.example/ubuntu bionic main\n"
|
||||
"\n"
|
||||
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
|
||||
"https://download.docker.com/linux/ubuntu jammy stable # moved by hand\n"
|
||||
"deb-src [ arch=amd64 ] http://archive.ubuntu.com/ubuntu noble main\n"
|
||||
"deb https://repo.example/flat ./\n"
|
||||
)
|
||||
found = parse_apt_releases(_framed(_NOBLE, {"/etc/apt/sources.list": text}))
|
||||
assert [
|
||||
(s["line"], s["types"], s["uri"], s["suites"], s["components"])
|
||||
for s in found["sources"]
|
||||
] == [
|
||||
(3, ["deb"], "https://download.docker.com/linux/ubuntu", ["jammy"], ["stable"]),
|
||||
(4, ["deb-src"], "http://archive.ubuntu.com/ubuntu", ["noble"], ["main"]),
|
||||
(5, ["deb"], "https://repo.example/flat", ["./"], []),
|
||||
]
|
||||
|
||||
def test_deb822_paragraphs_one_per_uri(self):
|
||||
text = (
|
||||
"Types: deb deb-src\n"
|
||||
"URIs: http://a.example/ubuntu http://b.example/ubuntu\n"
|
||||
"Suites: focal\n"
|
||||
" focal-updates\n"
|
||||
"Components: main\n"
|
||||
"\n"
|
||||
"# a comment between paragraphs\n"
|
||||
"Enabled: no\n"
|
||||
"Types: deb\n"
|
||||
"URIs: http://c.example/ubuntu\n"
|
||||
"Suites: bionic\n"
|
||||
"Components: main\n"
|
||||
)
|
||||
found = parse_apt_releases(_framed(_NOBLE, {"/etc/apt/sources.list.d/x.sources": text}))
|
||||
assert [(s["line"], s["types"], s["uri"], s["suites"]) for s in found["sources"]] == [
|
||||
(3, ["deb", "deb-src"], "http://a.example/ubuntu", ["focal", "focal-updates"]),
|
||||
(3, ["deb", "deb-src"], "http://b.example/ubuntu", ["focal", "focal-updates"]),
|
||||
]
|
||||
|
||||
def test_debian_and_a_derivative(self):
|
||||
assert parse_apt_releases(_framed(_TRIXIE, {}))["codename"] == "trixie"
|
||||
mint = parse_apt_releases(_framed(_MINT, {}))
|
||||
assert (mint["os_id"], mint["os_like"], mint["codename"], mint["ubuntu_codename"]) == (
|
||||
"linuxmint",
|
||||
["ubuntu", "debian"],
|
||||
"virginia",
|
||||
"jammy",
|
||||
)
|
||||
|
||||
def test_without_os_release(self):
|
||||
found = parse_apt_releases("ARELEASE_BEGIN\n@@os\nARELEASE_END\n")
|
||||
assert (found["os_id"], found["codename"], found["ubuntu_codename"]) == ("", "", "")
|
||||
|
||||
def test_an_unreadable_file_is_left_out(self):
|
||||
out = _framed(_NOBLE, {}).replace(
|
||||
"ARELEASE_END", "@@unreadable /etc/apt/sources.list\nARELEASE_END"
|
||||
)
|
||||
assert parse_apt_releases(out)["sources"] == []
|
||||
|
||||
def test_no_intact_report(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_apt_releases("ARELEASE_BEGIN\n@@os\n")
|
||||
|
||||
|
||||
@needs_apt
|
||||
class TestTheCommand:
|
||||
def test_reads_list_and_sources_files_as_apt_names_them(self, tmp_path):
|
||||
(tmp_path / "sources.list.d").mkdir()
|
||||
(tmp_path / "sources.list").write_text("")
|
||||
(tmp_path / "sources.list.d" / _Z2M_DOCKER_FILE).write_text(_Z2M_DOCKER)
|
||||
(tmp_path / "sources.list.d" / "ubuntu.sources").write_text(_UBUNTU_SOURCES)
|
||||
(tmp_path / "sources.list.d" / "old.list.distUpgrade").write_text(_Z2M_DOCKER)
|
||||
(tmp_path / "os-release").write_text(_NOBLE)
|
||||
command = apt_releases_command(
|
||||
apt_config=(
|
||||
f"apt-config -o Dir::Etc::sourcelist={tmp_path}/sources.list "
|
||||
f"-o Dir::Etc::sourceparts={tmp_path}/sources.list.d/"
|
||||
),
|
||||
os_release=str(tmp_path / "os-release"),
|
||||
)
|
||||
done = subprocess.run(["sh", "-c", command], capture_output=True, text=True, check=False)
|
||||
found = parse_apt_releases(done.stdout)
|
||||
assert found["codename"] == "noble"
|
||||
assert [(s["file"].rsplit("/", 1)[-1], s["suites"]) for s in found["sources"]] == [
|
||||
(_Z2M_DOCKER_FILE, ["focal"]),
|
||||
("ubuntu.sources", ["noble", "noble-updates", "noble-backports"]),
|
||||
("ubuntu.sources", ["noble-security"]),
|
||||
]
|
||||
|
||||
|
||||
class TestRewriteRelease:
|
||||
def test_a_list_line(self):
|
||||
assert rewrite_release(_Z2M_DOCKER, "focal", "noble", deb822=False) == (
|
||||
"deb http://HTTPS///download.docker.com/linux/ubuntu noble stable\n",
|
||||
1,
|
||||
)
|
||||
|
||||
def test_suffixes_options_and_a_uri_that_names_the_release(self):
|
||||
text = (
|
||||
"deb [signed-by=/k.asc] http://repo.example/focal/ focal-security main # focal\n"
|
||||
"# deb http://repo.example/focal/ focal main\n"
|
||||
"deb http://repo.example/other noble main\n"
|
||||
"deb http://repo.example/flat ./\n"
|
||||
)
|
||||
assert rewrite_release(text, "focal", "noble", deb822=False) == (
|
||||
"deb [signed-by=/k.asc] http://repo.example/focal/ noble-security main # focal\n"
|
||||
"# deb http://repo.example/focal/ focal main\n"
|
||||
"deb http://repo.example/other noble main\n"
|
||||
"deb http://repo.example/flat ./\n",
|
||||
1,
|
||||
)
|
||||
|
||||
def test_a_release_that_is_only_a_prefix_is_not_it(self):
|
||||
text = "deb http://repo.example/x focalfossa main\n"
|
||||
assert rewrite_release(text, "focal", "noble", deb822=False) == (text, 0)
|
||||
|
||||
def test_deb822_suites_and_continuations_of_enabled_paragraphs(self):
|
||||
text = (
|
||||
"Types: deb\n"
|
||||
"URIs: http://a.example/focal\n"
|
||||
"Suites: focal focal-updates\n"
|
||||
" focal-backports\n"
|
||||
"Components: main\n"
|
||||
"\n"
|
||||
"Enabled: no\n"
|
||||
"Types: deb\n"
|
||||
"URIs: http://b.example/ubuntu\n"
|
||||
"Suites: focal\n"
|
||||
"Components: main\n"
|
||||
)
|
||||
new, changed = rewrite_release(text, "focal", "noble", deb822=True)
|
||||
assert changed == 3
|
||||
assert new == (
|
||||
"Types: deb\n"
|
||||
"URIs: http://a.example/focal\n"
|
||||
"Suites: noble noble-updates\n"
|
||||
" noble-backports\n"
|
||||
"Components: main\n"
|
||||
"\n"
|
||||
"Enabled: no\n"
|
||||
"Types: deb\n"
|
||||
"URIs: http://b.example/ubuntu\n"
|
||||
"Suites: focal\n"
|
||||
"Components: main\n"
|
||||
)
|
||||
|
||||
def test_the_text_without_a_final_newline_stays_so(self):
|
||||
assert rewrite_release("deb http://a/ focal main", "focal", "noble", deb822=False) == (
|
||||
"deb http://a/ noble main",
|
||||
1,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize("bad", ["", "Noble", "no ble", "noble;", "../x"])
|
||||
def test_a_name_that_is_no_codename(self, bad):
|
||||
with pytest.raises(ValueError):
|
||||
rewrite_release(_Z2M_DOCKER, bad, "noble", deb822=False)
|
||||
with pytest.raises(ValueError):
|
||||
rewrite_release(_Z2M_DOCKER, "focal", bad, deb822=False)
|
||||
|
||||
|
||||
_OK = "[fetch]\nGet:1 x\n[rc]\n0\n"
|
||||
_NOT_FOUND = "[fetch]\nE: Failed to fetch {url} 404 Not Found [IP: 1.2.3.4 80]\n[rc]\n100\n"
|
||||
|
||||
|
||||
class _Host(AptReleaseMixin):
|
||||
"""An apt host whose files are a dict and whose fetches answer from a set."""
|
||||
|
||||
def __init__(self, files, os_release=_NOBLE, served=(), apt=True):
|
||||
self.files = dict(files)
|
||||
self.os_release = os_release
|
||||
self.served = set(served)
|
||||
self.apt = apt
|
||||
self.fetched: list[str] = []
|
||||
self.written: list[str] = []
|
||||
|
||||
def _is_apt_host(self):
|
||||
return self.apt
|
||||
|
||||
def _run_apt_command(self, command):
|
||||
if "ARELEASE_" in command:
|
||||
return _framed(self.os_release, self.files)
|
||||
if command.startswith("base64 "):
|
||||
path = command.split(" ", 1)[1].strip("'")
|
||||
return base64.b64encode(self.files[path].encode()).decode() + "\n"
|
||||
url = command.split("download-file ", 1)[1].split()[0].strip("'")
|
||||
self.fetched.append(url)
|
||||
body = _OK if url in self.served else _NOT_FOUND.format(url=url)
|
||||
return f"APROXY_BEGIN\n{body}APROXY_END\n"
|
||||
|
||||
def _write_apt_file(self, path, content):
|
||||
self.written.append(path)
|
||||
self.files[path] = content
|
||||
return {"ok": True, "output": f"wrote {path}"}
|
||||
|
||||
|
||||
_DOCKER_PATH = f"/etc/apt/sources.list.d/{_Z2M_DOCKER_FILE}"
|
||||
_DOCKER_NOBLE = "http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease"
|
||||
|
||||
|
||||
class TestTheMixin:
|
||||
def test_reads_the_host(self):
|
||||
found = _Host({_DOCKER_PATH: _Z2M_DOCKER}).get_apt_source_releases()
|
||||
assert found is not None
|
||||
assert found["codename"] == "noble"
|
||||
assert found["sources"][0]["suites"] == ["focal"]
|
||||
|
||||
def test_none_without_apt(self):
|
||||
assert _Host({}, apt=False).get_apt_source_releases() is None
|
||||
|
||||
def test_moves_the_file_once_the_vendor_serves_the_release(self):
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
|
||||
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
|
||||
assert result["success"], result["output"]
|
||||
assert host.files[_DOCKER_PATH] == (
|
||||
"deb http://HTTPS///download.docker.com/linux/ubuntu noble stable\n"
|
||||
)
|
||||
assert host.fetched == [_DOCKER_NOBLE]
|
||||
assert result["rewritten"] == 1
|
||||
assert [s["step"] for s in result["steps"]] == ["read", "check", "rewrite"]
|
||||
|
||||
def test_the_release_file_when_there_is_no_inrelease(self):
|
||||
release = _DOCKER_NOBLE.replace("InRelease", "Release")
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={release})
|
||||
assert host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")["success"]
|
||||
assert host.fetched == [_DOCKER_NOBLE, release]
|
||||
|
||||
def test_refuses_where_the_vendor_has_no_such_release(self):
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER})
|
||||
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
|
||||
assert not result["success"]
|
||||
assert host.written == []
|
||||
assert "noble" in result["output"] and "404" in result["output"]
|
||||
assert result["rewritten"] == 0
|
||||
|
||||
def test_one_suite_missing_leaves_the_whole_file(self):
|
||||
text = "deb http://a.example/u focal main\ndeb http://a.example/u focal-security main\n"
|
||||
served = {"http://a.example/u/dists/noble/InRelease"}
|
||||
host = _Host({"/etc/apt/sources.list": text}, served=served)
|
||||
result = host.set_apt_source_release("/etc/apt/sources.list", "focal", "noble")
|
||||
assert not result["success"]
|
||||
assert host.files["/etc/apt/sources.list"] == text
|
||||
|
||||
def test_a_file_apt_does_not_read_is_never_written(self):
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
|
||||
result = host.set_apt_source_release("/etc/shadow", "focal", "noble")
|
||||
assert not result["success"]
|
||||
assert host.written == []
|
||||
|
||||
def test_nothing_left_on_that_release(self):
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER.replace("focal", "noble")})
|
||||
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble")
|
||||
assert result["success"]
|
||||
assert result["rewritten"] == 0
|
||||
assert host.written == [] and host.fetched == []
|
||||
|
||||
def test_a_deb822_file(self):
|
||||
path = "/etc/apt/sources.list.d/docker.sources"
|
||||
text = "Types: deb\nURIs: https://download.docker.com/linux/ubuntu\nSuites: focal\nComponents: stable\n"
|
||||
served = {"https://download.docker.com/linux/ubuntu/dists/noble/InRelease"}
|
||||
host = _Host({path: text}, served=served)
|
||||
assert host.set_apt_source_release(path, "focal", "noble")["success"]
|
||||
assert "Suites: noble\n" in host.files[path]
|
||||
|
||||
def test_not_an_apt_host(self):
|
||||
result = _Host({}, apt=False).set_apt_source_release(_DOCKER_PATH, "focal", "noble")
|
||||
assert not result["success"]
|
||||
|
||||
def test_a_name_that_is_no_codename_changes_nothing(self):
|
||||
host = _Host({_DOCKER_PATH: _Z2M_DOCKER}, served={_DOCKER_NOBLE})
|
||||
result = host.set_apt_source_release(_DOCKER_PATH, "focal", "noble; rm -rf /")
|
||||
assert not result["success"]
|
||||
assert host.written == [] and host.fetched == []
|
||||
|
||||
|
||||
def test_every_apt_proxy_host_reads_its_releases():
|
||||
# Every driver with apt's proxy is an apt host: it reads its sources' releases too.
|
||||
assert issubclass(AptProxyMixin, AptReleaseMixin)
|
||||
@@ -0,0 +1,318 @@
|
||||
"""https sources through the cache: which can be, and the root commands that turn them.
|
||||
|
||||
apt-cacher-ng fetches ``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>``
|
||||
and stores it, where it allows port 443 (``AllowUserPorts=80 443``); a source
|
||||
fetched over https itself goes through the cache's tunnel and is never stored
|
||||
(NetOrk/netork#2555). The commands run here for real, on files in a temporary
|
||||
directory that ``apt-config -o`` points apt's own settings at.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.apt_proxy import parse_apt_proxy
|
||||
from napalm_device_types.apt_sources import (
|
||||
VIA_CACHE,
|
||||
cache_form,
|
||||
cacheable_https_hosts,
|
||||
is_host_name,
|
||||
revert_sources_command,
|
||||
rewrite_sources_command,
|
||||
)
|
||||
|
||||
|
||||
def _report(**over):
|
||||
report = {
|
||||
"proxy": "http://c:3142",
|
||||
"per_host": {},
|
||||
"auto_detect": None,
|
||||
"settings": [],
|
||||
"https_proxy": "http://c:3142",
|
||||
"https_per_host": {},
|
||||
"https_settings": [],
|
||||
"http_sources": ["http://deb.debian.org/debian/dists/trixie/"],
|
||||
"https_sources": [],
|
||||
"not_cached": [],
|
||||
}
|
||||
report.update(over)
|
||||
return report
|
||||
|
||||
|
||||
class TestCacheForm:
|
||||
def test_an_https_url(self):
|
||||
assert cache_form("https://download.docker.com/linux/ubuntu/dists/noble/") == (
|
||||
"http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/"
|
||||
)
|
||||
|
||||
def test_the_prefix(self):
|
||||
assert VIA_CACHE == "http://HTTPS///"
|
||||
|
||||
def test_not_an_https_url(self):
|
||||
with pytest.raises(ValueError):
|
||||
cache_form("http://deb.debian.org/debian/")
|
||||
|
||||
|
||||
class TestCacheableHttpsHosts:
|
||||
def test_each_host_once_sorted(self):
|
||||
report = _report(
|
||||
https_sources=[
|
||||
"https://packages.wazuh.com/4.x/apt/dists/stable/",
|
||||
"https://download.docker.com/linux/ubuntu/dists/noble/",
|
||||
"https://download.docker.com/linux/ubuntu/dists/noble-test/",
|
||||
]
|
||||
)
|
||||
assert cacheable_https_hosts(report) == ["download.docker.com", "packages.wazuh.com"]
|
||||
|
||||
def test_none_without_an_http_proxy(self):
|
||||
# The cache form goes through the http proxy: without one it goes nowhere.
|
||||
report = _report(proxy=None, https_sources=["https://h.example/dists/x/"])
|
||||
assert cacheable_https_hosts(report) == []
|
||||
|
||||
def test_a_host_apt_fetches_direct_stays(self):
|
||||
report = _report(
|
||||
per_host={"a.example": "DIRECT"},
|
||||
https_per_host={"B.example": "direct"},
|
||||
https_sources=[
|
||||
"https://a.example/dists/x/",
|
||||
"https://b.example/dists/x/",
|
||||
"https://c.example/dists/x/",
|
||||
],
|
||||
)
|
||||
assert cacheable_https_hosts(report) == ["c.example"]
|
||||
|
||||
def test_a_host_with_credentials_stays(self):
|
||||
report = _report(
|
||||
https_sources=["https://enterprise.proxmox.com/debian/pve/dists/trixie/"],
|
||||
not_cached=["enterprise.proxmox.com"],
|
||||
)
|
||||
assert cacheable_https_hosts(report) == []
|
||||
|
||||
def test_another_port_or_a_login_in_the_url_stays(self):
|
||||
# The cache allows port 443 alone, and a login in the URL is a credential too.
|
||||
report = _report(
|
||||
https_sources=[
|
||||
"https://h.example:8443/dists/x/",
|
||||
"https://u:p@i.example/dists/x/",
|
||||
"https://j.example:443/dists/x/",
|
||||
]
|
||||
)
|
||||
assert cacheable_https_hosts(report) == ["j.example"]
|
||||
|
||||
def test_a_report_from_before_not_cached(self):
|
||||
report = _report(https_sources=["https://h.example/dists/x/"])
|
||||
del report["not_cached"]
|
||||
assert cacheable_https_hosts(report) == ["h.example"]
|
||||
|
||||
|
||||
class TestNotCachedInTheReport:
|
||||
def test_the_dump_names_them(self):
|
||||
output = "\n".join(
|
||||
[
|
||||
"APROXY_BEGIN",
|
||||
"[dump]",
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Package-Proxy::Not-Cached "";',
|
||||
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";',
|
||||
'Package-Proxy::Not-Cached:: "esm.ubuntu.com";',
|
||||
"[sources]",
|
||||
"APROXY_END",
|
||||
]
|
||||
)
|
||||
assert parse_apt_proxy(output)["not_cached"] == [
|
||||
"enterprise.proxmox.com",
|
||||
"esm.ubuntu.com",
|
||||
]
|
||||
|
||||
def test_none(self):
|
||||
assert parse_apt_proxy("APROXY_BEGIN\n[dump]\nAPROXY_END")["not_cached"] == []
|
||||
|
||||
|
||||
def test_the_hosts_are_checked():
|
||||
with pytest.raises(ValueError):
|
||||
rewrite_sources_command(["h.example; rm -rf /"], "/etc/apt/apt.conf.d/00proxy")
|
||||
|
||||
|
||||
_APT_CONFIG = shutil.which("apt-config")
|
||||
needs_apt = pytest.mark.skipif(_APT_CONFIG is None, reason="apt-config is not on this host")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def etc(tmp_path):
|
||||
"""apt's sources, its credentials and the proxy file, under *tmp_path*."""
|
||||
(tmp_path / "sources.list.d").mkdir()
|
||||
(tmp_path / "auth.conf.d").mkdir()
|
||||
(tmp_path / "00proxy").write_text('Acquire::http::Proxy "http://c:3142";\n')
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _apt_config(etc):
|
||||
return (
|
||||
f"apt-config -o Dir::Etc::sourcelist={etc}/sources.list "
|
||||
f"-o Dir::Etc::sourceparts={etc}/sources.list.d/ "
|
||||
f"-o Dir::Etc::netrc={etc}/auth.conf -o Dir::Etc::netrcparts={etc}/auth.conf.d/"
|
||||
)
|
||||
|
||||
|
||||
def _run(command):
|
||||
done = subprocess.run(["sh", "-c", command], capture_output=True, text=True, check=False)
|
||||
assert done.returncode == 0, done.stderr
|
||||
return done.stdout
|
||||
|
||||
|
||||
_DOCKER_LIST = (
|
||||
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
|
||||
"https://download.docker.com/linux/ubuntu noble stable\n"
|
||||
)
|
||||
_DOCKER_SOURCES = (
|
||||
"Types: deb\n"
|
||||
"URIs: https://download.docker.com/linux/ubuntu\n"
|
||||
"Suites: noble\n"
|
||||
"Components: stable\n"
|
||||
"Signed-By: /etc/apt/keyrings/docker.asc\n"
|
||||
)
|
||||
|
||||
|
||||
@needs_apt
|
||||
class TestRewrite:
|
||||
def test_a_one_line_source(self, etc):
|
||||
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
|
||||
|
||||
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert (etc / "sources.list.d" / "docker.list").read_text() == (
|
||||
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
|
||||
"http://HTTPS///download.docker.com/linux/ubuntu noble stable\n"
|
||||
)
|
||||
|
||||
def test_a_deb822_source(self, etc):
|
||||
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
|
||||
|
||||
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert (etc / "sources.list.d" / "docker.sources").read_text() == (
|
||||
_DOCKER_SOURCES.replace("https://", "http://HTTPS///")
|
||||
)
|
||||
|
||||
def test_sources_list_itself_and_only_the_hosts_named(self, etc):
|
||||
(etc / "sources.list").write_text(
|
||||
"deb http://deb.debian.org/debian trixie main\n"
|
||||
"deb https://a.example/debian trixie main\n"
|
||||
"deb-src https://a.example/debian trixie main\n"
|
||||
"deb https://b.example/debian trixie main\n"
|
||||
"# deb https://a.example/old trixie main\n"
|
||||
)
|
||||
|
||||
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert (etc / "sources.list").read_text() == (
|
||||
"deb http://deb.debian.org/debian trixie main\n"
|
||||
"deb http://HTTPS///a.example/debian trixie main\n"
|
||||
"deb-src http://HTTPS///a.example/debian trixie main\n"
|
||||
"deb https://b.example/debian trixie main\n"
|
||||
"# deb https://a.example/old trixie main\n"
|
||||
)
|
||||
|
||||
def test_a_longer_name_or_another_port_stays(self, etc):
|
||||
text = (
|
||||
"deb https://a.example.org/debian trixie main\n"
|
||||
"deb https://a.example:8443/debian trixie main\n"
|
||||
)
|
||||
(etc / "sources.list").write_text(text)
|
||||
|
||||
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert (etc / "sources.list").read_text() == text
|
||||
|
||||
def test_a_host_with_credentials_stays_and_is_named(self, etc):
|
||||
(etc / "sources.list.d" / "pve.sources").write_text(
|
||||
"Types: deb\nURIs: https://enterprise.proxmox.com/debian/pve\nSuites: trixie\n"
|
||||
)
|
||||
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
|
||||
(etc / "auth.conf.d" / "pve.conf").write_text(
|
||||
"machine https://Enterprise.Proxmox.com/debian/pve\nlogin key\npassword secret\n"
|
||||
)
|
||||
|
||||
_run(
|
||||
rewrite_sources_command(
|
||||
["enterprise.proxmox.com", "download.docker.com"],
|
||||
f"{etc}/00proxy",
|
||||
_apt_config(etc),
|
||||
)
|
||||
)
|
||||
|
||||
assert "https://enterprise.proxmox.com" in (etc / "sources.list.d/pve.sources").read_text()
|
||||
assert "HTTPS///download.docker.com" in (etc / "sources.list.d/docker.list").read_text()
|
||||
assert (etc / "00proxy").read_text() == (
|
||||
'Acquire::http::Proxy "http://c:3142";\n'
|
||||
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";\n'
|
||||
)
|
||||
assert "secret" not in (etc / "00proxy").read_text()
|
||||
|
||||
def test_credentials_in_auth_conf_on_one_line(self, etc):
|
||||
(etc / "sources.list").write_text("deb https://esm.ubuntu.com/apps/ubuntu noble main\n")
|
||||
(etc / "auth.conf").write_text("machine esm.ubuntu.com/apps/ubuntu/ login a password b\n")
|
||||
|
||||
_run(rewrite_sources_command(["esm.ubuntu.com"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert "https://esm.ubuntu.com" in (etc / "sources.list").read_text()
|
||||
assert "esm.ubuntu.com" in (etc / "00proxy").read_text()
|
||||
|
||||
def test_a_file_without_the_host_is_left_untouched(self, etc):
|
||||
other = etc / "sources.list.d" / "other.list"
|
||||
other.write_text("deb http://deb.debian.org/debian trixie main\n")
|
||||
before = other.stat().st_ino # sed -i writes a new file; grep first leaves it be
|
||||
|
||||
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
assert other.stat().st_ino == before
|
||||
|
||||
def test_twice_is_once(self, etc):
|
||||
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
|
||||
command = rewrite_sources_command(
|
||||
["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)
|
||||
)
|
||||
|
||||
_run(command)
|
||||
_run(command)
|
||||
|
||||
assert (etc / "sources.list.d" / "docker.list").read_text().count("HTTPS///") == 1
|
||||
|
||||
|
||||
@needs_apt
|
||||
class TestRevert:
|
||||
def test_every_source_goes_back_to_https(self, etc):
|
||||
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
|
||||
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
|
||||
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
|
||||
|
||||
_run(revert_sources_command(_apt_config(etc)))
|
||||
|
||||
assert (etc / "sources.list.d" / "docker.list").read_text() == _DOCKER_LIST
|
||||
assert (etc / "sources.list.d" / "docker.sources").read_text() == _DOCKER_SOURCES
|
||||
|
||||
def test_nothing_to_revert(self, etc):
|
||||
(etc / "sources.list").write_text("deb http://deb.debian.org/debian trixie main\n")
|
||||
|
||||
_run(revert_sources_command(_apt_config(etc)))
|
||||
|
||||
assert (
|
||||
etc / "sources.list"
|
||||
).read_text() == "deb http://deb.debian.org/debian trixie main\n"
|
||||
|
||||
|
||||
def test_the_commands_are_valid_posix_sh():
|
||||
for command in (
|
||||
rewrite_sources_command(["a.example"], "/etc/apt/apt.conf.d/00proxy"),
|
||||
revert_sources_command(),
|
||||
):
|
||||
assert subprocess.run(["sh", "-n", "-c", command], check=False).returncode == 0
|
||||
|
||||
|
||||
def test_a_host_name():
|
||||
assert is_host_name("download.docker.com")
|
||||
assert not is_host_name('h"; Acquire::x "y')
|
||||
assert not is_host_name("-h.example")
|
||||
@@ -0,0 +1,172 @@
|
||||
"""ConfigRevisionsMixin: a device's own configuration history (NetOrk/netork#2511).
|
||||
|
||||
Some devices keep their earlier configurations themselves. An application restores a
|
||||
configuration there by reverting to the revision that holds it -- on OPNsense the only
|
||||
way, since no API takes an uploaded config.xml. Listing and reading are declared only,
|
||||
so ``callable(getattr(driver, "get_config_revisions", None))`` is the truthful answer;
|
||||
``revert_config_revision`` is a template: it refuses an id that is not in the history,
|
||||
lets the driver revert, and checks that the configuration in place now is the
|
||||
revision's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
ConfigRevertFailed,
|
||||
ConfigRevisionDict,
|
||||
ConfigRevisionsMixin,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
OSDriver,
|
||||
SwitchDriver,
|
||||
UnknownConfigRevision,
|
||||
)
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
assert get_type_hints(ConfigRevisionDict) == {
|
||||
"id": str,
|
||||
"time": str | None,
|
||||
"description": str | None,
|
||||
"user": str | None,
|
||||
"size": int | None,
|
||||
}
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, ConfigRevisionsMixin)
|
||||
|
||||
|
||||
def test_listing_and_reading_are_declared_not_implemented():
|
||||
assert not hasattr(ConfigRevisionsMixin, "get_config_revisions")
|
||||
assert not hasattr(ConfigRevisionsMixin, "get_config_revision")
|
||||
source = inspect.getsource(ConfigRevisionsMixin)
|
||||
assert "def get_config_revisions(self) -> list[ConfigRevisionDict]" in source
|
||||
assert "def get_config_revision(self, revision_id: str) -> str" in source
|
||||
assert "def _revert_to_config_revision(self, revision_id: str) -> None" in source
|
||||
|
||||
|
||||
def test_an_unknown_revision_is_a_value_error_that_names_it():
|
||||
error = UnknownConfigRevision("r9")
|
||||
|
||||
assert isinstance(error, ValueError)
|
||||
assert "'r9'" in str(error)
|
||||
|
||||
|
||||
def test_a_failed_revert_is_a_runtime_error():
|
||||
assert issubclass(ConfigRevertFailed, RuntimeError)
|
||||
|
||||
|
||||
def _revision(revision_id: str, time: str) -> ConfigRevisionDict:
|
||||
return {"id": revision_id, "time": time, "description": None, "user": None, "size": None}
|
||||
|
||||
|
||||
class _Device(ConfigRevisionsMixin):
|
||||
"""A device that keeps its history in memory, newest first, and saves a revert as a
|
||||
new newest revision with a stamp of its own on the first line."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.history = [
|
||||
(_revision("r3", "2026-11-13T21:00:00+00:00"), "#saved 3\nb\n"),
|
||||
(_revision("r2", "2026-11-12T21:00:00+00:00"), "#saved 2\na\n"),
|
||||
(_revision("r1", "2026-11-11T21:00:00+00:00"), "#saved 1\na\n"),
|
||||
]
|
||||
self.calls: list[str] = []
|
||||
|
||||
def get_config_revisions(self) -> list[ConfigRevisionDict]:
|
||||
self.calls.append("list")
|
||||
return [revision for revision, _ in self.history]
|
||||
|
||||
def get_config_revision(self, revision_id: str) -> str:
|
||||
self.calls.append(f"read {revision_id}")
|
||||
for revision, text in self.history:
|
||||
if revision["id"] == revision_id:
|
||||
return text
|
||||
raise UnknownConfigRevision(revision_id)
|
||||
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.calls.append(f"revert {revision_id}")
|
||||
body = self.get_config_revision(revision_id).split("\n", 1)[1]
|
||||
self.history.insert(0, (_revision("r4", "2026-11-14T08:00:00+00:00"), f"#saved 4\n{body}"))
|
||||
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
return text.split("\n", 1)[1]
|
||||
|
||||
|
||||
class TestRevertConfigRevision:
|
||||
def test_reverts_and_answers_with_the_revision_now_in_place(self):
|
||||
device = _Device()
|
||||
|
||||
now = device.revert_config_revision("r1")
|
||||
|
||||
assert now["id"] == "r4"
|
||||
assert device.calls == ["list", "read r1", "revert r1", "read r1", "list", "read r4"]
|
||||
|
||||
def test_the_revision_is_read_before_the_revert_which_may_rotate_it_out(self):
|
||||
class Rotating(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
super()._revert_to_config_revision(revision_id)
|
||||
self.history.pop() # the oldest, r1, drops out
|
||||
|
||||
assert Rotating().revert_config_revision("r1")["id"] == "r4"
|
||||
|
||||
def test_a_revision_not_in_the_history_is_refused_before_anything_changes(self):
|
||||
device = _Device()
|
||||
|
||||
with pytest.raises(UnknownConfigRevision, match="r9"):
|
||||
device.revert_config_revision("r9")
|
||||
|
||||
assert device.calls == ["list"]
|
||||
|
||||
def test_a_revert_the_device_did_not_take_fails(self):
|
||||
# OPNsense answers "reverted" even when it fell back to the previous file.
|
||||
class Deaf(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.calls.append(f"revert {revision_id}")
|
||||
|
||||
with pytest.raises(ConfigRevertFailed, match="r1"):
|
||||
Deaf().revert_config_revision("r1")
|
||||
|
||||
def test_a_history_that_is_empty_afterwards_fails(self):
|
||||
class Forgetful(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
self.history.clear()
|
||||
|
||||
with pytest.raises(ConfigRevertFailed):
|
||||
Forgetful().revert_config_revision("r1")
|
||||
|
||||
def test_what_a_save_stamps_anew_does_not_count(self):
|
||||
# r2 and r1 differ only in their stamp: reverting to r2 leaves r1's content.
|
||||
device = _Device()
|
||||
|
||||
assert device.revert_config_revision("r2")["id"] == "r4"
|
||||
|
||||
def test_a_refusal_of_the_device_reaches_the_caller(self):
|
||||
class Refusing(_Device):
|
||||
def _revert_to_config_revision(self, revision_id: str) -> None:
|
||||
raise ConfigRevertFailed("read-only user")
|
||||
|
||||
with pytest.raises(ConfigRevertFailed, match="read-only"):
|
||||
Refusing().revert_config_revision("r1")
|
||||
|
||||
|
||||
class TestConfigRevisionContent:
|
||||
def test_by_default_the_whole_text(self):
|
||||
class Plain(ConfigRevisionsMixin):
|
||||
pass
|
||||
|
||||
assert Plain().config_revision_content("a\nb\n") == "a\nb\n"
|
||||
|
||||
def test_without_a_stamp_every_save_differs(self):
|
||||
class Plain(_Device):
|
||||
def config_revision_content(self, text: str) -> str:
|
||||
return ConfigRevisionsMixin.config_revision_content(self, text)
|
||||
|
||||
with pytest.raises(ConfigRevertFailed):
|
||||
Plain().revert_config_revision("r1")
|
||||
@@ -19,30 +19,40 @@ from napalm_device_types.container_engine import (
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
|
||||
#: What each engine's CLI says to ``--version``.
|
||||
VERSIONS = {
|
||||
"docker": "Docker version 29.0.1, build 1a2b3c4",
|
||||
"podman": "podman version 5.2.2",
|
||||
}
|
||||
|
||||
|
||||
class FakeDriver(ContainerEngineMixin):
|
||||
"""A driver with a channel: records what it is asked to run."""
|
||||
"""A driver with a channel: records what it is asked to run.
|
||||
|
||||
def __init__(self, present=("docker",), binary=None):
|
||||
*present* are the engines whose CLI is installed; *says* overrides what an
|
||||
engine's CLI answers to ``--version`` (podman-docker's ``docker`` is podman).
|
||||
"""
|
||||
|
||||
def __init__(self, present=("docker",), binary=None, says=None):
|
||||
self.present = set(present)
|
||||
self.binary = binary
|
||||
self.says = dict(says or {})
|
||||
self.commands = []
|
||||
self.streams = []
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
|
||||
self.commands.append((command, privileged, timeout, stdin))
|
||||
found = any(command == f"command -v {b}" for b in self._binaries())
|
||||
return CommandResult(
|
||||
stdout="/usr/bin/x\n" if found else "", stderr="", exit_code=0 if found else 1
|
||||
)
|
||||
for engine in self.present:
|
||||
b = self._container_engine_binary(engine)
|
||||
if command == f"command -v {b} >/dev/null 2>&1 && {b} --version":
|
||||
answer = self.says.get(engine, VERSIONS[engine])
|
||||
return CommandResult(stdout=answer + "\n", stderr="", exit_code=0)
|
||||
return CommandResult(stdout="", stderr="", exit_code=1)
|
||||
|
||||
def open_stream(self, command, *, privileged=False):
|
||||
self.streams.append((command, privileged))
|
||||
return object()
|
||||
|
||||
def _binaries(self):
|
||||
return {self._container_engine_binary(e) for e in self.present}
|
||||
|
||||
def _container_engine_binary(self, engine):
|
||||
return self.binary or super()._container_engine_binary(engine)
|
||||
|
||||
@@ -123,6 +133,60 @@ def test_an_unknown_engine_is_refused():
|
||||
FakeDriver().open_container_engine("rkt")
|
||||
|
||||
|
||||
class TestPodman:
|
||||
"""Podman, reached the way Docker is (NetOrk/netork#818).
|
||||
|
||||
Its API is Podman's own service, which also speaks Docker's: netOrk reads the
|
||||
host's *rootful* Podman, the engine of the system's containers (Quadlet units
|
||||
in ``/etc/containers/systemd``, ``podman.socket``). A login user's rootless
|
||||
containers are that user's own and invisible to anybody else, so the
|
||||
connection to Podman runs as root, the way the driver gains root for any command.
|
||||
"""
|
||||
|
||||
def test_a_host_with_podman_lists_it_with_its_api(self):
|
||||
driver = FakeDriver(present=("podman",))
|
||||
|
||||
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
|
||||
|
||||
def test_a_host_with_both_lists_both(self):
|
||||
driver = FakeDriver(present=("docker", "podman"))
|
||||
|
||||
assert driver.container_engines() == [
|
||||
{"engine": "docker", "api": "docker-engine"},
|
||||
{"engine": "podman", "api": "podman-libpod"},
|
||||
]
|
||||
|
||||
def test_podman_docker_is_not_docker(self):
|
||||
"""podman-docker installs a ``docker`` that runs podman: listing it as
|
||||
Docker would read the login user's rootless Podman a second time."""
|
||||
driver = FakeDriver(present=("docker", "podman"), says={"docker": VERSIONS["podman"]})
|
||||
|
||||
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
|
||||
|
||||
def test_the_api_stream_runs_as_root(self):
|
||||
driver = FakeDriver(present=("podman",))
|
||||
|
||||
driver.open_container_engine("podman").open_api()
|
||||
|
||||
assert driver.streams == [("podman system dial-stdio", True)]
|
||||
|
||||
def test_the_cli_runs_as_root_too(self):
|
||||
driver = FakeDriver(present=("podman",))
|
||||
conn = driver.open_container_engine("podman")
|
||||
|
||||
conn.run_cli(["pull", "nginx"])
|
||||
conn.stream_cli(["pull", "nginx"])
|
||||
|
||||
assert driver.commands[-1][:2] == ("podman pull nginx", True)
|
||||
assert driver.streams == [("podman pull nginx", True)]
|
||||
|
||||
def test_the_connection_says_it_runs_as_root(self):
|
||||
driver = FakeDriver(present=("docker", "podman"))
|
||||
|
||||
assert driver.open_container_engine("podman").privileged is True
|
||||
assert driver.open_container_engine("docker").privileged is False
|
||||
|
||||
|
||||
def test_the_connection_names_its_engine():
|
||||
conn = FakeDriver().open_container_engine("docker")
|
||||
|
||||
|
||||
@@ -298,6 +298,156 @@ class TestApplyWithARefusedReservation:
|
||||
assert issubclass(DhcpApplyError, RuntimeError)
|
||||
|
||||
|
||||
#: What Kea answers through OPNsense's API for an address another reservation holds.
|
||||
_KEA_DUPLICATE = (
|
||||
"{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}"
|
||||
)
|
||||
|
||||
|
||||
class _KeaLikeServer(_FakeDhcpServer):
|
||||
"""A server that keeps each address unique at every write, as Kea does: a write
|
||||
taking an address another reservation still holds is refused."""
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if any(r["ip"] == reservation["ip"] and r["uuid"] != uuid for r in self.live):
|
||||
raise RuntimeError(_KEA_DUPLICATE)
|
||||
stored = {**reservation, "uuid": uuid or f"new-{len(self.live)}"}
|
||||
self.live = [r for r in self.live if r["uuid"] != uuid] + [stored] # type: ignore[list-item]
|
||||
return super().apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
|
||||
def ip_of(self, mac: str) -> str:
|
||||
return next(r["ip"] for r in self.live if r["mac"] == mac)
|
||||
|
||||
|
||||
_A, _B, _C, _N = (f"02:00:00:00:00:0{i}" for i in "abcd")
|
||||
|
||||
|
||||
def _at(mac: str, ip: str, uuid: str = "") -> DhcpReservationDict:
|
||||
return _reservation(mac=mac, ip=ip, uuid=uuid, subnet="192.0.2.0/24")
|
||||
|
||||
|
||||
class TestApplyOrderReleasesBeforeTaking:
|
||||
"""An address is released before another change takes it (#106)."""
|
||||
|
||||
def test_an_update_that_frees_an_address_goes_first(self):
|
||||
# The issue: B takes A's address, A moves on; desired lists B first.
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")])
|
||||
|
||||
lines = list(
|
||||
driver.apply_dhcp_reservationset([_at(_B, "192.0.2.10"), _at(_A, "192.0.2.20")])
|
||||
)
|
||||
|
||||
assert driver.ip_of(_A) == "192.0.2.20"
|
||||
assert driver.ip_of(_B) == "192.0.2.10"
|
||||
assert lines == [
|
||||
f"[update] {_A} -> 192.0.2.20 (ip)",
|
||||
f"[update] {_B} -> 192.0.2.10 (ip)",
|
||||
"[commit] applied 0 add(s), 2 update(s)",
|
||||
]
|
||||
|
||||
def test_a_chain_is_written_from_its_free_end(self):
|
||||
live = [
|
||||
_at(_A, "192.0.2.10", "ua"),
|
||||
_at(_B, "192.0.2.11", "ub"),
|
||||
_at(_C, "192.0.2.12", "uc"),
|
||||
]
|
||||
driver = _KeaLikeServer(live)
|
||||
|
||||
list(
|
||||
driver.apply_dhcp_reservationset(
|
||||
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.13")]
|
||||
)
|
||||
)
|
||||
|
||||
assert [e["reservation"]["mac"] for e in driver.applied] == [_C, _B, _A]
|
||||
assert [driver.ip_of(m) for m in (_A, _B, _C)] == ["192.0.2.11", "192.0.2.12", "192.0.2.13"]
|
||||
|
||||
def test_an_add_comes_after_the_update_that_frees_its_address(self):
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
|
||||
|
||||
lines = list(
|
||||
driver.apply_dhcp_reservationset([_at(_N, "192.0.2.10"), _at(_A, "192.0.2.20")])
|
||||
)
|
||||
|
||||
assert driver.ip_of(_N) == "192.0.2.10"
|
||||
assert driver.ip_of(_A) == "192.0.2.20"
|
||||
assert lines[-1] == "[commit] applied 1 add(s), 1 update(s)"
|
||||
|
||||
def test_addresses_match_however_they_are_spelled(self):
|
||||
driver = _FakeDhcpServer([_at(_A, "2001:DB8::10", "ua"), _at(_B, "2001:DB8::11", "ub")])
|
||||
|
||||
list(driver.apply_dhcp_reservationset([_at(_B, "2001:db8::10"), _at(_A, "2001:db8::20")]))
|
||||
|
||||
assert [e["reservation"]["mac"] for e in driver.applied] == [_A, _B]
|
||||
|
||||
def test_an_update_that_keeps_its_address_frees_nothing(self):
|
||||
# A hostname change is no release: the add for A's address stays refused.
|
||||
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 2"):
|
||||
list(
|
||||
driver.apply_dhcp_reservationset(
|
||||
[_at(_N, "192.0.2.10"), {**_at(_A, "192.0.2.10"), "hostname": "renamed"}]
|
||||
)
|
||||
)
|
||||
|
||||
assert driver.ip_of(_A) == "192.0.2.10"
|
||||
|
||||
|
||||
class TestApplyOrderWithASwap:
|
||||
"""Two reservations trading addresses: no order frees one first. Each is still
|
||||
tried -- a server that checks only at the commit takes them -- and a refusal says it
|
||||
was a swap. Nothing is moved to a temporary address: the contract has no way to know
|
||||
a free one, and a second write that failed would leave the reservation there."""
|
||||
|
||||
def _swap(self) -> list[DhcpReservationDict]:
|
||||
return [_at(_A, "192.0.2.11"), _at(_B, "192.0.2.10"), _at(_N, "192.0.2.30")]
|
||||
|
||||
def _live(self) -> list[DhcpReservationDict]:
|
||||
return [_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")]
|
||||
|
||||
def test_a_refused_swap_is_reported_as_one_and_the_rest_is_written(self):
|
||||
driver = _KeaLikeServer(self._live())
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="2 of 3"):
|
||||
for line in driver.apply_dhcp_reservationset(self._swap()):
|
||||
lines.append(line)
|
||||
|
||||
failed = [line for line in lines if line.startswith("[failed]")]
|
||||
assert len(failed) == 2
|
||||
assert all("Duplicate entry exists" in line for line in failed)
|
||||
assert f"swap with {_B}" in failed[0] and f"swap with {_A}" in failed[1]
|
||||
assert driver.ip_of(_A) == "192.0.2.10" and driver.ip_of(_B) == "192.0.2.11"
|
||||
assert driver.ip_of(_N) == "192.0.2.30"
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_a_server_that_takes_the_swap_gets_it(self):
|
||||
driver = _FakeDhcpServer(self._live())
|
||||
|
||||
lines = list(driver.apply_dhcp_reservationset(self._swap()))
|
||||
|
||||
assert len(driver.applied) == 3
|
||||
assert not any("swap" in line for line in lines)
|
||||
|
||||
def test_a_rotation_of_three_is_a_swap_too(self):
|
||||
driver = _KeaLikeServer([*self._live(), _at(_C, "192.0.2.12", "uc")])
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="3 of 3"):
|
||||
for line in driver.apply_dhcp_reservationset(
|
||||
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.10")]
|
||||
):
|
||||
lines.append(line)
|
||||
|
||||
by_mac = {line.split()[1]: line for line in lines if line.startswith("[failed]")}
|
||||
assert f"swap with {_B}, {_C}" in by_mac[_A]
|
||||
assert f"swap with {_A}, {_C}" in by_mac[_B]
|
||||
assert f"swap with {_A}, {_B}" in by_mac[_C]
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneReservation:
|
||||
def test_a_live_reservation_without_mac_is_never_matched(self):
|
||||
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
|
||||
|
||||
@@ -16,6 +16,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin
|
||||
from napalm_device_types.dhcp import DhcpApplyError
|
||||
from napalm_device_types.models import DhcpSubnetDict
|
||||
|
||||
|
||||
@@ -235,3 +236,92 @@ def test_calling_a_missing_device_method_fails_loudly() -> None:
|
||||
def test_a_subnet_that_cannot_name_one_raises(desired):
|
||||
with pytest.raises(ValueError):
|
||||
_FakeDhcpServer([]).diff_dhcp_subnets(desired)
|
||||
|
||||
|
||||
# ── apply: a refused subnet does not hold back the others (#113) ──────────────
|
||||
|
||||
|
||||
class _RefusingDhcpServer(_FakeDhcpServer):
|
||||
"""A server that refuses the subnets in *refused*, as Kea refuses one that
|
||||
overlaps a subnet it already serves."""
|
||||
|
||||
def __init__(self, live: list[DhcpSubnetDict] | None = None, *, refused: set[str]) -> None:
|
||||
super().__init__(live)
|
||||
self.refused = refused
|
||||
|
||||
def apply_dhcp_subnet(
|
||||
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if subnet["subnet"] in self.refused:
|
||||
raise RuntimeError(f"{subnet['subnet']} overlaps a subnet the server has")
|
||||
return super().apply_dhcp_subnet(subnet, uuid=uuid)
|
||||
|
||||
|
||||
def _three_subnets() -> list[DhcpSubnetDict]:
|
||||
return [
|
||||
_subnet(subnet="192.0.2.0/26"),
|
||||
_subnet(subnet="192.0.2.64/26"),
|
||||
_subnet(subnet="192.0.2.128/26"),
|
||||
]
|
||||
|
||||
|
||||
def test_a_refused_subnet_does_not_stop_the_others() -> None:
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset(_three_subnets()))
|
||||
|
||||
assert [e["subnet"]["subnet"] for e in fake.applied] == ["192.0.2.0/26", "192.0.2.128/26"]
|
||||
|
||||
|
||||
def test_what_was_written_is_committed() -> None:
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset(_three_subnets()))
|
||||
|
||||
assert fake.commits == 1
|
||||
|
||||
|
||||
def test_a_set_whose_every_write_is_refused_still_commits() -> None:
|
||||
# A run that stopped before its commit left saved writes behind, which the device
|
||||
# reports as configured: no later diff would commit them.
|
||||
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(fake.apply_dhcp_subnetset([_three_subnets()[1]]))
|
||||
|
||||
assert fake.commits == 1
|
||||
|
||||
|
||||
def test_progress_names_the_refused_subnet_and_why() -> None:
|
||||
fake = _RefusingDhcpServer(
|
||||
[_subnet(uuid="dev-1", subnet="192.0.2.128/26")], refused={"192.0.2.64/26"}
|
||||
)
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError) as raised:
|
||||
for line in fake.apply_dhcp_subnetset(
|
||||
[*_three_subnets()[:2], _subnet(subnet="192.0.2.128/26", description="Office")]
|
||||
):
|
||||
lines.append(line)
|
||||
|
||||
assert lines == [
|
||||
"[add] 192.0.2.0/26",
|
||||
"[failed] 192.0.2.64/26: 192.0.2.64/26 overlaps a subnet the server has",
|
||||
"[update] 192.0.2.128/26 (description)",
|
||||
"[commit] applied 1 add(s), 1 update(s)",
|
||||
]
|
||||
assert (raised.value.failed, raised.value.attempted) == (1, 3)
|
||||
assert str(raised.value) == "1 of 3 DHCP subnet change(s) were refused"
|
||||
|
||||
|
||||
def test_a_refused_update_is_counted_like_an_add() -> None:
|
||||
live = _subnet(uuid="dev-1", subnet="192.0.2.0/24")
|
||||
fake = _RefusingDhcpServer([live], refused={"192.0.2.0/24"})
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 1"):
|
||||
list(fake.apply_dhcp_subnetset([_subnet(subnet="192.0.2.0/24", description="Office")]))
|
||||
|
||||
assert fake.applied == []
|
||||
assert fake.commits == 1
|
||||
@@ -56,8 +56,26 @@ def test_a_firmware_offer_says_what_runs_and_what_is_offered():
|
||||
from napalm_device_types import FirmwareOfferDict, FirmwareOfferMixin
|
||||
|
||||
hints = get_type_hints(FirmwareOfferDict)
|
||||
assert set(hints) == {"current_version", "latest_version", "newer", "download_url"}
|
||||
assert set(hints) == {
|
||||
"current_version",
|
||||
"latest_version",
|
||||
"newer",
|
||||
"download_url",
|
||||
"installable",
|
||||
"latest_major_version",
|
||||
}
|
||||
assert FirmwareOfferDict.__required_keys__ == frozenset(
|
||||
{"current_version", "latest_version", "newer"}
|
||||
)
|
||||
assert not hasattr(FirmwareOfferMixin, "get_firmware_offer")
|
||||
|
||||
|
||||
def test_a_firmware_upgrade_is_started_and_followed_like_a_device_run_update():
|
||||
"""A driver that can install the firmware it offers starts it and the device
|
||||
runs it by itself, restarting at its end (OpenWrt's sysupgrade, NetOrk/netork#2615);
|
||||
the caller follows it through ``get_update_progress``. Declared only, so
|
||||
``hasattr`` tells a caller whether the driver offers it."""
|
||||
from napalm_device_types import FirmwareUpgradeMixin, UpdateMixin
|
||||
|
||||
assert not hasattr(FirmwareUpgradeMixin, "start_firmware_upgrade")
|
||||
assert not hasattr(UpdateMixin, "get_update_progress")
|
||||
@@ -259,6 +259,7 @@ class TestLldpAgent:
|
||||
"ensure_package",
|
||||
"configure_package_proxy",
|
||||
"grant_container_engine_access",
|
||||
"configure_zabbix_agent",
|
||||
],
|
||||
)
|
||||
def test_no_role_base_offers_a_fix(base, method):
|
||||
|
||||
@@ -37,9 +37,40 @@ class TestRules:
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
"template": "RSYSLOG_SyslogProtocol23Format",
|
||||
},
|
||||
]
|
||||
|
||||
def test_a_rule_without_a_template_sends_rsyslogs_default(self):
|
||||
"""None: RFC 3164, the sender's local time without a zone (NetOrk/netork#493)."""
|
||||
(rule,) = _rules("*.* @graylog.example.net:514\n")
|
||||
assert rule["template"] is None
|
||||
|
||||
def test_an_actions_template_parameter(self):
|
||||
(rule,) = _rules(
|
||||
'action(type="omfwd" target="10.0.0.7" Template="RSYSLOG_ForwardFormat")\n'
|
||||
)
|
||||
assert rule["template"] == "RSYSLOG_ForwardFormat"
|
||||
|
||||
def test_the_legacy_default_template_holds_for_the_rules_after_it(self):
|
||||
rules = _rules(
|
||||
"*.* @a\n$ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format\n"
|
||||
"*.* @@b:514\n*.* @c;RSYSLOG_TraditionalForwardFormat\n"
|
||||
)
|
||||
assert [(r["target"], r["template"]) for r in rules] == [
|
||||
("a", None),
|
||||
("b", "RSYSLOG_SyslogProtocol23Format"),
|
||||
("c", "RSYSLOG_TraditionalForwardFormat"),
|
||||
]
|
||||
|
||||
def test_the_forward_modules_template_holds_for_its_actions(self):
|
||||
text = (
|
||||
'module(load="builtin:omfwd" template="RSYSLOG_SyslogProtocol23Format")\n'
|
||||
'action(type="omfwd" target="10.0.0.7")\n'
|
||||
)
|
||||
(rule,) = _rules(text)
|
||||
assert rule["template"] == "RSYSLOG_SyslogProtocol23Format"
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
@@ -299,3 +299,52 @@ def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
assert not _root_commands(host)
|
||||
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
|
||||
|
||||
|
||||
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOldNetork netork@central"
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_once_the_new_one_is_in(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{OWN_KEY}\n{OLD_KEY}\n"
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
|
||||
|
||||
|
||||
def test_the_new_key_is_never_one_that_goes(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{NETORK_KEY}\n"
|
||||
|
||||
result = driver.add_service_account(
|
||||
"netork", NETORK_KEY, replaces=[f"{NETORK_KEY.split()[0]} {NETORK_KEY.split()[1]}"]
|
||||
)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{NETORK_KEY}\n"
|
||||
|
||||
|
||||
def test_a_replaced_key_stays_while_the_new_one_is_not_in(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = f"{OLD_KEY}\n"
|
||||
host.failing.add("as install")
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert "old keys" not in [s["step"] for s in result["steps"]]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OLD_KEY}\n"
|
||||
|
||||
|
||||
def test_replaced_keys_are_a_list_not_one_string(driver: _Driver) -> None:
|
||||
with pytest.raises(ValueError, match="list"):
|
||||
driver.add_service_account("netork", NETORK_KEY, replaces=OLD_KEY)
|
||||
@@ -14,8 +14,19 @@ from napalm_device_types import (
|
||||
UUID = "31303043-534D-C40C-7A72-6F1300000000"
|
||||
|
||||
|
||||
def _out(uuid: str, product_serial: str, board_serial: str) -> str:
|
||||
return f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\nIDEND\n"
|
||||
def _out(
|
||||
uuid: str,
|
||||
product_serial: str,
|
||||
board_serial: str,
|
||||
bios: tuple[str, str, str] = ("", "", ""),
|
||||
) -> str:
|
||||
vendor, version, date = bios
|
||||
return (
|
||||
f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\n{vendor}\n{version}\n{date}\nIDEND\n"
|
||||
)
|
||||
|
||||
|
||||
_NO_FIRMWARE = {"bios_vendor": "", "bios_version": "", "bios_date": ""}
|
||||
|
||||
|
||||
# ── parsing ───────────────────────────────────────────────────────────────────
|
||||
@@ -26,6 +37,7 @@ def test_reads_uuid_and_both_serials():
|
||||
"system_uuid": "31303043-534d-c40c-7a72-6f1300000000",
|
||||
"product_serial": "S123",
|
||||
"board_serial": "ZM155S026022",
|
||||
**_NO_FIRMWARE,
|
||||
}
|
||||
|
||||
|
||||
@@ -81,9 +93,57 @@ def test_unreadable_files_read_as_empty():
|
||||
"system_uuid": "",
|
||||
"product_serial": "",
|
||||
"board_serial": "BS1",
|
||||
**_NO_FIRMWARE,
|
||||
}
|
||||
|
||||
|
||||
# ── the system firmware ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_reads_the_system_firmware():
|
||||
# pve-dual, a Supermicro X10DRL-i
|
||||
out = _out(UUID, "", "ZM155S026022", ("American Megatrends Inc.", "3.4", "11/22/2021"))
|
||||
identity = parse_system_identity(out)
|
||||
assert identity["bios_vendor"] == "American Megatrends Inc."
|
||||
assert identity["bios_version"] == "3.4"
|
||||
assert identity["bios_date"] == "2021-11-22"
|
||||
|
||||
|
||||
def test_a_virtual_machines_firmware_is_read_like_any_other():
|
||||
out = _out(UUID, "", "", ("EDK II", "4.2023.08-4", "02/15/2024"))
|
||||
assert parse_system_identity(out)["bios_version"] == "4.2023.08-4"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("version", ["1", "0", "1.0", "A"])
|
||||
def test_a_short_firmware_version_is_no_placeholder(version):
|
||||
# Serials of one repeated character are placeholders, versions are not.
|
||||
out = _out(UUID, "", "", ("Vendor", version, ""))
|
||||
assert parse_system_identity(out)["bios_version"] == version
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["", "To be filled by O.E.M.", "Default string", "Not Specified"])
|
||||
def test_a_placeholder_firmware_version_reads_as_empty(value):
|
||||
out = _out(UUID, "", "", (value, value, ""))
|
||||
identity = parse_system_identity(out)
|
||||
assert identity["bios_vendor"] == ""
|
||||
assert identity["bios_version"] == ""
|
||||
|
||||
|
||||
@pytest.mark.parametrize("date", ["", "not a date", "13/45/2021", "2021-11-22"])
|
||||
def test_a_firmware_date_that_is_no_smbios_date_reads_as_empty(date):
|
||||
# sysfs writes the SMBIOS date as MM/DD/YYYY; anything else is no reading of it.
|
||||
out = _out(UUID, "", "", ("Vendor", "1.2", date))
|
||||
assert parse_system_identity(out)["bios_date"] == ""
|
||||
|
||||
|
||||
def test_output_of_the_identity_alone_has_no_firmware():
|
||||
# What a host answered before the firmware was read with it: three fields.
|
||||
old = f"IDBEGIN\n{UUID}\nS1\nB1\nIDEND\n"
|
||||
identity = parse_system_identity(old)
|
||||
assert identity["board_serial"] == "B1"
|
||||
assert {k: identity[k] for k in _NO_FIRMWARE} == _NO_FIRMWARE
|
||||
|
||||
|
||||
def test_output_without_its_frame_is_not_a_reading():
|
||||
with pytest.raises(ValueError):
|
||||
parse_system_identity("sudo: a password is required\n")
|
||||
@@ -115,6 +175,13 @@ def test_the_driver_only_carries_the_command_across():
|
||||
assert privileged is True
|
||||
|
||||
|
||||
def test_the_command_reads_the_three_dmi_files():
|
||||
for name in ("product_uuid", "product_serial", "board_serial"):
|
||||
def test_the_command_reads_the_dmi_files():
|
||||
for name in (
|
||||
"product_uuid",
|
||||
"product_serial",
|
||||
"board_serial",
|
||||
"bios_vendor",
|
||||
"bios_version",
|
||||
"bios_date",
|
||||
):
|
||||
assert f"/sys/class/dmi/id/{name}" in SYSTEM_IDENTITY_COMMAND
|
||||
@@ -0,0 +1,20 @@
|
||||
"""The contract of :class:`~napalm_device_types.updates.UpdateMixin`."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import napalm_device_types
|
||||
from napalm_device_types.updates import PackageIndexMissing
|
||||
|
||||
|
||||
def test_a_missing_index_is_its_own_failure():
|
||||
"""A reader can tell "no index to read from" from any other failure, and a
|
||||
caller that only knows failures still catches it as one."""
|
||||
exc = PackageIndexMissing("The package index is not loaded (run apk update first)")
|
||||
|
||||
assert isinstance(exc, RuntimeError)
|
||||
assert str(exc) == "The package index is not loaded (run apk update first)"
|
||||
|
||||
|
||||
def test_it_is_exported_from_the_package():
|
||||
assert napalm_device_types.PackageIndexMissing is PackageIndexMissing
|
||||
assert "PackageIndexMissing" in napalm_device_types.__all__
|
||||
@@ -0,0 +1,360 @@
|
||||
"""A native Zabbix agent: its configuration, its repository and the fix template
|
||||
(``ZabbixAgentMixin``, NetOrk/netork#2571).
|
||||
|
||||
Where a host has no container engine the application installs the agent from the
|
||||
host's own packages, and from Zabbix's repository only where those have none.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
ZabbixAgentMixin,
|
||||
check_zabbix_agent_settings,
|
||||
zabbix_agent_config,
|
||||
zabbix_repository_command,
|
||||
)
|
||||
|
||||
SETTINGS = {
|
||||
"server": "zabbix.example.com",
|
||||
"server_active": "zabbix.example.com",
|
||||
"listen_port": 10050,
|
||||
"allowed_sources": ["10.40.100.254"],
|
||||
"hostname": "ap-1og",
|
||||
"release": "7.4",
|
||||
}
|
||||
|
||||
SAMPLE = """# This is a configuration file for Zabbix agent daemon (Unix)
|
||||
LogFile=/tmp/zabbix_agentd.log
|
||||
|
||||
### Option: Server
|
||||
# Server=
|
||||
|
||||
Server=127.0.0.1
|
||||
ServerActive=127.0.0.1
|
||||
Hostname=Zabbix server
|
||||
# Include=/etc/zabbix_agentd.conf.d/*.conf
|
||||
"""
|
||||
|
||||
|
||||
class TestConfig:
|
||||
def test_the_lines_that_say_who_may_ask_get_the_caller_s_values(self):
|
||||
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
|
||||
|
||||
assert "\nServer=zabbix.example.com\n" in text
|
||||
assert "\nServerActive=zabbix.example.com\n" in text
|
||||
assert "\nHostname=ap-1og\n" in text
|
||||
assert "Server: 127.0.0.1 → zabbix.example.com" in changes
|
||||
assert "Hostname: Zabbix server → ap-1og" in changes
|
||||
|
||||
def test_comments_and_other_lines_stay_as_they_were(self):
|
||||
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
|
||||
|
||||
assert text.startswith("# This is a configuration file")
|
||||
assert "LogFile=/tmp/zabbix_agentd.log\n" in text
|
||||
assert "# Server=\n" in text
|
||||
assert text.endswith("# Include=/etc/zabbix_agentd.conf.d/*.conf\n" + "ListenPort=10050\n")
|
||||
|
||||
def test_a_key_the_file_does_not_set_is_added_at_the_end(self):
|
||||
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
|
||||
|
||||
assert text.endswith("ListenPort=10050\n")
|
||||
assert "ListenPort: (not set) → 10050" in changes
|
||||
|
||||
def test_a_file_that_already_says_it_changes_nothing(self):
|
||||
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
|
||||
|
||||
again, changes = zabbix_agent_config(text, SETTINGS)
|
||||
|
||||
assert again == text
|
||||
assert changes == []
|
||||
|
||||
def test_without_a_hostname_the_agent_keeps_its_own(self):
|
||||
settings = {k: v for k, v in SETTINGS.items() if k != "hostname"}
|
||||
|
||||
text, changes = zabbix_agent_config(SAMPLE, settings)
|
||||
|
||||
assert "\nHostname=Zabbix server\n" in text
|
||||
assert not any(c.startswith("Hostname") for c in changes)
|
||||
|
||||
def test_no_active_server_comments_the_line_out(self):
|
||||
text, changes = zabbix_agent_config(SAMPLE, {**SETTINGS, "server_active": ""})
|
||||
|
||||
assert "\n# ServerActive=127.0.0.1\n" in text
|
||||
assert "\nServerActive=" not in text
|
||||
assert "ServerActive: 127.0.0.1 → (not set)" in changes
|
||||
|
||||
def test_a_file_without_a_final_newline_gets_one_before_what_is_added(self):
|
||||
text, _ = zabbix_agent_config("Server=127.0.0.1", SETTINGS)
|
||||
|
||||
assert text.startswith("Server=zabbix.example.com\n")
|
||||
assert text.endswith("\n")
|
||||
|
||||
def test_every_line_of_a_key_set_twice_says_the_same(self):
|
||||
text, _ = zabbix_agent_config("Server=a\nServer=b\n", SETTINGS)
|
||||
|
||||
assert text.count("Server=zabbix.example.com\n") == 2
|
||||
|
||||
|
||||
class TestSettings:
|
||||
@pytest.mark.parametrize(
|
||||
"server",
|
||||
["", "zabbix.example.com\nUserParameter=x,id", "a b", "a;rm -rf /", "'a'"],
|
||||
)
|
||||
def test_a_server_that_is_not_addresses_is_refused(self, server):
|
||||
with pytest.raises(ValueError):
|
||||
check_zabbix_agent_settings({**SETTINGS, "server": server})
|
||||
|
||||
def test_several_servers_and_ports_pass(self):
|
||||
check_zabbix_agent_settings(
|
||||
{**SETTINGS, "server": "10.0.0.1,zabbix.example.com", "server_active": "[::1]:10051"}
|
||||
)
|
||||
|
||||
def test_a_hostname_with_a_newline_is_refused(self):
|
||||
with pytest.raises(ValueError):
|
||||
check_zabbix_agent_settings({**SETTINGS, "hostname": "ap\nServer=evil"})
|
||||
|
||||
@pytest.mark.parametrize("port", [0, 65536, "10050"])
|
||||
def test_a_listen_port_out_of_range_is_refused(self, port):
|
||||
with pytest.raises(ValueError):
|
||||
check_zabbix_agent_settings({**SETTINGS, "listen_port": port})
|
||||
|
||||
@pytest.mark.parametrize("release", ["7", "7.4.1", "latest", "7.4; id"])
|
||||
def test_a_release_that_is_not_major_dot_minor_is_refused(self, release):
|
||||
with pytest.raises(ValueError):
|
||||
check_zabbix_agent_settings({**SETTINGS, "release": release})
|
||||
|
||||
|
||||
def _os_release(**fields: str) -> str:
|
||||
return "".join(f'{key}="{value}"\n' for key, value in fields.items())
|
||||
|
||||
|
||||
class TestRepository:
|
||||
BASE = "https://repo.zabbix.com/zabbix"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("os_release", "url"),
|
||||
[
|
||||
(
|
||||
_os_release(ID="ubuntu", VERSION_ID="24.04", ID_LIKE="debian"),
|
||||
"7.4/release/ubuntu/pool/main/z/zabbix-release/"
|
||||
"zabbix-release_latest_7.4+ubuntu24.04_all.deb",
|
||||
),
|
||||
(
|
||||
_os_release(ID="debian", VERSION_ID="12"),
|
||||
"7.4/release/debian/pool/main/z/zabbix-release/"
|
||||
"zabbix-release_latest_7.4+debian12_all.deb",
|
||||
),
|
||||
(
|
||||
_os_release(ID="ol", VERSION_ID="9.4", ID_LIKE="fedora"),
|
||||
"7.4/release/oracle/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
|
||||
),
|
||||
(
|
||||
_os_release(ID="rhel", VERSION_ID="10.0", ID_LIKE="fedora"),
|
||||
"7.4/release/rhel/10/noarch/zabbix-release-latest-7.4.el10.noarch.rpm",
|
||||
),
|
||||
(
|
||||
_os_release(ID="almalinux", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
|
||||
"7.4/release/alma/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
|
||||
),
|
||||
(
|
||||
_os_release(ID="rocky", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
|
||||
"7.4/release/rocky/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_the_release_package_of_the_host_s_distribution(self, os_release, url):
|
||||
command = zabbix_repository_command(os_release, "7.4")
|
||||
|
||||
assert f"{self.BASE}/{url}" in command
|
||||
|
||||
def test_a_release_before_7_2_lives_in_the_old_layout(self):
|
||||
deb = zabbix_repository_command(_os_release(ID="ubuntu", VERSION_ID="24.04"), "7.0")
|
||||
rpm = zabbix_repository_command(_os_release(ID="ol", VERSION_ID="9.4"), "7.0")
|
||||
|
||||
assert f"{self.BASE}/7.0/ubuntu/pool/main/z/zabbix-release/" in deb
|
||||
assert f"{self.BASE}/7.0/oracle/9/x86_64/zabbix-release-latest-7.0.el9.noarch.rpm" in rpm
|
||||
|
||||
def test_a_deb_is_installed_with_dpkg_and_the_index_refreshed(self):
|
||||
command = zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4")
|
||||
|
||||
assert "dpkg -i" in command
|
||||
assert "apt-get update" in command
|
||||
|
||||
def test_an_rpm_is_installed_with_the_package_manager(self):
|
||||
command = zabbix_repository_command(_os_release(ID="rocky", VERSION_ID="9.5"), "7.4")
|
||||
|
||||
assert "dnf -y install" in command
|
||||
|
||||
def test_a_derivative_falls_back_to_what_it_is_like(self):
|
||||
command = zabbix_repository_command(
|
||||
_os_release(ID="centos", VERSION_ID="9", ID_LIKE="rhel fedora"), "7.4"
|
||||
)
|
||||
|
||||
assert "/7.4/release/rhel/9/noarch/" in command
|
||||
|
||||
def test_a_distribution_zabbix_has_no_repository_for_is_refused(self):
|
||||
with pytest.raises(ValueError, match="openwrt"):
|
||||
zabbix_repository_command(_os_release(ID="openwrt", VERSION_ID="25.12.2"), "7.4")
|
||||
|
||||
def test_a_release_that_is_not_major_dot_minor_is_refused(self):
|
||||
with pytest.raises(ValueError):
|
||||
zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4;id")
|
||||
|
||||
|
||||
class _Agent(ZabbixAgentMixin):
|
||||
"""The generic steps over ensure_package and manage_service; the driver's others."""
|
||||
|
||||
def __init__(self, refuse: frozenset[str] = frozenset(), repository: bool = True):
|
||||
self.calls: list = []
|
||||
self.refuse = refuse
|
||||
self.installs = 0
|
||||
if not repository:
|
||||
self._add_zabbix_repository = None # type: ignore[assignment]
|
||||
|
||||
def refresh_available_updates(self):
|
||||
self.calls.append(("refresh",))
|
||||
ok = "refresh" not in self.refuse
|
||||
return {"success": ok, "output": "" if ok else "Temporary failure resolving"}
|
||||
|
||||
def ensure_package(self, name):
|
||||
self.calls.append(("ensure_package", name))
|
||||
self.installs += 1
|
||||
# The distribution has no package; after the repository is added it has.
|
||||
ok = "distribution" not in self.refuse or self.installs > 1
|
||||
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
|
||||
|
||||
def _add_zabbix_repository(self, release):
|
||||
self.calls.append(("repository", release))
|
||||
ok = "repository" not in self.refuse
|
||||
return {"ok": ok, "output": f"Zabbix {release} repository added" if ok else "404"}
|
||||
|
||||
def _write_zabbix_agent_config(self, settings):
|
||||
self.calls.append(("configure", settings["server"]))
|
||||
return {"ok": "configure" not in self.refuse, "output": "written"}
|
||||
|
||||
def _allow_zabbix_agent_sources(self, settings):
|
||||
self.calls.append(("firewall", settings["allowed_sources"]))
|
||||
return {"ok": "firewall" not in self.refuse, "output": "open"}
|
||||
|
||||
def manage_service(self, name, action):
|
||||
self.calls.append((action, name))
|
||||
ok = action not in self.refuse
|
||||
return {"success": ok, "output": "" if ok else f"cannot {action}"}
|
||||
|
||||
|
||||
class TestZabbixAgentMixin:
|
||||
def test_install_configure_open_enable_and_restart(self):
|
||||
agent = _Agent()
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls == [
|
||||
("refresh",),
|
||||
("ensure_package", "zabbix-agent"),
|
||||
("configure", "zabbix.example.com"),
|
||||
("firewall", ["10.40.100.254"]),
|
||||
("enable", "zabbix-agent"),
|
||||
("restart", "zabbix-agent"),
|
||||
]
|
||||
assert [s["step"] for s in result["steps"]] == ["install", "configure", "firewall", "start"]
|
||||
|
||||
def test_without_a_package_in_the_distribution_zabbix_s_repository_is_added(self):
|
||||
agent = _Agent(refuse=frozenset({"distribution"}))
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls[:4] == [
|
||||
("refresh",),
|
||||
("ensure_package", "zabbix-agent"),
|
||||
("repository", "7.4"),
|
||||
("ensure_package", "zabbix-agent"),
|
||||
]
|
||||
assert "Zabbix 7.4 repository added" in result["output"]
|
||||
|
||||
def test_without_a_release_there_is_no_fallback(self):
|
||||
settings = {k: v for k, v in SETTINGS.items() if k != "release"}
|
||||
agent = _Agent(refuse=frozenset({"distribution"}))
|
||||
|
||||
result = agent.configure_zabbix_agent(settings)
|
||||
|
||||
assert result["success"] is False
|
||||
assert ("repository", "7.4") not in agent.calls
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
def test_a_driver_that_cannot_add_the_repository_fails_with_the_distribution_s_answer(self):
|
||||
agent = _Agent(refuse=frozenset({"distribution"}), repository=False)
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is False
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
def test_a_repository_that_cannot_be_added_configures_nothing(self):
|
||||
agent = _Agent(refuse=frozenset({"distribution", "repository"}))
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is False
|
||||
assert [c[0] for c in agent.calls] == ["refresh", "ensure_package", "repository"]
|
||||
assert "404" in result["output"]
|
||||
|
||||
def test_the_package_index_is_loaded_first(self):
|
||||
agent = _Agent()
|
||||
|
||||
agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert agent.calls[0] == ("refresh",)
|
||||
|
||||
def test_an_index_that_will_not_load_still_lets_the_install_try(self):
|
||||
agent = _Agent(refuse=frozenset({"refresh"}))
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is True
|
||||
assert ("ensure_package", "zabbix-agent") in agent.calls
|
||||
assert "Temporary failure resolving" in result["steps"][0]["output"]
|
||||
|
||||
def test_a_driver_without_an_index_to_load_installs_straight_away(self):
|
||||
agent = _Agent()
|
||||
agent.refresh_available_updates = None # type: ignore[assignment]
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls[0] == ("ensure_package", "zabbix-agent")
|
||||
|
||||
def test_a_closed_firewall_starts_nothing(self):
|
||||
agent = _Agent(refuse=frozenset({"firewall"}))
|
||||
|
||||
assert agent.configure_zabbix_agent(SETTINGS)["success"] is False
|
||||
assert ("restart", "zabbix-agent") not in agent.calls
|
||||
|
||||
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
|
||||
agent = _Agent(refuse=frozenset({"enable"}))
|
||||
|
||||
result = agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert result["success"] is False
|
||||
assert ("restart", "zabbix-agent") not in agent.calls
|
||||
assert "cannot enable" in result["output"]
|
||||
|
||||
def test_the_package_and_service_names_are_the_driver_s(self):
|
||||
agent = _Agent()
|
||||
agent.ZABBIX_AGENT_PACKAGE = "zabbix-agentd"
|
||||
agent.ZABBIX_AGENT_SERVICE = "zabbix_agentd"
|
||||
|
||||
agent.configure_zabbix_agent(SETTINGS)
|
||||
|
||||
assert ("ensure_package", "zabbix-agentd") in agent.calls
|
||||
assert ("restart", "zabbix_agentd") in agent.calls
|
||||
|
||||
def test_settings_that_do_not_hold_are_refused_before_anything_changes(self):
|
||||
agent = _Agent()
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
agent.configure_zabbix_agent({**SETTINGS, "server": "a\nb"})
|
||||
assert agent.calls == []
|
||||
Reference in new issue
Block a user