Compare commits

...
16 Commits
Author SHA1 Message Date
christianmanivong eb1964c4b3 Merge pull request 'chore(release): 4.40.0' (#125) from chore/release-4.40.0 into main
driver / types (push) Successful in 57s
driver / standard (push) Successful in 17s
driver / lint (push) Successful in 12s
driver / test (3.12) (push) Successful in 1m14s
driver / test (3.13) (push) Successful in 1m1s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 33s
driver / security (push) Successful in 57s
driver / publish (push) Successful in 27s
CI / driver (push) Successful in 3m10s
2026-11-17 04:06:06 +00:00
christianmanivong 3622d0f825 chore(release): 4.40.0
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 57s
CI / driver (pull_request) Successful in 2m42s
driver / publish (pull_request) Skipped
2026-11-17 05:00:46 +01:00
christianmanivong caf7ef4cc8 Merge pull request 'feat(fixes): ZabbixAgentMixin, a native Zabbix agent for a host without a container engine (NetOrk/netork#2571)' (#124) from feat/zabbix-agent-native into main
driver / standard (push) Successful in 16s
driver / lint (push) Successful in 14s
driver / types (push) Successful in 55s
driver / test (3.12) (push) Successful in 1m14s
driver / lowest (push) Successful in 38s
driver / test (3.13) (push) Successful in 1m9s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m2s
CI / driver (push) Successful in 2m43s
driver / publish (push) Skipped
2026-11-17 04:00:27 +00:00
christianmanivong 3422b7d030 Merge remote-tracking branch 'origin/main' into feat/zabbix-agent-native
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Successful in 12s
driver / types (pull_request) Successful in 55s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / test (3.13) (pull_request) Successful in 59s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m38s
driver / publish (pull_request) Skipped
# Conflicts:
#	CHANGELOG.md
2026-11-17 04:55:56 +01:00
christianmanivong f1940aa918 Merge pull request 'feat(container-engine): reach rootful Podman like Docker (NetOrk/netork#818)' (#123) from feature/podman-engine-netork-818 into main
driver / lint (push) Successful in 12s
driver / standard (push) Successful in 15s
driver / types (push) Successful in 49s
driver / test (3.12) (push) Successful in 1m8s
driver / test (3.13) (push) Successful in 58s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 27s
driver / security (push) Successful in 55s
CI / driver (push) Successful in 2m18s
driver / publish (push) Skipped
2026-11-17 03:54:09 +00:00
christianmanivong 5217c6e333 feat(fixes): ZabbixAgentMixin, a native Zabbix agent for a host without a container engine (NetOrk/netork#2571)
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 49s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 34s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / build (pull_request) Successful in 24s
driver / security (pull_request) Successful in 55s
CI / driver (pull_request) Successful in 2m20s
driver / publish (pull_request) Skipped
configure_zabbix_agent(settings) loads the package index where the driver can,
installs ZABBIX_AGENT_PACKAGE from the host's own packages and, where those have
none and settings["release"] names the server's release line, from Zabbix's
repository (the driver's _add_zabbix_repository); then the driver writes the
configuration and opens the firewall, and the service is enabled and restarted.
Settings that do not hold raise ValueError before anything changes.

zabbix_agent.py holds what is the same on every host, without I/O: the settings
check, the agent's configuration with Server, ServerActive, ListenPort and
Hostname set and the rest kept, and the root command that adds Zabbix's
repository for Debian, Ubuntu, Raspberry Pi OS and the RHEL family, in the
layout before and since 7.2. LldpAgentMixin and ZabbixAgentMixin share the
start step (enable_and_restart).
2026-11-17 04:50:42 +01:00
christianmanivong 7051199589 feat(container-engine): reach rootful Podman like Docker (NetOrk/netork#818)
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 55s
driver / test (3.13) (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m13s
driver / lowest (pull_request) Successful in 37s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m27s
driver / publish (pull_request) Skipped
container_engines() lists Podman with the API podman-libpod, and
open_container_engine("podman") reaches it with podman system dial-stdio
(Podman 4.0 and later). It is the host's rootful Podman: the engine of the
system's containers and of Quadlet's units, whose socket is root's. So the
connection runs every call as root (ContainerEngineConnection.privileged).

An engine is listed only when its CLI names itself on --version:
podman-docker's docker is podman, and listing it as Docker would read the
login user's rootless Podman a second time.
2026-11-16 23:26:55 +01:00
christianmanivong 7c0e4fffc5 Merge pull request 'feat(apt): send https sources through the cache in the form it stores (netork#2555)' (#122) from fix/apt-https-cache into main
driver / lint (push) Successful in 13s
driver / standard (push) Successful in 16s
driver / types (push) Successful in 58s
driver / test (3.13) (push) Successful in 1m14s
driver / test (3.12) (push) Successful in 1m21s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m3s
CI / driver (push) Successful in 2m33s
driver / publish (push) Skipped
2026-11-15 19:23:46 +00:00
christianmanivong cb0f1cc05c feat(apt): send https sources through the cache in the form it stores (NetOrk/netork#2555)
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 1m1s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 35s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 2m29s
driver / publish (pull_request) Skipped
apt-cacher-ng tunnels a source fetched over https and stores nothing of it. It
fetches http://HTTPS///<host>/<path> over https itself, with the certificate
verified, and stores it, where port 443 is allowed (AllowUserPorts=80 443).

- apt_sources: cacheable_https_hosts(report), and the root commands that rewrite
  the https sources of .list and deb822 .sources files to that form and back.
  A host apt has credentials for in auth.conf stays on https; the rewrite names
  it in the proxy file as Package-Proxy::Not-Cached, which the report reads.
- configure_package_proxy(url): a step "cache" before "https". It rewrites the
  sources, fetches one of them through url, and turns them all back to https
  where url cannot. The "fetch" step proves the proxy with a plain http source.
- configure_package_proxy(""): the sources go back to https before the proxy
  file goes; without the cache apt would look for a host called HTTPS.
2026-11-15 20:07:27 +01:00
christianmanivong b4929990f1 Merge pull request 'chore(release): 4.39.0' (#121) from chore/release-4.39.0 into main
driver / types (push) Successful in 54s
driver / standard (push) Successful in 17s
driver / lint (push) Successful in 13s
driver / test (3.12) (push) Successful in 1m12s
driver / lowest (push) Successful in 36s
driver / test (3.13) (push) Successful in 1m2s
driver / build (push) Successful in 26s
driver / security (push) Successful in 1m20s
driver / publish (push) Successful in 29s
CI / driver (push) Successful in 3m32s
2026-11-14 22:56:28 +00:00
christianmanivong 3db6c6385c Merge pull request 'feat(updates): PackageIndexMissing, for a host without a package index' (#120) from feat/package-index-missing into main
driver / lint (push) Successful in 12s
driver / standard (push) Successful in 14s
driver / types (push) Successful in 53s
driver / test (3.13) (push) Successful in 1m1s
driver / test (3.12) (push) Successful in 1m7s
driver / lowest (push) Successful in 34s
driver / build (push) Successful in 28s
driver / security (push) Successful in 1m0s
CI / driver (push) Successful in 2m15s
driver / publish (push) Skipped
2026-11-14 22:52:11 +00:00
christianmanivong c5154d954b chore(release): 4.39.0
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Successful in 12s
driver / types (pull_request) Successful in 49s
driver / test (3.12) (pull_request) Successful in 1m3s
driver / lowest (pull_request) Successful in 33s
driver / test (3.13) (pull_request) Successful in 59s
driver / build (pull_request) Successful in 24s
driver / security (pull_request) Successful in 50s
CI / driver (pull_request) Successful in 2m19s
driver / publish (pull_request) Skipped
2026-11-14 23:52:11 +01:00
christianmanivong 211d0c02eb feat(updates): PackageIndexMissing, for a host without a package index
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / lowest (pull_request) Successful in 36s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 52s
CI / driver (pull_request) Successful in 2m12s
driver / publish (pull_request) Skipped
OpenWrt keeps its package index in RAM and loses it on every reboot.
get_available_updates() raises then, rightly, but only as a bare
RuntimeError, so a caller cannot tell it from a read that failed and
reports a communication error for a state a refresh cures.

PackageIndexMissing (a RuntimeError, so existing callers still catch it)
names that case; refresh_available_updates() is the way out.

Refs NetOrk/netork#2527
2026-11-14 23:45:38 +01:00
christianmanivong 74d2f42b0e Merge pull request 'feat(system-identity): read the system firmware with the identity' (#119) from feat/system-firmware into main
driver / standard (push) Successful in 17s
driver / lint (push) Successful in 16s
driver / types (push) Successful in 56s
driver / test (3.12) (push) Successful in 1m13s
driver / test (3.13) (push) Successful in 1m10s
driver / lowest (push) Successful in 43s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m6s
CI / driver (push) Successful in 2m52s
driver / publish (push) Skipped
2026-11-13 22:11:26 +00:00
christianmanivong c25f39768f refactor(system-identity): keep SystemIdentityDict with its mixin
driver / standard (pull_request) Successful in 18s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 57s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 2m51s
driver / publish (pull_request) Skipped
models.py is over its line limit and may not grow; the dict has no other
user than system_identity.py, as clusters.py and apt_proxy.py keep theirs.

NetOrk/netork#2518
2026-11-13 23:04:18 +01:00
christianmanivong 9ecc6823c8 feat(system-identity): read the system firmware with the identity
driver / lint (pull_request) Successful in 13s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 52s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 58s
CI / driver (pull_request) Failing after 2m17s
driver / publish (pull_request) Skipped
get_system_identity() also reads bios_vendor, bios_version and bios_date
from /sys/class/dmi/id, in the same round trip as the SMBIOS identity.
The date is given as ISO 8601; anything that is no MM/DD/YYYY reads as
empty. Only the vendors' text defaults are placeholders for the firmware:
a version of "1" is a version, unlike a serial of one repeated character.

Output of the three identity fields alone still parses, with the firmware
empty.

NetOrk/netork#2518
2026-11-13 23:00:23 +01:00
21 changed files with 1870 additions and 88 deletions

No files matched your search

+55
View File
@@ -7,9 +7,64 @@ recorded here; their history is the git log.
## [Unreleased]
## [4.40.0] - 2026-11-17
### Added
- `ZabbixAgentMixin.configure_zabbix_agent(settings)`: a native Zabbix agent, for a host
without a container engine (NetOrk/netork#2571). Install (`ensure_package` with
`ZABBIX_AGENT_PACKAGE`, and where the host's own packages have none and
`settings["release"]` names the server's release line, the driver's
`_add_zabbix_repository(release)` and a second try), configure (the driver's
`_write_zabbix_agent_config`), firewall (the driver's `_allow_zabbix_agent_sources`),
start (`manage_service` enable and restart of `ZABBIX_AGENT_SERVICE`). Settings that do
not hold raise `ValueError` before anything changes.
- `ZabbixAgentSettingsDict`; `zabbix_agent_config(text, settings)`, the agent's
configuration with `Server`, `ServerActive`, `ListenPort` and `Hostname` set and the rest
kept; `zabbix_repository_command(os_release, release)`, the root command that adds
Zabbix's repository for Debian, Ubuntu, Raspberry Pi OS, RHEL, CentOS, Oracle Linux,
AlmaLinux and Rocky Linux, in the layout before and since 7.2;
`check_zabbix_agent_settings(settings)`.
- `fixes.enable_and_restart(manage_service, service)`, the start step `LldpAgentMixin` and
`ZabbixAgentMixin` share.
- Podman as a container engine (NetOrk/netork#818): `container_engines()` lists it with the
API `podman-libpod`, and `open_container_engine("podman")` reaches it with `podman system
dial-stdio` (Podman 4.0 and later). It is the host's rootful Podman, so the connection
runs every call as root (`ContainerEngineConnection.privileged`, `ROOT_ENGINES`).
- `apt_sources`: apt's https sources through the package cache, in the form apt-cacher-ng
fetches over https itself and stores (`http://HTTPS///<host>/…`, with `AllowUserPorts=80
443` on the cache). `cacheable_https_hosts(report)` names the hosts this applies to, and
`rewrite_sources_command` / `revert_sources_command` are the root commands, for `.list`
and deb822 `.sources` files. A host apt keeps credentials for in `auth.conf` stays on
https, because apt finds them by the host's name and the cache is shared. The rewrite
names such a host in the proxy file as `Package-Proxy::Not-Cached` (NetOrk/netork#2555).
- `get_package_proxy_report()` reports those hosts as `not_cached`.
### Changed
- `container_engines()` lists an engine only when its CLI names itself on `--version`:
podman-docker's `docker` is podman, and is no longer listed as Docker.
- `configure_package_proxy(url)` has a step `cache` before `https`. It rewrites the https
sources it can, fetches one of them through *url*, and turns them all back to https
where *url* cannot. The `fetch` step proves the proxy with a plain http source only.
- `configure_package_proxy("")` turns the sources back to https (step `sources`) before it
removes the proxy file. Without the cache, apt would look for a host called `HTTPS`.
## [4.39.0] - 2026-11-14
### Added
- `PackageIndexMissing` (a `RuntimeError`): what `get_available_updates()` raises when
the host holds no package index to read from -- OpenWrt's, which lives in RAM, after
every reboot. `refresh_available_updates()` loads it again, so a caller can tell this
from a failed read and refresh instead of reporting an error (NetOrk/netork#2527).
## [4.38.0] - 2026-11-13
### Added
- `get_system_identity()` also reads the host's system firmware (BIOS or UEFI) from the
same SMBIOS tables in the same round trip: `bios_vendor`, `bios_version` and
`bios_date` (ISO 8601, from sysfs's `MM/DD/YYYY`; anything else reads as empty). Only
the vendors' text defaults (`To be filled by O.E.M.`, `Default string`) are
placeholders there -- a version of `1` is a version. Output of the three identity
fields alone still parses, with the firmware empty (NetOrk/netork#2518).
- `ConfigRevisionsMixin`: a device's own configuration history (NetOrk/netork#2511).
`get_config_revisions()` (newest first, the first in place now, as
`ConfigRevisionDict`: id, UTC time, description, user, size) and
+10 -3
View File
@@ -124,7 +124,10 @@ application wants made on a device runs as fixed steps the driver implements, wi
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
interfaces, enable and restart; the driver writes only the interfaces),
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
`ZabbixAgentMixin.configure_zabbix_agent(settings)` (install from the host's packages or
else Zabbix's repository, configure, firewall, enable and restart; the driver writes the
configuration and opens the firewall), `EnsurePackageMixin.ensure_package(name)`,
`PackageProxyMixin.configure_package_proxy(url)`,
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
change in one call overrides the template method. A driver mixes in the fixes its device
@@ -286,6 +289,9 @@ O.E.M.`, `Default string`, all-zero or the common `03000200-…-000700080009` UU
empty: matching on them would join unrelated machines. `is_placeholder_identity` is the
same filter for a BMC driver's side. `product_uuid` and `product_serial` are root's, so a
driver supplies `_run_system_identity_command(command, privileged=)`.
The same round trip reads the host's system firmware (BIOS or UEFI): `bios_vendor`,
`bios_version` and `bios_date` (`YYYY-MM-DD`, from sysfs's `MM/DD/YYYY`). Only the vendors'
text defaults read as empty there; a short version such as `1` is a version.
`OpenSshServerMixin` (`harden_ssh_server`, `get_ssh_server_algorithms`) takes algorithms
out of what an OpenSSH server offers -- the ones ssh-audit flags. It reads the lists sshd
@@ -400,10 +406,11 @@ but not revert sets `revert_config_revision = None`.
The mixin sits in `DeviceTypeDriver` and only declares, so `hasattr(driver, "run_command")` is true exactly where a driver implements it. For SSH drivers, `channel.run_on_transport` and `channel.open_stream_on_transport` are the implementation over a paramiko exec channel: no PTY, stderr kept apart, a real exit code. How a command gains root stays with the driver.
**`ContainerEngineMixin` is deliberately different from `SystemdServicesMixin`.** The systemd mixin owns its command and its parse; this one owns neither.
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}]`).
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}, {"engine": "podman", "api": "podman-libpod"}]`). An engine is listed when its CLI answers `--version` with its own name; podman-docker's `docker` says it is podman and is left out.
- `open_container_engine(engine)` returns a `ContainerEngineConnection`:
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`);
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`, `podman system dial-stdio`);
- `run_cli(args)` and `stream_cli(args)` run the engine's CLI with arguments the caller chooses. With `privileged=True` the driver runs it the way `run_command` gains root, for an engine that refuses the login user.
- **Podman is reached as root** (`ROOT_ENGINES`, `connection.privileged`): the host's rootful Podman, the engine of the system's containers and of Quadlet's units, whose API `podman.socket` serves. A login user's rootless containers are that user's own and invisible to others. `podman system dial-stdio` exists from Podman 4.0 on and needs the socket listening (`systemctl enable --now podman.socket`) (NetOrk/netork#818).
- The driver decides only *how* the engine is reached. Its hook `_container_engine_binary(engine)` returns, for QNAP, the Container Station path, and the caller never sees it.
**What runs on the engine, and what to do with it, is netOrk's** (NetOrk/netork#765): the container model, the Engine API requests, compose, updates. Above the connection everything is specific to the service, so this package abstracts the connection and nothing more.
+1
View File
@@ -80,6 +80,7 @@ The same shape replaces every action in today's `run_device_action`:
| `fix_apt_proxy` | `configure_package_proxy(url)` |
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
| the Zabbix agent where no container engine runs (NetOrk/netork#2571) | `configure_zabbix_agent(settings)` |
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
driver that inherits a template it cannot run on its device, as a NAS driver built on the
+14 -1
View File
@@ -144,6 +144,7 @@ from napalm_device_types.fixes import (
LldpAgentMixin,
PackageProxyMixin,
SnmpAgentMixin,
ZabbixAgentMixin,
run_fix_steps,
)
from napalm_device_types.health_metrics import HealthMetricsMixin
@@ -335,7 +336,7 @@ from napalm_device_types.time_sync import (
check_ntp_servers,
)
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
from napalm_device_types.updates import UpdateMixin
from napalm_device_types.updates import PackageIndexMissing, UpdateMixin
from napalm_device_types.virtual_ips import (
VIRTUAL_IPS_COMMAND,
VirtualIpsMixin,
@@ -343,6 +344,12 @@ from napalm_device_types.virtual_ips import (
)
from napalm_device_types.vm_migration import MigrationPreflightDict
from napalm_device_types.vm_stop_request import VmStopRequestMixin
from napalm_device_types.zabbix_agent import (
ZabbixAgentSettingsDict,
check_zabbix_agent_settings,
zabbix_agent_config,
zabbix_repository_command,
)
__all__ = [
"BACKUP_JOB_KINDS",
@@ -392,6 +399,11 @@ __all__ = [
"SnmpAgentMixin",
"SnmpAgentSettingsDict",
"run_fix_steps",
"ZabbixAgentMixin",
"ZabbixAgentSettingsDict",
"check_zabbix_agent_settings",
"zabbix_agent_config",
"zabbix_repository_command",
"SnmpHealthProfile",
"SnmpInterfacesSpec",
"SnmpMemorySpec",
@@ -505,6 +517,7 @@ __all__ = [
"SystemdServicesMixin",
"SystemdUnavailable",
"parse_systemd_services",
"PackageIndexMissing",
"UpdateMixin",
"VIRTUAL_IPS_COMMAND",
"VirtualIpsMixin",
+106 -17
View File
@@ -35,8 +35,12 @@ page is the right one, the caller says (``expect``).
out every other line that sets a different one, reads the result back from
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
last step is what proves it: apt reached a mirror through the proxy it is now set
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
to. The source chosen is one apt does not fetch ``DIRECT``. Then the https sources
go through the proxy in the form it stores (:mod:`~napalm_device_types.apt_sources`),
and one of them is fetched; where the proxy cannot, they go back to https. Last it
fetches one https source left the same way; where the proxy refuses it, https goes
``DIRECT``. Removing the proxy turns the sources back to https first: without the
proxy apt cannot fetch the stored form.
"""
from __future__ import annotations
@@ -49,6 +53,14 @@ from urllib.parse import urlsplit
from typing_extensions import TypedDict
from napalm_device_types.apt_sources import (
NOT_CACHED_KEY,
VIA_CACHE,
cacheable_https_hosts,
is_host_name,
revert_sources_command,
rewrite_sources_command,
)
from napalm_device_types.fixes import PackageProxyMixin, run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
from napalm_device_types.terminal import strip_terminal_codes
@@ -87,6 +99,9 @@ class AptProxyReportDict(TypedDict):
#: The base URIs of the host's sources over http and over https.
http_sources: list[str]
https_sources: list[str]
#: The hosts the cache rewrite left on https, for their credentials
#: (:data:`~napalm_device_types.apt_sources.NOT_CACHED_KEY`).
not_cached: list[str]
class ProxyFetchDict(TypedDict):
@@ -123,7 +138,8 @@ PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
#: them early.
APT_PROXY_COMMAND = (
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
"echo '[dump]'; apt-config dump 2>/dev/null "
"| grep -i -E '^(Acquire::https?::Proxy|Package-Proxy::)'; "
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
@@ -133,6 +149,7 @@ APT_PROXY_COMMAND = (
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
)
_NOT_CACHED = re.compile(rf'^{re.escape(NOT_CACHED_KEY)}::\s+"([^"]+)"', re.I)
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
@@ -293,6 +310,7 @@ def parse_apt_proxy(output: str) -> AptProxyReportDict:
"https_settings": _settings(lines, _https_setting),
"http_sources": _of_scheme(sources, "http"),
"https_sources": _of_scheme(sources, "https"),
"not_cached": [m.group(1) for m in map(_NOT_CACHED.match, sections.get("dump", [])) if m],
}
@@ -498,16 +516,20 @@ class AptProxyMixin(PackageProxyMixin):
def configure_package_proxy(self, url: str) -> FixResultDict:
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
The https sources go through *url* in the form it stores, where it can; the
rest goes through *url* too, unless *url* refuses it: then ``DIRECT``.
"""
if not url:
return run_fix_steps([("configure", self._remove_package_proxy)])
return run_fix_steps(
[("sources", self._revert_sources), ("configure", self._remove_package_proxy)]
)
return run_fix_steps(
[
("configure", lambda: self._write_package_proxy(url)),
("conflicts", lambda: self._comment_out_other_proxies(url)),
("verify", lambda: self._verify_package_proxy(url)),
("fetch", lambda: self._fetch_through_package_proxy(url)),
("cache", lambda: self._cache_https_sources(url)),
("https", lambda: self._https_through_package_proxy(url)),
]
)
@@ -515,6 +537,11 @@ class AptProxyMixin(PackageProxyMixin):
def _not_apt(self) -> FixStepOutcomeDict:
return {"ok": False, "output": "not an apt host"}
def _revert_sources(self) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
return self._run_apt_root_command(revert_sources_command(), "sources back on https")
def _remove_package_proxy(self) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
@@ -522,7 +549,9 @@ class AptProxyMixin(PackageProxyMixin):
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
)
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
def _write_package_proxy(
self, url: str, https_direct: bool = False, not_cached: list[str] | None = None
) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
if not PROXY_URL.match(url):
@@ -530,6 +559,10 @@ class AptProxyMixin(PackageProxyMixin):
content = f'Acquire::http::Proxy "{url}";\n'
if https_direct:
content += 'Acquire::https::Proxy "DIRECT";\n'
# Host names the report read from this file; one a line could not carry is left out.
content += "".join(
f'{NOT_CACHED_KEY}:: "{host}";\n' for host in not_cached or [] if is_host_name(host)
)
return self._write_apt_file(self.APT_PROXY_FILE, content)
def _report(self) -> AptProxyReportDict:
@@ -566,16 +599,7 @@ class AptProxyMixin(PackageProxyMixin):
}
def _fetch_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
report = self._report()
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
source = next(
(
s
for s in report["http_sources"]
if "/dists/" in s and (urlsplit(s).hostname or "").lower() not in direct
),
None,
)
source = _plain_http_source(self._report())
if source is None:
return {"ok": True, "output": "no http source to fetch through the proxy; not tried"}
target = source.rstrip("/") + "/InRelease"
@@ -584,6 +608,49 @@ class AptProxyMixin(PackageProxyMixin):
return {"ok": True, "output": f"fetched {target} through {url}"}
return {"ok": False, "output": f"{target}: {fetched['error']}"}
def _cache_https_sources(self, url: str) -> FixStepOutcomeDict:
"""The https sources through *url* in the form it stores; back to https where it cannot."""
hosts = cacheable_https_hosts(self._report())
if not hosts:
return {"ok": True, "output": "no https source to cache"}
rewritten = self._run_apt_root_command(
rewrite_sources_command(hosts, self.APT_PROXY_FILE), "sources rewritten"
)
if not rewritten["ok"]:
return rewritten
return self._fetch_cached(url, hosts, self._report())
def _fetch_cached(
self, url: str, hosts: list[str], report: AptProxyReportDict
) -> FixStepOutcomeDict:
"""One rewritten source through *url*: it stays where *url* fetched it."""
left = sorted(set(hosts) & set(report["not_cached"]))
named = f"; left on https, apt has credentials for them: {', '.join(left)}" if left else ""
target = next(
(s.rstrip("/") + "/InRelease" for s in report["http_sources"] if _cached(s)), None
)
if target is None:
return {"ok": True, "output": f"no https source to cache{named}"}
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
if fetched["reachable"] and not fetched["error"]:
cached = ", ".join(h for h in hosts if h not in left)
return {"ok": True, "output": f"https sources cached through {url}: {cached}{named}"}
return self._back_to_https(url, f"{target}: {fetched['error']}", named)
def _back_to_https(self, url: str, why: str, named: str) -> FixStepOutcomeDict:
"""*url* did not fetch the stored form: every source goes back to https."""
back = self._run_apt_root_command(revert_sources_command(), "sources back on https")
if not back["ok"]:
return {
"ok": False,
"output": f"{why}; turning the sources back to https failed: {back['output']}",
}
return {
"ok": True,
"output": f"{url} does not fetch https sources itself ({why}); "
f"they stay on https, not cached{named}",
}
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
@@ -609,7 +676,9 @@ class AptProxyMixin(PackageProxyMixin):
def _https_direct(self, url: str) -> FixStepOutcomeDict:
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
written = self._write_package_proxy(url, https_direct=True)
written = self._write_package_proxy(
url, https_direct=True, not_cached=self._report()["not_cached"]
)
if not written["ok"]:
return written
later = self._later_https_lines(self._report(), url)
@@ -635,6 +704,26 @@ class AptProxyMixin(PackageProxyMixin):
]
def _plain_http_source(report: AptProxyReportDict) -> str | None:
"""A suite's http source the proxy takes, not in the cache's https form: its proof."""
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
return next(
(
s
for s in report["http_sources"]
if "/dists/" in s
and not s.startswith(VIA_CACHE)
and (urlsplit(s).hostname or "").lower() not in direct
),
None,
)
def _cached(source: str) -> bool:
"""A source of a suite in the form the cache stores."""
return source.startswith(VIA_CACHE) and "/dists/" in source
def _https_source(
report: AptProxyReportDict, through: Callable[[str | None], bool]
) -> tuple[str, str] | None:
+172
View File
@@ -0,0 +1,172 @@
"""apt's https sources through the package cache, so that the cache stores them.
A source fetched over https goes through apt-cacher-ng's tunnel, if at all: the
TLS runs end to end and nothing of it is stored. The cache fetches
``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>`` itself, with the
certificate verified, and stores it -- where it allows port 443
(``AllowUserPorts=80 443``) (NetOrk/netork#2555). apt fetches that form through
its http proxy, so it works only while the cache is that proxy: without one apt
looks for a host called ``HTTPS``.
**Which hosts** (:func:`cacheable_https_hosts`): the hosts of the https sources,
on port 443, while apt has an http proxy for every host and none of its own for
that one. Not a host with a login in the URL, nor one whose credentials apt keeps
in ``auth.conf``: apt finds those by the host's name, which the rewrite changes,
and paid content does not belong in a cache other hosts share. Only root reads
``auth.conf``, so the rewrite judges that, leaves such a host on https and names
it in the proxy file as ``Package-Proxy::Not-Cached``, which apt ignores and
``apt-config dump`` shows to the report.
**The rewrite** (:func:`rewrite_sources_command`) turns ``https://<host>`` into
the cache form on the lines that name sources: ``deb`` and ``deb-src`` in a
``.list`` file, ``URIs:`` in a deb822 ``.sources`` file. A file without the host
is not touched. :func:`revert_sources_command` turns every source back.
"""
from __future__ import annotations
import re
import shlex
from typing import TYPE_CHECKING
from urllib.parse import urlsplit
if TYPE_CHECKING: # pragma: no cover - the report's type, without an import cycle
from napalm_device_types.apt_proxy import AptProxyReportDict
#: The prefix apt-cacher-ng fetches over https.
VIA_CACHE = "http://HTTPS///"
#: The key in the proxy file that names the hosts left on https for their credentials.
NOT_CACHED_KEY = "Package-Proxy::Not-Cached"
_LABEL = r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?"
_HOST = re.compile(rf"^{_LABEL}(?:\.{_LABEL})*$")
_NO_PROXY = frozenset({"", "direct"})
#: Where apt's sources and credentials are, as shell variables; apt's defaults
#: when apt-config cannot say.
_LOCATE = (
'eval "$({apt_config} shell sl Dir::Etc::sourcelist/f sp Dir::Etc::sourceparts/d '
'nr Dir::Etc::netrc/f np Dir::Etc::netrcparts/d 2>/dev/null)"; '
'sl="${{sl:-/etc/apt/sources.list}}"; sp="${{sp:-/etc/apt/sources.list.d/}}"; '
'nr="${{nr:-/etc/apt/auth.conf}}"; np="${{np:-/etc/apt/auth.conf.d/}}"; '
)
#: The lines that name sources: ``deb``/``deb-src`` and deb822's ``URIs:``.
_SOURCE_LINES = "/^[[:space:]]*(deb|deb-src|[Uu][Rr][Ii][Ss]:)/"
#: The hosts apt has credentials for: each word after ``machine``, without its
#: scheme, port and path, in lower case.
_CREDENTIAL_HOSTS = (
'creds=$(cat "$nr" "$np"* 2>/dev/null | tr -s " \\t\\r" "\\n" '
"| awk 'p {{ print; p = 0; next }} $0 == \"machine\" {{ p = 1 }}' "
"| sed -e 's|^[A-Za-z+]*://||' -e 's|[:/].*||' | tr A-Z a-z); "
)
_REWRITE = (
"( " + _LOCATE + _CREDENTIAL_HOSTS + "for h in {hosts}; do "
'l=$(printf "%s" "$h" | tr A-Z a-z); '
'if printf "%s\\n" "$creds" | grep -qxF -- "$l"; then '
'grep -qF -- "\\"$h\\"" {proxy_file} 2>/dev/null || '
'printf \'{key}:: "%s";\\n\' "$h" >> {proxy_file} || exit 1; continue; fi; '
"e=$(printf '%s' \"$h\" | sed 's/[.]/[.]/g'); "
'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
'[ -f "$f" ] && grep -qF -- "https://$h" "$f" || continue; '
'sed -i -E "{lines} s#([[:space:]:])https://$e([/[:space:]]|\\$)#\\1{via}$h\\2#g" "$f" '
"|| exit 1; done; done )"
)
_REVERT = (
"( " + _LOCATE + 'for f in "$sl" "$sp"*.list "$sp"*.sources; do '
'[ -f "$f" ] && grep -qF -- "{via}" "$f" || continue; '
"sed -i -E '{lines} s#([[:space:]:]){via}#\\1https://#g' \"$f\" || exit 1; done )"
)
def cache_form(url: str) -> str:
"""*url*, an https URL, in the form apt-cacher-ng fetches over https and stores.
:raises ValueError: if *url* is not an https URL.
"""
if not url.startswith("https://"):
raise ValueError(f"not an https URL: {url!r}")
return VIA_CACHE + url[len("https://") :]
def _per_host_direct(report: AptProxyReportDict) -> set[str]:
hosts = {**report["per_host"], **report["https_per_host"]}
return {h.lower() for h, v in hosts.items() if v.lower() in _NO_PROXY}
def _host_of(source: str) -> str | None:
"""The host of an https source the cache can fetch: port 443, no login in the URL."""
try:
parts = urlsplit(source)
port = parts.port
except ValueError:
return None
if parts.username is not None or port not in (None, 443) or not parts.hostname:
return None
return parts.hostname
def cacheable_https_hosts(report: AptProxyReportDict) -> list[str]:
"""The hosts of the https sources the rewrite would send through the cache, sorted.
None while apt has no http proxy for every host: the cache form goes through it.
Not a host apt fetches ``DIRECT``, nor one the rewrite left on https for its
credentials (``not_cached``), nor one on another port or with a login in the URL.
"""
if not report["proxy"]:
return []
skip = _per_host_direct(report) | {h.lower() for h in report.get("not_cached", [])}
hosts = {_host_of(s) for s in report["https_sources"]}
return sorted(h for h in hosts if h and h.lower() not in skip)
def is_host_name(host: str) -> bool:
"""Whether *host* is a DNS name: letters, digits and hyphens between dots."""
return bool(_HOST.match(host))
def _checked(hosts: list[str]) -> str:
for host in hosts:
if not is_host_name(host):
raise ValueError(f"not a host name: {host!r}")
return " ".join(hosts)
def rewrite_sources_command(
hosts: list[str], proxy_file: str, apt_config: str = "apt-config"
) -> str:
"""The root command that sends the https sources of *hosts* through the cache.
A host apt has credentials for stays on https and is named in *proxy_file*.
*apt_config* is how apt-config is called; a test points apt's paths elsewhere.
:raises ValueError: if one of *hosts* is not a host name.
"""
return _REWRITE.format(
apt_config=apt_config,
hosts=_checked(hosts),
proxy_file=shlex.quote(proxy_file),
key=NOT_CACHED_KEY,
lines=_SOURCE_LINES,
via=VIA_CACHE,
)
def revert_sources_command(apt_config: str = "apt-config") -> str:
"""The root command that turns every source in the cache form back to https."""
return _REVERT.format(apt_config=apt_config, lines=_SOURCE_LINES, via=VIA_CACHE)
__all__ = [
"NOT_CACHED_KEY",
"VIA_CACHE",
"cache_form",
"cacheable_https_hosts",
"is_host_name",
"revert_sources_command",
"rewrite_sources_command",
]
+59 -17
View File
@@ -23,6 +23,22 @@ Mixed in by a driver, not by a role base: a Windows host is an
:class:`~napalm_device_types.os.OSDriver` too, and without a shell channel it
has no ``dial-stdio`` to offer. ``hasattr(driver, "open_container_engine")``
stays truthful.
**Which engine a CLI is.** An engine is listed when its CLI answers
``--version`` with its own name: ``Docker version 29.0.1, build …``, ``podman
version 5.2.2``. podman-docker installs a ``docker`` that is podman; listed as
Docker it would be read twice, the second time as the login user's rootless
Podman (NetOrk/netork#818).
**Podman as root.** The Podman this reaches is the host's *rootful* one: the
engine of the system's containers, whose units Quadlet generates from
``/etc/containers/systemd`` and whose API ``podman.socket`` serves at
``/run/podman/podman.sock``. Rootless containers belong to the user who runs
them, in that user's own storage, and no other login sees them. So the
connection to Podman runs every call as root (``privileged``), the way the
driver gains root for any command: root itself, or sudo. ``podman system
dial-stdio`` exists from Podman 4.0 on; it reaches the API socket, which has
to be listening (``systemctl enable --now podman.socket``).
"""
from __future__ import annotations
@@ -34,8 +50,11 @@ from typing import TYPE_CHECKING, Any, cast
from napalm_device_types.channel import ByteStream, CommandResult
from napalm_device_types.models import ContainerEngineDict, FixResultDict
#: Engines this package knows how to reach, and the API each speaks.
ENGINES: dict[str, str] = {"docker": "docker-engine"}
#: Engines this package knows how to reach, and the API each speaks. Podman's
#: is its own service, which serves Docker's API next to libpod's.
ENGINES: dict[str, str] = {"docker": "docker-engine", "podman": "podman-libpod"}
#: Engines reached as root: rootful Podman's socket is root's (see above).
ROOT_ENGINES: frozenset[str] = frozenset({"podman"})
_PROBE_TIMEOUT = 15
_CLI_TIMEOUT = 120
@@ -52,17 +71,24 @@ class ContainerEngineConnection:
arguments; binary, quoting and the channel are the driver's.
"""
def __init__(self, driver: Any, engine: str, binary: str) -> None:
def __init__(self, driver: Any, engine: str, binary: str, *, privileged: bool = False) -> None:
self._driver = driver
self._binary = binary
self.engine = engine
#: Every call runs as root: the engine is root's (rootful Podman).
self.privileged = privileged
def _command(self, args: Sequence[str]) -> str:
return shlex.join([self._binary, *args])
def open_api(self) -> ByteStream:
"""A stream to the engine's API: HTTP/1.1 over ``system dial-stdio``."""
return cast(ByteStream, self._driver.open_stream(self._command(["system", "dial-stdio"])))
return cast(
ByteStream,
self._driver.open_stream(
self._command(["system", "dial-stdio"]), privileged=self.privileged
),
)
def run_cli(
self,
@@ -80,7 +106,10 @@ class ContainerEngineConnection:
return cast(
CommandResult,
self._driver.run_command(
self._command(args), privileged=privileged, timeout=timeout, stdin=stdin
self._command(args),
privileged=privileged or self.privileged,
timeout=timeout,
stdin=stdin,
),
)
@@ -96,7 +125,8 @@ class ContainerEngineConnection:
return cast(
ByteStream,
self._driver.open_stream(
f"{command} 2>&1" if merge_stderr else command, privileged=privileged
f"{command} 2>&1" if merge_stderr else command,
privileged=privileged or self.privileged,
),
)
@@ -134,20 +164,32 @@ class ContainerEngineMixin:
* engine (string) - e.g. "docker"
* api (string) - what its API is, e.g. "docker-engine"
An engine is listed when its CLI exists on the host. Whether the login
user may use it is decided by the first API call, which says
"permission denied" in the stream's stderr.
An engine is listed when its CLI exists on the host and names itself
when asked for its version: podman-docker's ``docker`` says it is podman,
and is left out. Whether the login user may use the engine is decided by
the first API call, which says "permission denied" in the stream's stderr.
"""
found: list[ContainerEngineDict] = []
for engine, api in ENGINES.items():
probe = shlex.join(["command", "-v", self._container_engine_binary(engine)])
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
if result.exit_code == 0 and result.stdout.strip():
found.append({"engine": engine, "api": api})
return found
return [
{"engine": engine, "api": api}
for engine, api in ENGINES.items()
if self._answers_as(engine)
]
def _answers_as(self, engine: str) -> bool:
"""Whether *engine*'s CLI is on the host and says it is *engine*."""
binary = shlex.quote(self._container_engine_binary(engine))
probe = f"command -v {binary} >/dev/null 2>&1 && {binary} --version"
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
words = result.stdout.split()
return result.exit_code == 0 and bool(words) and words[0].lower() == engine
def open_container_engine(self, engine: str) -> ContainerEngineConnection:
"""The connection to *engine*; :class:`ContainerEngineUnavailable` if unknown."""
if engine not in ENGINES:
raise ContainerEngineUnavailable(f"no way to reach container engine {engine!r}")
return ContainerEngineConnection(self, engine, self._container_engine_binary(engine))
return ContainerEngineConnection(
self,
engine,
self._container_engine_binary(engine),
privileged=engine in ROOT_ENGINES,
)
+126 -7
View File
@@ -16,6 +16,7 @@ They replace ``run_device_action`` (docs/STRUCTURE.md):
``fix_apt_proxy`` :meth:`PackageProxyMixin.configure_package_proxy`
``fix_docker_permissions`` ``ContainerEngineMixin.grant_container_engine_access``
``install_lldpd`` and co. :meth:`EnsurePackageMixin.ensure_package`
(new) :meth:`ZabbixAgentMixin.configure_zabbix_agent`
========================== ===================================================
"""
@@ -31,6 +32,7 @@ from napalm_device_types.models import (
PackageDict,
SnmpAgentSettingsDict,
)
from napalm_device_types.zabbix_agent import ZabbixAgentSettingsDict, check_zabbix_agent_settings
#: One step of a fix: its name and the call that does it.
FixStep = tuple[str, Callable[[], FixStepOutcomeDict]]
@@ -58,6 +60,23 @@ def run_fix_steps(steps: Sequence[FixStep]) -> FixResultDict:
}
def enable_and_restart(
manage_service: Callable[[str, str], dict[str, Any]], service: str
) -> FixStepOutcomeDict:
"""Enable *service* at boot, then restart it so it reads its configuration; not
restarted when it cannot be enabled."""
for action in ("enable", "restart"):
result = manage_service(service, action)
if not result.get("success"):
output = str(result.get("output", "")).strip()
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
return {"ok": True, "output": f"{service} enabled and restarted"}
def _joined(*parts: object) -> str:
return "\n".join(str(p).strip() for p in parts if p and str(p).strip())
class SnmpAgentMixin:
"""Run an SNMP agent on the device that answers the caller.
@@ -171,13 +190,7 @@ class LldpAgentMixin:
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
def _start_lldp_agent(self) -> FixStepOutcomeDict:
service = self.LLDP_AGENT_SERVICE
for action in ("enable", "restart"):
result = self.manage_service(service, action)
if not result.get("success"):
output = str(result.get("output", "")).strip()
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
return {"ok": True, "output": f"{service} enabled and restarted"}
return enable_and_restart(self.manage_service, self.LLDP_AGENT_SERVICE)
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
@@ -193,6 +206,110 @@ class LldpAgentMixin:
...
class ZabbixAgentMixin:
"""Run a native Zabbix agent that answers the caller's server.
For a host without a container engine, where the agent cannot run as a
container. The steps: load the package index where the driver can
(``refresh_available_updates``), install the agent from the host's own packages -- and
where those have none, from Zabbix's repository for ``settings["release"]``
(:func:`~napalm_device_types.zabbix_agent.zabbix_repository_command`) --, write
its configuration (:func:`~napalm_device_types.zabbix_agent.zabbix_agent_config`),
let the allowed sources through the host's firewall, enable and restart it.
Installing and starting are generic, over ``ensure_package`` and
``manage_service`` with ``ZABBIX_AGENT_PACKAGE`` and ``ZABBIX_AGENT_SERVICE``.
Whether Zabbix reaches it is the caller's to ask Zabbix.
The settings are checked before anything changes; ``ValueError`` names the one
that does not hold. A driver that cannot add a repository sets
``_add_zabbix_repository`` to ``None``: its hosts then take the distribution's
package or none.
"""
#: The package that brings the agent, as ``ensure_package`` takes it.
ZABBIX_AGENT_PACKAGE: ClassVar[str] = "zabbix-agent"
#: The agent's service, as ``manage_service`` takes it.
ZABBIX_AGENT_SERVICE: ClassVar[str] = "zabbix-agent"
def configure_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixResultDict:
"""Install, configure, open, enable and restart the agent with *settings*.
:raises ValueError: for settings the agent's configuration cannot take,
before anything changes.
"""
check_zabbix_agent_settings(settings)
return run_fix_steps(
[
("install", lambda: self._install_zabbix_agent(settings)),
("configure", lambda: self._write_zabbix_agent_config(settings)),
("firewall", lambda: self._allow_zabbix_agent_sources(settings)),
("start", self._start_zabbix_agent),
]
)
def _install_zabbix_agent(self, settings: ZabbixAgentSettingsDict) -> FixStepOutcomeDict:
index = self._load_package_index()
first = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
if first["success"]:
return {"ok": True, "output": _joined(index, first.get("output"))}
release = settings.get("release")
add = getattr(self, "_add_zabbix_repository", None)
if not release or not callable(add):
return {"ok": False, "output": _joined(index, first.get("output"))}
added = add(release)
if not added["ok"]:
return {"ok": False, "output": _joined(index, added.get("output"))}
second = self.ensure_package(self.ZABBIX_AGENT_PACKAGE)
return {
"ok": bool(second["success"]),
"output": _joined(index, added.get("output"), second.get("output")),
}
def _load_package_index(self) -> str:
"""Load the package index where the driver can (OpenWrt keeps it in RAM, gone
after a restart); what went wrong, or "". The install tries either way."""
refresh = getattr(self, "refresh_available_updates", None)
if not callable(refresh):
return ""
loaded = refresh()
if loaded.get("success"):
return ""
return f"package index not loaded: {str(loaded.get('output', '')).strip()}"
def _start_zabbix_agent(self) -> FixStepOutcomeDict:
return enable_and_restart(self.manage_service, self.ZABBIX_AGENT_SERVICE)
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def ensure_package(self, name: str) -> FixResultDict: ...
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
def refresh_available_updates(self) -> dict[str, Any]: ...
# The steps a driver implements:
def _add_zabbix_repository(self, release: str) -> FixStepOutcomeDict:
"""Add Zabbix's repository for *release* and refresh the package index
(``zabbix_repository_command`` builds the command for most hosts)."""
...
def _write_zabbix_agent_config(
self, settings: ZabbixAgentSettingsDict
) -> FixStepOutcomeDict:
"""Write *settings* into the agent's configuration file, keeping the rest of
it (``zabbix_agent_config``), and whatever else the device needs before the
agent may start."""
...
def _allow_zabbix_agent_sources(
self, settings: ZabbixAgentSettingsDict
) -> FixStepOutcomeDict:
"""Let ``settings["allowed_sources"]`` reach tcp/``listen_port``; ``ok`` with
nothing to do where the host has no firewall of its own, or for []."""
...
class PackageProxyMixin:
"""Point the device's package manager at a caching proxy."""
@@ -215,5 +332,7 @@ __all__ = [
"LldpAgentMixin",
"PackageProxyMixin",
"SnmpAgentMixin",
"ZabbixAgentMixin",
"enable_and_restart",
"run_fix_steps",
]
-12
View File
@@ -1440,15 +1440,3 @@ class ManagementProtocolsDict(TypedDict):
snmp: ProtocolStateDict
kvm: ProtocolStateDict
virtual_media: ProtocolStateDict
class SystemIdentityDict(TypedDict):
"""A host's SMBIOS identity, what ties it to its BMC.
See :mod:`napalm_device_types.system_identity`.
Each value is empty when unknown or a vendor placeholder."""
system_uuid: str # lowercase
product_serial: str
board_serial: str
+66 -15
View File
@@ -1,6 +1,7 @@
"""The SMBIOS identity of a host: system UUID, product serial, board serial.
"""The SMBIOS identity of a host: system UUID, product serial, board serial -- and
its system firmware (BIOS or UEFI): vendor, version, release date.
What ties a server to its BMC. The BMC reports the server's serial number and
The identity is what ties a server to its BMC. The BMC reports the server's serial number and
system UUID; the host reads the same values from its firmware tables. Both sides
read them the same way on every Linux host, so the command, its parse and the
placeholder filter live here once, and a driver only carries the command across.
@@ -13,27 +14,54 @@ Matching on those would join unrelated machines, so they read as empty.
**Root.** ``product_uuid`` and ``product_serial`` are readable by root only; without
it they come back empty, which is "not known", not a mismatch.
**The firmware** comes from the same tables in the same round trip, so a driver that
reads the identity reads it too. It is a fact about the host, not part of its identity:
its placeholders are the vendors' text defaults, never a short value (a version of
``1`` is a version), and sysfs's ``MM/DD/YYYY`` release date is given as ISO 8601.
"""
from __future__ import annotations
import re
from typing import TYPE_CHECKING
from datetime import date
from typing import TYPE_CHECKING, TypedDict
from napalm_device_types.models import SystemIdentityDict
from napalm_device_types.terminal import strip_terminal_codes
class SystemIdentityDict(TypedDict):
"""A host's SMBIOS identity, what ties it to its BMC, and its system firmware.
Each value is empty when unknown or a vendor placeholder."""
system_uuid: str # lowercase
product_serial: str
board_serial: str
bios_vendor: str # the system firmware (BIOS or UEFI): its vendor,
bios_version: str # its version,
bios_date: str # its release date, YYYY-MM-DD
_BEGIN = "IDBEGIN"
_END = "IDEND"
_DMI = "/sys/class/dmi/id"
#: Reads the three files in one round trip, framed so a banner or prompt around the
#: output cannot shift a field.
SYSTEM_IDENTITY_COMMAND = (
f"printf '{_BEGIN}\\n%s\\n%s\\n%s\\n{_END}\\n' "
f'"$(cat {_DMI}/product_uuid 2>/dev/null)" '
f'"$(cat {_DMI}/product_serial 2>/dev/null)" '
f'"$(cat {_DMI}/board_serial 2>/dev/null)"'
#: The files read, in the order the output carries them.
_FILES = (
"product_uuid",
"product_serial",
"board_serial",
"bios_vendor",
"bios_version",
"bios_date",
)
#: Reads the files in one round trip, framed so a banner or prompt around the output
#: cannot shift a field.
_FIELDS = "%s\\n" * len(_FILES)
SYSTEM_IDENTITY_COMMAND = f"printf '{_BEGIN}\\n{_FIELDS}{_END}\\n' " + " ".join(
f'"$(cat {_DMI}/{name} 2>/dev/null)"' for name in _FILES
)
#: Lowercased defaults board vendors leave in place.
@@ -80,6 +108,22 @@ def _clean(value: str) -> str:
return "" if is_placeholder_identity(value) else value
def _clean_text(value: str) -> str:
"""A firmware text, empty for a vendor's text default; a short value stays."""
value = value.strip()
return "" if value.lower() in _PLACEHOLDERS else value
def _iso_date(value: str) -> str:
"""sysfs's ``MM/DD/YYYY`` as ``YYYY-MM-DD``; anything else is no date."""
month, _, rest = value.strip().partition("/")
day, _, year = rest.partition("/")
try:
return date(int(year), int(month), int(day)).isoformat()
except ValueError:
return ""
def parse_system_identity(output: str) -> SystemIdentityDict:
"""The identity from :data:`SYSTEM_IDENTITY_COMMAND`'s output, placeholders empty.
@@ -91,12 +135,16 @@ def parse_system_identity(output: str) -> SystemIdentityDict:
end = lines.index(_END, start)
except ValueError:
raise ValueError("system identity: no intact IDBEGIN/IDEND frame") from None
fields = (lines[start:end] + ["", "", ""])[:3]
uuid, product_serial, board_serial = (_clean(f) for f in fields)
fields = (lines[start:end] + [""] * len(_FILES))[: len(_FILES)]
uuid, product_serial, board_serial = (_clean(f) for f in fields[:3])
bios_vendor, bios_version, bios_date = fields[3:]
return {
"system_uuid": uuid.lower(),
"product_serial": product_serial,
"board_serial": board_serial,
"bios_vendor": _clean_text(bios_vendor),
"bios_version": _clean_text(bios_version),
"bios_date": _iso_date(bios_date),
}
@@ -119,12 +167,15 @@ class SystemIdentityMixin:
def get_system_identity(self) -> SystemIdentityDict:
"""
Returns the host's SMBIOS identity, each value empty when unknown or a vendor
placeholder:
Returns the host's SMBIOS identity and system firmware, each value empty when
unknown or a vendor placeholder:
* system_uuid (str) - lowercase
* product_serial (str)
* board_serial (str)
* bios_vendor (str) - the system firmware's vendor
* bios_version (str)
* bios_date (str) - its release date, ``YYYY-MM-DD``
:raises ValueError: if the output carried no intact reading.
"""
+11
View File
@@ -17,6 +17,15 @@ from typing import TYPE_CHECKING, Any
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict, UpdateProgressDict
class PackageIndexMissing(RuntimeError):
"""The host holds no package index to read pending updates from.
Not a fault: OpenWrt keeps its index in RAM and loses it on every reboot.
:meth:`UpdateMixin.refresh_available_updates` loads it again, so a caller
that sees this can refresh instead of reporting a failed read.
"""
class UpdateMixin:
if TYPE_CHECKING:
@@ -37,6 +46,8 @@ class UpdateMixin:
read -- no package index yet, an API that did not answer -- raises
instead: netOrk keeps "pending since" per package, and an empty
list for "don't know" would reset every one of those clocks.
Without a package index it raises :exc:`PackageIndexMissing`, which
:meth:`refresh_available_updates` cures.
Example::
+210
View File
@@ -0,0 +1,210 @@
"""A native Zabbix agent: its configuration and Zabbix's package repository.
What :class:`~napalm_device_types.fixes.ZabbixAgentMixin` needs that is the same on
every host, without I/O: which settings an agent may be given
(:func:`check_zabbix_agent_settings`), the agent's configuration file with them
(:func:`zabbix_agent_config`), and the command that adds Zabbix's repository where
the host's own packages have no agent (:func:`zabbix_repository_command`). A driver
reads and writes the file and runs the command.
**Which lines.** ``Server`` (who may ask, for passive checks), ``ServerActive``
(where active checks go), ``ListenPort`` and, when the caller names one,
``Hostname`` -- the host's technical name in Zabbix, which an active agent sends.
Every other line stays as it was; a key the file does not set is added at its end.
**The repository.** Zabbix publishes a ``zabbix-release`` package per release line
and distribution that adds its repository and key. Since 7.2 it lives under
``<release>/release/<distribution>/``; up to 7.0 under ``<release>/<distribution>/``,
with the rpm in an ``x86_64`` directory. The release line is the server's: an agent
may be older than its server, not newer.
"""
from __future__ import annotations
import re
import shlex
from typing import NotRequired, TypedDict
class ZabbixAgentSettingsDict(TypedDict):
"""What the caller wants a native Zabbix agent to be (``ZabbixAgentMixin``)."""
server: str # Server: the addresses that may ask (passive checks), comma-separated
server_active: str # ServerActive: where active checks go, comma-separated; "" for none
listen_port: int # ListenPort
# addresses or CIDRs let through to the port; [] changes no firewall
allowed_sources: list[str]
# Hostname: the host's technical name in Zabbix; absent keeps the agent's
hostname: NotRequired[str]
# the server's release line ("7.4"): Zabbix's repository to fall back to where the
# host's own packages have no agent; absent: no fallback
release: NotRequired[str]
#: Addresses, comma-separated, as ``Server`` and ``ServerActive`` take them: names,
#: IPv4, ``[IPv6]`` and ports. Nothing a config line or a shell word breaks out of.
_ADDRESSES = re.compile(r"^[A-Za-z0-9.:\[\]-]+(,[A-Za-z0-9.:\[\]-]+)*$")
#: What Zabbix allows in a technical host name.
_HOSTNAME = re.compile(r"^[A-Za-z0-9._ -]+$")
#: A release line, as Zabbix's repository names it.
_RELEASE = re.compile(r"^(\d+)\.(\d+)$")
_KEYS = ("Server", "ServerActive", "ListenPort", "Hostname")
_LINE = re.compile(r"^(\s*)(" + "|".join(_KEYS) + r")\s*=\s*(.*?)\s*$")
REPOSITORY = "https://repo.zabbix.com/zabbix"
#: The distributions Zabbix publishes packages for, by ``/etc/os-release``'s ``ID``:
#: the name in the repository's path, and whether they take a deb.
_DISTRIBUTIONS: dict[str, tuple[str, bool]] = {
"ubuntu": ("ubuntu", True),
"debian": ("debian", True),
"raspbian": ("raspbian", True),
"rhel": ("rhel", False),
"centos": ("rhel", False),
"ol": ("oracle", False),
"almalinux": ("alma", False),
"rocky": ("rocky", False),
}
def check_zabbix_agent_settings(settings: ZabbixAgentSettingsDict) -> None:
"""Refuse what an agent's configuration or a shell line cannot take.
:raises ValueError: naming the setting.
"""
_check_addresses("Server", str(settings.get("server", "")), required=True)
_check_addresses("ServerActive", str(settings.get("server_active", "")), required=False)
hostname = settings.get("hostname")
if hostname is not None and not _HOSTNAME.match(hostname):
raise ValueError(f"not a Zabbix host name: {hostname!r}")
port = settings.get("listen_port")
if not isinstance(port, int) or isinstance(port, bool) or not 1 <= port <= 65535:
raise ValueError(f"not a port: {port!r}")
release = settings.get("release")
if release is not None and not _RELEASE.match(release):
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
def _check_addresses(key: str, value: str, *, required: bool) -> None:
if (value or required) and not _ADDRESSES.match(value):
raise ValueError(f"not addresses an agent's {key} takes: {value!r}")
def _wanted(settings: ZabbixAgentSettingsDict) -> dict[str, str]:
wanted = {
"Server": settings["server"],
"ServerActive": settings.get("server_active", ""),
"ListenPort": str(settings["listen_port"]),
}
if settings.get("hostname"):
wanted["Hostname"] = str(settings["hostname"])
return wanted
def zabbix_agent_config(text: str, settings: ZabbixAgentSettingsDict) -> tuple[str, list[str]]:
"""*text*, an agent's configuration file, with *settings*; and what changed, one
``"Key: old → new"`` each.
An empty ``server_active`` comments ``ServerActive`` out: the agent then runs
passive checks only.
"""
wanted = _wanted(settings)
lines: list[str] = []
changes: list[str] = []
seen: set[str] = set()
for line in text.splitlines(keepends=True):
match = _LINE.match(line.rstrip("\r\n"))
if not match or match.group(2) not in wanted:
lines.append(line)
continue
seen.add(match.group(2))
lines.append(_set_line(line, match, wanted, changes))
_add_missing(
lines, [key for key in _KEYS if wanted.get(key) and key not in seen], wanted, changes
)
return "".join(lines), changes
def _add_missing(
lines: list[str], keys: list[str], wanted: dict[str, str], changes: list[str]
) -> None:
"""*keys*, which the file does not set, at its end."""
if keys and lines and not lines[-1].endswith("\n"):
lines[-1] += "\n"
for key in keys:
lines.append(f"{key}={wanted[key]}\n")
changes.append(f"{key}: (not set) → {wanted[key]}")
def _set_line(line: str, match: re.Match[str], wanted: dict[str, str], changes: list[str]) -> str:
"""*line*, which sets a key the caller wants, as the caller wants it."""
indent, key, old = match.groups()
end = "\n" if line.endswith("\n") else ""
new = wanted[key]
if not new:
changes.append(f"{key}: {old} → (not set)")
return f"{indent}# {key}={old}{end}"
if old == new:
return line
changes.append(f"{key}: {old} → {new}")
return f"{indent}{key}={new}{end}"
def _os_release(text: str) -> dict[str, str]:
fields = {}
for line in text.splitlines():
key, sep, value = line.partition("=")
if sep:
fields[key.strip()] = value.strip().strip("'\"")
return fields
def _distribution(fields: dict[str, str]) -> tuple[str, bool]:
for candidate in [fields.get("ID", ""), *fields.get("ID_LIKE", "").split()]:
if candidate in _DISTRIBUTIONS:
return _DISTRIBUTIONS[candidate]
raise ValueError(f"Zabbix publishes no repository for {fields.get('ID') or 'this system'}")
def zabbix_repository_command(os_release: str, release: str) -> str:
"""The root command that adds Zabbix's *release* repository to the host whose
``/etc/os-release`` is *os_release*, and refreshes the package index.
:raises ValueError: for a release line that is not ``major.minor``, or a
distribution Zabbix publishes nothing for.
"""
match = _RELEASE.match(release)
if not match:
raise ValueError(f"not a Zabbix release line (major.minor): {release!r}")
fields = _os_release(os_release)
name, deb = _distribution(fields)
version = fields.get("VERSION_ID", "")
new_layout = (int(match.group(1)), int(match.group(2))) >= (7, 2)
base = f"{REPOSITORY}/{release}/" + ("release/" if new_layout else "") + name
if deb:
# Raspberry Pi OS's packages are named after the Debian release they follow.
label = "debian" if name == "raspbian" else name
filename = f"zabbix-release_latest_{release}+{label}{version}_all.deb"
url = f"{base}/pool/main/z/zabbix-release/{filename}"
package = shlex.quote(url)
return (
"t=$(mktemp -d) && "
f'{{ curl -fsSL -o "$t/zabbix-release.deb" {package} '
f'|| wget -qO "$t/zabbix-release.deb" {package}; }} && '
'dpkg -i "$t/zabbix-release.deb" && apt-get update; '
# The download's directory goes either way; the status is the install's.
's=$?; rm -rf "$t"; [ "$s" -eq 0 ]'
)
major = version.split(".")[0]
arch = "noarch" if new_layout else "x86_64"
url = f"{base}/{major}/{arch}/zabbix-release-latest-{release}.el{major}.noarch.rpm"
return f"dnf -y install {shlex.quote(url)}"
__all__ = [
"REPOSITORY",
"ZabbixAgentSettingsDict",
"check_zabbix_agent_settings",
"zabbix_agent_config",
"zabbix_repository_command",
]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.38.0"
version = "4.40.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.12"
+1 -1
View File
@@ -31,7 +31,7 @@
},
"mypy_relaxed": [],
"python_file_lines": {
"napalm_device_types/models.py": 1454
"napalm_device_types/models.py": 1442
},
"untyped_public": []
}
+195 -1
View File
@@ -25,6 +25,7 @@ from napalm_device_types.apt_proxy import (
parse_apt_proxy,
same_proxy_url,
)
from napalm_device_types.apt_sources import NOT_CACHED_KEY, VIA_CACHE
from napalm_device_types.fixes import PackageProxyMixin
@@ -302,6 +303,13 @@ _CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
E: Download Failed"""
#: A cache that does not fetch the cache form itself: ``AllowUserPorts`` is 80 alone
#: (apt-cacher-ng 3.7.5, NetOrk/netork#2555).
_PROHIBITED_PORT = """Err:1 http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease
403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
E: Failed to fetch http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/InRelease 403 Configuration error (confusing proxy mode) or prohibited port (see AllowUserPorts) [IP: 127.0.0.1 39143]
E: Download Failed"""
def _fetch(output, rc, match=None):
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
@@ -422,9 +430,21 @@ class _Host(AptProxyMixin):
"""
def __init__(
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
self,
files=None,
fetch_rc=0,
apt=True,
https_sources=(),
tunnel=True,
direct_rc=0,
cache_https=False,
credentials=(),
):
self.files = dict(files or {})
#: Whether the proxy fetches the cache form itself (AllowUserPorts 443), and
#: the hosts apt keeps credentials for.
self.cache_https = cache_https
self.credentials = set(credentials)
self.fetch_rc = fetch_rc
self.apt = apt
self.https_sources = list(https_sources)
@@ -459,6 +479,12 @@ class _Host(AptProxyMixin):
value, https = self._effective(), self._effective("https")
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
dump += [
line
for path, text in self.files.items()
for line in text.splitlines()
if line.startswith(NOT_CACHED_KEY)
]
return _report(
dump=dump,
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
@@ -466,6 +492,8 @@ class _Host(AptProxyMixin):
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
)
self.fetched.append(command)
if re.search(r"download-file '?http://HTTPS///", command):
return _fetch(_FETCHED, 0) if self.cache_https else _fetch(_PROHIBITED_PORT, 100)
if re.search(r"download-file '?https://", command):
return self._fetch_https(command)
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
@@ -482,6 +510,11 @@ class _Host(AptProxyMixin):
def _run_apt_root_command(self, command, done):
self.root.append(command)
if NOT_CACHED_KEY in command:
return self._rewrite(re.search(r"for h in ([^;]+); do", command).group(1).split(), done)
if "#\\1https://#g" in command:
self.https_sources = [s.replace(VIA_CACHE, "https://") for s in self.https_sources]
return {"ok": True, "output": done}
path = command.rsplit(" ", 1)[-1].strip("'")
for m in re.finditer(r"(\d+)\{/[^}]*\}", command):
n = int(m.group(1))
@@ -492,6 +525,19 @@ class _Host(AptProxyMixin):
self.files.pop(command.split()[-1].strip("'"), None)
return {"ok": True, "output": done}
def _rewrite(self, hosts, done):
for host in hosts:
if host in self.credentials:
proxy = self.files.get(f"{_D}00proxy", "")
self.files[f"{_D}00proxy"] = proxy + f'{NOT_CACHED_KEY}:: "{host}";\n'
continue
prefix = f"https://{host}/"
self.https_sources = [
VIA_CACHE + s[len("https://") :] if s.startswith(prefix) else s
for s in self.https_sources
]
return {"ok": True, "output": done}
class TestCheckHttpsSource:
def test_one_source_the_way_apt_fetches_it(self):
@@ -569,8 +615,10 @@ class TestConfigurePackageProxy:
"conflicts",
"verify",
"fetch",
"cache",
"https",
]
assert "no https source" in _step(result, "cache")["output"]
assert "no https source" in result["steps"][-1]["output"]
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
@@ -766,6 +814,144 @@ class TestConfigurePackageProxy:
assert host.files == {}
class TestCacheHttpsSources:
"""NetOrk/netork#2555: the cache stores an https source only in its own form."""
def test_a_cache_that_fetches_https_gets_them(self):
host = _Host(https_sources=[_DOCKER], cache_https=True)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.https_sources == [f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"]
assert _step(result, "cache")["output"] == (
"https sources cached through http://c:3142: download.docker.com"
)
assert any(VIA_CACHE in c for c in host.fetched)
assert "no https source goes through" in _step(result, "https")["output"]
def test_a_cache_that_does_not_turns_them_back(self):
host = _Host(https_sources=[_DOCKER])
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.https_sources == [_DOCKER]
output = _step(result, "cache")["output"]
assert "http://c:3142 does not fetch https sources itself" in output
assert "prohibited port (see AllowUserPorts)" in output
assert "they stay on https, not cached" in output
assert _step(result, "https")["output"] == (
f"fetched {_DOCKER}InRelease through http://c:3142"
)
def test_a_host_with_credentials_stays_on_https_and_is_named(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(
https_sources=[pve, _DOCKER],
cache_https=True,
credentials={"enterprise.proxmox.com"},
)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert pve in host.https_sources
output = _step(result, "cache")["output"]
assert "cached through http://c:3142: download.docker.com" in output
assert "left on https, apt has credentials for them: enterprise.proxmox.com" in output
assert host.get_package_proxy_report()["not_cached"] == ["enterprise.proxmox.com"]
def test_only_hosts_with_credentials(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(https_sources=[pve], cache_https=True, credentials={"enterprise.proxmox.com"})
result = host.configure_package_proxy("http://c:3142")
assert _step(result, "cache")["output"] == (
"no https source to cache; "
"left on https, apt has credentials for them: enterprise.proxmox.com"
)
def test_the_https_direct_fallback_keeps_the_hosts_left_on_https(self):
pve = "https://enterprise.proxmox.com/debian/pve/dists/trixie/"
host = _Host(https_sources=[pve], tunnel=False, credentials={"enterprise.proxmox.com"})
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.files[f"{_D}00proxy"] == (
'Acquire::http::Proxy "http://c:3142";\n'
'Acquire::https::Proxy "DIRECT";\n'
f'{NOT_CACHED_KEY}:: "enterprise.proxmox.com";\n'
)
def test_a_rewrite_that_fails_stops_the_fix(self):
host = _Host(https_sources=[_DOCKER], cache_https=True)
real = host._run_apt_root_command
def denied(command, done):
if NOT_CACHED_KEY in command:
return {"ok": False, "output": "sed: denied"}
return real(command, done)
host._run_apt_root_command = denied
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
assert result["steps"][-1] == {"step": "cache", "ok": False, "output": "sed: denied"}
def test_a_turn_back_that_fails_says_so(self):
host = _Host(https_sources=[_DOCKER])
real = host._run_apt_root_command
def denied(command, done):
if "#\\1https://#g" in command:
return {"ok": False, "output": "sed: denied"}
return real(command, done)
host._run_apt_root_command = denied
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
output = result["steps"][-1]["output"]
assert "turning the sources back to https failed: sed: denied" in output
def test_the_proof_of_the_proxy_is_a_plain_http_source(self):
"""A source already in the cache form says nothing about plain http."""
host = _Host(https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"])
host.configure_package_proxy("http://c:3142")
fetch = next(c for c in host.fetched if "InRelease" in c)
assert "deb.debian.org" in fetch
def test_removing_the_proxy_turns_the_sources_back_first(self):
host = _Host(
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
https_sources=[f"{VIA_CACHE}download.docker.com/linux/ubuntu/dists/noble/"],
)
result = host.configure_package_proxy("")
assert result["success"] is True
assert [s["step"] for s in result["steps"]] == ["sources", "configure"]
assert host.https_sources == [_DOCKER]
assert host.files == {}
def test_a_turn_back_that_fails_keeps_the_proxy(self):
"""Without the cache apt looks for a host called HTTPS: the proxy stays."""
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
host._run_apt_root_command = lambda command, done: {"ok": False, "output": "denied"}
result = host.configure_package_proxy("")
assert result["success"] is False
assert f"{_D}00proxy" in host.files
class TestWhatTheParserSkips:
def test_a_line_that_is_no_grep_output(self):
assert (
@@ -824,3 +1010,11 @@ def test_the_comment_out_command_for_https():
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
)
def test_a_line_left_on_https_that_is_no_host_name_is_not_written_back():
host = _Host()
host._write_package_proxy("http://c:3142", not_cached=["ok.example", 'x"; Foo "1'])
assert host.files[f"{_D}00proxy"] == (
f'Acquire::http::Proxy "http://c:3142";\n{NOT_CACHED_KEY}:: "ok.example";\n'
)
+318
View File
@@ -0,0 +1,318 @@
"""https sources through the cache: which can be, and the root commands that turn them.
apt-cacher-ng fetches ``http://HTTPS///<host>/<path>`` from ``https://<host>/<path>``
and stores it, where it allows port 443 (``AllowUserPorts=80 443``); a source
fetched over https itself goes through the cache's tunnel and is never stored
(NetOrk/netork#2555). The commands run here for real, on files in a temporary
directory that ``apt-config -o`` points apt's own settings at.
"""
from __future__ import annotations
import shutil
import subprocess
import pytest
from napalm_device_types.apt_proxy import parse_apt_proxy
from napalm_device_types.apt_sources import (
VIA_CACHE,
cache_form,
cacheable_https_hosts,
is_host_name,
revert_sources_command,
rewrite_sources_command,
)
def _report(**over):
report = {
"proxy": "http://c:3142",
"per_host": {},
"auto_detect": None,
"settings": [],
"https_proxy": "http://c:3142",
"https_per_host": {},
"https_settings": [],
"http_sources": ["http://deb.debian.org/debian/dists/trixie/"],
"https_sources": [],
"not_cached": [],
}
report.update(over)
return report
class TestCacheForm:
def test_an_https_url(self):
assert cache_form("https://download.docker.com/linux/ubuntu/dists/noble/") == (
"http://HTTPS///download.docker.com/linux/ubuntu/dists/noble/"
)
def test_the_prefix(self):
assert VIA_CACHE == "http://HTTPS///"
def test_not_an_https_url(self):
with pytest.raises(ValueError):
cache_form("http://deb.debian.org/debian/")
class TestCacheableHttpsHosts:
def test_each_host_once_sorted(self):
report = _report(
https_sources=[
"https://packages.wazuh.com/4.x/apt/dists/stable/",
"https://download.docker.com/linux/ubuntu/dists/noble/",
"https://download.docker.com/linux/ubuntu/dists/noble-test/",
]
)
assert cacheable_https_hosts(report) == ["download.docker.com", "packages.wazuh.com"]
def test_none_without_an_http_proxy(self):
# The cache form goes through the http proxy: without one it goes nowhere.
report = _report(proxy=None, https_sources=["https://h.example/dists/x/"])
assert cacheable_https_hosts(report) == []
def test_a_host_apt_fetches_direct_stays(self):
report = _report(
per_host={"a.example": "DIRECT"},
https_per_host={"B.example": "direct"},
https_sources=[
"https://a.example/dists/x/",
"https://b.example/dists/x/",
"https://c.example/dists/x/",
],
)
assert cacheable_https_hosts(report) == ["c.example"]
def test_a_host_with_credentials_stays(self):
report = _report(
https_sources=["https://enterprise.proxmox.com/debian/pve/dists/trixie/"],
not_cached=["enterprise.proxmox.com"],
)
assert cacheable_https_hosts(report) == []
def test_another_port_or_a_login_in_the_url_stays(self):
# The cache allows port 443 alone, and a login in the URL is a credential too.
report = _report(
https_sources=[
"https://h.example:8443/dists/x/",
"https://u:p@i.example/dists/x/",
"https://j.example:443/dists/x/",
]
)
assert cacheable_https_hosts(report) == ["j.example"]
def test_a_report_from_before_not_cached(self):
report = _report(https_sources=["https://h.example/dists/x/"])
del report["not_cached"]
assert cacheable_https_hosts(report) == ["h.example"]
class TestNotCachedInTheReport:
def test_the_dump_names_them(self):
output = "\n".join(
[
"APROXY_BEGIN",
"[dump]",
'Acquire::http::Proxy "http://c:3142";',
'Package-Proxy::Not-Cached "";',
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";',
'Package-Proxy::Not-Cached:: "esm.ubuntu.com";',
"[sources]",
"APROXY_END",
]
)
assert parse_apt_proxy(output)["not_cached"] == [
"enterprise.proxmox.com",
"esm.ubuntu.com",
]
def test_none(self):
assert parse_apt_proxy("APROXY_BEGIN\n[dump]\nAPROXY_END")["not_cached"] == []
def test_the_hosts_are_checked():
with pytest.raises(ValueError):
rewrite_sources_command(["h.example; rm -rf /"], "/etc/apt/apt.conf.d/00proxy")
_APT_CONFIG = shutil.which("apt-config")
needs_apt = pytest.mark.skipif(_APT_CONFIG is None, reason="apt-config is not on this host")
@pytest.fixture
def etc(tmp_path):
"""apt's sources, its credentials and the proxy file, under *tmp_path*."""
(tmp_path / "sources.list.d").mkdir()
(tmp_path / "auth.conf.d").mkdir()
(tmp_path / "00proxy").write_text('Acquire::http::Proxy "http://c:3142";\n')
return tmp_path
def _apt_config(etc):
return (
f"apt-config -o Dir::Etc::sourcelist={etc}/sources.list "
f"-o Dir::Etc::sourceparts={etc}/sources.list.d/ "
f"-o Dir::Etc::netrc={etc}/auth.conf -o Dir::Etc::netrcparts={etc}/auth.conf.d/"
)
def _run(command):
done = subprocess.run(["sh", "-c", command], capture_output=True, text=True, check=False)
assert done.returncode == 0, done.stderr
return done.stdout
_DOCKER_LIST = (
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
"https://download.docker.com/linux/ubuntu noble stable\n"
)
_DOCKER_SOURCES = (
"Types: deb\n"
"URIs: https://download.docker.com/linux/ubuntu\n"
"Suites: noble\n"
"Components: stable\n"
"Signed-By: /etc/apt/keyrings/docker.asc\n"
)
@needs_apt
class TestRewrite:
def test_a_one_line_source(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list.d" / "docker.list").read_text() == (
"deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] "
"http://HTTPS///download.docker.com/linux/ubuntu noble stable\n"
)
def test_a_deb822_source(self, etc):
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list.d" / "docker.sources").read_text() == (
_DOCKER_SOURCES.replace("https://", "http://HTTPS///")
)
def test_sources_list_itself_and_only_the_hosts_named(self, etc):
(etc / "sources.list").write_text(
"deb http://deb.debian.org/debian trixie main\n"
"deb https://a.example/debian trixie main\n"
"deb-src https://a.example/debian trixie main\n"
"deb https://b.example/debian trixie main\n"
"# deb https://a.example/old trixie main\n"
)
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list").read_text() == (
"deb http://deb.debian.org/debian trixie main\n"
"deb http://HTTPS///a.example/debian trixie main\n"
"deb-src http://HTTPS///a.example/debian trixie main\n"
"deb https://b.example/debian trixie main\n"
"# deb https://a.example/old trixie main\n"
)
def test_a_longer_name_or_another_port_stays(self, etc):
text = (
"deb https://a.example.org/debian trixie main\n"
"deb https://a.example:8443/debian trixie main\n"
)
(etc / "sources.list").write_text(text)
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert (etc / "sources.list").read_text() == text
def test_a_host_with_credentials_stays_and_is_named(self, etc):
(etc / "sources.list.d" / "pve.sources").write_text(
"Types: deb\nURIs: https://enterprise.proxmox.com/debian/pve\nSuites: trixie\n"
)
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
(etc / "auth.conf.d" / "pve.conf").write_text(
"machine https://Enterprise.Proxmox.com/debian/pve\nlogin key\npassword secret\n"
)
_run(
rewrite_sources_command(
["enterprise.proxmox.com", "download.docker.com"],
f"{etc}/00proxy",
_apt_config(etc),
)
)
assert "https://enterprise.proxmox.com" in (etc / "sources.list.d/pve.sources").read_text()
assert "HTTPS///download.docker.com" in (etc / "sources.list.d/docker.list").read_text()
assert (etc / "00proxy").read_text() == (
'Acquire::http::Proxy "http://c:3142";\n'
'Package-Proxy::Not-Cached:: "enterprise.proxmox.com";\n'
)
assert "secret" not in (etc / "00proxy").read_text()
def test_credentials_in_auth_conf_on_one_line(self, etc):
(etc / "sources.list").write_text("deb https://esm.ubuntu.com/apps/ubuntu noble main\n")
(etc / "auth.conf").write_text("machine esm.ubuntu.com/apps/ubuntu/ login a password b\n")
_run(rewrite_sources_command(["esm.ubuntu.com"], f"{etc}/00proxy", _apt_config(etc)))
assert "https://esm.ubuntu.com" in (etc / "sources.list").read_text()
assert "esm.ubuntu.com" in (etc / "00proxy").read_text()
def test_a_file_without_the_host_is_left_untouched(self, etc):
other = etc / "sources.list.d" / "other.list"
other.write_text("deb http://deb.debian.org/debian trixie main\n")
before = other.stat().st_ino # sed -i writes a new file; grep first leaves it be
_run(rewrite_sources_command(["a.example"], f"{etc}/00proxy", _apt_config(etc)))
assert other.stat().st_ino == before
def test_twice_is_once(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
command = rewrite_sources_command(
["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)
)
_run(command)
_run(command)
assert (etc / "sources.list.d" / "docker.list").read_text().count("HTTPS///") == 1
@needs_apt
class TestRevert:
def test_every_source_goes_back_to_https(self, etc):
(etc / "sources.list.d" / "docker.list").write_text(_DOCKER_LIST)
(etc / "sources.list.d" / "docker.sources").write_text(_DOCKER_SOURCES)
_run(rewrite_sources_command(["download.docker.com"], f"{etc}/00proxy", _apt_config(etc)))
_run(revert_sources_command(_apt_config(etc)))
assert (etc / "sources.list.d" / "docker.list").read_text() == _DOCKER_LIST
assert (etc / "sources.list.d" / "docker.sources").read_text() == _DOCKER_SOURCES
def test_nothing_to_revert(self, etc):
(etc / "sources.list").write_text("deb http://deb.debian.org/debian trixie main\n")
_run(revert_sources_command(_apt_config(etc)))
assert (
etc / "sources.list"
).read_text() == "deb http://deb.debian.org/debian trixie main\n"
def test_the_commands_are_valid_posix_sh():
for command in (
rewrite_sources_command(["a.example"], "/etc/apt/apt.conf.d/00proxy"),
revert_sources_command(),
):
assert subprocess.run(["sh", "-n", "-c", command], check=False).returncode == 0
def test_a_host_name():
assert is_host_name("download.docker.com")
assert not is_host_name('h"; Acquire::x "y')
assert not is_host_name("-h.example")
+73 -9
View File
@@ -19,30 +19,40 @@ from napalm_device_types.container_engine import (
ContainerEngineUnavailable,
)
#: What each engine's CLI says to ``--version``.
VERSIONS = {
"docker": "Docker version 29.0.1, build 1a2b3c4",
"podman": "podman version 5.2.2",
}
class FakeDriver(ContainerEngineMixin):
"""A driver with a channel: records what it is asked to run."""
"""A driver with a channel: records what it is asked to run.
def __init__(self, present=("docker",), binary=None):
*present* are the engines whose CLI is installed; *says* overrides what an
engine's CLI answers to ``--version`` (podman-docker's ``docker`` is podman).
"""
def __init__(self, present=("docker",), binary=None, says=None):
self.present = set(present)
self.binary = binary
self.says = dict(says or {})
self.commands = []
self.streams = []
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
self.commands.append((command, privileged, timeout, stdin))
found = any(command == f"command -v {b}" for b in self._binaries())
return CommandResult(
stdout="/usr/bin/x\n" if found else "", stderr="", exit_code=0 if found else 1
)
for engine in self.present:
b = self._container_engine_binary(engine)
if command == f"command -v {b} >/dev/null 2>&1 && {b} --version":
answer = self.says.get(engine, VERSIONS[engine])
return CommandResult(stdout=answer + "\n", stderr="", exit_code=0)
return CommandResult(stdout="", stderr="", exit_code=1)
def open_stream(self, command, *, privileged=False):
self.streams.append((command, privileged))
return object()
def _binaries(self):
return {self._container_engine_binary(e) for e in self.present}
def _container_engine_binary(self, engine):
return self.binary or super()._container_engine_binary(engine)
@@ -123,6 +133,60 @@ def test_an_unknown_engine_is_refused():
FakeDriver().open_container_engine("rkt")
class TestPodman:
"""Podman, reached the way Docker is (NetOrk/netork#818).
Its API is Podman's own service, which also speaks Docker's: netOrk reads the
host's *rootful* Podman, the engine of the system's containers (Quadlet units
in ``/etc/containers/systemd``, ``podman.socket``). A login user's rootless
containers are that user's own and invisible to anybody else, so the
connection to Podman runs as root, the way the driver gains root for any command.
"""
def test_a_host_with_podman_lists_it_with_its_api(self):
driver = FakeDriver(present=("podman",))
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
def test_a_host_with_both_lists_both(self):
driver = FakeDriver(present=("docker", "podman"))
assert driver.container_engines() == [
{"engine": "docker", "api": "docker-engine"},
{"engine": "podman", "api": "podman-libpod"},
]
def test_podman_docker_is_not_docker(self):
"""podman-docker installs a ``docker`` that runs podman: listing it as
Docker would read the login user's rootless Podman a second time."""
driver = FakeDriver(present=("docker", "podman"), says={"docker": VERSIONS["podman"]})
assert driver.container_engines() == [{"engine": "podman", "api": "podman-libpod"}]
def test_the_api_stream_runs_as_root(self):
driver = FakeDriver(present=("podman",))
driver.open_container_engine("podman").open_api()
assert driver.streams == [("podman system dial-stdio", True)]
def test_the_cli_runs_as_root_too(self):
driver = FakeDriver(present=("podman",))
conn = driver.open_container_engine("podman")
conn.run_cli(["pull", "nginx"])
conn.stream_cli(["pull", "nginx"])
assert driver.commands[-1][:2] == ("podman pull nginx", True)
assert driver.streams == [("podman pull nginx", True)]
def test_the_connection_says_it_runs_as_root(self):
driver = FakeDriver(present=("docker", "podman"))
assert driver.open_container_engine("podman").privileged is True
assert driver.open_container_engine("docker").privileged is False
def test_the_connection_names_its_engine():
conn = FakeDriver().open_container_engine("docker")
+1
View File
@@ -259,6 +259,7 @@ class TestLldpAgent:
"ensure_package",
"configure_package_proxy",
"grant_container_engine_access",
"configure_zabbix_agent",
],
)
def test_no_role_base_offers_a_fix(base, method):
+71 -4
View File
@@ -14,8 +14,19 @@ from napalm_device_types import (
UUID = "31303043-534D-C40C-7A72-6F1300000000"
def _out(uuid: str, product_serial: str, board_serial: str) -> str:
return f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\nIDEND\n"
def _out(
uuid: str,
product_serial: str,
board_serial: str,
bios: tuple[str, str, str] = ("", "", ""),
) -> str:
vendor, version, date = bios
return (
f"IDBEGIN\n{uuid}\n{product_serial}\n{board_serial}\n{vendor}\n{version}\n{date}\nIDEND\n"
)
_NO_FIRMWARE = {"bios_vendor": "", "bios_version": "", "bios_date": ""}
# ── parsing ───────────────────────────────────────────────────────────────────
@@ -26,6 +37,7 @@ def test_reads_uuid_and_both_serials():
"system_uuid": "31303043-534d-c40c-7a72-6f1300000000",
"product_serial": "S123",
"board_serial": "ZM155S026022",
**_NO_FIRMWARE,
}
@@ -81,9 +93,57 @@ def test_unreadable_files_read_as_empty():
"system_uuid": "",
"product_serial": "",
"board_serial": "BS1",
**_NO_FIRMWARE,
}
# ── the system firmware ───────────────────────────────────────────────────────
def test_reads_the_system_firmware():
# pve-dual, a Supermicro X10DRL-i
out = _out(UUID, "", "ZM155S026022", ("American Megatrends Inc.", "3.4", "11/22/2021"))
identity = parse_system_identity(out)
assert identity["bios_vendor"] == "American Megatrends Inc."
assert identity["bios_version"] == "3.4"
assert identity["bios_date"] == "2021-11-22"
def test_a_virtual_machines_firmware_is_read_like_any_other():
out = _out(UUID, "", "", ("EDK II", "4.2023.08-4", "02/15/2024"))
assert parse_system_identity(out)["bios_version"] == "4.2023.08-4"
@pytest.mark.parametrize("version", ["1", "0", "1.0", "A"])
def test_a_short_firmware_version_is_no_placeholder(version):
# Serials of one repeated character are placeholders, versions are not.
out = _out(UUID, "", "", ("Vendor", version, ""))
assert parse_system_identity(out)["bios_version"] == version
@pytest.mark.parametrize("value", ["", "To be filled by O.E.M.", "Default string", "Not Specified"])
def test_a_placeholder_firmware_version_reads_as_empty(value):
out = _out(UUID, "", "", (value, value, ""))
identity = parse_system_identity(out)
assert identity["bios_vendor"] == ""
assert identity["bios_version"] == ""
@pytest.mark.parametrize("date", ["", "not a date", "13/45/2021", "2021-11-22"])
def test_a_firmware_date_that_is_no_smbios_date_reads_as_empty(date):
# sysfs writes the SMBIOS date as MM/DD/YYYY; anything else is no reading of it.
out = _out(UUID, "", "", ("Vendor", "1.2", date))
assert parse_system_identity(out)["bios_date"] == ""
def test_output_of_the_identity_alone_has_no_firmware():
# What a host answered before the firmware was read with it: three fields.
old = f"IDBEGIN\n{UUID}\nS1\nB1\nIDEND\n"
identity = parse_system_identity(old)
assert identity["board_serial"] == "B1"
assert {k: identity[k] for k in _NO_FIRMWARE} == _NO_FIRMWARE
def test_output_without_its_frame_is_not_a_reading():
with pytest.raises(ValueError):
parse_system_identity("sudo: a password is required\n")
@@ -115,6 +175,13 @@ def test_the_driver_only_carries_the_command_across():
assert privileged is True
def test_the_command_reads_the_three_dmi_files():
for name in ("product_uuid", "product_serial", "board_serial"):
def test_the_command_reads_the_dmi_files():
for name in (
"product_uuid",
"product_serial",
"board_serial",
"bios_vendor",
"bios_version",
"bios_date",
):
assert f"/sys/class/dmi/id/{name}" in SYSTEM_IDENTITY_COMMAND
+20
View File
@@ -0,0 +1,20 @@
"""The contract of :class:`~napalm_device_types.updates.UpdateMixin`."""
from __future__ import annotations
import napalm_device_types
from napalm_device_types.updates import PackageIndexMissing
def test_a_missing_index_is_its_own_failure():
"""A reader can tell "no index to read from" from any other failure, and a
caller that only knows failures still catches it as one."""
exc = PackageIndexMissing("The package index is not loaded (run apk update first)")
assert isinstance(exc, RuntimeError)
assert str(exc) == "The package index is not loaded (run apk update first)"
def test_it_is_exported_from_the_package():
assert napalm_device_types.PackageIndexMissing is PackageIndexMissing
assert "PackageIndexMissing" in napalm_device_types.__all__
+360
View File
@@ -0,0 +1,360 @@
"""A native Zabbix agent: its configuration, its repository and the fix template
(``ZabbixAgentMixin``, NetOrk/netork#2571).
Where a host has no container engine the application installs the agent from the
host's own packages, and from Zabbix's repository only where those have none.
"""
from __future__ import annotations
import pytest
from napalm_device_types import (
ZabbixAgentMixin,
check_zabbix_agent_settings,
zabbix_agent_config,
zabbix_repository_command,
)
SETTINGS = {
"server": "zabbix.example.com",
"server_active": "zabbix.example.com",
"listen_port": 10050,
"allowed_sources": ["10.40.100.254"],
"hostname": "ap-1og",
"release": "7.4",
}
SAMPLE = """# This is a configuration file for Zabbix agent daemon (Unix)
LogFile=/tmp/zabbix_agentd.log
### Option: Server
# Server=
Server=127.0.0.1
ServerActive=127.0.0.1
Hostname=Zabbix server
# Include=/etc/zabbix_agentd.conf.d/*.conf
"""
class TestConfig:
def test_the_lines_that_say_who_may_ask_get_the_caller_s_values(self):
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
assert "\nServer=zabbix.example.com\n" in text
assert "\nServerActive=zabbix.example.com\n" in text
assert "\nHostname=ap-1og\n" in text
assert "Server: 127.0.0.1 → zabbix.example.com" in changes
assert "Hostname: Zabbix server → ap-1og" in changes
def test_comments_and_other_lines_stay_as_they_were(self):
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
assert text.startswith("# This is a configuration file")
assert "LogFile=/tmp/zabbix_agentd.log\n" in text
assert "# Server=\n" in text
assert text.endswith("# Include=/etc/zabbix_agentd.conf.d/*.conf\n" + "ListenPort=10050\n")
def test_a_key_the_file_does_not_set_is_added_at_the_end(self):
text, changes = zabbix_agent_config(SAMPLE, SETTINGS)
assert text.endswith("ListenPort=10050\n")
assert "ListenPort: (not set) → 10050" in changes
def test_a_file_that_already_says_it_changes_nothing(self):
text, _ = zabbix_agent_config(SAMPLE, SETTINGS)
again, changes = zabbix_agent_config(text, SETTINGS)
assert again == text
assert changes == []
def test_without_a_hostname_the_agent_keeps_its_own(self):
settings = {k: v for k, v in SETTINGS.items() if k != "hostname"}
text, changes = zabbix_agent_config(SAMPLE, settings)
assert "\nHostname=Zabbix server\n" in text
assert not any(c.startswith("Hostname") for c in changes)
def test_no_active_server_comments_the_line_out(self):
text, changes = zabbix_agent_config(SAMPLE, {**SETTINGS, "server_active": ""})
assert "\n# ServerActive=127.0.0.1\n" in text
assert "\nServerActive=" not in text
assert "ServerActive: 127.0.0.1 → (not set)" in changes
def test_a_file_without_a_final_newline_gets_one_before_what_is_added(self):
text, _ = zabbix_agent_config("Server=127.0.0.1", SETTINGS)
assert text.startswith("Server=zabbix.example.com\n")
assert text.endswith("\n")
def test_every_line_of_a_key_set_twice_says_the_same(self):
text, _ = zabbix_agent_config("Server=a\nServer=b\n", SETTINGS)
assert text.count("Server=zabbix.example.com\n") == 2
class TestSettings:
@pytest.mark.parametrize(
"server",
["", "zabbix.example.com\nUserParameter=x,id", "a b", "a;rm -rf /", "'a'"],
)
def test_a_server_that_is_not_addresses_is_refused(self, server):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "server": server})
def test_several_servers_and_ports_pass(self):
check_zabbix_agent_settings(
{**SETTINGS, "server": "10.0.0.1,zabbix.example.com", "server_active": "[::1]:10051"}
)
def test_a_hostname_with_a_newline_is_refused(self):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "hostname": "ap\nServer=evil"})
@pytest.mark.parametrize("port", [0, 65536, "10050"])
def test_a_listen_port_out_of_range_is_refused(self, port):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "listen_port": port})
@pytest.mark.parametrize("release", ["7", "7.4.1", "latest", "7.4; id"])
def test_a_release_that_is_not_major_dot_minor_is_refused(self, release):
with pytest.raises(ValueError):
check_zabbix_agent_settings({**SETTINGS, "release": release})
def _os_release(**fields: str) -> str:
return "".join(f'{key}="{value}"\n' for key, value in fields.items())
class TestRepository:
BASE = "https://repo.zabbix.com/zabbix"
@pytest.mark.parametrize(
("os_release", "url"),
[
(
_os_release(ID="ubuntu", VERSION_ID="24.04", ID_LIKE="debian"),
"7.4/release/ubuntu/pool/main/z/zabbix-release/"
"zabbix-release_latest_7.4+ubuntu24.04_all.deb",
),
(
_os_release(ID="debian", VERSION_ID="12"),
"7.4/release/debian/pool/main/z/zabbix-release/"
"zabbix-release_latest_7.4+debian12_all.deb",
),
(
_os_release(ID="ol", VERSION_ID="9.4", ID_LIKE="fedora"),
"7.4/release/oracle/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
(
_os_release(ID="rhel", VERSION_ID="10.0", ID_LIKE="fedora"),
"7.4/release/rhel/10/noarch/zabbix-release-latest-7.4.el10.noarch.rpm",
),
(
_os_release(ID="almalinux", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
"7.4/release/alma/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
(
_os_release(ID="rocky", VERSION_ID="9.5", ID_LIKE="rhel centos fedora"),
"7.4/release/rocky/9/noarch/zabbix-release-latest-7.4.el9.noarch.rpm",
),
],
)
def test_the_release_package_of_the_host_s_distribution(self, os_release, url):
command = zabbix_repository_command(os_release, "7.4")
assert f"{self.BASE}/{url}" in command
def test_a_release_before_7_2_lives_in_the_old_layout(self):
deb = zabbix_repository_command(_os_release(ID="ubuntu", VERSION_ID="24.04"), "7.0")
rpm = zabbix_repository_command(_os_release(ID="ol", VERSION_ID="9.4"), "7.0")
assert f"{self.BASE}/7.0/ubuntu/pool/main/z/zabbix-release/" in deb
assert f"{self.BASE}/7.0/oracle/9/x86_64/zabbix-release-latest-7.0.el9.noarch.rpm" in rpm
def test_a_deb_is_installed_with_dpkg_and_the_index_refreshed(self):
command = zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4")
assert "dpkg -i" in command
assert "apt-get update" in command
def test_an_rpm_is_installed_with_the_package_manager(self):
command = zabbix_repository_command(_os_release(ID="rocky", VERSION_ID="9.5"), "7.4")
assert "dnf -y install" in command
def test_a_derivative_falls_back_to_what_it_is_like(self):
command = zabbix_repository_command(
_os_release(ID="centos", VERSION_ID="9", ID_LIKE="rhel fedora"), "7.4"
)
assert "/7.4/release/rhel/9/noarch/" in command
def test_a_distribution_zabbix_has_no_repository_for_is_refused(self):
with pytest.raises(ValueError, match="openwrt"):
zabbix_repository_command(_os_release(ID="openwrt", VERSION_ID="25.12.2"), "7.4")
def test_a_release_that_is_not_major_dot_minor_is_refused(self):
with pytest.raises(ValueError):
zabbix_repository_command(_os_release(ID="debian", VERSION_ID="12"), "7.4;id")
class _Agent(ZabbixAgentMixin):
"""The generic steps over ensure_package and manage_service; the driver's others."""
def __init__(self, refuse: frozenset[str] = frozenset(), repository: bool = True):
self.calls: list = []
self.refuse = refuse
self.installs = 0
if not repository:
self._add_zabbix_repository = None # type: ignore[assignment]
def refresh_available_updates(self):
self.calls.append(("refresh",))
ok = "refresh" not in self.refuse
return {"success": ok, "output": "" if ok else "Temporary failure resolving"}
def ensure_package(self, name):
self.calls.append(("ensure_package", name))
self.installs += 1
# The distribution has no package; after the repository is added it has.
ok = "distribution" not in self.refuse or self.installs > 1
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
def _add_zabbix_repository(self, release):
self.calls.append(("repository", release))
ok = "repository" not in self.refuse
return {"ok": ok, "output": f"Zabbix {release} repository added" if ok else "404"}
def _write_zabbix_agent_config(self, settings):
self.calls.append(("configure", settings["server"]))
return {"ok": "configure" not in self.refuse, "output": "written"}
def _allow_zabbix_agent_sources(self, settings):
self.calls.append(("firewall", settings["allowed_sources"]))
return {"ok": "firewall" not in self.refuse, "output": "open"}
def manage_service(self, name, action):
self.calls.append((action, name))
ok = action not in self.refuse
return {"success": ok, "output": "" if ok else f"cannot {action}"}
class TestZabbixAgentMixin:
def test_install_configure_open_enable_and_restart(self):
agent = _Agent()
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls == [
("refresh",),
("ensure_package", "zabbix-agent"),
("configure", "zabbix.example.com"),
("firewall", ["10.40.100.254"]),
("enable", "zabbix-agent"),
("restart", "zabbix-agent"),
]
assert [s["step"] for s in result["steps"]] == ["install", "configure", "firewall", "start"]
def test_without_a_package_in_the_distribution_zabbix_s_repository_is_added(self):
agent = _Agent(refuse=frozenset({"distribution"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls[:4] == [
("refresh",),
("ensure_package", "zabbix-agent"),
("repository", "7.4"),
("ensure_package", "zabbix-agent"),
]
assert "Zabbix 7.4 repository added" in result["output"]
def test_without_a_release_there_is_no_fallback(self):
settings = {k: v for k, v in SETTINGS.items() if k != "release"}
agent = _Agent(refuse=frozenset({"distribution"}))
result = agent.configure_zabbix_agent(settings)
assert result["success"] is False
assert ("repository", "7.4") not in agent.calls
assert "no candidate" in result["output"]
def test_a_driver_that_cannot_add_the_repository_fails_with_the_distribution_s_answer(self):
agent = _Agent(refuse=frozenset({"distribution"}), repository=False)
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert "no candidate" in result["output"]
def test_a_repository_that_cannot_be_added_configures_nothing(self):
agent = _Agent(refuse=frozenset({"distribution", "repository"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert [c[0] for c in agent.calls] == ["refresh", "ensure_package", "repository"]
assert "404" in result["output"]
def test_the_package_index_is_loaded_first(self):
agent = _Agent()
agent.configure_zabbix_agent(SETTINGS)
assert agent.calls[0] == ("refresh",)
def test_an_index_that_will_not_load_still_lets_the_install_try(self):
agent = _Agent(refuse=frozenset({"refresh"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert ("ensure_package", "zabbix-agent") in agent.calls
assert "Temporary failure resolving" in result["steps"][0]["output"]
def test_a_driver_without_an_index_to_load_installs_straight_away(self):
agent = _Agent()
agent.refresh_available_updates = None # type: ignore[assignment]
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is True
assert agent.calls[0] == ("ensure_package", "zabbix-agent")
def test_a_closed_firewall_starts_nothing(self):
agent = _Agent(refuse=frozenset({"firewall"}))
assert agent.configure_zabbix_agent(SETTINGS)["success"] is False
assert ("restart", "zabbix-agent") not in agent.calls
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
agent = _Agent(refuse=frozenset({"enable"}))
result = agent.configure_zabbix_agent(SETTINGS)
assert result["success"] is False
assert ("restart", "zabbix-agent") not in agent.calls
assert "cannot enable" in result["output"]
def test_the_package_and_service_names_are_the_driver_s(self):
agent = _Agent()
agent.ZABBIX_AGENT_PACKAGE = "zabbix-agentd"
agent.ZABBIX_AGENT_SERVICE = "zabbix_agentd"
agent.configure_zabbix_agent(SETTINGS)
assert ("ensure_package", "zabbix-agentd") in agent.calls
assert ("restart", "zabbix_agentd") in agent.calls
def test_settings_that_do_not_hold_are_refused_before_anything_changes(self):
agent = _Agent()
with pytest.raises(ValueError):
agent.configure_zabbix_agent({**SETTINGS, "server": "a\nb"})
assert agent.calls == []